From Cecil at decebal.nl  Fri May  1 00:19:26 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 06:19:26 +0200
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <mailman.135.1430413555.3680.python-list@python.org>
 <mhtnpg$fsf$1@dont-email.me> <87mw1ptpr0.fsf@Equus.decebal.nl>
 <mailman.146.1430427228.3680.python-list@python.org>
Message-ID: <87iocdrktt.fsf@Equus.decebal.nl>

Op Thursday 30 Apr 2015 22:53 CEST schreef Mark Lawrence:

> On 30/04/2015 19:50, Cecil Westerhof wrote:
>> Op Thursday 30 Apr 2015 19:12 CEST schreef Rob Gaddi:
>>
>>> This also leads to a unrelated question, Cecil. Given that you
>>> really are just starting to get your Python feet under you, why
>>> are you using Python2? Python3 is the standard now, Python2 is
>>> really just given legacy support. I'd understand if you were
>>> trying to maintain an old codebase with lots of legacy code that
>>> was having "problematic" migrations, but with the opportunity to
>>> start fresh? Start fresh. You'll be happier for it.
>>
>> I try to write code that works with both. Wen looking around in the
>> Netherlands there are more job opportunities with Python 2 as with
>> Python 3. So at the moment I find Python 2 more important, without
>> forgetting Python 3.
>>
>
> You might find this useful then in you haven't already seen it
> https://docs.python.org/3/howto/pyporting.html
>
> I must also confess to being highly impressed, it's a breath of
> fresh air having an apprentice Pythonista who is looking at doing
> things the Pythonic way :)

When in Rome, do as the Romans do.

Besides: there probably is a reason for the Pythonic way.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Fri May  1 00:36:01 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 06:36:01 +0200
Subject: Is my implementation of happy number OK
References: <87oam5vc8k.fsf@Equus.decebal.nl>
 <mailman.140.1430420006.3680.python-list@python.org>
 <slrnmk5esv.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.155.1430437953.3680.python-list@python.org>
Message-ID: <87a8xosymm.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 01:52 CEST schreef Dave Angel:

> On 04/30/2015 07:31 PM, Jon Ribbens wrote:
>> On 2015-04-30, Dave Angel <davea at davea.name> wrote:
>>> Finally, I did some testing on Jon Ribben's version. His was
>>> substantially faster for smaller sets, and about the same for
>>> 10*7. So it's likely it'll be slower than yours and mine for
>>> 10**8.
>>
>> You know what they say about assumptions. Actually, my version is
>> six times faster for 10**8 (running under Python 3.4).
>>
>>> But the real reason I didn't like it was it produced a much larger
>>> set of happy_numbers, which could clog memory a lot sooner.  For
>>> 10**7 items, I had 3250 happy members, and 19630 unhappy.  And Jon
>>> had 1418854 happy members.
>>
>> Er, what? You're complaining that mine is less efficient by not
>> producing the wrong output?
>>
>
> It's not intended as a criticism; you solved a different problem.
> The problem Cecil was solving was to determine if a particular
> number is happy. The problem you solved was to make a list of all
> values under a particular limit that are happy.

I can also produce a list. I have the following function for that:
    def happy_number_list(n):
        found = []

        for i in range(1, n + 1):
            if happy_number(i):
                found.append(i)
        return found

But the idea is that if you only want to know if a certain number is
happy or not, it is a waist to determine it for every number.

I have to look at the speed increase to determine if it would be
interesting to make a dedicated version for it.
>
> Both produce identical results for the Cecil purpose, and yours is
> faster if one wants all the values. But if one wants a sampling of
> values, his function will fetch them quickly, and even if you want
> them all, his function will use much less memory.
>
> He keeps only one permutation of each value in the set, for
> substantial savings in space. For example, he might just keep 28,
> while you keep 28 and 82, 208, 280, 802, and 820.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From breamoreboy at yahoo.co.uk  Fri May  1 00:41:37 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 01 May 2015 05:41:37 +0100
Subject: l = range(int(1E9))
In-Reply-To: <87iocdrktt.fsf@Equus.decebal.nl>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <mailman.135.1430413555.3680.python-list@python.org>
 <mhtnpg$fsf$1@dont-email.me> <87mw1ptpr0.fsf@Equus.decebal.nl>
 <mailman.146.1430427228.3680.python-list@python.org>
 <87iocdrktt.fsf@Equus.decebal.nl>
Message-ID: <mhv064$ib0$1@ger.gmane.org>

On 01/05/2015 05:19, Cecil Westerhof wrote:
> Op Thursday 30 Apr 2015 22:53 CEST schreef Mark Lawrence:
>
>> On 30/04/2015 19:50, Cecil Westerhof wrote:
>>> Op Thursday 30 Apr 2015 19:12 CEST schreef Rob Gaddi:
>>>
>>>> This also leads to a unrelated question, Cecil. Given that you
>>>> really are just starting to get your Python feet under you, why
>>>> are you using Python2? Python3 is the standard now, Python2 is
>>>> really just given legacy support. I'd understand if you were
>>>> trying to maintain an old codebase with lots of legacy code that
>>>> was having "problematic" migrations, but with the opportunity to
>>>> start fresh? Start fresh. You'll be happier for it.
>>>
>>> I try to write code that works with both. Wen looking around in the
>>> Netherlands there are more job opportunities with Python 2 as with
>>> Python 3. So at the moment I find Python 2 more important, without
>>> forgetting Python 3.
>>>
>>
>> You might find this useful then in you haven't already seen it
>> https://docs.python.org/3/howto/pyporting.html
>>
>> I must also confess to being highly impressed, it's a breath of
>> fresh air having an apprentice Pythonista who is looking at doing
>> things the Pythonic way :)
>
> When in Rome, do as the Romans do.
>
> Besides: there probably is a reason for the Pythonic way.
>

No probably about it, the threat of the Comfy Chair...

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From steve+comp.lang.python at pearwood.info  Fri May  1 00:42:04 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 01 May 2015 14:42:04 +1000
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
Message-ID: <5543041d$0$12996$c3e8da3$5496439d@news.astraweb.com>

On Fri, 1 May 2015 02:06 am, Cecil Westerhof wrote:

> If I execute:
>     l = range(int(1E9)


Others have already answered your questions about memory. Let me answer the
question you didn't ask about style :-)


Don't use "l" as a variable name, as it looks too much like 1. Better to use
L, or even better, a meaningful name.

Rather than convert a float 1E9 to an int at runtime, better to use an int:

range(10**9)


With recent versions of CPython, the compiler has a keyhole optimiser which
does constant folding. For implementations of Python which don't do
constant folding, 10**9 is likely to be faster than int(1E9) -- but even if
it isn't, does it matter? It will be very fast one way of the other. The
important thing is that 10**9 expresses the intention to use an integer in
a more direct fashion than using 1E9.



-- 
Steven



From steve+comp.lang.python at pearwood.info  Fri May  1 00:46:33 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 01 May 2015 14:46:33 +1000
Subject: Nuitka Progress in Spring 2015
References: <mailman.147.1430427684.3680.python-list@python.org>
Message-ID: <5543052a$0$12996$c3e8da3$5496439d@news.astraweb.com>

On Fri, 1 May 2015 07:01 am, Mark Lawrence wrote:

> Sales and marketing doesn't appear to be Kay Hayen's great strength so
> I'm taking a massive liberty and flagging Nuitka up here anyway.
> 
> http://nuitka.net/posts/nuitka-progress-spring-2015.html

Anyone care to summarise the highlights for those of us who just crashed
their web browser for the second time today? (F*%$ing javascript...)




-- 
Steven



From Cecil at decebal.nl  Fri May  1 01:04:22 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 07:04:22 +0200
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <mailman.135.1430413555.3680.python-list@python.org>
 <mhtnpg$fsf$1@dont-email.me> <87mw1ptpr0.fsf@Equus.decebal.nl>
 <mailman.146.1430427228.3680.python-list@python.org>
 <mhu6df$6bs$2@dont-email.me>
 <CAPTjJmrs=a=L9TiRUnBPs561bEQFsoBv+3oaQmez5Yi-1gEW6A@mail.gmail.com>
 <mailman.152.1430435562.3680.python-list@python.org>
Message-ID: <87383gsxbd.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 01:12 CEST schreef Ben Finney:

> Chris Angelico <rosuav at gmail.com> writes:
>
>> Very easily and simply: Python 3 and Python 2 will always install
>> separately, and the only possible conflicts are over the "python"
>> command in PATH and which program is associated with ".py" files.
>
> Calling ?python? is now ambiguous, and with Python 2 slipping
> inexorably into the past, increasingly the ?python? command is the
> wrong choice for code that we want to survive in the future.
>
> I am seeing a growing call, with which I agree, to recommend
> explicitly calling ?python2? or ?python3? as commands.
>
> That includes when we type it for direct invocation, or when we set
> it as the command for automatic execution of a program (e.g. in the
> ?shebang? line of a program).
>
> Use the command ?python2? or ?python3? to be explicit about which
> Python version you intend to run.

Good tip. I used python and python3, but there is also a python2. I
learn myself to use that instead of python.

By the way: I also see python3.4 and python3.4m. Any idea where the m
stands for?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Fri May  1 01:13:43 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 07:13:43 +0200
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <5543041d$0$12996$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87y4l8ribc.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 06:42 CEST schreef Steven D'Aprano:

> On Fri, 1 May 2015 02:06 am, Cecil Westerhof wrote:
>
>> If I execute:
>> l = range(int(1E9)
>
>
> Others have already answered your questions about memory. Let me
> answer the question you didn't ask about style :-)

That can be very useful.


> Don't use "l" as a variable name, as it looks too much like 1.
> Better to use L, or even better, a meaningful name.
>
> Rather than convert a float 1E9 to an int at runtime, better to use
> an int:
>
> range(10**9)
>
>
> With recent versions of CPython, the compiler has a keyhole
> optimiser which does constant folding. For implementations of Python
> which don't do constant folding, 10**9 is likely to be faster than
> int(1E9) -- but even if it isn't, does it matter? It will be very
> fast one way of the other. The important thing is that 10**9
> expresses the intention to use an integer in a more direct fashion
> than using 1E9.

It was just a short hack to show the problem. In real code I would
never use l. But I'll remember to use 10**9 instead of 1E9.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Fri May  1 01:19:51 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 07:19:51 +0200
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <mailman.139.1430418355.3680.python-list@python.org>
 <mhttda$pf$1@speranza.aioe.org>
 <mailman.141.1430420396.3680.python-list@python.org>
 <87zj5ps73z.fsf@Equus.decebal.nl>
 <mailman.149.1430430673.3680.python-list@python.org>
Message-ID: <87wq0sri14.fsf@Equus.decebal.nl>

Op Thursday 30 Apr 2015 23:41 CEST schreef Tim Chase:

> On 2015-04-30 22:18, Cecil Westerhof wrote:
>> Op Thursday 30 Apr 2015 20:59 CEST schreef Dave Angel:
>>> ulimit is your friend if you've got a program that wants to gobble
>>> up all of swap space.
>>
>> Yes, my system is openSUSE 64 bit. I really should look into
>> ulimit. The default is:
> [snip]
>> max memory size         (kbytes, -m) unlimited
>
> Note that AFAIK, "ulimit -m" doesn't work on Linux
>
> http://unix.stackexchange.com/questions/129587/does-ulimit-m-not-work-on-modern-linux
>
> Based on some quick testing[1], it doesn't appear to work on OpenBSD
> or FreeBSD either.

Yes, as I already found out you need to use -v. But it would be nice
that there was an indication that -m is obsolete.

Depending on the available memory you could use something like:
    ulimit -v $((4 * 1024 * 1024))

I can then do:
    l = range(int(1E8))
but:
    l = range(int(1E9))
gives MemoryError.

And if I can also do (from another thread):
    happy_number_list(int(1E8))
if I did not something memory consuming.


And if a (Python) process should use significantly less, it could be a
good idea to tweak the ulimit call.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From steve+comp.lang.python at pearwood.info  Fri May  1 01:20:24 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 01 May 2015 15:20:24 +1000
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
Message-ID: <55430d19$0$12979$c3e8da3$5496439d@news.astraweb.com>

On Fri, 1 May 2015 03:20 am, Ben Finney wrote:

> Jon Ribbens <jon+usenet at unequivocal.co.uk> writes:
> 
>> On 2015-04-30, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> > If I execute:
>> > l = range(int(1E9)
>> >
>> > The python process gobbles up all the memory and is killed. [?] Is
>> > there a way to circumvent Python claiming all the memory?
> 
> You seem to be asking for a way to stop a program doing exactly what
> it's written to do. I don't know what kind of answer you expect.

How about a way to stop a program from doing what you didn't intend it to
do? As we all know quite well, computers are quite stupid, they do what
they are told, not what we want.

It's not just to protect against bugs and mistakes. It's also to protect one
user from another user, or a single user from hostile code. Just because
Cecil tells the computer "use up 12GB of RAM in one great big list" doesn't
mean that the other users of that computer have to acquiesce to that
request.


>> It's your operating system's job to handle processes.
> 
> Indeed. In this case, the program is written to gobble up memory, and
> the operating system kills it. To ?circumvent? that behaviour surely
> reveals the problem: that the operating system isn't handling processes
> very well.

Indeed indeed :-) 

Some programming language virtual machines limit how much memory they will
use. The CPython VM isn't one of those, although I understand that both
Jython and IronPython are. (I may be wrong -- corrections are welcome.)
That means you have to use the OS to limit how much memory CPython will
use, if you can.




-- 
Steven



From ben+python at benfinney.id.au  Fri May  1 01:45:47 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 01 May 2015 15:45:47 +1000
Subject: Nuitka Progress in Spring 2015
References: <mailman.147.1430427684.3680.python-list@python.org>
 <5543052a$0$12996$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <85vbgcg8ac.fsf@benfinney.id.au>

Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:

> On Fri, 1 May 2015 07:01 am, Mark Lawrence wrote:
>
> > http://nuitka.net/posts/nuitka-progress-spring-2015.html
>
> Anyone care to summarise the highlights

Nuitka Progress in Spring 2015

* SSA (Single State Assignment Form): will allow Nuitka to be an
  *optimising* compiler for Python.

* Scalability: Nuitka uses less memory now, and generates smaller code
  output.

* Compatibility: will pass the Python 3.4 test suite. Python 3.5 support
  is coming along. Problems with the import system are gone after a
  complete rewrite of that code.

* Performance: improvements are spotty, but real. More progress needs to
  be made.

* Standalone: steady improvement but no big news.

* Debian Jessie released this month, and contains a packaged Nuitka.

* Funding continues; please send more, this is an independent project.

* EuroPython 2015 will have an appearance by the Nuitka team.

* Collaborators: the infrastructure is there now to join and work on
  developing Nuitka.


> for those of us who just crashed their web browser for the second time
> today? (F*%$ing javascript...)

<URL:http://vimeo.com/93691338>

-- 
 \      ?A lie can be told in a few words. Debunking that lie can take |
  `\   pages. That is why my book? is five hundred pages long.? ?Chris |
_o__)                                                Rodda, 2011-05-05 |
Ben Finney



From steve+comp.lang.python at pearwood.info  Fri May  1 01:56:56 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 01 May 2015 15:56:56 +1000
Subject: Python is not bad ;-)
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net>
Message-ID: <554315a9$0$13003$c3e8da3$5496439d@news.astraweb.com>

On Thu, 30 Apr 2015 08:16 pm, Marko Rauhamaa wrote:

> Still, Python has features that defy effective optimization. Most
> notably, Python's dot notation translates into a hash table lookup -- or
> worse.


Effective optimization may be difficult, but it isn't impossible. PyPy has a
very effective Just In Time optimizer.

http://speed.pypy.org/

PyPy currently ranges from approximately 1.25 times to 50 times faster than
CPython, with an average of over 7 times faster.

And now PyPy also has a version that runs without the GIL:

http://morepypy.blogspot.com.au/2015/03/pypy-stm-251-released.html


-- 
Steven



From Cecil at decebal.nl  Fri May  1 02:08:49 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 08:08:49 +0200
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <55430d19$0$12979$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87sibgrfri.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 07:20 CEST schreef Steven D'Aprano:

> Some programming language virtual machines limit how much memory
> they will use. The CPython VM isn't one of those, although I
> understand that both Jython and IronPython are. (I may be wrong --

Jython runs in the JVM, so Jython is.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Fri May  1 02:27:04 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 08:27:04 +0200
Subject: Looks like Python 3 files do not confirm to Python 3
Message-ID: <87mw1orex3.fsf@Equus.decebal.nl>

On my system in:
/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py

it says:
    try:
        from ndg.httpsclient.subj_alt_name import SubjectAltName
        from pyasn1.codec.der import decoder as der_decoder
        SUBJ_ALT_NAME_SUPPORT = True
    except ImportError, e:
        SUBJ_ALT_NAME_SUPPORT = False
        SUBJ_ALT_NAME_SUPPORT_MSG = (
            'SubjectAltName support is disabled - check pyasn1 package '
            'installation to enable'
        )
        import warnings
        warnings.warn(SUBJ_ALT_NAME_SUPPORT_MSG)

which gives:
    File "/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py", line 17
      except ImportError, e:

Does my system have outdated files, or are there still Python 3 files
that do not conform to Python 3?

The only real python program I use at the moment uses:
    import urllib3.contrib.pyopenssl

and then you get this error.


So another reason to stay with Python 2. (While still writing code
that works in Python3.)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From steve+comp.lang.python at pearwood.info  Fri May  1 03:00:37 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 01 May 2015 17:00:37 +1000
Subject: Looks like Python 3 files do not confirm to Python 3
References: <87mw1orex3.fsf@Equus.decebal.nl>
Message-ID: <55432496$0$12994$c3e8da3$5496439d@news.astraweb.com>

On Fri, 1 May 2015 04:27 pm, Cecil Westerhof wrote:

> On my system in:
> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py

ndg is a third-party package, not part of the Python 3 standard library.

> it says:
>     try:
>         from ndg.httpsclient.subj_alt_name import SubjectAltName
>         from pyasn1.codec.der import decoder as der_decoder
>         SUBJ_ALT_NAME_SUPPORT = True
>     except ImportError, e:
[...]
> which gives:
>     File
>     "/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py",
>     line 17
>       except ImportError, e:

I think you're missing the last line of the error. I'm guessing it was
probably NameError: name 'e' is not defined.


> Does my system have outdated files, or are there still Python 3 files
> that do not conform to Python 3?

I would say, either you have accidentally installed a Python 2 file in your
Python 3 library, or it is simply a bug in ndg.


-- 
Steven



From steve+comp.lang.python at pearwood.info  Fri May  1 03:03:50 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 01 May 2015 17:03:50 +1000
Subject: Python is not bad ;-)
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
Message-ID: <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>

On Thu, 30 Apr 2015 09:30 pm, Cecil Westerhof wrote:

> Tail recursion would nice to have also.

People coming from functional languages like Lisp and Haskell often say
that, but how many recursive algorithms naturally take a tail-call form?
Not that many.

I suppose that it would be nice if Python let you optionally use tail-call
optimization, but that might be tricky in practice.



-- 
Steven



From Cecil at decebal.nl  Fri May  1 03:16:30 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 09:16:30 +0200
Subject: Looks like Python 3 files do not confirm to Python 3
References: <87mw1orex3.fsf@Equus.decebal.nl>
Message-ID: <87ioccrcmp.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 08:27 CEST schreef Cecil Westerhof:

> On my system in:
> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>
> it says:
> try:
> from ndg.httpsclient.subj_alt_name import SubjectAltName
> from pyasn1.codec.der import decoder as der_decoder
> SUBJ_ALT_NAME_SUPPORT = True
> except ImportError, e:
> SUBJ_ALT_NAME_SUPPORT = False
> SUBJ_ALT_NAME_SUPPORT_MSG = (
> 'SubjectAltName support is disabled - check pyasn1 package '
> 'installation to enable'
> )
> import warnings
> warnings.warn(SUBJ_ALT_NAME_SUPPORT_MSG)
>
> which gives: File
> "/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py",
> line 17 except ImportError, e:

It is solved by making it:
    except (ImportError) as e:

The same for:
    /usr/lib/python3.4/site-packages/ndg/httpsclient/subj_alt_name.py

I would not mind to solve those and others if I could contribute them
to the Python community. What is the procedure for that?

>
> Does my system have outdated files, or are there still Python 3
> files that do not conform to Python 3?
>
> The only real python program I use at the moment uses:
> import urllib3.contrib.pyopenssl
>
> and then you get this error.
>
>
> So another reason to stay with Python 2. (While still writing code
> that works in Python3.)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Fri May  1 03:22:54 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 09:22:54 +0200
Subject: Rewriting to Python 3
Message-ID: <87egn0rcc1.fsf@Equus.decebal.nl>

On my system I have:
    PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
in:
    /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py

In Python 3 that gives:
    TypeError: unsupported operand type(s) for +: 'dict_keys' and 'dict_values'

How should I rewrite this?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From jon+usenet at unequivocal.co.uk  Fri May  1 03:23:05 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Fri, 1 May 2015 07:23:05 +0000 (UTC)
Subject: Is my implementation of happy number OK
References: <87oam5vc8k.fsf@Equus.decebal.nl>
 <mailman.140.1430420006.3680.python-list@python.org>
 <slrnmk5esv.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.155.1430437953.3680.python-list@python.org>
Message-ID: <slrnmk6agn.apd.jon+usenet@frosty.unequivocal.co.uk>

On 2015-04-30, Dave Angel <davea at davea.name> wrote:
> On 04/30/2015 07:31 PM, Jon Ribbens wrote:
>> On 2015-04-30, Dave Angel <davea at davea.name> wrote:
>>> But the real reason I didn't like it was it produced a much larger
>>> set of happy_numbers, which could clog memory a lot sooner.  For
>>> 10**7 items, I had 3250 happy members, and 19630 unhappy.  And Jon
>>> had 1418854 happy members.
>>
>> Er, what? You're complaining that mine is less efficient by not
>> producing the wrong output?
>
> It's not intended as a criticism;  you solved a different problem.  The 
> problem Cecil was solving was to determine if a particular number is 
> happy.  The problem you solved was to make a list of all values under a 
> particular limit that are happy.
>
> Both produce identical results for the Cecil purpose, and yours is 
> faster if one wants all the values.  But if one wants a sampling of 
> values, his function will fetch them quickly, and even if you want them 
> all, his function will use much less memory.

I must admit, I'm still not understanding. If you want to know only
whether or not int(1e7) is happy then the calculation takes no
measurable time or memory.


From Cecil at decebal.nl  Fri May  1 03:36:04 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 09:36:04 +0200
Subject: Looks like Python 3 files do not confirm to Python 3
References: <87mw1orex3.fsf@Equus.decebal.nl>
 <55432496$0$12994$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87a8xorbq3.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 09:00 CEST schreef Steven D'Aprano:

> On Fri, 1 May 2015 04:27 pm, Cecil Westerhof wrote:
>
>> On my system in:
>> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>
> ndg is a third-party package, not part of the Python 3 standard
> library.

Oops, barking up to the wrong tree. Sorry.


>> it says:
>> try:
>> from ndg.httpsclient.subj_alt_name import SubjectAltName
>> from pyasn1.codec.der import decoder as der_decoder
>> SUBJ_ALT_NAME_SUPPORT = True
>> except ImportError, e:
> [...]
>> which gives: File
>> "/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py",
>> line 17 except ImportError, e:
>
> I think you're missing the last line of the error. I'm guessing it
> was probably NameError: name 'e' is not defined.

Nope:
        except ImportError, e:
                          ^
    SyntaxError: invalid syntax

It is solved by:
    except (ImportError) as e:

But then:
        PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
    TypeError: unsupported operand type(s) for +: 'dict_keys' and 'dict_values'


>> Does my system have outdated files, or are there still Python 3
>> files that do not conform to Python 3?
>
> I would say, either you have accidentally installed a Python 2 file
> in your Python 3 library, or it is simply a bug in ndg.

I will contact the ndg people.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Fri May  1 03:47:05 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 09:47:05 +0200
Subject: Python is not bad ;-)
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87618crb7q.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 09:03 CEST schreef Steven D'Aprano:

> On Thu, 30 Apr 2015 09:30 pm, Cecil Westerhof wrote:
>
>> Tail recursion would nice to have also.
>
> People coming from functional languages like Lisp and Haskell often
> say that, but how many recursive algorithms naturally take a
> tail-call form? Not that many.

When I was playing with Scala there where a few cases where tail
recursion made a significant performance boost.

Is some time ago, so I do not remember which.


> I suppose that it would be nice if Python let you optionally use
> tail-call optimization, but that might be tricky in practice.

That is the difference between Scala and Clojure. Scala does it
silently, while by Clojure you have to say you want it. When a chance
breaks tail recursion Scala just compiles, but your code becomes
slower, Clojure does not compile. I prefer the Clojure variant.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From ben+python at benfinney.id.au  Fri May  1 03:56:14 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 01 May 2015 17:56:14 +1000
Subject: Rewriting to Python 3
References: <87egn0rcc1.fsf@Equus.decebal.nl>
Message-ID: <85r3r0g28x.fsf@benfinney.id.au>

Cecil Westerhof <Cecil at decebal.nl> writes:

> On my system I have:
>     PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
> in:
>     /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py

Are you in contact with the author of that third-party library? Why is
it in your Python 3.4 site-packages?

> In Python 3 that gives:
>     TypeError: unsupported operand type(s) for +: 'dict_keys' and 'dict_values'
>
> How should I rewrite this?

My attempt:

    PARSER_RE_STR = "/({keys}|{values})=".format(
            keys=DN_LUT.keys(), values=DN_LUT.values())

Explicit is better than implicit, especially in mini-languages like
string interpolation.

-- 
 \      ?Often, the surest way to convey misinformation is to tell the |
  `\               strict truth.? ?Mark Twain, _Following the Equator_ |
_o__)                                                                  |
Ben Finney



From steve+comp.lang.python at pearwood.info  Fri May  1 04:03:02 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 01 May 2015 18:03:02 +1000
Subject: Jedi is now a static analysis library
Message-ID: <55433337$0$12987$c3e8da3$5496439d@news.astraweb.com>

Jedi, the refactoring and static analysis tool, is now able to be used as a
library for your own code. See the announcement here:

https://mail.python.org/pipermail/code-quality/2015-April/000534.html


Jedi itself:

https://github.com/davidhalter/jedi/



-- 
Steven



From breamoreboy at yahoo.co.uk  Fri May  1 04:05:27 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 01 May 2015 09:05:27 +0100
Subject: Looks like Python 3 files do not confirm to Python 3
In-Reply-To: <87ioccrcmp.fsf@Equus.decebal.nl>
References: <87mw1orex3.fsf@Equus.decebal.nl> <87ioccrcmp.fsf@Equus.decebal.nl>
Message-ID: <mhvc4a$456$1@ger.gmane.org>

On 01/05/2015 08:16, Cecil Westerhof wrote:
> Op Friday 1 May 2015 08:27 CEST schreef Cecil Westerhof:
>
>> On my system in:
>> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>>
>> it says:
>> try:
>> from ndg.httpsclient.subj_alt_name import SubjectAltName
>> from pyasn1.codec.der import decoder as der_decoder
>> SUBJ_ALT_NAME_SUPPORT = True
>> except ImportError, e:
>> SUBJ_ALT_NAME_SUPPORT = False
>> SUBJ_ALT_NAME_SUPPORT_MSG = (
>> 'SubjectAltName support is disabled - check pyasn1 package'
>> 'installation to enable'
>> )
>> import warnings
>> warnings.warn(SUBJ_ALT_NAME_SUPPORT_MSG)
>>
>> which gives: File
>> "/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py",
>> line 17 except ImportError, e:
>
> It is solved by making it:
>      except (ImportError) as e:
>
> The same for:
>      /usr/lib/python3.4/site-packages/ndg/httpsclient/subj_alt_name.py
>
> I would not mind to solve those and others if I could contribute them
> to the Python community. What is the procedure for that?
>

You'll have to find out who supports ndg as it's not standard Python. 
The big clue is the site-packages folder name, that's used for third 
party packages.  From searching I think it's this 
https://github.com/cedadev/ndg_httpsclient/ but please don't quote me on 
that.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From __peter__ at web.de  Fri May  1 04:09:24 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 01 May 2015 10:09:24 +0200
Subject: Rewriting to Python 3
References: <87egn0rcc1.fsf@Equus.decebal.nl>
Message-ID: <mhvcbm$79j$1@ger.gmane.org>

Cecil Westerhof wrote:

> On my system I have:
>     PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
> in:
>     /usr/lib/python3.4/site-
packages/ndg/httpsclient/ssl_peer_verification.py
> 
> In Python 3 that gives:
>     TypeError: unsupported operand type(s) for +: 'dict_keys' and
>     'dict_values'
> 
> How should I rewrite this?


There's a tool called 2to3 -- it doesn't produce perfect code but it can 
cope with the mechanical changes:
 
$ cat demo.py    
d = dict("aA bB cC".split())

try:
    print d.keys() + d.values()
except Exception, e:
    print e
$ 2to3 -w demo.py
[...]
$ cat demo.py
d = dict("aA bB cC".split())

try:
    print(list(d.keys()) + list(d.values()))
except Exception as e:
    print(e)
$ python3 demo.py
['c', 'a', 'b', 'C', 'A', 'B']
$



From breamoreboy at yahoo.co.uk  Fri May  1 04:10:27 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 01 May 2015 09:10:27 +0100
Subject: Rewriting to Python 3
In-Reply-To: <87egn0rcc1.fsf@Equus.decebal.nl>
References: <87egn0rcc1.fsf@Equus.decebal.nl>
Message-ID: <mhvcdm$874$1@ger.gmane.org>

On 01/05/2015 08:22, Cecil Westerhof wrote:
> On my system I have:
>      PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
> in:
>      /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>
> In Python 3 that gives:
>      TypeError: unsupported operand type(s) for +: 'dict_keys' and 'dict_values'
>
> How should I rewrite this?
>

I'd run the entire package through the 2to3 tool that's part of the 
standard library, or uninstall that package and get an up to date 
version that actually supports Python 3.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Fri May  1 04:13:29 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 1 May 2015 18:13:29 +1000
Subject: Rewriting to Python 3
In-Reply-To: <87egn0rcc1.fsf@Equus.decebal.nl>
References: <87egn0rcc1.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmo8tFrEVsus90mucNDP3LTYvfknyDOGauNq77pVZjFJQA@mail.gmail.com>

On Fri, May 1, 2015 at 5:22 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> On my system I have:
>     PARSER_RE_STR = '/(%s)=' % '|'.join(DN_LUT.keys() + DN_LUT.values())
> in:
>     /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>
> In Python 3 that gives:
>     TypeError: unsupported operand type(s) for +: 'dict_keys' and 'dict_values'
>
> How should I rewrite this?

The keys() and values() methods used to return lists, now they return
views. If you explicitly listify them, it should work; alternatively,
pipe-join each one separately and then pipe-join the result:

PARSER_RE_STR = '/(%s|%s)=' % ('|'.join(DN_LUT.keys()),
'|'.join(DN_LUT.values()))

Note that this is slightly different from the above; in the case of an
empty dict, the original produces empty parentheses, but my
alternative will place a junk pipe in there.

ChrisA


From steve+comp.lang.python at pearwood.info  Fri May  1 04:27:08 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 01 May 2015 18:27:08 +1000
Subject: Is my implementation of happy number OK
References: <87oam5vc8k.fsf@Equus.decebal.nl>
 <mailman.140.1430420006.3680.python-list@python.org>
 <slrnmk5esv.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.155.1430437953.3680.python-list@python.org>
 <slrnmk6agn.apd.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <554338dd$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Fri, 1 May 2015 05:23 pm, Jon Ribbens wrote:

> On 2015-04-30, Dave Angel <davea at davea.name> wrote:
>> On 04/30/2015 07:31 PM, Jon Ribbens wrote:
>>> On 2015-04-30, Dave Angel <davea at davea.name> wrote:
>>>> But the real reason I didn't like it was it produced a much larger
>>>> set of happy_numbers, which could clog memory a lot sooner.  For
>>>> 10**7 items, I had 3250 happy members, and 19630 unhappy.  And Jon
>>>> had 1418854 happy members.
>>>
>>> Er, what? You're complaining that mine is less efficient by not
>>> producing the wrong output?
>>
>> It's not intended as a criticism;  you solved a different problem.  The
>> problem Cecil was solving was to determine if a particular number is
>> happy.  The problem you solved was to make a list of all values under a
>> particular limit that are happy.
>>
>> Both produce identical results for the Cecil purpose, and yours is
>> faster if one wants all the values.  But if one wants a sampling of
>> values, his function will fetch them quickly, and even if you want them
>> all, his function will use much less memory.
> 
> I must admit, I'm still not understanding. If you want to know only
> whether or not int(1e7) is happy then the calculation takes no
> measurable time or memory.

Oh, I'm sure somebody would be able to measure it...

Rather than 10**7, how about trying (10**500 + 2). Is it happy?

Using the Python code from Wikipedia:
https://en.wikipedia.org/wiki/Happy_number

SQUARE = dict([(c, int(c)**2) for c in "0123456789"])
def is_happy(n):
  while (n > 1) and (n != 4):
    n = sum(SQUARE[d] for d in str(n))
  return n == 1


I can calculate whether n=10**500 + 2 is happy in less than a millisecond on
my computer. Using your version doesn't even work, as it would require
significantly more than 1000 terrabytes of RAM just to hold the results. I
don't have that much memory.

Your version is a reasonable answer to a different question, but it doesn't
scale to very large inputs.



-- 
Steven



From Cecil at decebal.nl  Fri May  1 05:18:38 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 11:18:38 +0200
Subject: Looks like Python 3 files do not confirm to Python 3
References: <87mw1orex3.fsf@Equus.decebal.nl> <87ioccrcmp.fsf@Equus.decebal.nl>
 <mailman.2.1430467545.3347.python-list@python.org>
Message-ID: <87sibgpsep.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 10:05 CEST schreef Mark Lawrence:

> On 01/05/2015 08:16, Cecil Westerhof wrote:
>> Op Friday 1 May 2015 08:27 CEST schreef Cecil Westerhof:
>>
>>> On my system in:
>>> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>>>
>>> it says:
>>> try:
>>> from ndg.httpsclient.subj_alt_name import SubjectAltName
>>> from pyasn1.codec.der import decoder as der_decoder
>>> SUBJ_ALT_NAME_SUPPORT = True
>>> except ImportError, e:
>>> SUBJ_ALT_NAME_SUPPORT = False
>>> SUBJ_ALT_NAME_SUPPORT_MSG = (
>>> 'SubjectAltName support is disabled - check pyasn1 package'
>>> 'installation to enable'
>>> )
>>> import warnings
>>> warnings.warn(SUBJ_ALT_NAME_SUPPORT_MSG)
>>>
>>> which gives: File
>>> "/usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py",
>>> line 17 except ImportError, e:
>>
>> It is solved by making it:
>> except (ImportError) as e:
>>
>> The same for:
>> /usr/lib/python3.4/site-packages/ndg/httpsclient/subj_alt_name.py
>>
>> I would not mind to solve those and others if I could contribute
>> them to the Python community. What is the procedure for that?
>>
>
> You'll have to find out who supports ndg as it's not standard
> Python. The big clue is the site-packages folder name, that's used
> for third party packages. From searching I think it's this
> https://github.com/cedadev/ndg_httpsclient/ but please don't quote
> me on that.

Yeah, I was barking up to the wrong tree.

I found an email address in the sources. The copyright is from 2012,
so hopefully it is still supported.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From the.loeki at gmail.com  Fri May  1 05:42:25 2015
From: the.loeki at gmail.com (the.loeki at gmail.com)
Date: Fri, 1 May 2015 02:42:25 -0700 (PDT)
Subject: Am I missing something here? ipaddress vs socket
Message-ID: <f46cdd6e-5c94-4370-a368-cce0bd68c997@googlegroups.com>

Hi all,

Given the following code:

import ipaddress
import socket

ip = ipaddress.ip_address(mystring)
sock_family = ip.????
socket = socket.socket(sock_family, socket.SOCK_STREAM)

Am I crazy or is this undoable?

sock.AF_INET == 2
sock.AF_INET6 == 10
ip.version == 4 or 6




From newzodiak at hotmail.com  Fri May  1 05:56:25 2015
From: newzodiak at hotmail.com (newton avetisyan)
Date: Fri, 1 May 2015 09:56:25 +0000
Subject: No subject
Message-ID: <SNT406-EAS323AA516A560663F9C05B1EA1D50@phx.gbl>








Sent from Windows Mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150501/53cd1a28/attachment.html>

From rosuav at gmail.com  Fri May  1 06:11:04 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 1 May 2015 20:11:04 +1000
Subject: Am I missing something here? ipaddress vs socket
In-Reply-To: <f46cdd6e-5c94-4370-a368-cce0bd68c997@googlegroups.com>
References: <f46cdd6e-5c94-4370-a368-cce0bd68c997@googlegroups.com>
Message-ID: <CAPTjJmrAfC4fsQy3CAt5iEc80cDHXsLGNi0V56BQBh9w-pCrOA@mail.gmail.com>

On Fri, May 1, 2015 at 7:42 PM,  <the.loeki at gmail.com> wrote:
> Hi all,
>
> Given the following code:
>
> import ipaddress
> import socket
>
> ip = ipaddress.ip_address(mystring)
> sock_family = ip.????
> socket = socket.socket(sock_family, socket.SOCK_STREAM)
>
> Am I crazy or is this undoable?
>
> sock.AF_INET == 2
> sock.AF_INET6 == 10
> ip.version == 4 or 6

Are you trying to look up a name to get an address? Or just look up an
address? The easiest way would be to use a ternary if:

sock_family = sock.AF_INET if ip.version == 4 else sock.AF_INET6

But you may find it convenient to use a dedicated function for
establishing a connection, which could look up an AAAA or A record for
a name, then proceed through all addresses, attempting connections in
turn. I'm fairly sure one exists in Python, but I can't right now
remember the name.

ChrisA


From robin at reportlab.com  Fri May  1 06:29:12 2015
From: robin at reportlab.com (Robin Becker)
Date: Fri, 01 May 2015 11:29:12 +0100
Subject: GAE environment differences
Message-ID: <55435578.50003@chamonix.reportlab.co.uk>

A user reports getting KeyError/ImportError when executing GAE dev/live code 
looking like


import os
.......
fn = os.path.expanduser(os.path.join('~','.reportlab_settings'))
......

This works fine in cpython; so I assume that the GAE environment must be doing 
module resolution differently.

Is that the case or is it that os.path or os.path.expanduser doesn't exist?
-- 
Robin Becker



From alain at dpt-info.u-strasbg.fr  Fri May  1 06:31:01 2015
From: alain at dpt-info.u-strasbg.fr (Alain Ketterlin)
Date: Fri, 01 May 2015 12:31:01 +0200
Subject: Am I missing something here? ipaddress vs socket
References: <f46cdd6e-5c94-4370-a368-cce0bd68c997@googlegroups.com>
Message-ID: <87r3r07foa.fsf@dpt-info.u-strasbg.fr>

the.loeki at gmail.com writes:

> Given the following code:
>
> import ipaddress
> import socket
>
> ip = ipaddress.ip_address(mystring)
> sock_family = ip.????
> socket = socket.socket(sock_family, socket.SOCK_STREAM)
>
> Am I crazy or is this undoable?
>
> sock.AF_INET == 2
> sock.AF_INET6 == 10
> ip.version == 4 or 6

Check the value of the "version" attribute. Or use socket.getaddrinfo(),
which is more convenient if all you need is an address parser.

-- Alain.


From the.loeki at gmail.com  Fri May  1 06:41:43 2015
From: the.loeki at gmail.com (Ronald van Zantvoort)
Date: Fri, 1 May 2015 03:41:43 -0700 (PDT)
Subject: Am I missing something here? ipaddress vs socket
In-Reply-To: <mailman.6.1430475067.3347.python-list@python.org>
References: <f46cdd6e-5c94-4370-a368-cce0bd68c997@googlegroups.com>
 <mailman.6.1430475067.3347.python-list@python.org>
Message-ID: <c8a5569c-66aa-4b96-829e-36307d6947cb@googlegroups.com>

Hi Chris & Alain, thanks for  responding :)

On Friday, 1 May 2015 12:11:20 UTC+2, Chris Angelico  wrote:
> On Fri, May 1, 2015 at 7:42 PM,  Ronald van Zantvoort wrote:
> > Hi all,
> >
> > Given the following code:
> >
> > import ipaddress
> > import socket
> >
> > ip = ipaddress.ip_address(mystring)
> > sock_family = ip.????
> > socket = socket.socket(sock_family, socket.SOCK_STREAM)
> >
> > Am I crazy or is this undoable?
> >
> > sock.AF_INET == 2
> > sock.AF_INET6 == 10
> > ip.version == 4 or 6
> 
> Are you trying to look up a name to get an address? Or just look up an
> address? The easiest way would be to use a ternary if:
> 
> sock_family = sock.AF_INET if ip.version == 4 else sock.AF_INET6

mystring is already an IP address, sorry for not mentioning that.
The point is that ternary if; I've been scouring around the internet and there's a million different variations on the same idea. 
I'm completely flabberghasted it's not a simple property within the object though, e.g. ip.sock_family.

Was this a deliberate decision by the Python devs or can I try an RFE for that?

> But you may find it convenient to use a dedicated function for
> establishing a connection, which could look up an AAAA or A record for
> a name, then proceed through all addresses, attempting connections in
> turn. I'm fairly sure one exists in Python, but I can't right now
> remember the name.
> 
> ChrisA

Well you've already got socket.create_connection(), which (I think) does perform that lookup.


From kylotan at gmail.com  Fri May  1 07:01:26 2015
From: kylotan at gmail.com (Ben Sizer)
Date: Fri, 1 May 2015 04:01:26 -0700 (PDT)
Subject: ImportError with pickle (Python 2.7.9),
 possibly platform dependent
In-Reply-To: <mailman.108.1430354695.3680.python-list@python.org>
References: <494551ca-532f-4d4d-aff0-a3932416c8f4@googlegroups.com>
 <mailman.108.1430354695.3680.python-list@python.org>
Message-ID: <20a5c7bf-2163-4b7a-8495-30ce23239903@googlegroups.com>

On Thursday, 30 April 2015 01:45:05 UTC+1, Chris Angelico  wrote:
> On Thu, Apr 30, 2015 at 2:01 AM, Ben Sizer wrote:
> > 1) There clearly is a module named OMDBMap, and it's importable - it's there in the 2nd line of the traceback.
> >
> > Does anybody have any suggestions on how I can go about debugging this? Or refactoring it to avoid whatever is happening here?
> 
> Are you half way through importing it when this load() call happens?
> That might cause some issues.

No, we already imported OMDBMap at the top of OMDBSetup.

> Has your current directory been changed anywhere in there?

Good question. It turns out that the current directory seems to be $HOME when loading, but is the script directory during saving. This will be because the Linux server is running under mod_wsgi, whereas we run the save script in-place. Our Windows and Mac tests run via Flask's built-in server so the working directory is likely to be the same whether we're running the script that does pickle.dump or the whole app that does pickle.load.

> What happens if you catch this exception and print out sys.modules at
> that point?

Another good question, and this gives us the answer. The module lists are quite different, as I'd expect because the load happens in the context of the full application whereas the dump happens as a standalone script. But literally every module that is in the 'before dump' list is in the 'before load' list - except OMDBMap. Like the error says! What /is/ in the 'before load' list however is "my_wsgi_app.OMDBMap". The module has been imported, but the pickle algorithm is unable to reconcile the module in the WSGI app's namespace with the module referenced in the pickle file.

So... I don't know how to fix this, but I do now know why it fails, and I have a satisfactory answer for why it is acting differently on the Linux server (and that is just because that is the only one running under WSGI). Two out of three isn't bad!

Thanks,
-- 
Ben Sizer



From PointedEars at web.de  Fri May  1 07:15:29 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Fri, 01 May 2015 13:15:29 +0200
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <mailman.135.1430413555.3680.python-list@python.org>
 <mhtnpg$fsf$1@dont-email.me> <87mw1ptpr0.fsf@Equus.decebal.nl>
 <mailman.146.1430427228.3680.python-list@python.org>
 <mhu6df$6bs$2@dont-email.me>
 <CAPTjJmrs=a=L9TiRUnBPs561bEQFsoBv+3oaQmez5Yi-1gEW6A@mail.gmail.com>
 <mailman.152.1430435562.3680.python-list@python.org>
 <87383gsxbd.fsf@Equus.decebal.nl>
Message-ID: <3374990.nA8q5WTVfI@PointedEars.de>

Cecil Westerhof wrote:

> By the way: I also see python3.4 and python3.4m. Any idea where the m
> stands for?

I googled for ?python3.4m? and found as second result

<http://stackoverflow.com/questions/16675865/difference-between-python3-and-python3m-executibles>

In a nutshell: python3.4m was built with configure option ?--with-pymalloc? 
which causes the resulting implementation to use ?Pymalloc, a specialized 
object allocator written by Vladimir Marangozov, [?] a feature added to 
Python 2.1. Pymalloc is intended to be faster than the system malloc() and 
to have less memory overhead for allocation patterns typical of Python 
programs. The allocator uses C's malloc() function to get large pools of 
memory and then fulfills smaller memory requests from these pools.?

See also: <https://github.com/Homebrew/homebrew/issues/32402> (first result)

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From rosuav at gmail.com  Fri May  1 08:06:12 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 1 May 2015 22:06:12 +1000
Subject: Am I missing something here? ipaddress vs socket
In-Reply-To: <c8a5569c-66aa-4b96-829e-36307d6947cb@googlegroups.com>
References: <f46cdd6e-5c94-4370-a368-cce0bd68c997@googlegroups.com>
 <mailman.6.1430475067.3347.python-list@python.org>
 <c8a5569c-66aa-4b96-829e-36307d6947cb@googlegroups.com>
Message-ID: <CAPTjJmr-aSYrE54F9fmxfd4RW_Xbv8+y2=TV20AG1=-=tYWr+w@mail.gmail.com>

On Fri, May 1, 2015 at 8:41 PM, Ronald van Zantvoort
<the.loeki at gmail.com> wrote:
> mystring is already an IP address, sorry for not mentioning that.
> The point is that ternary if; I've been scouring around the internet and there's a million different variations on the same idea.
> I'm completely flabberghasted it's not a simple property within the object though, e.g. ip.sock_family.
>
> Was this a deliberate decision by the Python devs or can I try an RFE for that?

You could ask. At very worst, you'll find someone explain to you why
it shouldn't be done. I suggest using the python-ideas mailing list;
people there are usually reasonably friendly, though occasionally you
find a caustic response to a dumb idea like "Python should use braces
for indentation" :)

>> But you may find it convenient to use a dedicated function for
>> establishing a connection, which could look up an AAAA or A record for
>> a name, then proceed through all addresses, attempting connections in
>> turn. I'm fairly sure one exists in Python, but I can't right now
>> remember the name.
>>
> Well you've already got socket.create_connection(), which (I think) does perform that lookup.

Yeah, I went looking for it and came across that, but wasn't sure if
that was it.

Now I look more closely (ie: look at the actual source code), I see
that it probably is the function I was thinking of. Might need a docs
patch to make it clear that this iterates over getaddrinfo() and
attempts them all in succession.

ChrisA


From rosuav at gmail.com  Fri May  1 08:09:28 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 1 May 2015 22:09:28 +1000
Subject: ImportError with pickle (Python 2.7.9),
 possibly platform dependent
In-Reply-To: <20a5c7bf-2163-4b7a-8495-30ce23239903@googlegroups.com>
References: <494551ca-532f-4d4d-aff0-a3932416c8f4@googlegroups.com>
 <mailman.108.1430354695.3680.python-list@python.org>
 <20a5c7bf-2163-4b7a-8495-30ce23239903@googlegroups.com>
Message-ID: <CAPTjJmoG0ax05HjnWgTpggkV8iAqE3T7O3i4Z0LORMJQ5xjoFQ@mail.gmail.com>

On Fri, May 1, 2015 at 9:01 PM, Ben Sizer <kylotan at gmail.com> wrote:
> Another good question, and this gives us the answer. The module lists are quite different, as I'd expect because the load happens in the context of the full application whereas the dump happens as a standalone script. But literally every module that is in the 'before dump' list is in the 'before load' list - except OMDBMap. Like the error says! What /is/ in the 'before load' list however is "my_wsgi_app.OMDBMap". The module has been imported, but the pickle algorithm is unable to reconcile the module in the WSGI app's namespace with the module referenced in the pickle file.
>
> So... I don't know how to fix this, but I do now know why it fails, and I have a satisfactory answer for why it is acting differently on the Linux server (and that is just because that is the only one running under WSGI). Two out of three isn't bad!
>

Cool! That's part way. So, can you simply stuff OMDBMap into
sys.modules prior to loading? It might be a bit of a hack, but it
should work for testing, at least. Conversely, you could change the
dump script to import via the name my_wsgi_app to make it consistent.

ChrisA


From rosuav at gmail.com  Fri May  1 08:15:41 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 1 May 2015 22:15:41 +1000
Subject: GAE environment differences
In-Reply-To: <55435578.50003@chamonix.reportlab.co.uk>
References: <55435578.50003@chamonix.reportlab.co.uk>
Message-ID: <CAPTjJmqUeu7AXm6MAiBPwabFv+GRgFOV7B1BUpeLLO7GhiXRvw@mail.gmail.com>

On Fri, May 1, 2015 at 8:29 PM, Robin Becker <robin at reportlab.com> wrote:
> A user reports getting KeyError/ImportError when executing GAE dev/live code
> looking like
>
>
> import os
> .......
> fn = os.path.expanduser(os.path.join('~','.reportlab_settings'))
> ......
>
> This works fine in cpython; so I assume that the GAE environment must be
> doing module resolution differently.
>
> Is that the case or is it that os.path or os.path.expanduser doesn't exist?

Best thing to do is to ask the user to post the complete traceback.
You might need to use "import os.path" but normally I would expect
that not to be an issue.

ChrisA


From Cecil at decebal.nl  Fri May  1 08:22:51 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 14:22:51 +0200
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <mailman.135.1430413555.3680.python-list@python.org>
 <mhtnpg$fsf$1@dont-email.me> <87mw1ptpr0.fsf@Equus.decebal.nl>
 <mailman.146.1430427228.3680.python-list@python.org>
 <mhu6df$6bs$2@dont-email.me>
 <CAPTjJmrs=a=L9TiRUnBPs561bEQFsoBv+3oaQmez5Yi-1gEW6A@mail.gmail.com>
 <mailman.152.1430435562.3680.python-list@python.org>
 <87383gsxbd.fsf@Equus.decebal.nl> <3374990.nA8q5WTVfI@PointedEars.de>
Message-ID: <87vbgco5b8.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 13:15 CEST schreef Thomas Lahn:

> Cecil Westerhof wrote:
>
>> By the way: I also see python3.4 and python3.4m. Any idea where the
>> m stands for?
>
> I googled for ?python3.4m? and found as second result

Eh, I could/should have done that myself. :-(
Nice that you do not burn me. :-D


> <http://stackoverflow.com/questions/16675865/difference-between-python3-and-python3m-executibles>
>
> In a nutshell: python3.4m was built with configure option
> ?--with-pymalloc? which causes the resulting implementation to use
> ?Pymalloc, a specialized object allocator written by Vladimir
> Marangozov, [?] a feature added to Python 2.1. Pymalloc is intended
> to be faster than the system malloc() and to have less memory
> overhead for allocation patterns typical of Python programs. The
> allocator uses C's malloc() function to get large pools of memory
> and then fulfills smaller memory requests from these pools.?
>
> See also: <https://github.com/Homebrew/homebrew/issues/32402> (first
> result)

Something to look in later. Looks interesting, but have a ?few? things
that are more important.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From subhabrata.banerji at gmail.com  Fri May  1 08:28:39 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Fri, 1 May 2015 05:28:39 -0700 (PDT)
Subject: Fast way of extracting files from various folders
Message-ID: <6ba8934e-2f1a-4bcf-b72a-0dd276182ca2@googlegroups.com>

Dear Group,

I have several millions of documents in several folders and subfolders in my machine.
I tried to write a script as follows, to extract all the .doc files and to convert them in text, but it seems it is taking too much of time. 

import os
from fnmatch import fnmatch
import win32com.client
import zipfile, re
def listallfiles2(n):
    root = 'C:\Cand_Res'
    pattern = "*.doc"
    list1=[]
    for path, subdirs, files in os.walk(root):
        for name in files:
            if fnmatch(name, pattern):
                file_name1=os.path.join(path, name)
                if ".doc" in file_name1:
                    #EXTRACTING ONLY .DOC FILES
                    if ".docx" not in file_name1:
                        #print "It is A Doc file$$:",file_name1
                        try:
                            doc = win32com.client.GetObject(file_name1)
                            text = doc.Range().Text
                            text1=text.encode('ascii','ignore')
                            text_word=text1.split()
                            #print "Text for Document File Is:",text1
                            list1.append(text_word)
                            print "It is a Doc file"
                        except:
                            print "DOC ISSUE"

But it seems it is taking too much of time, to convert to text and to append to list. Is there any way I may do it fast? I am using Python2.7 on Windows 7 Professional Edition. Apology for any indentation error. 

If any one may kindly suggest a solution.

Regards,
Subhabrata Banerjee. 



From __peter__ at web.de  Fri May  1 08:34:13 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 01 May 2015 14:34:13 +0200
Subject: ImportError with pickle (Python 2.7.9),
 possibly platform dependent
References: <494551ca-532f-4d4d-aff0-a3932416c8f4@googlegroups.com>
 <mailman.108.1430354695.3680.python-list@python.org>
 <20a5c7bf-2163-4b7a-8495-30ce23239903@googlegroups.com>
Message-ID: <mhvrs7$omo$1@ger.gmane.org>

Ben Sizer wrote:

> On Thursday, 30 April 2015 01:45:05 UTC+1, Chris Angelico  wrote:
>> On Thu, Apr 30, 2015 at 2:01 AM, Ben Sizer wrote:
>> > 1) There clearly is a module named OMDBMap, and it's importable - it's
>> > there in the 2nd line of the traceback.
>> >
>> > Does anybody have any suggestions on how I can go about debugging this?
>> > Or refactoring it to avoid whatever is happening here?
>> 
>> Are you half way through importing it when this load() call happens?
>> That might cause some issues.
> 
> No, we already imported OMDBMap at the top of OMDBSetup.
> 
>> Has your current directory been changed anywhere in there?
> 
> Good question. It turns out that the current directory seems to be $HOME
> when loading, but is the script directory during saving. This will be
> because the Linux server is running under mod_wsgi, whereas we run the
> save script in-place. Our Windows and Mac tests run via Flask's built-in
> server so the working directory is likely to be the same whether we're
> running the script that does pickle.dump or the whole app that does
> pickle.load.
> 
>> What happens if you catch this exception and print out sys.modules at
>> that point?
> 
> Another good question, and this gives us the answer. The module lists are
> quite different, as I'd expect because the load happens in the context of
> the full application whereas the dump happens as a standalone script. But
> literally every module that is in the 'before dump' list is in the 'before
> load' list - except OMDBMap. Like the error says! What /is/ in the 'before
> load' list however is "my_wsgi_app.OMDBMap". The module has been imported,
> but the pickle algorithm is unable to reconcile the module in the WSGI
> app's namespace with the module referenced in the pickle file.
> 
> So... I don't know how to fix this, but I do now know why it fails, and I
> have a satisfactory answer for why it is acting differently on the Linux
> server (and that is just because that is the only one running under WSGI).
> Two out of three isn't bad!

How about moving OMDBMap.py into the parent folder of my_wsgi_app.__file__ 
or any other folder in sys.path?



From robin at reportlab.com  Fri May  1 09:04:19 2015
From: robin at reportlab.com (Robin Becker)
Date: Fri, 01 May 2015 14:04:19 +0100
Subject: GAE environment differences
In-Reply-To: <CAPTjJmqUeu7AXm6MAiBPwabFv+GRgFOV7B1BUpeLLO7GhiXRvw@mail.gmail.com>
References: <55435578.50003@chamonix.reportlab.co.uk>
 <CAPTjJmqUeu7AXm6MAiBPwabFv+GRgFOV7B1BUpeLLO7GhiXRvw@mail.gmail.com>
Message-ID: <554379D3.1080202@chamonix.reportlab.co.uk>

On 01/05/2015 13:15, Chris Angelico wrote:
> On Fri, May 1, 2015 at 8:29 PM, Robin Becker <robin at reportlab.com> wrote:
>
>
> Best thing to do is to ask the user to post the complete traceback.
> You might need to use "import os.path" but normally I would expect
> that not to be an issue.



> jamesbynd said:
>
> here's a traceback:
>
> ```
> #!python
>
> ImportError: No module named pwd
> (12 additional frame(s) were not displayed)
> ...
>   File "/base/data/home/apps/e~yt-maggi-2015-eu/testing.383971015313322618/vendor/xhtml2pdf/context.py", line 5, in <module>
>     from reportlab.lib.styles import ParagraphStyle
>   File "/base/data/home/apps/e~yt-maggi-2015-eu/testing.383971015313322618/vendor/reportlab/lib/styles.py", line 28, in <module>
>     from reportlab.rl_config import canvas_basefontname as _baseFontName, baseUnderlineProportion as _baseUnderlineProportion
>   File "/base/data/home/apps/e~yt-maggi-2015-eu/testing.383971015313322618/vendor/reportlab/rl_config.py", line 131, in <mod
>  ule>
>     _startUp()
>   File "/base/data/home/apps/e~yt-maggi-2015-eu/testing.383971015313322618/vendor/reportlab/rl_config.py", line 99, in _startUp
>     d = os.path.expanduser(d)   #appengine fails with KeyError
>   File "python2.7/posixpath.py", line 268, in expanduser
>     import pwd
> ```
the user suggests that even though claims are made that you can use a 
filesystem, but stuff like pwd is missing. Apparently the user module has no 
meaning, but there is a users module? I guess I'll need to keep patching 
reportlab when GAE users find these glitches.
-- 
Robin Becker



From rosuav at gmail.com  Fri May  1 09:22:02 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 1 May 2015 23:22:02 +1000
Subject: GAE environment differences
In-Reply-To: <554379D3.1080202@chamonix.reportlab.co.uk>
References: <55435578.50003@chamonix.reportlab.co.uk>
 <CAPTjJmqUeu7AXm6MAiBPwabFv+GRgFOV7B1BUpeLLO7GhiXRvw@mail.gmail.com>
 <554379D3.1080202@chamonix.reportlab.co.uk>
Message-ID: <CAPTjJmr=2Kj_HHx9E07L_aJ2hDgR9142rQuZxvxOF4+36_kV0w@mail.gmail.com>

On Fri, May 1, 2015 at 11:04 PM, Robin Becker <robin at reportlab.com> wrote:
>>   File "python2.7/posixpath.py", line 268, in expanduser
>>     import pwd
>> ```
>
> the user suggests that even though claims are made that you can use a
> filesystem, but stuff like pwd is missing. Apparently the user module has no
> meaning, but there is a users module? I guess I'll need to keep patching
> reportlab when GAE users find these glitches.

According to my Python installation, the pwd module is built-in. But
you could bypass its usage by making sure $HOME is set before calling
expanduser.

What happens if you fire up Python and just type "import pwd"? Does
that work? If not, does it give any more useful error message?

ChrisA


From torriem at gmail.com  Fri May  1 09:25:16 2015
From: torriem at gmail.com (Michael Torrie)
Date: Fri, 01 May 2015 07:25:16 -0600
Subject: l = range(int(1E9))
In-Reply-To: <87iocdrktt.fsf@Equus.decebal.nl>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <mailman.135.1430413555.3680.python-list@python.org>
 <mhtnpg$fsf$1@dont-email.me> <87mw1ptpr0.fsf@Equus.decebal.nl>
 <mailman.146.1430427228.3680.python-list@python.org>
 <87iocdrktt.fsf@Equus.decebal.nl>
Message-ID: <55437EBC.7010202@gmail.com>

On 04/30/2015 10:19 PM, Cecil Westerhof wrote:
>> I must also confess to being highly impressed, it's a breath of
>> fresh air having an apprentice Pythonista who is looking at doing
>> things the Pythonic way :)
> 
> When in Rome, do as the Romans do.
> 
> Besides: there probably is a reason for the Pythonic way.

You maybe surprised, then, by how many people that stop by this list
wanting to program Java, C# or C++ in Python and get frustrated and
upset when the list tries to steer them in a Pythonic direction.  Most
of them leave upset and never return to the list or Python.




From __peter__ at web.de  Fri May  1 09:31:00 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 01 May 2015 15:31 +0200
Subject: GAE environment differences
References: <55435578.50003@chamonix.reportlab.co.uk>
 <CAPTjJmqUeu7AXm6MAiBPwabFv+GRgFOV7B1BUpeLLO7GhiXRvw@mail.gmail.com>
 <554379D3.1080202@chamonix.reportlab.co.uk>
 <CAPTjJmr=2Kj_HHx9E07L_aJ2hDgR9142rQuZxvxOF4+36_kV0w@mail.gmail.com>
Message-ID: <mhvv6k$7fa$1@ger.gmane.org>

Chris Angelico wrote:

> On Fri, May 1, 2015 at 11:04 PM, Robin Becker <robin at reportlab.com> wrote:
>>>   File "python2.7/posixpath.py", line 268, in expanduser
>>>     import pwd
>>> ```
>>
>> the user suggests that even though claims are made that you can use a
>> filesystem, but stuff like pwd is missing. Apparently the user module has
>> no meaning, but there is a users module? I guess I'll need to keep
>> patching reportlab when GAE users find these glitches.
> 
> According to my Python installation, the pwd module is built-in. But
> you could bypass its usage by making sure $HOME is set before calling
> expanduser.
> 
> What happens if you fire up Python and just type "import pwd"? Does
> that work? If not, does it give any more useful error message?

I found (guess how)

https://cloud.google.com/appengine/kb/libraries

which states:

"""
As a result, these are the C modules that are not usable with the Python 2.7 
runtime:

ctypes
sqlite
ssl/_ssl
fcntl
spwd
pwd
grp
syslog
select
_socket
"""



From the.loeki at gmail.com  Fri May  1 09:37:50 2015
From: the.loeki at gmail.com (Ronald van Zantvoort)
Date: Fri, 1 May 2015 06:37:50 -0700 (PDT)
Subject: Am I missing something here? ipaddress vs socket
In-Reply-To: <mailman.8.1430481980.3347.python-list@python.org>
References: <f46cdd6e-5c94-4370-a368-cce0bd68c997@googlegroups.com>
 <mailman.6.1430475067.3347.python-list@python.org>
 <c8a5569c-66aa-4b96-829e-36307d6947cb@googlegroups.com>
 <mailman.8.1430481980.3347.python-list@python.org>
Message-ID: <8a3d28f9-52d4-48e3-9da3-dc64dd773d16@googlegroups.com>

On Friday, 1 May 2015 14:06:34 UTC+2, Chris Angelico  wrote:
> On Fri, May 1, 2015 at 8:41 PM, Ronald van Zantvoort wrote:
> > mystring is already an IP address, sorry for not mentioning that.
> > The point is that ternary if; I've been scouring around the internet and there's a million different variations on the same idea.
> > I'm completely flabberghasted it's not a simple property within the object though, e.g. ip.sock_family.
> >
> > Was this a deliberate decision by the Python devs or can I try an RFE for that?
> 
> You could ask. At very worst, you'll find someone explain to you why
> it shouldn't be done. I suggest using the python-ideas mailing list;
> people there are usually reasonably friendly, though occasionally you
> find a caustic response to a dumb idea like "Python should use braces
> for indentation" :)

There are no dumb ideas, just dumb people ;)

But seriously, I'll go see over there, thanks :) It's just that I'm amazed that this isn't in there yet; it's 2 simple properties added in for convenience and IMHO very Pythonic to be able to do aforementioned code.

Thanks again pplz :)


From rosuav at gmail.com  Fri May  1 09:46:04 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 1 May 2015 23:46:04 +1000
Subject: Am I missing something here? ipaddress vs socket
In-Reply-To: <8a3d28f9-52d4-48e3-9da3-dc64dd773d16@googlegroups.com>
References: <f46cdd6e-5c94-4370-a368-cce0bd68c997@googlegroups.com>
 <mailman.6.1430475067.3347.python-list@python.org>
 <c8a5569c-66aa-4b96-829e-36307d6947cb@googlegroups.com>
 <mailman.8.1430481980.3347.python-list@python.org>
 <8a3d28f9-52d4-48e3-9da3-dc64dd773d16@googlegroups.com>
Message-ID: <CAPTjJmpNH2HJiimbGX7BsnWuwcZMET_Cgkx6mKNNbFbyObvHQw@mail.gmail.com>

On Fri, May 1, 2015 at 11:37 PM, Ronald van Zantvoort
<the.loeki at gmail.com> wrote:
> But seriously, I'll go see over there, thanks :) It's just that I'm amazed that this isn't in there yet; it's 2 simple properties added in for convenience and IMHO very Pythonic to be able to do aforementioned code.
>

There are lots of things that are so obvious they get added with
little argument, but someone still has to think of them :)

ChrisA


From the.loeki at gmail.com  Fri May  1 09:48:36 2015
From: the.loeki at gmail.com (Ronald van Zantvoort)
Date: Fri, 1 May 2015 06:48:36 -0700 (PDT)
Subject: Am I missing something here? ipaddress vs socket
In-Reply-To: <mailman.17.1430487966.3347.python-list@python.org>
References: <f46cdd6e-5c94-4370-a368-cce0bd68c997@googlegroups.com>
 <mailman.6.1430475067.3347.python-list@python.org>
 <c8a5569c-66aa-4b96-829e-36307d6947cb@googlegroups.com>
 <mailman.8.1430481980.3347.python-list@python.org>
 <8a3d28f9-52d4-48e3-9da3-dc64dd773d16@googlegroups.com>
 <mailman.17.1430487966.3347.python-list@python.org>
Message-ID: <0bf34717-3a90-48da-a1a8-05b4ebd504c5@googlegroups.com>

On Friday, 1 May 2015 15:46:32 UTC+2, Chris Angelico  wrote:
> On Fri, May 1, 2015 at 11:37 PM, Ronald van Zantvoort wrote:
> > But seriously, I'll go see over there, thanks :) It's just that I'm amazed that this isn't in there yet; it's 2 simple properties added in for convenience and IMHO very Pythonic to be able to do aforementioned code.
> >
> 
> There are lots of things that are so obvious they get added with
> little argument, but someone still has to think of them :)
> 
> ChrisA

Hehehe right you are. I've posted an idea to python-ideas, we'll see how it goes from there.
Thanks again!


From Cecil at decebal.nl  Fri May  1 10:12:23 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 16:12:23 +0200
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <mailman.135.1430413555.3680.python-list@python.org>
 <mhtnpg$fsf$1@dont-email.me> <87mw1ptpr0.fsf@Equus.decebal.nl>
 <mailman.146.1430427228.3680.python-list@python.org>
 <87iocdrktt.fsf@Equus.decebal.nl>
 <mailman.15.1430486728.3347.python-list@python.org>
Message-ID: <87oam4o08o.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 15:25 CEST schreef Michael Torrie:

> On 04/30/2015 10:19 PM, Cecil Westerhof wrote:
>>> I must also confess to being highly impressed, it's a breath of
>>> fresh air having an apprentice Pythonista who is looking at doing
>>> things the Pythonic way :)
>>
>> When in Rome, do as the Romans do.
>>
>> Besides: there probably is a reason for the Pythonic way.
>
> You maybe surprised, then, by how many people that stop by this list
> wanting to program Java, C# or C++ in Python and get frustrated and
> upset when the list tries to steer them in a Pythonic direction.
> Most of them leave upset and never return to the list or Python.

I am not really surprised. I know that this is how it often works. :-(
But I like to be different. :-D

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Fri May  1 10:41:28 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 16:41:28 +0200
Subject: Rewriting to Python 3
References: <87egn0rcc1.fsf@Equus.decebal.nl>
Message-ID: <87k2wsnyw7.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 09:22 CEST schreef Cecil Westerhof:

> On my system I have: PARSER_RE_STR = '/(%s)=' %
> '|'.join(DN_LUT.keys() + DN_LUT.values()) in:
> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>
> In Python 3 that gives: TypeError: unsupported operand type(s) for
> +: 'dict_keys' and 'dict_values'
>
> How should I rewrite this?

I am not the only person that has a problem:
    https://github.com/cedadev/ndg_httpsclient/issues/1

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Fri May  1 10:52:43 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 01 May 2015 16:52:43 +0200
Subject: Rewriting to Python 3
References: <87egn0rcc1.fsf@Equus.decebal.nl> <87k2wsnyw7.fsf@Equus.decebal.nl>
Message-ID: <87bni4nydg.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 16:41 CEST schreef Cecil Westerhof:

> Op Friday 1 May 2015 09:22 CEST schreef Cecil Westerhof:
>
>> On my system I have: PARSER_RE_STR = '/(%s)=' %
>> '|'.join(DN_LUT.keys() + DN_LUT.values()) in:
>> /usr/lib/python3.4/site-packages/ndg/httpsclient/ssl_peer_verification.py
>>
>> In Python 3 that gives: TypeError: unsupported operand type(s) for
>> +: 'dict_keys' and 'dict_values'
>>
>> How should I rewrite this?
>
> I am not the only person that has a problem:
> https://github.com/cedadev/ndg_httpsclient/issues/1

Just personally contacting does make a difference. He made a patch. I
am going to test a later today.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From ian.g.kelly at gmail.com  Fri May  1 11:03:32 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 1 May 2015 09:03:32 -0600
Subject: Is my implementation of happy number OK
In-Reply-To: <554338dd$0$13004$c3e8da3$5496439d@news.astraweb.com>
References: <87oam5vc8k.fsf@Equus.decebal.nl>
 <mailman.140.1430420006.3680.python-list@python.org>
 <slrnmk5esv.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.155.1430437953.3680.python-list@python.org>
 <slrnmk6agn.apd.jon+usenet@frosty.unequivocal.co.uk>
 <554338dd$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CALwzid=iNnSRspO9F1KAt=9=x3U=oOB0n=agQqs=yOVHA6wmmw@mail.gmail.com>

On Fri, May 1, 2015 at 2:27 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> Rather than 10**7, how about trying (10**500 + 2). Is it happy?
>
> Using the Python code from Wikipedia:
> https://en.wikipedia.org/wiki/Happy_number
>
> SQUARE = dict([(c, int(c)**2) for c in "0123456789"])
> def is_happy(n):
>   while (n > 1) and (n != 4):
>     n = sum(SQUARE[d] for d in str(n))
>   return n == 1
>
>
> I can calculate whether n=10**500 + 2 is happy in less than a millisecond on
> my computer.

Not really the most exciting example, since the following number in
the sequence would be 5. But a random sequence of 500 non-zero digits
doesn't seem to take substantially longer.


From irmen.NOSPAM at xs4all.nl  Fri May  1 12:36:39 2015
From: irmen.NOSPAM at xs4all.nl (Irmen de Jong)
Date: Fri, 01 May 2015 18:36:39 +0200
Subject: Fast way of extracting files from various folders
In-Reply-To: <6ba8934e-2f1a-4bcf-b72a-0dd276182ca2@googlegroups.com>
References: <6ba8934e-2f1a-4bcf-b72a-0dd276182ca2@googlegroups.com>
Message-ID: <5543ab93$0$2901$e4fe514c@news.xs4all.nl>

On 1-5-2015 14:28, subhabrata.banerji at gmail.com wrote:
> Dear Group,
> 
> I have several millions of documents in several folders and subfolders in my machine.
> I tried to write a script as follows, to extract all the .doc files and to convert them in text, but it seems it is taking too much of time. 
> 

[snip]

> But it seems it is taking too much of time, to convert to text and to append to list. Is there any way I may do it fast? I am using Python2.7 on Windows 7 Professional Edition. Apology for any indentation error. 
> 
> If any one may kindly suggest a solution.

Have you profiled and identified the part of your script that is slow?

On first sight though your python code, while not optimal, contains no immediate
performance issues. It is likely the COM interop call to Winword and getting the text
via that interface that is slow. Imagine opening word for "several million documents",
no wonder it doesn't perform.

Investigate tools like antiword, wv, docx2txt. I suspect they're quite a bit faster than
relying on Word itself.


Irmen




From auriocus at gmx.de  Fri May  1 13:56:02 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Fri, 01 May 2015 19:56:02 +0200
Subject: Python is not bad ;-)
In-Reply-To: <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mi0elj$ujt$1@dont-email.me>

Am 01.05.15 um 09:03 schrieb Steven D'Aprano:
> On Thu, 30 Apr 2015 09:30 pm, Cecil Westerhof wrote:
>
>> Tail recursion would nice to have also.
>
> People coming from functional languages like Lisp and Haskell often say
> that, but how many recursive algorithms naturally take a tail-call form?
> Not that many.

That is because tailcall optimization is used in functional languages 
mainly as a replacement for loops. IOW, in non-functional languages you 
can simply use an infinite loop to do the same (in most cases).

> I suppose that it would be nice if Python let you optionally use tail-call
> optimization, but that might be tricky in practice.
>

In Tcl 8.6 there is an explicit "tailcall something" command which calls 
something without creating a new stack frame, i.e. it returns to the 
caller after something returns. Useful sometimes, but more a 
micro-optimization than a vital feature.

	Christian


From __peter__ at web.de  Fri May  1 14:13:20 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 01 May 2015 20:13:20 +0200
Subject: Is my implementation of happy number OK
References: <87oam5vc8k.fsf@Equus.decebal.nl>
 <mailman.140.1430420006.3680.python-list@python.org>
 <slrnmk5esv.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.155.1430437953.3680.python-list@python.org>
 <slrnmk6agn.apd.jon+usenet@frosty.unequivocal.co.uk>
 <554338dd$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <CALwzid=iNnSRspO9F1KAt=9=x3U=oOB0n=agQqs=yOVHA6wmmw@mail.gmail.com>
Message-ID: <mi0fo1$5cf$1@ger.gmane.org>

Ian Kelly wrote:

> On Fri, May 1, 2015 at 2:27 AM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>> Rather than 10**7, how about trying (10**500 + 2). Is it happy?
>>
>> Using the Python code from Wikipedia:
>> https://en.wikipedia.org/wiki/Happy_number
>>
>> SQUARE = dict([(c, int(c)**2) for c in "0123456789"])
>> def is_happy(n):
>>   while (n > 1) and (n != 4):
>>     n = sum(SQUARE[d] for d in str(n))
>>   return n == 1
>>
>>
>> I can calculate whether n=10**500 + 2 is happy in less than a millisecond
>> on my computer.
> 
> Not really the most exciting example, since the following number in
> the sequence would be 5. But a random sequence of 500 non-zero digits
> doesn't seem to take substantially longer.

I may be missing something, but isn't the '+ 2' in '10**500 + 2' just a 
distraction? 10**500 would bring across nicely what the approach Steven took 
from Wikipeda can do in one iteration where Jon's cannot even find enough 
memory.

Also, with 500 digits and 0 <= d <= 9 

sum(d*d for d in digits) <= 81 * 500

should hold, i. e. on the second iteration the maximum number of digits is 
already down to five in the worst case. By repeating this simple reasoning 
you can tell that all interesting examples must be below 1000 as 
num_digits(3*81) == 3. 

A computer that cannot calculate a lookup table with all 3-digit cases 
should be almost as hard to find as the one needed by Jon ;)




From tjreedy at udel.edu  Fri May  1 22:00:40 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 01 May 2015 22:00:40 -0400
Subject: l = range(int(1E9))
In-Reply-To: <87383gsxbd.fsf@Equus.decebal.nl>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <mailman.135.1430413555.3680.python-list@python.org>
 <mhtnpg$fsf$1@dont-email.me> <87mw1ptpr0.fsf@Equus.decebal.nl>
 <mailman.146.1430427228.3680.python-list@python.org>
 <mhu6df$6bs$2@dont-email.me>
 <CAPTjJmrs=a=L9TiRUnBPs561bEQFsoBv+3oaQmez5Yi-1gEW6A@mail.gmail.com>
 <mailman.152.1430435562.3680.python-list@python.org>
 <87383gsxbd.fsf@Equus.decebal.nl>
Message-ID: <mi1b4d$pmo$1@ger.gmane.org>

On 5/1/2015 1:04 AM, Cecil Westerhof wrote:

> By the way: I also see python3.4 and python3.4m. Any idea where the m
> stands for?

I never heard of that in 18 years of Python, and thought it must be an 
error, but putting 'python3.4b' into google search return this.

https://stackoverflow.com/questions/16675865/difference-between-python3-and-python3m-executibles

See first answer.
-- 
Terry Jan Reedy



From greg.ewing at canterbury.ac.nz  Sat May  2 03:44:29 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Sat, 02 May 2015 19:44:29 +1200
Subject: Python is not bad ;-)
In-Reply-To: <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <cqjditFd6n5U1@mid.individual.net>

Steven D'Aprano wrote:

> People coming from functional languages like Lisp and Haskell often say
> that, but how many recursive algorithms naturally take a tail-call form?
> Not that many.

And the ones that do tend to be the ones that are
better expressed iteratively in Python. So Python
doesn't really need tail call optimisation.

Also, Guido has said that he doesn't like it,
because of the detrimental effect it has on stack
tracebacks.

-- 
Greg


From Cecil at decebal.nl  Sat May  2 04:26:13 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 02 May 2015 10:26:13 +0200
Subject: Python is not bad ;-)
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87383fo062.fsf@Equus.decebal.nl>

Op Friday 1 May 2015 09:03 CEST schreef Steven D'Aprano:

> On Thu, 30 Apr 2015 09:30 pm, Cecil Westerhof wrote:
>
>> Tail recursion would nice to have also.
>
> People coming from functional languages like Lisp and Haskell often
> say that, but how many recursive algorithms naturally take a
> tail-call form? Not that many.

One example:
    def factorial(x, y = 1):
        return y if x == 1 else factorial(x - 1, x * y)

    def factorial_iterative(x):
        result = 1
        for i in range(2, x + 1):
            result *= i
        return result

Executing factorial(985) 100.000 times takes 54 seconds.
While executing factorial_iterative(985) takes 34 seconds.
Also you can not call factorial with a value that is much bigger
because of recursion depth. You can call factorial_iterative with
1.000.000.

I made also a version that simulates tail recursion:
    def factorial_tail_recursion(x):
        y = 1
        while True:
            if x == 1:
                return y
            y *= x
            x -= 1

This is that a lot less efficient as the iterative version. It takes 43
seconds. But it is a lot better as the normal recursive version: about
25%. The iterative version is about 25% more efficient as the tail
recursion version.

With larger values it decreases. Calculating onetime for 5.000.000
takes 117 and 131 seconds. Just 10% faster.

That is mostly because the tail recursion version starts multiplying
at the high end. I wrote a second version:
    def factorial_tail_recursion2(x):
        y = 1
        z = 1
        while True:
            if x == z:
                return y
            y *= z
            z += 1

This has almost the performance of the iterative version: 34 and 121
seconds.

So I made a new recursive version:
    def factorial_recursive(x, y = 1, z = 1):
        return y if x == z else factorial_recursive(x, x * y, z + 1)

But this take almost the same tame as the other. Probably the
recursive calls are more important as the multiplications.


I find factorial a lot cleaner code as factorial_iterative, so here
tail recursion would be beneficial.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From subhabrata.banerji at gmail.com  Sat May  2 05:00:21 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Sat, 2 May 2015 02:00:21 -0700 (PDT)
Subject: Fast way of extracting files from various folders
In-Reply-To: <6ba8934e-2f1a-4bcf-b72a-0dd276182ca2@googlegroups.com>
References: <6ba8934e-2f1a-4bcf-b72a-0dd276182ca2@googlegroups.com>
Message-ID: <fc6342cd-1bcd-4f66-903b-1e99f8fbd4a1@googlegroups.com>

On Friday, May 1, 2015 at 5:58:50 PM UTC+5:30, subhabrat... at gmail.com wrote:
> Dear Group,
> 
> I have several millions of documents in several folders and subfolders in my machine.
> I tried to write a script as follows, to extract all the .doc files and to convert them in text, but it seems it is taking too much of time. 
> 
> import os
> from fnmatch import fnmatch
> import win32com.client
> import zipfile, re
> def listallfiles2(n):
>     root = 'C:\Cand_Res'
>     pattern = "*.doc"
>     list1=[]
>     for path, subdirs, files in os.walk(root):
>         for name in files:
>             if fnmatch(name, pattern):
>                 file_name1=os.path.join(path, name)
>                 if ".doc" in file_name1:
>                     #EXTRACTING ONLY .DOC FILES
>                     if ".docx" not in file_name1:
>                         #print "It is A Doc file$$:",file_name1
>                         try:
>                             doc = win32com.client.GetObject(file_name1)
>                             text = doc.Range().Text
>                             text1=text.encode('ascii','ignore')
>                             text_word=text1.split()
>                             #print "Text for Document File Is:",text1
>                             list1.append(text_word)
>                             print "It is a Doc file"
>                         except:
>                             print "DOC ISSUE"
> 
> But it seems it is taking too much of time, to convert to text and to append to list. Is there any way I may do it fast? I am using Python2.7 on Windows 7 Professional Edition. Apology for any indentation error. 
> 
> If any one may kindly suggest a solution.
> 
> Regards,
> Subhabrata Banerjee.

Thanks. You are right conversions are taking time. I would surely check. Rest part is okay. Regards, Subhabrata Banerjee. 


From marko at pacujo.net  Sat May  2 05:10:52 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 02 May 2015 12:10:52 +0300
Subject: Python is not bad ;-)
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
Message-ID: <87egmzl4yr.fsf@elektro.pacujo.net>

Cecil Westerhof <Cecil at decebal.nl>:
> I find factorial a lot cleaner code as factorial_iterative, so here
> tail recursion would be beneficial.

I would just call math.factorial() and be done with it.

Note: Scheme is my favorite language and I use tail recursion all the
time. I also think eliminating tail recursion is such a low-hanging
fruit that even CPython should just pick it. However, I wouldn't use the
fibonacci sequence to justify anything at all about a programming
language.

It's not about performance (that's rarely a very useful argument when it
comes to Python). It's only that every now and then a very natural tail
recursion crops up and it's just silly to have to refactor perfectly
good code because of a stack overflow.


Marko


From kevin.p.dwyer at gmail.com  Sat May  2 05:14:00 2015
From: kevin.p.dwyer at gmail.com (Kev Dwyer)
Date: Sat, 02 May 2015 10:14 +0100
Subject: GAE environment differences
References: <55435578.50003@chamonix.reportlab.co.uk>
 <CAPTjJmqUeu7AXm6MAiBPwabFv+GRgFOV7B1BUpeLLO7GhiXRvw@mail.gmail.com>
 <554379D3.1080202@chamonix.reportlab.co.uk>
Message-ID: <mi24gq$p4u$1@ger.gmane.org>

Robin Becker wrote:

> On 01/05/2015 13:15, Chris Angelico wrote:
>> On Fri, May 1, 2015 at 8:29 PM, Robin Becker <robin at reportlab.com> wrote:
>>
>>
>> Best thing to do is to ask the user to post the complete traceback.
>> You might need to use "import os.path" but normally I would expect
>> that not to be an issue.
> 
> 
> 
>> jamesbynd said:
>>
>> here's a traceback:
>>
>> ```
>> #!python
>>
>> ImportError: No module named pwd
>> (12 additional frame(s) were not displayed)
>> ...
>>   File
>>   "/base/data/home/apps/e~yt-maggi-2015-
eu/testing.383971015313322618/vendor/xhtml2pdf/context.py",
>>   line 5, in <module>
>>     from reportlab.lib.styles import ParagraphStyle
>>   File
>>   "/base/data/home/apps/e~yt-maggi-2015-
eu/testing.383971015313322618/vendor/reportlab/lib/styles.py",
>>   line 28, in <module>
>>     from reportlab.rl_config import canvas_basefontname as _baseFontName,
>>     baseUnderlineProportion as _baseUnderlineProportion
>>   File
>>   "/base/data/home/apps/e~yt-maggi-2015-
eu/testing.383971015313322618/vendor/reportlab/rl_config.py",
>>   line 131, in <mod
>>  ule>
>>     _startUp()
>>   File
>>   "/base/data/home/apps/e~yt-maggi-2015-
eu/testing.383971015313322618/vendor/reportlab/rl_config.py",
>>   line 99, in _startUp
>>     d = os.path.expanduser(d)   #appengine fails with KeyError
>>   File "python2.7/posixpath.py", line 268, in expanduser
>>     import pwd
>> ```
> the user suggests that even though claims are made that you can use a
> filesystem, but stuff like pwd is missing. Apparently the user module has
> no meaning, but there is a users module? I guess I'll need to keep
> patching reportlab when GAE users find these glitches.

For what it's worth, we use reportlab on GAE to generate a simple pdf
and the above error is the only one that I've encountered.  For us it was 
enough to trap the ImportError.

Thanks for all your work on reportlab,

Kev



From __peter__ at web.de  Sat May  2 05:22:08 2015
From: __peter__ at web.de (Peter Otten)
Date: Sat, 02 May 2015 11:22:08 +0200
Subject: Fast way of extracting files from various folders
References: <6ba8934e-2f1a-4bcf-b72a-0dd276182ca2@googlegroups.com>
Message-ID: <mi2503$aa$1@ger.gmane.org>

subhabrata.banerji at gmail.com wrote:

> I have several millions of documents in several folders and subfolders in
> my machine. I tried to write a script as follows, to extract all the .doc
> files and to convert them in text, but it seems it is taking too much of
> time.
> 
> import os
> from fnmatch import fnmatch
> import win32com.client
> import zipfile, re
> def listallfiles2(n):
>     root = 'C:\Cand_Res'
>     pattern = "*.doc"
>     list1=[]
>     for path, subdirs, files in os.walk(root):
>         for name in files:
>             if fnmatch(name, pattern):
>                 file_name1=os.path.join(path, name)
>                 if ".doc" in file_name1:
>                     #EXTRACTING ONLY .DOC FILES
>                     if ".docx" not in file_name1:
>                         #print "It is A Doc file$$:",file_name1
>                         try:
>                             doc = win32com.client.GetObject(file_name1)
>                             text = doc.Range().Text
>                             text1=text.encode('ascii','ignore')
>                             text_word=text1.split()
>                             #print "Text for Document File Is:",text1
>                             list1.append(text_word)
>                             print "It is a Doc file"
>                         except:
>                             print "DOC ISSUE"
> 
> But it seems it is taking too much of time, to convert to text and to
> append to list. Is there any way I may do it fast? I am using Python2.7 on
> Windows 7 Professional Edition. Apology for any indentation error.
> 
> If any one may kindly suggest a solution.

It will not help the first time through your documents, but if you write the 
words for the word documents in one .txt file per .doc, and the original 
files rarely change you can read from the .txt files when you run your 
script a second time. Just make sure that the .txt is younger than the 
corresponding .doc by checking the file time.

In short: use a caching strategy.



From jaronstreak at gmail.com  Sat May  2 05:23:14 2015
From: jaronstreak at gmail.com (jaronstreak at gmail.com)
Date: Sat, 2 May 2015 02:23:14 -0700 (PDT)
Subject: convert pdf to excel xls file
In-Reply-To: <mailman.62.1427100644.10327.python-list@python.org>
References: <mailman.62.1427100644.10327.python-list@python.org>
Message-ID: <5b36ac7d-53fd-4271-80a6-5ba3a9f14c7a@googlegroups.com>


This is a very common problem when we need to convert excel files to pdf format. I have faced this problem many time. Because of my job nature i always need to convert many excel sheets because i could not send them to clients directly. It always took a long time to convert them manually. Then one of my friend suggested me to use one converter tool to convert XLS to PDF very easily. It saves my lots of time and efforts. 

Source: Convert XLS to PDF - http://www.coolutils.com/XLS-to-PDF


From rosuav at gmail.com  Sat May  2 05:25:11 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 2 May 2015 19:25:11 +1000
Subject: Python is not bad ;-)
In-Reply-To: <87egmzl4yr.fsf@elektro.pacujo.net>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net>
 <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
 <87egmzl4yr.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmoWbCff+p5MZQTpjSLfd1qJYuU4QChweC3kaECKiS4b+w@mail.gmail.com>

On Sat, May 2, 2015 at 7:10 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Note: Scheme is my favorite language and I use tail recursion all the
> time. I also think eliminating tail recursion is such a low-hanging
> fruit that even CPython should just pick it. However, I wouldn't use the
> fibonacci sequence to justify anything at all about a programming
> language.

Actually, it isn't such low-hanging fruit. As has been mentioned in
this thread already, Guido is against anything that disrupts
tracebacks, and optimizing tail recursion while maintaining traceback
integrity is rather harder.

In the situations where it really is simple, you can always make the
change in your own code anyway. Often, the process of converting
recursion into tail recursion warps the code to the point where it's
abusing recursion to implement iteration anyway, so just make it
iterative.

ChrisA


From Cecil at decebal.nl  Sat May  2 05:33:56 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 02 May 2015 11:33:56 +0200
Subject: Python is not bad ;-)
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
Message-ID: <87twvvmigr.fsf@Equus.decebal.nl>

Op Saturday 2 May 2015 10:26 CEST schreef Cecil Westerhof:

> Op Friday 1 May 2015 09:03 CEST schreef Steven D'Aprano:
>
>> On Thu, 30 Apr 2015 09:30 pm, Cecil Westerhof wrote:
>>
>>> Tail recursion would nice to have also.
>>
>> People coming from functional languages like Lisp and Haskell often
>> say that, but how many recursive algorithms naturally take a
>> tail-call form? Not that many.
>
> One example:
> def factorial(x, y = 1):
> return y if x == 1 else factorial(x - 1, x * y)
>
> def factorial_iterative(x):
> result = 1
> for i in range(2, x + 1):
> result *= i
> return result
>
> Executing factorial(985) 100.000 times takes 54 seconds.
> While executing factorial_iterative(985) takes 34 seconds.
> Also you can not call factorial with a value that is much bigger
> because of recursion depth. You can call factorial_iterative with
> 1.000.000.
>
> I made also a version that simulates tail recursion:
> def factorial_tail_recursion(x):
> y = 1
> while True:
> if x == 1:
> return y
> y *= x
> x -= 1
>
> This is that a lot less efficient as the iterative version. It takes
> 43 seconds. But it is a lot better as the normal recursive version:
> about 25%. The iterative version is about 25% more efficient as the
> tail recursion version.
>
> With larger values it decreases. Calculating onetime for 5.000.000
> takes 117 and 131 seconds. Just 10% faster.
>
> That is mostly because the tail recursion version starts multiplying
> at the high end. I wrote a second version:
> def factorial_tail_recursion2(x):
> y = 1
> z = 1
> while True:
> if x == z:
> return y
> y *= z
> z += 1
>
> This has almost the performance of the iterative version: 34 and 121
> seconds.
>
> So I made a new recursive version:
> def factorial_recursive(x, y = 1, z = 1):
> return y if x == z else factorial_recursive(x, x * y, z + 1)

Stupid me 'x == z' should be 'z > x'

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From marko at pacujo.net  Sat May  2 05:58:37 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 02 May 2015 12:58:37 +0300
Subject: Python is not bad ;-)
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl> <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
Message-ID: <87a8xnl2r6.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> Guido is against anything that disrupts tracebacks, and optimizing
> tail recursion while maintaining traceback integrity is rather harder.

Tail recursion could be suppressed during debugging. Optimized code can
play all kinds of non-obvious tricks with the execution frame.

> In the situations where it really is simple, you can always make the
> change in your own code anyway. Often, the process of converting
> recursion into tail recursion warps the code to the point where it's
> abusing recursion to implement iteration anyway, so just make it
> iterative.

While you shouldn't actively replace Python iteration with recursion, I
strongly disagree that naturally occurring tail recursion is abuse or
should be avoided in any manner.


Marko


From davea at davea.name  Sat May  2 06:22:20 2015
From: davea at davea.name (Dave Angel)
Date: Sat, 02 May 2015 06:22:20 -0400
Subject: Python is not bad ;-)
In-Reply-To: <87a8xnl2r6.fsf@elektro.pacujo.net>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl> <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net>
Message-ID: <5544A55C.2070504@davea.name>

On 05/02/2015 05:58 AM, Marko Rauhamaa wrote:
> Chris Angelico <rosuav at gmail.com>:
>
>> Guido is against anything that disrupts tracebacks, and optimizing
>> tail recursion while maintaining traceback integrity is rather harder.
>
> Tail recursion could be suppressed during debugging. Optimized code can
> play all kinds of non-obvious tricks with the execution frame.
>
>> In the situations where it really is simple, you can always make the
>> change in your own code anyway. Often, the process of converting
>> recursion into tail recursion warps the code to the point where it's
>> abusing recursion to implement iteration anyway, so just make it
>> iterative.
>
> While you shouldn't actively replace Python iteration with recursion, I
> strongly disagree that naturally occurring tail recursion is abuse or
> should be avoided in any manner.
>

When you strongly disagree, make sure you're disagreeing with what Chris 
actually said.  The key phrase in his message was
     "converting recursion into tail recursion"

NOT  "converting iteration into recursion"
and NOT "naturally occurring tail recursion"


-- 
DaveA


From Cecil at decebal.nl  Sat May  2 06:29:22 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 02 May 2015 12:29:22 +0200
Subject: Python is not bad ;-)
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl> <87egmzl4yr.fsf@elektro.pacujo.net>
Message-ID: <87pp6jmfwd.fsf@Equus.decebal.nl>

Op Saturday 2 May 2015 11:10 CEST schreef Marko Rauhamaa:

> Cecil Westerhof <Cecil at decebal.nl>:
>> I find factorial a lot cleaner code as factorial_iterative, so here
>> tail recursion would be beneficial.
>
> I would just call math.factorial() and be done with it.

You missed the point. I did not need a factorial function, I wanted to
show that tail optimisation could be beneficial.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From davea at davea.name  Sat May  2 06:35:11 2015
From: davea at davea.name (Dave Angel)
Date: Sat, 02 May 2015 06:35:11 -0400
Subject: Python is not bad ;-)
In-Reply-To: <87twvvmigr.fsf@Equus.decebal.nl>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl> <87twvvmigr.fsf@Equus.decebal.nl>
Message-ID: <5544A85F.5080805@davea.name>

On 05/02/2015 05:33 AM, Cecil Westerhof wrote:

Please check your email settings.  Your messages that you type seem to 
be indented properly, but those that are quoting earlier messages (even 
your own) are not.  See below.  I suspect there's some problem with how 
your email program processes html messages.

> Op Saturday 2 May 2015 10:26 CEST schreef Cecil Westerhof:
>>
>> That is mostly because the tail recursion version starts multiplying
>> at the high end. I wrote a second version:
>> def factorial_tail_recursion2(x):
>> y = 1
>> z = 1
>> while True:
>> if x == z:
>> return y
>> y *= z
>> z += 1
>>
>> This has almost the performance of the iterative version: 34 and 121
>> seconds.
>>
>> So I made a new recursive version:
>> def factorial_recursive(x, y = 1, z = 1):
>> return y if x == z else factorial_recursive(x, x * y, z + 1)
>
> Stupid me 'x == z' should be 'z > x'
>

I can't see how that is worth doing. The recursive version is already a 
distortion of the definition of factorial that I learned.  And to force 
it to be recursive and also contort it so it does the operations in the 
same order as the iterative version, just to gain performance?

If you want performance on factorial, write it iteratively, in as 
straightforward a way as possible.  Or just call the library function.

Recursion is a very useful tool in a developer's toolbox.  But the only 
reason I would use it for factorial is to provide a simple demonstration 
to introduce the concept to a beginner.


-- 
DaveA


From rosuav at gmail.com  Sat May  2 06:42:14 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 2 May 2015 20:42:14 +1000
Subject: Python is not bad ;-)
In-Reply-To: <5544A55C.2070504@davea.name>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net>
 <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
 <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net> <5544A55C.2070504@davea.name>
Message-ID: <CAPTjJmoK7AD=Xwy35Ux6EPZqJjJ77X2PbYNnOwyA-Zc8kYLC3g@mail.gmail.com>

On Sat, May 2, 2015 at 8:22 PM, Dave Angel <davea at davea.name> wrote:
> On 05/02/2015 05:58 AM, Marko Rauhamaa wrote:
>>
>> Chris Angelico <rosuav at gmail.com>:
>>
>>> Guido is against anything that disrupts tracebacks, and optimizing
>>> tail recursion while maintaining traceback integrity is rather harder.
>>
>>
>> Tail recursion could be suppressed during debugging. Optimized code can
>> play all kinds of non-obvious tricks with the execution frame.
>>
>>> In the situations where it really is simple, you can always make the
>>> change in your own code anyway. Often, the process of converting
>>> recursion into tail recursion warps the code to the point where it's
>>> abusing recursion to implement iteration anyway, so just make it
>>> iterative.
>>
>>
>> While you shouldn't actively replace Python iteration with recursion, I
>> strongly disagree that naturally occurring tail recursion is abuse or
>> should be avoided in any manner.
>>
>
> When you strongly disagree, make sure you're disagreeing with what Chris
> actually said.  The key phrase in his message was
>     "converting recursion into tail recursion"
>
> NOT  "converting iteration into recursion"
> and NOT "naturally occurring tail recursion"

Indeed. I'll clarify my statement.

When a function needs to do further actions after the tail call, the
usual solution is to carry the information through parameters.

def stupid_sum_iter(x):
    """Calculate sum(range(x+1))"""
    tot = 0
    while x:
        tot += x
        x -= 1
    return tot

def stupid_sum_recur(x):
    return x and x + stupid_sum_recur(x - 1)

def stupid_sum_tail_recur(x, tot=0):
    if not x: return tot
    return stupid_sum_tail_recur(x - 1, tot + x)

Converting the recursive form into optimizable tail recursion requires
an accumulator parameter, which means it's virtually the same as the
iterative version. Naturally-occurring tail recursion does exist, but
it's far from all cases of recursion.

ChrisA


From subhabrata.banerji at gmail.com  Sat May  2 06:42:14 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Sat, 2 May 2015 03:42:14 -0700 (PDT)
Subject: Problem in Handling MySql Data.
Message-ID: <b8422ad3-e9e4-475c-9545-403fc41b9e74@googlegroups.com>

Dear Group,

I am trying to write the following script to call around 0.3 million files from a remote server. 
It is generally working fine, but could work only upto 65 to 70 files. After this, it is just
printing the file names and not processing anything. If anyone may kindly suggest what I am
doing wrong?


import pymysql
import pymysql.cursors
import os
import win32com.client
from gensim.models import Word2Vec
import nltk
from nltk.corpus import stopwords
import pyPdf
from pyth.plugins.rtf15.reader import Rtf15Reader
from pyth.plugins.plaintext.writer import PlaintextWriter
import nltk
import zipfile, re
import time

#READING ONE DOC FILE FROM REMOTE LOCATION
def readfilesq9(n):
    connection = pymysql.connect(host='xxx.xxx.x.xxx',
                             user='abcd',
                             passwd='pwd1',
                             db='rep_db',
                             charset='utf8mb4',
                             cursorclass=pymysql.cursors.DictCursor)
    list1=[]
    with connection.cursor() as cursor:
        # Read a single record
        sql = "SELECT candidateid,cnd.FirstName, cnd.LastName,Concat('\\xxx.xxx.x.xxx\File\Cand_Res/',orgguid,'/',DATE_FORMAT(cnd.createddate,'%Y%m'),'/',candidateguid,'/',Resume) as ResumePath  from candidate cnd join mstorganization org on cnd.orgid = org.OrgId where Resume <> '' and Resume is not null order by cnd.modifieddate limit 100000"
        cursor.execute(sql)
        result = cursor.fetchall()
        #print result
        #list1=[]
        for i in result:
            try:
                #print i
                item_1=i.items()
                item_2=item_1[2][1]
                print item_2
                item_3=item_2.index("/")
                file1=item_2[item_2:]
                string1='\\\\xxx.xxx.x.xxx\\Resumes\\Cand_Res'
                file1e=file1.encode('ascii', 'ignore')
                urls=file1e.replace("/","\\")
                file_full=string1+urls
                time.sleep(1)
                #osp="C:\\Python27"
                os1=os.path.abspath(os.curdir)
                osp2=os.path.join(os1,file_full)
                print "Path1:",osp2
                file_name1=osp2
                print "Path:",file_name1
                #IDENTIFICATION OF FILE KIND
                #DOC CONVERSION
                if ".doc" in file_name1:
                    #EXTRACTING ONLY .DOC FILES
                    if ".docx" not in file_name1:
                        #print "It is A Doc file$$:",file_name
                        try:
                            doc = win32com.client.GetObject(file_name1)
                            text = doc.Range().Text
                            text1=text.encode('ascii','ignore')
                            text_word=text1.split()
                            #print "The Text Word is:",text_word
                            #print "Text for Document File Is:",text1
                            list1.append(text_word)
                            #print "List for Doc File Is:",list3
                            #print "It is a Doc file"
                        except:
                            print "DOC ISSUE"
                    #EXTRACTING ONLY .DOCX FILES
                    elif ".docx" in file_name1:
                        #print "It is DOCX FILE:",file_name
                        docx1=zipfile.ZipFile(file_name1)
                        content = docx1.read('word/document.xml').decode('utf-8')
                        cleaned = re.sub('<(.|\n)*?>','',content).encode('ascii','ignore')
                        cleaned_word=cleaned.split()
                        #print "The Cleaned Document Is:",cleaned
                        list1.append(cleaned_word)
                        #print "List for DocX file Is:",list4
                    else:
                        print "NONE1"
                else:
                    print "It is not a Doc file" 
                
                
            except:
                print "OOPS1"

I am using Python2.7.6 on Enthought Canopy. It is not my default Python. My default Python is in location,"C:\\Python27".
I am using MySql and Windows 7 Professional. Apology for any indentation error.

Regards,
Subhabrata Banerjee. 



From subhabrata.banerji at gmail.com  Sat May  2 06:44:25 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Sat, 2 May 2015 03:44:25 -0700 (PDT)
Subject: Fast way of extracting files from various folders
In-Reply-To: <mailman.3.1430558542.12865.python-list@python.org>
References: <6ba8934e-2f1a-4bcf-b72a-0dd276182ca2@googlegroups.com>
 <mailman.3.1430558542.12865.python-list@python.org>
Message-ID: <19811322-c2af-44e0-b84b-fd97127a9f0c@googlegroups.com>

On Saturday, May 2, 2015 at 2:52:32 PM UTC+5:30, Peter Otten wrote:
>  wrote:
> 
> > I have several millions of documents in several folders and subfolders in
> > my machine. I tried to write a script as follows, to extract all the .doc
> > files and to convert them in text, but it seems it is taking too much of
> > time.
> > 
> > import os
> > from fnmatch import fnmatch
> > import win32com.client
> > import zipfile, re
> > def listallfiles2(n):
> >     root = 'C:\Cand_Res'
> >     pattern = "*.doc"
> >     list1=[]
> >     for path, subdirs, files in os.walk(root):
> >         for name in files:
> >             if fnmatch(name, pattern):
> >                 file_name1=os.path.join(path, name)
> >                 if ".doc" in file_name1:
> >                     #EXTRACTING ONLY .DOC FILES
> >                     if ".docx" not in file_name1:
> >                         #print "It is A Doc file$$:",file_name1
> >                         try:
> >                             doc = win32com.client.GetObject(file_name1)
> >                             text = doc.Range().Text
> >                             text1=text.encode('ascii','ignore')
> >                             text_word=text1.split()
> >                             #print "Text for Document File Is:",text1
> >                             list1.append(text_word)
> >                             print "It is a Doc file"
> >                         except:
> >                             print "DOC ISSUE"
> > 
> > But it seems it is taking too much of time, to convert to text and to
> > append to list. Is there any way I may do it fast? I am using Python2.7 on
> > Windows 7 Professional Edition. Apology for any indentation error.
> > 
> > If any one may kindly suggest a solution.
> 
> It will not help the first time through your documents, but if you write the 
> words for the word documents in one .txt file per .doc, and the original 
> files rarely change you can read from the .txt files when you run your 
> script a second time. Just make sure that the .txt is younger than the 
> corresponding .doc by checking the file time.
> 
> In short: use a caching strategy.

Thanks Peter. I'll surely check on that. Regards, Subhabrata Banerjee. 


From rosuav at gmail.com  Sat May  2 07:05:39 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 2 May 2015 21:05:39 +1000
Subject: Problem in Handling MySql Data.
In-Reply-To: <b8422ad3-e9e4-475c-9545-403fc41b9e74@googlegroups.com>
References: <b8422ad3-e9e4-475c-9545-403fc41b9e74@googlegroups.com>
Message-ID: <CAPTjJmr+Q5Q1oQ2ydWt+Sy4aEk6MpzAkvedyRsRabmz+8Xn9Ag@mail.gmail.com>

On Sat, May 2, 2015 at 8:42 PM,  <subhabrata.banerji at gmail.com> wrote:
> Dear Group,
>
> I am trying to write the following script to call around 0.3 million files from a remote server.
> It is generally working fine, but could work only upto 65 to 70 files. After this, it is just
> printing the file names and not processing anything. If anyone may kindly suggest what I am
> doing wrong?
>
>             except:
>                 print "OOPS1"

Get rid of these lines (and the similar ones elsewhere), then run the
script again. You'll get a useful error message, most likely.

ChrisA


From auriocus at gmx.de  Sat May  2 07:07:52 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Sat, 02 May 2015 13:07:52 +0200
Subject: Python is not bad ;-)
In-Reply-To: <87a8xnl2r6.fsf@elektro.pacujo.net>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>	<mailman.121.1430385051.3680.python-list@python.org>	<87383hj4zj.fsf@elektro.pacujo.net>
 <87d22lx38x.fsf@Equus.decebal.nl>	<55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>	<87383fo062.fsf@Equus.decebal.nl>
 <87egmzl4yr.fsf@elektro.pacujo.net>	<mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net>
Message-ID: <mi2b49$pip$1@dont-email.me>

Am 02.05.15 um 11:58 schrieb Marko Rauhamaa:
> Chris Angelico <rosuav at gmail.com>:
>
>> Guido is against anything that disrupts tracebacks, and optimizing
>> tail recursion while maintaining traceback integrity is rather harder.
>
> Tail recursion could be suppressed during debugging. Optimized code can
> play all kinds of non-obvious tricks with the execution frame.
>
>> In the situations where it really is simple, you can always make the
>> change in your own code anyway. Often, the process of converting
>> recursion into tail recursion warps the code to the point where it's
>> abusing recursion to implement iteration anyway, so just make it
>> iterative.
>
> While you shouldn't actively replace Python iteration with recursion, I
> strongly disagree that naturally occurring tail recursion is abuse or
> should be avoided in any manner.

Could you show me an example of naturally occuring tail recursion? I 
can't think of any. Or is it maybe one involving mutual recursion?

I need to add, I grew up with imperative programming, and as such got 
recursion as the solution to problems that are too complex for 
iteration, i.e. tree traversal and such, and exactly these are never 
tail-recursive.


	Christian


From Cecil at decebal.nl  Sat May  2 07:12:18 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 02 May 2015 13:12:18 +0200
Subject: Python is not bad ;-)
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl> <87twvvmigr.fsf@Equus.decebal.nl>
 <mailman.6.1430562930.12865.python-list@python.org>
Message-ID: <87lhh7mdwt.fsf@Equus.decebal.nl>

Op Saturday 2 May 2015 12:35 CEST schreef Dave Angel:

> On 05/02/2015 05:33 AM, Cecil Westerhof wrote:
>
> Please check your email settings. Your messages that you type seem
> to be indented properly, but those that are quoting earlier messages
> (even your own) are not. See below. I suspect there's some problem
> with how your email program processes html messages.
>
>> Op Saturday 2 May 2015 10:26 CEST schreef Cecil Westerhof:
>>>
>>> That is mostly because the tail recursion version starts
>>> multiplying at the high end. I wrote a second version: def
>>> factorial_tail_recursion2(x): y = 1 z = 1 while True: if x == z:
>>> return y y *= z z += 1
>>>
>>> This has almost the performance of the iterative version: 34 and
>>> 121 seconds.
>>>
>>> So I made a new recursive version:
>>> def factorial_recursive(x, y = 1, z = 1):
>>> return y if x == z else factorial_recursive(x, x * y, z + 1)
>>
>> Stupid me 'x == z' should be 'z > x'
>>
>
> I can't see how that is worth doing. The recursive version is
> already a distortion of the definition of factorial that I learned.
> And to force it to be recursive and also contort it so it does the
> operations in the same order as the iterative version, just to gain
> performance?
>
> If you want performance on factorial, write it iteratively, in as
> straightforward a way as possible. Or just call the library
> function.
>
> Recursion is a very useful tool in a developer's toolbox. But the
> only reason I would use it for factorial is to provide a simple
> demonstration to introduce the concept to a beginner.

And that was what I was doing here: showing that tail recursion can
have benefits.

By the way I have seen factorial mostly implemented recursive.

Also I am now testing with very large values. The version where I use
both tail recursion versions are faster as the iterative version.
Calculating 500.000:
iterative:                  137 seconds
tail recursion optimised:   120 seconds
tail recursion simple:      130 seconds

You have to be careful that you do not fall into the trap that you are
sure your way is the best way. I have had the same on the Scala list.
I told there that for a certain function it was better to use an
iterative version as a tail recursive function. They did not want to
believe it at first.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From rosuav at gmail.com  Sat May  2 07:21:44 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 2 May 2015 21:21:44 +1000
Subject: Python is not bad ;-)
In-Reply-To: <mi2b49$pip$1@dont-email.me>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net>
 <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
 <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net> <mi2b49$pip$1@dont-email.me>
Message-ID: <CAPTjJmoE6JvfUo3fOZi+hXPsY3+-k9N2Ob=3mZcFUCWFft-zoQ@mail.gmail.com>

On Sat, May 2, 2015 at 9:07 PM, Christian Gollwitzer <auriocus at gmx.de> wrote:
> I need to add, I grew up with imperative programming, and as such got
> recursion as the solution to problems that are too complex for iteration,
> i.e. tree traversal and such, and exactly these are never tail-recursive.

Tree traversal can be forking-recursive:

def walk(func):
    if self.left and self.left.walk(func): return 1
    if func(self.payload): return 1
    if self.right: return self.right.walk(func)

The left walk is non-tail-recursive, but the right walk is. This
particular style looks rather less clean with iteration:

def walk(func):
    while True:
        if self.left and self.left.walk(func): return 1
        if func(self.payload): return 1
        if not self.right: break
        self = self.right

I'd call this an example of fairly natural tail recursion.

ChrisA


From auriocus at gmx.de  Sat May  2 07:32:59 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Sat, 02 May 2015 13:32:59 +0200
Subject: Python is not bad ;-)
In-Reply-To: <mailman.9.1430565706.12865.python-list@python.org>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl> <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net> <mi2b49$pip$1@dont-email.me>
 <mailman.9.1430565706.12865.python-list@python.org>
Message-ID: <mi2cjc$uq2$1@dont-email.me>

Am 02.05.15 um 13:21 schrieb Chris Angelico:
> On Sat, May 2, 2015 at 9:07 PM, Christian Gollwitzer <auriocus at gmx.de> wrote:
>> I need to add, I grew up with imperative programming, and as such got
>> recursion as the solution to problems that are too complex for iteration,
>> i.e. tree traversal and such, and exactly these are never tail-recursive.
>
> Tree traversal can be forking-recursive:
>
> def walk(func):
>      if self.left and self.left.walk(func): return 1
>      if func(self.payload): return 1
>      if self.right: return self.right.walk(func)
>
> The left walk is non-tail-recursive, but the right walk is. This
> particular style looks rather less clean with iteration:

Accepted. In case your tree is heavily imbalanced, i.e. it is a list 
pretending to be a tree, the tail-recursion can cut down the memory 
usage significantly. It does not help, though, in the general case of a 
balanced tree or if the tree leans to the left side. That's why I still 
think it is a microoptimization, which helps only in some specific cases.

	Christian


From marko at pacujo.net  Sat May  2 07:42:31 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 02 May 2015 14:42:31 +0300
Subject: Python is not bad ;-)
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl> <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net> <mi2b49$pip$1@dont-email.me>
 <mailman.9.1430565706.12865.python-list@python.org>
 <mi2cjc$uq2$1@dont-email.me>
Message-ID: <87383fkxy0.fsf@elektro.pacujo.net>

Christian Gollwitzer <auriocus at gmx.de>:

> That's why I still think it is a microoptimization, which helps only
> in some specific cases.

It isn't done for performance. It's done to avoid a stack overflow
exception.


Marko


From DLipman~NoSpam~ at Verizon.Net  Sat May  2 08:03:00 2015
From: DLipman~NoSpam~ at Verizon.Net (David H. Lipman)
Date: Sat, 2 May 2015 08:03:00 -0400
Subject: convert pdf to excel xls file
In-Reply-To: <5b36ac7d-53fd-4271-80a6-5ba3a9f14c7a@googlegroups.com>
References: <mailman.62.1427100644.10327.python-list@python.org>
 <5b36ac7d-53fd-4271-80a6-5ba3a9f14c7a@googlegroups.com>
Message-ID: <zpOdnVC6RLZoIdnInZ2dnUU7-aednZ2d@giganews.com>

From: <jaronstreak at gmail.com>

> 
> This is a very common problem 

That problem is called spam coupled with Google who facilitating it.


-- 
Dave
Multi-AV Scanning Tool - http://multi-av.thespykiller.co.uk
http://www.pctipp.ch/downloads/dl/35905.asp


From invalid at invalid.invalid  Sat May  2 09:42:25 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Sat, 2 May 2015 13:42:25 +0000 (UTC)
Subject: convert pdf to excel xls file
References: <mailman.62.1427100644.10327.python-list@python.org>
 <5b36ac7d-53fd-4271-80a6-5ba3a9f14c7a@googlegroups.com>
 <zpOdnVC6RLZoIdnInZ2dnUU7-aednZ2d@giganews.com>
Message-ID: <mi2k81$orf$1@reader1.panix.com>

On 2015-05-02, David H. Lipman <DLipman~NoSpam~@Verizon.Net> wrote:
> From: <jaronstreak at gmail.com>
>
>> 
>> This is a very common problem 
>
> That problem is called spam coupled with Google who facilitating it.

The problem is called "seeing responses to spam that had bee properly
filtered out".

--
Grant





From liik.joonas at gmail.com  Sat May  2 09:50:58 2015
From: liik.joonas at gmail.com (Joonas Liik)
Date: Sat, 2 May 2015 16:50:58 +0300
Subject: Python is not bad ;-)
In-Reply-To: <87383fkxy0.fsf@elektro.pacujo.net>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net>
 <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
 <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net> <mi2b49$pip$1@dont-email.me>
 <mailman.9.1430565706.12865.python-list@python.org>
 <mi2cjc$uq2$1@dont-email.me> <87383fkxy0.fsf@elektro.pacujo.net>
Message-ID: <CAB1GNpSqvkwVJ+fjx3pv1DGh595+0VVKF4C_voOPq4VymSA_tQ@mail.gmail.com>

I agree, stack overflow is literally the main issue that ive run in to
(tree traversal)
I've yet to refactor recursion to iterative for speed, but i have done so
to avoid hitting the stack size limit.

Tree traversal and similar problems in particular lend themselves well to
recursion and are not quite as trivial to do in an iterative fashion.

I've also found myself constructing an ad-hoc trampoline at least once just
to sneak past the stack limit.
(probably very evil but it worked and it wasn't nearly as ugly as it sounds
like so ill live with it..)


On 2 May 2015 at 14:42, Marko Rauhamaa <marko at pacujo.net> wrote:

> Christian Gollwitzer <auriocus at gmx.de>:
>
> > That's why I still think it is a microoptimization, which helps only
> > in some specific cases.
>
> It isn't done for performance. It's done to avoid a stack overflow
> exception.
>
>
> Marko
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150502/f4151ff6/attachment.html>

From Cecil at decebal.nl  Sat May  2 10:20:29 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 02 May 2015 16:20:29 +0200
Subject: Throw the cat among the pigeons
Message-ID: <87h9rvm576.fsf@Equus.decebal.nl>

I am throwing the cat among the pigeons. ;-)

In another thread I mentioned that I liked to have tail recursion in
Python. To be clear not automatic, but asked for.

Looking at the replies I did hit a nerve. But I still want to
continue.

Some things are better expressed recursively for the people reading
the code. But there are two problems with that:
- You can get out of stack space
- It is less efficient

Most of the time the first problem is the most important.

When I write factorial (I know it is already written, but I use it as
an example to show a point), the recursive variant can not be called
with 1.000 without tail recursion. So for functions that could go very
deep, tail recursion would be a blessing.

By the way: I think that even if the recursion does not go further as
500, it is still a good idea to use tail recursion. Why use stack
space when it is not necessary?

But to my surprise tail recursion could even be more efficient. I
wrote two different versions of factorial with self implemented tail
recursion. For bigger values both are more efficient. And I expect
that if the tail recursion is done by the compiler instead of by hand,
it will be a little faster still.

This is the output a run of my code:
    15:05:50: Start with the time needed to calculate 100000 times
    15:05:50: Timing factorial_iterative         (985): 32.265420768002514
    15:06:22: Timing factorial_recursive         (985): 58.381072121992474
    15:07:21: Timing factorial_recursive_old     (985): 64.46238571999129
    15:08:25: Timing factorial_tail_recursion    (985): 40.43312480399618
    15:09:06: Timing factorial_tail_recursion_old(985): 39.70765891499468

    15:09:45: Start with the time needed to calculate 1 times
              No recursive, because without tail recursion you would run out of stack space
    15:09:45: Timing factorial_iterative         (100000): 3.9112528519763146
    15:09:49: Timing factorial_tail_recursion    (100000): 3.928693111985922
    15:09:53: Timing factorial_tail_recursion_old(100000): 4.305187558988109

    15:09:58: Timing factorial_iterative         (200000): 18.081113666004967
    15:10:16: Timing factorial_tail_recursion    (200000): 16.660855480993632
    15:10:32: Timing factorial_tail_recursion_old(200000): 18.169589380006073

    15:10:51: Timing factorial_iterative         (300000): 41.79109025900834
    15:11:32: Timing factorial_tail_recursion    (300000): 38.368264676013496
    15:12:11: Timing factorial_tail_recursion_old(300000): 41.646923307009274

    15:12:52: Timing factorial_iterative         (400000): 78.35287749301642
    15:14:11: Timing factorial_tail_recursion    (400000): 73.17889478098368
    15:15:24: Timing factorial_tail_recursion_old(400000): 89.64840986899799

    15:16:53: Timing factorial_iterative         (500000): 154.76221033901675
    15:19:28: Timing factorial_tail_recursion    (500000): 130.3837693700043
    15:21:39: Timing factorial_tail_recursion_old(500000): 131.41286378499353

    15:23:50: These result show that tail recursion can be interesting
              They show also that the way you use tail recursion is important

As said the most important reason is that code is often more elegant
when written recursively, but you cannot do that if it is possible
that the recursion can go very deep. But if recursively code would be
more elegant and faster, then it would really be interesting to have.

I would not opt for automatically using tail recursion, but only when
the programmer says so. And if it is not possible, that should be an
error.

To make sure it was not a fluke, I ran it again:
    16:01:30: Start with the time needed to calculate 100000 times
    16:01:30: Timing factorial_iterative         (985): 31.465190444985637
    16:02:01: Timing factorial_recursive         (985): 54.562154764978914
    16:02:56: Timing factorial_recursive_old     (985): 55.56128695001826
    16:03:52: Timing factorial_tail_recursion    (985): 36.27355203201296
    16:04:28: Timing factorial_tail_recursion_old(985): 40.36879472099827

    16:05:08: Start with the time needed to calculate 1 times
              No recursive, because without tail recursion you would run out of stack space
    16:05:08: Timing factorial_iterative         (100000): 3.764512833993649
    16:05:12: Timing factorial_tail_recursion    (100000): 3.8083034529990982
    16:05:16: Timing factorial_tail_recursion_old(100000): 4.107901128008962

    16:05:20: Timing factorial_iterative         (200000): 16.076719653996406
    16:05:36: Timing factorial_tail_recursion    (200000): 16.108007609989727
    16:05:52: Timing factorial_tail_recursion_old(200000): 17.71343147099833

    16:06:10: Timing factorial_iterative         (300000): 37.82596729800571
    16:06:48: Timing factorial_tail_recursion    (300000): 40.308226338995155
    16:07:28: Timing factorial_tail_recursion_old(300000): 41.254319412022596

    16:08:09: Timing factorial_iterative         (400000): 77.01277641401975
    16:09:26: Timing factorial_tail_recursion    (400000): 73.4060631209868
    16:10:40: Timing factorial_tail_recursion_old(400000): 80.26402168802451

    16:12:00: Timing factorial_iterative         (500000): 131.84731978402124
    16:14:12: Timing factorial_tail_recursion    (500000): 125.31950747498195
    16:16:17: Timing factorial_tail_recursion_old(500000): 133.39186109701404

    16:18:30: These result show that tail recursion can be interesting
              They show also that the way you use tail recursion is important


-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From bc at freeuk.com  Sat May  2 11:26:40 2015
From: bc at freeuk.com (BartC)
Date: Sat, 02 May 2015 16:26:40 +0100
Subject: l = range(int(1E9))
In-Reply-To: <mailman.136.1430414427.3680.python-list@python.org>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
Message-ID: <W061x.692563$396.195269@fx12.am4>

On 30/04/2015 18:20, Ben Finney wrote:
> Jon Ribbens <jon+usenet at unequivocal.co.uk> writes:

>> If you use xrange() instead of range() then you will get an iterator
>> which will return each of the numbers in turn without any need to
>> create an enormous list of all of them.
>
> If you use Python 3 instead of the obsolescent Python 2, the ?range?
> callable has this sensible behaviour by default.

When I first looked at Python 20 or so years ago this seemed to be the 
standard way of writing a for-loop:

for i in range(N):
    ....

I remember being completely astonished at the time that 'range' actually 
created a list of values from 0 to N-1.

Python was already known to be slow yet it deliberately crippled itself 
by using just about the slowest method imaginable of executing a simple 
iterative loop? By first creating a list of a million objects!

And sometimes you weren't even interested in the values but just wanted 
to execute something N times so it was a wasted effort.

That was eventually fixed with xrange, but why do it like that in the 
first place?

(At the time, I was creating bytecode languages as part of the 
applications I was writing. An empty for-loop executed just one bytecode 
per iteration, and it was also the fastest bytecode instruction. An 
empty for-loop executed three Python bytecodes per iteration last time I 
looked. It seems that Python used to like making a rod for its own back.)

-- 
Bartc


From ian.g.kelly at gmail.com  Sat May  2 11:31:57 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 2 May 2015 09:31:57 -0600
Subject: Python is not bad ;-)
In-Reply-To: <5544A85F.5080805@davea.name>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
 <87twvvmigr.fsf@Equus.decebal.nl> <5544A85F.5080805@davea.name>
Message-ID: <CALwzidkgzEeGQ3L85a0xngOkrZ9t==76Kukdu0Eur-Gdvk_FcQ@mail.gmail.com>

On Sat, May 2, 2015 at 4:35 AM, Dave Angel <davea at davea.name> wrote:
> I can't see how that is worth doing. The recursive version is already a
> distortion of the definition of factorial that I learned.  And to force it
> to be recursive and also contort it so it does the operations in the same
> order as the iterative version, just to gain performance?
>
> If you want performance on factorial, write it iteratively, in as
> straightforward a way as possible.  Or just call the library function.

Or if you really want to write it functionally:

from functools import reduce
from operator import mul

def product(iterable):
    return reduce(mul, iterable, 1)

def factorial(n):
    return product(range(1, n+1))

For Python 2, delete the first import and replace range with xrange.


From pander.musubi at gmail.com  Sat May  2 11:35:39 2015
From: pander.musubi at gmail.com (Pander Musubi)
Date: Sat, 2 May 2015 08:35:39 -0700 (PDT)
Subject: Custom alphabetical sort
In-Reply-To: <40d108ec-b019-4829-a969-c8ef513866f1@googlegroups.com>
References: <40d108ec-b019-4829-a969-c8ef513866f1@googlegroups.com>
Message-ID: <e383c003-1fff-4fd9-b6a2-222e0a436e6d@googlegroups.com>

On Monday, 24 December 2012 16:32:56 UTC+1, Pander Musubi  wrote:
> Hi all,
> 
> I would like to sort according to this order:
> 
> (' ', '.', '\'', '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'A', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', 'b', 'B', 'c', 'C', '?', '?', 'd', 'D', 'e', 'E', '?', '?', '?', '?', '?', '?', '?', '?', 'f', 'F', 'g', 'G', 'h', 'H', 'i', 'I', '?', '?', '?', '?', '?', '?', '?', '?', 'j', 'J', 'k', 'K', 'l', 'L', 'm', 'M', 'n', '?', 'N', '?', 'o', 'O', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', 'p', 'P', 'q', 'Q', 'r', 'R', 's', 'S', 't', 'T', 'u', 'U', '?', '?', '?', '?', '?', '?', '?', '?', 'v', 'V', 'w', 'W', 'x', 'X', 'y', 'Y', 'z', 'Z')
> 
> How can I do this? The default sorted() does not give the desired result.
> 
> Thanks,
> 
> Pander

Meanwhile Python 3 supports locale aware sorting, see https://docs.python.org/3/howto/sorting.html


From breamoreboy at yahoo.co.uk  Sat May  2 11:40:44 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 02 May 2015 16:40:44 +0100
Subject: l = range(int(1E9))
In-Reply-To: <W061x.692563$396.195269@fx12.am4>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
Message-ID: <mi2r61$aqc$1@ger.gmane.org>

On 02/05/2015 16:26, BartC wrote:
> On 30/04/2015 18:20, Ben Finney wrote:
>> Jon Ribbens <jon+usenet at unequivocal.co.uk> writes:
>
>>> If you use xrange() instead of range() then you will get an iterator
>>> which will return each of the numbers in turn without any need to
>>> create an enormous list of all of them.
>>
>> If you use Python 3 instead of the obsolescent Python 2, the ?range?
>> callable has this sensible behaviour by default.
>
> When I first looked at Python 20 or so years ago this seemed to be the
> standard way of writing a for-loop:
>
> for i in range(N):
>     ....
>
> I remember being completely astonished at the time that 'range' actually
> created a list of values from 0 to N-1.
>
> Python was already known to be slow yet it deliberately crippled itself
> by using just about the slowest method imaginable of executing a simple
> iterative loop? By first creating a list of a million objects!
>
> And sometimes you weren't even interested in the values but just wanted
> to execute something N times so it was a wasted effort.
>
> That was eventually fixed with xrange, but why do it like that in the
> first place?
>
> (At the time, I was creating bytecode languages as part of the
> applications I was writing. An empty for-loop executed just one bytecode
> per iteration, and it was also the fastest bytecode instruction. An
> empty for-loop executed three Python bytecodes per iteration last time I
> looked. It seems that Python used to like making a rod for its own back.)
>

I first started maybe 14 years ago and the standard way of writing a for 
loop was, and still is:-

for item in items:

When did this change, or has it always been this way and you were simply 
using an idiom from other languages?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From ian.g.kelly at gmail.com  Sat May  2 11:45:11 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 2 May 2015 09:45:11 -0600
Subject: Python is not bad ;-)
In-Reply-To: <87383fkxy0.fsf@elektro.pacujo.net>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
 <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net> <mi2b49$pip$1@dont-email.me>
 <mailman.9.1430565706.12865.python-list@python.org>
 <mi2cjc$uq2$1@dont-email.me> <87383fkxy0.fsf@elektro.pacujo.net>
Message-ID: <CALwzidn_4EyKA-AekPqwk2+1+0Yd3ONsTdKx73bfpzZkyXj++w@mail.gmail.com>

On Sat, May 2, 2015 at 5:42 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Christian Gollwitzer <auriocus at gmx.de>:
>
>> That's why I still think it is a microoptimization, which helps only
>> in some specific cases.
>
> It isn't done for performance. It's done to avoid a stack overflow
> exception.

If your tree is balanced, then the number of items you would need to
have to get a stack overflow exception would be approximately 2 **
1000, which you can't possibly hope to fit into memory.

If your tree is unbalanced and you're getting a stack overflow
exception, then maybe you should think about balancing it.


From liik.joonas at gmail.com  Sat May  2 11:53:24 2015
From: liik.joonas at gmail.com (Joonas Liik)
Date: Sat, 2 May 2015 18:53:24 +0300
Subject: Python is not bad ;-)
In-Reply-To: <CALwzidn_4EyKA-AekPqwk2+1+0Yd3ONsTdKx73bfpzZkyXj++w@mail.gmail.com>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net>
 <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
 <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net> <mi2b49$pip$1@dont-email.me>
 <mailman.9.1430565706.12865.python-list@python.org>
 <mi2cjc$uq2$1@dont-email.me> <87383fkxy0.fsf@elektro.pacujo.net>
 <CALwzidn_4EyKA-AekPqwk2+1+0Yd3ONsTdKx73bfpzZkyXj++w@mail.gmail.com>
Message-ID: <CAB1GNpRRL8qwA+Oy7EgNb_5myYdBgo6triAB_550YgvvUaC2+A@mail.gmail.com>

Balancing of trees is kind of irrelevant when "tree" means "search space"
no?
And you definitely dont need to keep the entire tree in memory at the same
time.

By cropping unsuitable branches early (and not keeping the entire tree in
memory)
it is quite easy to have more than 1000 of call stack and still have
reasonable
preformance. (some/many nodes have 0 or 1 children)

Also should not-running-out-of-call-stack really be the main reason to
balance trees?
That sounds like an optimisation to me ..

On 2 May 2015 at 18:45, Ian Kelly <ian.g.kelly at gmail.com> wrote:

> On Sat, May 2, 2015 at 5:42 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> > Christian Gollwitzer <auriocus at gmx.de>:
> >
> >> That's why I still think it is a microoptimization, which helps only
> >> in some specific cases.
> >
> > It isn't done for performance. It's done to avoid a stack overflow
> > exception.
>
> If your tree is balanced, then the number of items you would need to
> have to get a stack overflow exception would be approximately 2 **
> 1000, which you can't possibly hope to fit into memory.
>
> If your tree is unbalanced and you're getting a stack overflow
> exception, then maybe you should think about balancing it.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150502/da03bcff/attachment.html>

From rosuav at gmail.com  Sat May  2 11:55:44 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 3 May 2015 01:55:44 +1000
Subject: Python is not bad ;-)
In-Reply-To: <CALwzidn_4EyKA-AekPqwk2+1+0Yd3ONsTdKx73bfpzZkyXj++w@mail.gmail.com>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net>
 <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
 <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net> <mi2b49$pip$1@dont-email.me>
 <mailman.9.1430565706.12865.python-list@python.org>
 <mi2cjc$uq2$1@dont-email.me> <87383fkxy0.fsf@elektro.pacujo.net>
 <CALwzidn_4EyKA-AekPqwk2+1+0Yd3ONsTdKx73bfpzZkyXj++w@mail.gmail.com>
Message-ID: <CAPTjJmqmVuFmZbhtiiOj2haVjFuHOYrBVcNLSe6ohUUS1btNBA@mail.gmail.com>

On Sun, May 3, 2015 at 1:45 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Sat, May 2, 2015 at 5:42 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
>> Christian Gollwitzer <auriocus at gmx.de>:
>>
>>> That's why I still think it is a microoptimization, which helps only
>>> in some specific cases.
>>
>> It isn't done for performance. It's done to avoid a stack overflow
>> exception.
>
> If your tree is balanced, then the number of items you would need to
> have to get a stack overflow exception would be approximately 2 **
> 1000, which you can't possibly hope to fit into memory.
>
> If your tree is unbalanced and you're getting a stack overflow
> exception, then maybe you should think about balancing it.

That's assuming it's a search tree, where you *can* just "think about
balancing it". What if it's a parse tree? Let's say you're walking the
AST of a Python module, looking for all functions that contain 'yield'
or 'yield from' (ie generator functions). To do that, you need to walk
the entire depth of the tree, no matter how far that goes. I'm not
sure how complex a piece of code would have to be to hit 1000, but it
wouldn't be hard to have each level of tree cost you two or three
stack entries, so that could come down to just a few hundred.

However, while that _is_ more likely to produce an unbalanced tree,
it's also that much less capable of being converted into tail
recursion.

ChrisA


From bc at freeuk.com  Sat May  2 12:17:57 2015
From: bc at freeuk.com (BartC)
Date: Sat, 02 May 2015 17:17:57 +0100
Subject: l = range(int(1E9))
In-Reply-To: <mailman.12.1430581268.12865.python-list@python.org>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
Message-ID: <_M61x.477467$Ek.357048@fx07.am4>

On 02/05/2015 16:40, Mark Lawrence wrote:
> On 02/05/2015 16:26, BartC wrote:
>> On 30/04/2015 18:20, Ben Finney wrote:
>>> Jon Ribbens <jon+usenet at unequivocal.co.uk> writes:
>>
>>>> If you use xrange() instead of range() then you will get an iterator
>>>> which will return each of the numbers in turn without any need to
>>>> create an enormous list of all of them.
>>>
>>> If you use Python 3 instead of the obsolescent Python 2, the ?range?
>>> callable has this sensible behaviour by default.
>>
>> When I first looked at Python 20 or so years ago this seemed to be the
>> standard way of writing a for-loop:
>>
>> for i in range(N):
>>     ....
>>
>> I remember being completely astonished at the time that 'range' actually
>> created a list of values from 0 to N-1.

> I first started maybe 14 years ago and the standard way of writing a for
> loop was, and still is:-
>
> for item in items:
>
> When did this change, or has it always been this way and you were simply
> using an idiom from other languages?

Your example is the equivalent of 'forall' in other languages, where you 
iterate over the values of some collection of data.

I agree that most for-loops in Pythonic code probably fall into that 
category.

But for looping over a simple integer range, then using 'range' to 
denote the range (and build a list as it used to do), was how it was 
done. And earlier on people would have been porting coding code to 
Python at which point a straightforward 'for i=a to b' loop suddenly 
acquired a substantial overhead it didn't have before!

-- 
Bartc






From breamoreboy at yahoo.co.uk  Sat May  2 12:39:20 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 02 May 2015 17:39:20 +0100
Subject: l = range(int(1E9))
In-Reply-To: <_M61x.477467$Ek.357048@fx07.am4>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
Message-ID: <mi2ujt$vqf$1@ger.gmane.org>

On 02/05/2015 17:17, BartC wrote:
> On 02/05/2015 16:40, Mark Lawrence wrote:
>> On 02/05/2015 16:26, BartC wrote:
>>> On 30/04/2015 18:20, Ben Finney wrote:
>>>> Jon Ribbens <jon+usenet at unequivocal.co.uk> writes:
>>>
>>>>> If you use xrange() instead of range() then you will get an iterator
>>>>> which will return each of the numbers in turn without any need to
>>>>> create an enormous list of all of them.
>>>>
>>>> If you use Python 3 instead of the obsolescent Python 2, the ?range?
>>>> callable has this sensible behaviour by default.
>>>
>>> When I first looked at Python 20 or so years ago this seemed to be the
>>> standard way of writing a for-loop:
>>>
>>> for i in range(N):
>>>     ....
>>>
>>> I remember being completely astonished at the time that 'range' actually
>>> created a list of values from 0 to N-1.
>
>> I first started maybe 14 years ago and the standard way of writing a for
>> loop was, and still is:-
>>
>> for item in items:
>>
>> When did this change, or has it always been this way and you were simply
>> using an idiom from other languages?
>
> Your example is the equivalent of 'forall' in other languages, where you
> iterate over the values of some collection of data.
>
> I agree that most for-loops in Pythonic code probably fall into that
> category.
>
> But for looping over a simple integer range, then using 'range' to
> denote the range (and build a list as it used to do), was how it was
> done. And earlier on people would have been porting coding code to
> Python at which point a straightforward 'for i=a to b' loop suddenly
> acquired a substantial overhead it didn't have before!
>

All you are saying is that they didn't bother reading the docs and 
learning how to write a *PYTHON* for loop.  Failing that don't bother 
using range, just directly convert your (say) C loop into Python.  I 
really don't see any issue here at all.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From ian.g.kelly at gmail.com  Sat May  2 13:00:04 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 2 May 2015 11:00:04 -0600
Subject: Python is not bad ;-)
In-Reply-To: <CAPTjJmqmVuFmZbhtiiOj2haVjFuHOYrBVcNLSe6ohUUS1btNBA@mail.gmail.com>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
 <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net> <mi2b49$pip$1@dont-email.me>
 <mailman.9.1430565706.12865.python-list@python.org>
 <mi2cjc$uq2$1@dont-email.me> <87383fkxy0.fsf@elektro.pacujo.net>
 <CALwzidn_4EyKA-AekPqwk2+1+0Yd3ONsTdKx73bfpzZkyXj++w@mail.gmail.com>
 <CAPTjJmqmVuFmZbhtiiOj2haVjFuHOYrBVcNLSe6ohUUS1btNBA@mail.gmail.com>
Message-ID: <CALwzidkCAFmF7hPkrfq9k0zbzycLgQj-koOzqtdCBXkQ=Agykg@mail.gmail.com>

On Sat, May 2, 2015 at 9:55 AM, Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, May 3, 2015 at 1:45 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> On Sat, May 2, 2015 at 5:42 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
>>> Christian Gollwitzer <auriocus at gmx.de>:
>>>
>>>> That's why I still think it is a microoptimization, which helps only
>>>> in some specific cases.
>>>
>>> It isn't done for performance. It's done to avoid a stack overflow
>>> exception.
>>
>> If your tree is balanced, then the number of items you would need to
>> have to get a stack overflow exception would be approximately 2 **
>> 1000, which you can't possibly hope to fit into memory.
>>
>> If your tree is unbalanced and you're getting a stack overflow
>> exception, then maybe you should think about balancing it.
>
> That's assuming it's a search tree, where you *can* just "think about
> balancing it". What if it's a parse tree? Let's say you're walking the
> AST of a Python module, looking for all functions that contain 'yield'
> or 'yield from' (ie generator functions). To do that, you need to walk
> the entire depth of the tree, no matter how far that goes. I'm not
> sure how complex a piece of code would have to be to hit 1000, but it
> wouldn't be hard to have each level of tree cost you two or three
> stack entries, so that could come down to just a few hundred.

Or you just iterate over the ast.walk generator, which uses a deque
rather than recursion.


From ian.g.kelly at gmail.com  Sat May  2 13:17:17 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 2 May 2015 11:17:17 -0600
Subject: Python is not bad ;-)
In-Reply-To: <CAB1GNpRRL8qwA+Oy7EgNb_5myYdBgo6triAB_550YgvvUaC2+A@mail.gmail.com>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl>
 <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net> <mi2b49$pip$1@dont-email.me>
 <mailman.9.1430565706.12865.python-list@python.org>
 <mi2cjc$uq2$1@dont-email.me> <87383fkxy0.fsf@elektro.pacujo.net>
 <CALwzidn_4EyKA-AekPqwk2+1+0Yd3ONsTdKx73bfpzZkyXj++w@mail.gmail.com>
 <CAB1GNpRRL8qwA+Oy7EgNb_5myYdBgo6triAB_550YgvvUaC2+A@mail.gmail.com>
Message-ID: <CALwzidkMR8OoOWmJF9Jp+_aJfMPOPDYjycRwtjs-eooSLwsnzw@mail.gmail.com>

On Sat, May 2, 2015 at 9:53 AM, Joonas Liik <liik.joonas at gmail.com> wrote:

Top-posting is heavily frowned at on this list, so please don't do it.

> Balancing of trees is kind of irrelevant when "tree" means "search space"
> no?

I think it's relatively rare that DFS is truly the best algorithm for
such a search. For one thing, "search space" often means "graph", not
"tree". And for any other type of search, you'll want/need to
implement it iteratively rather than recursively anyway.

> And you definitely dont need to keep the entire tree in memory at the same
> time.

You could harness every single storage device on the planet and you
would still not have nearly enough capacity to fill a balanced search
tree to a depth of 1000.

> Also should not-running-out-of-call-stack really be the main reason to
> balance trees?
> That sounds like an optimisation to me ..

It is. My point was that if your unbalanced search tree is getting to
a depth of 1000, then it's probably long past time for you to start
thinking about optimizing it *anyway*.


From breamoreboy at yahoo.co.uk  Sat May  2 13:22:07 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 02 May 2015 18:22:07 +0100
Subject: Python is not bad ;-)
In-Reply-To: <CAB1GNpSqvkwVJ+fjx3pv1DGh595+0VVKF4C_voOPq4VymSA_tQ@mail.gmail.com>
References: <87mw1q9jqw.fsf@Equus.decebal.nl>
 <mailman.121.1430385051.3680.python-list@python.org>
 <87383hj4zj.fsf@elektro.pacujo.net> <87d22lx38x.fsf@Equus.decebal.nl>
 <55432557$0$12994$c3e8da3$5496439d@news.astraweb.com>
 <87383fo062.fsf@Equus.decebal.nl> <87egmzl4yr.fsf@elektro.pacujo.net>
 <mailman.4.1430558719.12865.python-list@python.org>
 <87a8xnl2r6.fsf@elektro.pacujo.net> <mi2b49$pip$1@dont-email.me>
 <mailman.9.1430565706.12865.python-list@python.org>
 <mi2cjc$uq2$1@dont-email.me> <87383fkxy0.fsf@elektro.pacujo.net>
 <CAB1GNpSqvkwVJ+fjx3pv1DGh595+0VVKF4C_voOPq4VymSA_tQ@mail.gmail.com>
Message-ID: <mi3143$6ud$1@ger.gmane.org>

On 02/05/2015 14:50, Joonas Liik wrote:

[top posting fixed]

>
> On 2 May 2015 at 14:42, Marko Rauhamaa <marko at pacujo.net> wrote:
>
>> Christian Gollwitzer <auriocus at gmx.de>:
>>
>>> That's why I still think it is a microoptimization, which helps only
>>> in some specific cases.
>>
>> It isn't done for performance. It's done to avoid a stack overflow
>> exception.
>>
>>
>> Marko
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
> I agree, stack overflow is literally the main issue that ive run in to
> (tree traversal)
> I've yet to refactor recursion to iterative for speed, but i have done so
> to avoid hitting the stack size limit.
>
> Tree traversal and similar problems in particular lend themselves well to
> recursion and are not quite as trivial to do in an iterative fashion.
>
> I've also found myself constructing an ad-hoc trampoline at least once just
> to sneak past the stack limit.
> (probably very evil but it worked and it wasn't nearly as ugly as it sounds
> like so ill live with it..)
>
>

Please do not top post on this list.  Some of the threads get very long 
and are difficult if not impossible to follow when replies are top 
posted.  Interspersing your replies is the preferred method here, 
failing that simply reply at the bottom, thanks.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From gherron at digipen.edu  Sat May  2 13:28:44 2015
From: gherron at digipen.edu (Gary Herron)
Date: Sat, 02 May 2015 10:28:44 -0700
Subject: Try Except Specific Error Messages
In-Reply-To: <trinity-d2e41874-219e-46a2-8768-44616fc2dce5-1430436471933@3capp-mailcom-lxa05>
References: <trinity-d2e41874-219e-46a2-8768-44616fc2dce5-1430436471933@3capp-mailcom-lxa05>
Message-ID: <5545094C.3030103@digipen.edu>

On 04/30/2015 04:27 PM, brandon wallace wrote:
> Hi,
>   
> I am try to get more specific error messages using try/except.
> I ran this code with the cable unplugged to see the error message. I got
>
> #!/usr/bin/env python3
>
> import urllib.request
>
> webpage = urllib.request.urlopen("http://fakewebsite.com/")
> text = webpage.read().decode("utf8")
>
>
> I got two errors. This:
> [....]
> socket.gaierror: [Errno -2] Name or service not known
>
> and this:
> [....]
> urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

Note the error message here.  The exception is named 
"urllib.error.URLError" NOT "URLError" as you use in the next bit of code.

You could import that specific error if you wanted to access it that way:
      from urllib.error import URLError
otherwise you should use the fully qualified name urllib.error.URLError.

>
> I tried this but got more error messages.
As a side note, that is never a useful thing to say in this group. Take 
the time to tell is the actual errors message you got.  That way I don't 
have to waste my time running your code to see what error message you 
are getting.


> try:
>      webpage = urllib.request.urlopen("http://fakewebsite.com/")
>      text = webpage.read().decode("utf8")
> except URLError as err:
>      print("URLError: " + str(err))
>
> How do I wrap urllib.request with try/except?


-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418



From zondervanz at gmail.com  Sat May  2 14:30:21 2015
From: zondervanz at gmail.com (Dr. John Q. Hacker)
Date: Sat, 2 May 2015 13:30:21 -0500
Subject: code blocks
Message-ID: <CADL-Dvj=HCr8hEDZdRY9JLSKvrTd-EfvApOP5iNCU+J8YS5NTg@mail.gmail.com>

Hello,

I'm thinking how interesting it would be to add code blocks to Python, so
that arbitrary strings of code can be passed around.   It would open up
some interesting possibilities for self-modifying code and generic
programming.

Since Python has already a plethora of ambiguous string designations, one
of them could be set aside specificially for code blocks:

"""for i in n: print i"""

For any variables, like "n", it would access the scope in which it was
running.  When you tried to print a triple-double-quoted code block,
perhaps it could invoke the code.

My suggestion would be to use triple double-quoted strings.

You probably already know that Ruby has code blocks.

zipher
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150502/26f079e0/attachment.html>

From bc at freeuk.com  Sat May  2 14:34:41 2015
From: bc at freeuk.com (BartC)
Date: Sat, 02 May 2015 19:34:41 +0100
Subject: l = range(int(1E9))
In-Reply-To: <mailman.15.1430584786.12865.python-list@python.org>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
Message-ID: <bN81x.386497$G53.218869@fx13.am4>

On 02/05/2015 17:39, Mark Lawrence wrote:
> On 02/05/2015 17:17, BartC wrote:
>> On 02/05/2015 16:40, Mark Lawrence wrote:


>>> for item in items:
>>>
>>> When did this change, or has it always been this way and you were simply
>>> using an idiom from other languages?
>>
>> Your example is the equivalent of 'forall' in other languages, where you
>> iterate over the values of some collection of data.
>>
>> I agree that most for-loops in Pythonic code probably fall into that
>> category.
>>
>> But for looping over a simple integer range, then using 'range' to
>> denote the range (and build a list as it used to do), was how it was
>> done. And earlier on people would have been porting coding code to
>> Python at which point a straightforward 'for i=a to b' loop suddenly
>> acquired a substantial overhead it didn't have before!
>>
>
> All you are saying is that they didn't bother reading the docs and
> learning how to write a *PYTHON* for loop.  Failing that don't bother
> using range, just directly convert your (say) C loop into Python.  I
> really don't see any issue here at all.

OK, so it's the programmer's fault if as fundamental a concept as a 
for-loop ranging over integers is implemented inefficiently. He has to 
transform it into high-level terms, or has to reconstruct it somehow 
using a while-loop and an incrementing loop index.

Now I understand (why Python has been beset for so long with performance 
problems, if that is a typical attitude!).

BTW, why did they introduce the 'xrange' thing; for fun? Or did someone 
realise there might have been an issue after all?

(I tried an empty loop counting to 1 billion in Python 2.x, using 'for i 
in range'. It ran out of memory. Counting to 100 million instead, it 
worked, but still used a massive 1.5GB RAM while doing so (and took 6 
seconds to count to 100M, not too bad for Python)

Outside Python, it might typically take a few seconds to count to 1 
billion, and would use virtually no memory.

Why would anyone want to loop over all those numbers instead of 
iterating over actual data? For any number of reasons. Counting how many 
happy numbers are in that range for one!)

-- 
Bartc


From breamoreboy at yahoo.co.uk  Sat May  2 15:15:24 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 02 May 2015 20:15:24 +0100
Subject: l = range(int(1E9))
In-Reply-To: <bN81x.386497$G53.218869@fx13.am4>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
Message-ID: <mi37og$85q$1@ger.gmane.org>

On 02/05/2015 19:34, BartC wrote:
> On 02/05/2015 17:39, Mark Lawrence wrote:
>> On 02/05/2015 17:17, BartC wrote:
>>> On 02/05/2015 16:40, Mark Lawrence wrote:
>
>
>>>> for item in items:
>>>>
>>>> When did this change, or has it always been this way and you were
>>>> simply
>>>> using an idiom from other languages?
>>>
>>> Your example is the equivalent of 'forall' in other languages, where you
>>> iterate over the values of some collection of data.
>>>
>>> I agree that most for-loops in Pythonic code probably fall into that
>>> category.
>>>
>>> But for looping over a simple integer range, then using 'range' to
>>> denote the range (and build a list as it used to do), was how it was
>>> done. And earlier on people would have been porting coding code to
>>> Python at which point a straightforward 'for i=a to b' loop suddenly
>>> acquired a substantial overhead it didn't have before!
>>>
>>
>> All you are saying is that they didn't bother reading the docs and
>> learning how to write a *PYTHON* for loop.  Failing that don't bother
>> using range, just directly convert your (say) C loop into Python.  I
>> really don't see any issue here at all.
>
> OK, so it's the programmer's fault if as fundamental a concept as a
> for-loop ranging over integers is implemented inefficiently. He has to
> transform it into high-level terms, or has to reconstruct it somehow
> using a while-loop and an incrementing loop index.
>
> Now I understand (why Python has been beset for so long with performance
> problems, if that is a typical attitude!).
>
> BTW, why did they introduce the 'xrange' thing; for fun? Or did someone
> realise there might have been an issue after all?
>
> (I tried an empty loop counting to 1 billion in Python 2.x, using 'for i
> in range'. It ran out of memory. Counting to 100 million instead, it
> worked, but still used a massive 1.5GB RAM while doing so (and took 6
> seconds to count to 100M, not too bad for Python)
>
> Outside Python, it might typically take a few seconds to count to 1
> billion, and would use virtually no memory.
>
> Why would anyone want to loop over all those numbers instead of
> iterating over actual data? For any number of reasons. Counting how many
> happy numbers are in that range for one!)
>

I give up.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From marko at pacujo.net  Sat May  2 15:15:49 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 02 May 2015 22:15:49 +0300
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
Message-ID: <87mw1mkcyi.fsf@elektro.pacujo.net>

BartC <bc at freeuk.com>:

> (I tried an empty loop counting to 1 billion in Python 2.x, using 'for
> i in range'. It ran out of memory. Counting to 100 million instead, it
> worked, but still used a massive 1.5GB RAM while doing so (and took 6
> seconds to count to 100M, not too bad for Python)
>
> Outside Python, it might typically take a few seconds to count to 1
> billion, and would use virtually no memory.
>
> Why would anyone want to loop over all those numbers instead of
> iterating over actual data? For any number of reasons. Counting how
> many happy numbers are in that range for one!)

You're doing it wrong. Here's the Pythonic way to iterate over a billion
numbers:

    def loop(i):
        if i < 1000000000:
            do_stuff(i)
            keep_iterating(i + 1)

    loop(0)


Marko


From davea at davea.name  Sat May  2 15:25:36 2015
From: davea at davea.name (Dave Angel)
Date: Sat, 02 May 2015 15:25:36 -0400
Subject: Custom alphabetical sort
In-Reply-To: <e383c003-1fff-4fd9-b6a2-222e0a436e6d@googlegroups.com>
References: <40d108ec-b019-4829-a969-c8ef513866f1@googlegroups.com>
 <e383c003-1fff-4fd9-b6a2-222e0a436e6d@googlegroups.com>
Message-ID: <554524B0.8000304@davea.name>

On 05/02/2015 11:35 AM, Pander Musubi wrote:
> On Monday, 24 December 2012 16:32:56 UTC+1, Pander Musubi  wrote:
>> Hi all,
>>
>> I would like to sort according to this order:
>>
>> (' ', '.', '\'', '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'A', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', 'b', 'B', 'c', 'C', '?', '?', 'd', 'D', 'e', 'E', '?', '?', '?', '?', '?', '?', '?', '?', 'f', 'F', 'g', 'G', 'h', 'H', 'i', 'I', '?', '?', '?', '?', '?', '?', '?', '?', 'j', 'J', 'k', 'K', 'l', 'L', 'm', 'M', 'n', '?', 'N', '?', 'o', 'O', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', 'p', 'P', 'q', 'Q', 'r', 'R', 's', 'S', 't', 'T', 'u', 'U', '?', '?', '?', '?', '?', '?', '?', '?', 'v', 'V', 'w', 'W', 'x', 'X', 'y', 'Y', 'z', 'Z')
>>
>> How can I do this? The default sorted() does not give the desired result.
>>
>> Thanks,
>>
>> Pander
>
> Meanwhile Python 3 supports locale aware sorting, see https://docs.python.org/3/howto/sorting.html
>

You're aware that the message you're responding to is 16 months old?

And answered pretty thoroughly, starting with the fact that the OP's 
desired order didn't match any particular locale.



-- 
DaveA


From marko at pacujo.net  Sat May  2 15:26:56 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 02 May 2015 22:26:56 +0300
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
Message-ID: <87iocakcfz.fsf@elektro.pacujo.net>

Mark Lawrence <breamoreboy at yahoo.co.uk>:

> On 02/05/2015 19:34, BartC wrote:
>> [...]
>> [...]
>> [...]
>> [...]
>> [...]
>> [...]
>> [...]
>> [...]
>> [...]
>> [...]
>> [...]
>> [...]
>> [...]
>> [...]
[etc etc etc]
>
> I give up.

Mark, you do a commendable job admonishing the forum participants
against top-posting. Let me bring up another highly recommended
practice: trimming, or pruning, your quotes.

   The proper order is
 
      > Quote 1 (properly pruned)
 
      Your response 1
 
      > Quote 2 (properly pruned)
 
      Your response 2

   <URL: http://lipas.uwasa.fi/~ts/http/quote.html>


Marko


From joel.goldstick at gmail.com  Sat May  2 15:29:17 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Sat, 2 May 2015 15:29:17 -0400
Subject: Custom alphabetical sort
In-Reply-To: <554524B0.8000304@davea.name>
References: <40d108ec-b019-4829-a969-c8ef513866f1@googlegroups.com>
 <e383c003-1fff-4fd9-b6a2-222e0a436e6d@googlegroups.com>
 <554524B0.8000304@davea.name>
Message-ID: <CAPM-O+yFysg7yHUU8pLz52Rd=-932aCi0dxW9-yfKyB0VV-CxQ@mail.gmail.com>

On Sat, May 2, 2015 at 3:25 PM, Dave Angel <davea at davea.name> wrote:
> On 05/02/2015 11:35 AM, Pander Musubi wrote:
>>
>> On Monday, 24 December 2012 16:32:56 UTC+1, Pander Musubi  wrote:
>>>
>>> Hi all,
>>>
>>> I would like to sort according to this order:
>>>
>>> (' ', '.', '\'', '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
>>> 'a', 'A', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', 'b', 'B', 'c',
>>> 'C', '?', '?', 'd', 'D', 'e', 'E', '?', '?', '?', '?', '?', '?', '?', '?',
>>> 'f', 'F', 'g', 'G', 'h', 'H', 'i', 'I', '?', '?', '?', '?', '?', '?', '?',
>>> '?', 'j', 'J', 'k', 'K', 'l', 'L', 'm', 'M', 'n', '?', 'N', '?', 'o', 'O',
>>> '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', 'p', 'P', 'q', 'Q', 'r',
>>> 'R', 's', 'S', 't', 'T', 'u', 'U', '?', '?', '?', '?', '?', '?', '?', '?',
>>> 'v', 'V', 'w', 'W', 'x', 'X', 'y', 'Y', 'z', 'Z')
>>>
>>> How can I do this? The default sorted() does not give the desired result.
>>>
>>> Thanks,
>>>
>>> Pander
>>
>>
>> Meanwhile Python 3 supports locale aware sorting, see
>> https://docs.python.org/3/howto/sorting.html
>>
>
> You're aware that the message you're responding to is 16 months old?
>
> And answered pretty thoroughly, starting with the fact that the OP's desired
> order didn't match any particular locale.
>
>
>
> --
> DaveA
> --
> https://mail.python.org/mailman/listinfo/python-list

Dave, he is the OP.  Talking to himself?

-- 
Joel Goldstick
http://joelgoldstick.com


From lucas.bertolotti at yahoo.com  Sat May  2 15:50:08 2015
From: lucas.bertolotti at yahoo.com (lbertolotti)
Date: Sat, 2 May 2015 12:50:08 -0700 (PDT)
Subject: Canopy editor-Variables browser and help
Message-ID: <475013a6-6dcb-414e-8b99-d3af617cd2c7@googlegroups.com>

Can I:
1.Enable a variable browser in Canopy editor similar to the Spyder editor?
2.Writing a function say log( gives me the function help, but I can't read the whole documentation
3.Eclipse had a auto-complete, can I enable this in Canopy?
4.Perhaps I should try using another editor? I was having some problems with Spyder pythonpath...


From lucas.bertolotti at yahoo.com  Sat May  2 15:51:04 2015
From: lucas.bertolotti at yahoo.com (lbertolotti)
Date: Sat, 2 May 2015 12:51:04 -0700 (PDT)
Subject: Canopy editor
Message-ID: <cbf81829-1d80-43c9-b7c7-3d9e2467d10c@googlegroups.com>

I having some problems with Python editor Canopy:


From bc at freeuk.com  Sat May  2 15:51:57 2015
From: bc at freeuk.com (BartC)
Date: Sat, 02 May 2015 20:51:57 +0100
Subject: l = range(int(1E9))
In-Reply-To: <mailman.23.1430594149.12865.python-list@python.org>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
Message-ID: <CV91x.402510$zE7.111881@fx40.am4>

On 02/05/2015 20:15, Mark Lawrence wrote:
> On 02/05/2015 19:34, BartC wrote:

>> OK, so it's the programmer's fault if as fundamental a concept as a
>> for-loop ranging over integers is implemented inefficiently. He has to
>> transform it into high-level terms, or has to reconstruct it somehow
>> using a while-loop and an incrementing loop index.

> I give up.

So do I, I think, if no-one is willing to admit that the original way of 
implementing range() was a glaring mistake.

-- 
Bartc





From vasudevram at gmail.com  Sat May  2 16:02:01 2015
From: vasudevram at gmail.com (vasudevram)
Date: Sat, 2 May 2015 13:02:01 -0700 (PDT)
Subject: Inner workings of this Python feature: Can a Python data structure
 reference itself?
Message-ID: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>

Hi group,

Please refer to this blog post about code showing that a Python data structure can be self-referential:

http://jugad2.blogspot.in/2015/05/can-python-data-structure-reference.html

Gotten a couple of comments on it already, but interested in hearing thoughts of Python core dev team members or others who can comment on the internals of how this feature operates, why, etc.

Thanks
Vasudev Ram
Site: www.dancingbison.com
Python posts: jugad2.blogspot.com/search/label/python




From python.list at tim.thechases.com  Sat May  2 16:10:17 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Sat, 2 May 2015 15:10:17 -0500
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
Message-ID: <20150502151017.58536bb3@bigbox.christie.dr>

On 2015-05-02 13:02, vasudevram wrote:
> Hi group,
> 
> Please refer to this blog post about code showing that a Python
> data structure can be self-referential:
> 
> http://jugad2.blogspot.in/2015/05/can-python-data-structure-reference.html
> 
> Gotten a couple of comments on it already, but interested in
> hearing thoughts of Python core dev team members or others who can
> comment on the internals of how this feature operates, why, etc.
> 
> Thanks
> Vasudev Ram
> Site: www.dancingbison.com
> Python posts: jugad2.blogspot.com/search/label/python
> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list



From python.list at tim.thechases.com  Sat May  2 16:17:23 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Sat, 2 May 2015 15:17:23 -0500
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
Message-ID: <20150502151723.3a36e33b@bigbox.christie.dr>

[dangit, had Control down when I hit <enter> and it sent prematurely]

On 2015-05-02 13:02, vasudevram wrote:
> http://jugad2.blogspot.in/2015/05/can-python-data-structure-reference.html
>
> https://docs.python.org/2/reference/datamodel.html
>
> and saw this excerpt:
> 
> [ CPython implementation detail: CPython currently uses a
> reference-counting scheme with (optional) delayed
> detection of cyclically linked garbage, which collects
> most objects as soon as they become unreachable, but is
> not guaranteed to collect garbage containing circular
> references. ]
> 
> Not sure whether it is relevant to the topic at hand,
> since, on the one hand, it uses the words "cyclically
> linked", but on the other, it says "garbage collection".

The gotcha happens in a case where you do something like this:

  lst = []
  lst.append(lst)  # create a cycle
  del lst

This creates a cycle, then makes it unreachable, but the list is
still referenced by itself, so the reference count never drops to
zero (where it would get GC'd), and thus that item lingers around in
memory.

If you know that you're creating such cyclical structures, it's best
to manually unlink them before freeing them:

  lst = []
  lst.append(lst) # create the cycle
  lst[:] = []   # break the cycle
  # or lst.remove(lst) # though this takes more care
  del lst

-tkc






From Cecil at decebal.nl  Sat May  2 17:06:38 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 02 May 2015 23:06:38 +0200
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <mailman.26.1430597813.12865.python-list@python.org>
Message-ID: <87a8xmn0yp.fsf@Equus.decebal.nl>

Op Saturday 2 May 2015 22:17 CEST schreef Tim Chase:

> [dangit, had Control down when I hit <enter> and it sent
> prematurely]
>
> On 2015-05-02 13:02, vasudevram wrote:
>> http://jugad2.blogspot.in/2015/05/can-python-data-structure-reference.html
>>
>> https://docs.python.org/2/reference/datamodel.html
>>
>> and saw this excerpt:
>>
>> [ CPython implementation detail: CPython currently uses a
>> reference-counting scheme with (optional) delayed
>> detection of cyclically linked garbage, which collects
>> most objects as soon as they become unreachable, but is
>> not guaranteed to collect garbage containing circular
>> references. ]
>>
>> Not sure whether it is relevant to the topic at hand,
>> since, on the one hand, it uses the words "cyclically
>> linked", but on the other, it says "garbage collection".
>
> The gotcha happens in a case where you do something like this:
>
> lst = []
> lst.append(lst)  # create a cycle
> del lst
>
> This creates a cycle, then makes it unreachable, but the list is
> still referenced by itself, so the reference count never drops to
> zero (where it would get GC'd), and thus that item lingers around in
> memory.

Maybe look at Java? If my memory is correct, the JVM gc reclaims those
kind of things also.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From vasudevram at gmail.com  Sat May  2 17:07:44 2015
From: vasudevram at gmail.com (vasudevram)
Date: Sat, 2 May 2015 14:07:44 -0700 (PDT)
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <mailman.26.1430597813.12865.python-list@python.org>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <mailman.26.1430597813.12865.python-list@python.org>
Message-ID: <d2a9b8d5-beac-4470-81bf-4654d7f68630@googlegroups.com>

On Sunday, May 3, 2015 at 1:47:04 AM UTC+5:30, Tim Chase wrote:
> [dangit, had Control down when I hit <enter> and it sent prematurely]
> 
> On 2015-05-02 13:02, vasudevram wrote:
> > http://jugad2.blogspot.in/2015/05/can-python-data-structure-reference.html
> >
> > https://docs.python.org/2/reference/datamodel.html
> >
> > and saw this excerpt:
> > 
> > [ CPython implementation detail: CPython currently uses a
> > reference-counting scheme with (optional) delayed
> > detection of cyclically linked garbage, which collects
> > most objects as soon as they become unreachable, but is
> > not guaranteed to collect garbage containing circular
> > references. ]
> > 
> > Not sure whether it is relevant to the topic at hand,
> > since, on the one hand, it uses the words "cyclically
> > linked", but on the other, it says "garbage collection".
> 
> The gotcha happens in a case where you do something like this:
> 
>   lst = []
>   lst.append(lst)  # create a cycle
>   del lst
> 
> This creates a cycle, then makes it unreachable, but the list is
> still referenced by itself, so the reference count never drops to
> zero (where it would get GC'd), and thus that item lingers around in
> memory.
> 
> If you know that you're creating such cyclical structures, it's best
> to manually unlink them before freeing them:
> 
>   lst = []
>   lst.append(lst) # create the cycle
>   lst[:] = []   # break the cycle
>   # or lst.remove(lst) # though this takes more care
>   del lst
> 
> -tkc

Thanks for the reply. Will check that out.


From joel.goldstick at gmail.com  Sat May  2 17:21:51 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Sat, 2 May 2015 17:21:51 -0400
Subject: l = range(int(1E9))
In-Reply-To: <CV91x.402510$zE7.111881@fx40.am4>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
Message-ID: <CAPM-O+wMuxryqEa8SLPZP8z_FH4dHGi7CAGcuXACVRqK9vVkTQ@mail.gmail.com>

>> I give up.
>
>
> So do I, I think, if no-one is willing to admit that the original way of
> implementing range() was a glaring mistake.
>
>
> --
> Bartc
>
It doesn't matter for small ranges.  Anyway it was fixed.
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Joel Goldstick
http://joelgoldstick.com


From jon+usenet at unequivocal.co.uk  Sat May  2 17:23:57 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sat, 2 May 2015 21:23:57 +0000 (UTC)
Subject: Is my implementation of happy number OK
References: <87oam5vc8k.fsf@Equus.decebal.nl>
 <mailman.140.1430420006.3680.python-list@python.org>
 <slrnmk5esv.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.155.1430437953.3680.python-list@python.org>
 <slrnmk6agn.apd.jon+usenet@frosty.unequivocal.co.uk>
 <554338dd$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <CALwzid=iNnSRspO9F1KAt=9=x3U=oOB0n=agQqs=yOVHA6wmmw@mail.gmail.com>
 <mailman.0.1430504017.20872.python-list@python.org>
Message-ID: <slrnmkag5c.apd.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-01, Peter Otten <__peter__ at web.de> wrote:
> A computer that cannot calculate a lookup table with all 3-digit cases 
> should be almost as hard to find as the one needed by Jon ;)

... or anyone else writing code to return the set of happy numbers.


From ian.g.kelly at gmail.com  Sat May  2 17:31:47 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 2 May 2015 15:31:47 -0600
Subject: l = range(int(1E9))
In-Reply-To: <CV91x.402510$zE7.111881@fx40.am4>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
Message-ID: <CALwzidnAGmimhH_Kztt6KA6x+WLnA6M-WjrZvEzGEj7Xxr0VmQ@mail.gmail.com>

On Sat, May 2, 2015 at 1:51 PM, BartC <bc at freeuk.com> wrote:
> On 02/05/2015 20:15, Mark Lawrence wrote:
>>
>> On 02/05/2015 19:34, BartC wrote:
>
>
>>> OK, so it's the programmer's fault if as fundamental a concept as a
>>> for-loop ranging over integers is implemented inefficiently. He has to
>>> transform it into high-level terms, or has to reconstruct it somehow
>>> using a while-loop and an incrementing loop index.
>
>
>> I give up.
>
>
> So do I, I think, if no-one is willing to admit that the original way of
> implementing range() was a glaring mistake.

range() was and is a *convenience* function. In the real world, the
vast majority of for loops are over arrays or other containers, not
integers, and those that aren't are usually very small. In non-toy
code, using a for loop to count to a billion is highly unusual.

So yeah, for a programmer porting code to Python who needed to loop
over an array, the correct approach would be to actually loop over the
*array* in place of the indices of the array. I don't know why you
make this out to be such a big deal; it's a simple conversion.

Would it have been better if range() had been implemented as xrange()
from the beginning? Sure, that would have been great. Except for one
small detail: the iterator protocol didn't exist back then. That
wasn't introduced until PEP 234 in Python 2.1, which means that the
xrange() function wasn't even *possible* before then.

I don't think anybody would claim that Python was perfect when it was
first introduced (nor is it perfect now). Like all other software, it
has improved over time as a result of iterative refinement.


From ian.g.kelly at gmail.com  Sat May  2 17:32:54 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 2 May 2015 15:32:54 -0600
Subject: l = range(int(1E9))
In-Reply-To: <rkb1x.386614$JH2.12955@fx11.am4>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <5543041d$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <rkb1x.386614$JH2.12955@fx11.am4>
Message-ID: <CALwzidkpvE_WkxD44TXgC=+c19xUkDuE4fXUShhhOrQdOSGx_A@mail.gmail.com>

On Sat, May 2, 2015 at 3:28 PM, Tony the Tiger <tony at tiger.invalid> wrote:
> On Fri, 01 May 2015 14:42:04 +1000, Steven D'Aprano wrote:
>
>> use "l" as a variable name, as it looks too much like 1
>
> If you use a better font, they are very different. Besides, a variable
> name cannot start with a digit (nor can it be a single digit), so it's a
> given that it's an 'l'.

Of course it can be a single digit. You're assuming that the person
reading the code already somehow knows that this is a variable name
and not an integer literal.


From jon+usenet at unequivocal.co.uk  Sat May  2 17:40:24 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sat, 2 May 2015 21:40:24 +0000 (UTC)
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
Message-ID: <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-02, BartC <bc at freeuk.com> wrote:
> So do I, I think, if no-one is willing to admit that the original way of 
> implementing range() was a glaring mistake.

I think the issue is that nobody else here thinks the "original way"
of iterating was to use range(), for anything other than extremely
small ranges.

For information, it looks like xrange() was added on 26 Oct 1993,
which pre-dates Python 1.0.


From lucas.bertolotti at yahoo.com  Sat May  2 18:12:46 2015
From: lucas.bertolotti at yahoo.com (lbertolotti)
Date: Sat, 2 May 2015 15:12:46 -0700 (PDT)
Subject: Python xlrd
Message-ID: <4c202871-d102-4bb0-85c4-3868dac02d03@googlegroups.com>

Ubuntu terminal gives me:

import xlrd

I get "ImportError: No module named xlrd"

Any ideas? I installed it by:

sudo apt-get install python-xlrd


From breamoreboy at yahoo.co.uk  Sat May  2 18:26:00 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 02 May 2015 23:26:00 +0100
Subject: l = range(int(1E9))
In-Reply-To: <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <mi3its$g42$1@ger.gmane.org>

On 02/05/2015 22:40, Jon Ribbens wrote:
> On 2015-05-02, BartC <bc at freeuk.com> wrote:
>> So do I, I think, if no-one is willing to admit that the original way of
>> implementing range() was a glaring mistake.
>
> I think the issue is that nobody else here thinks the "original way"
> of iterating was to use range(), for anything other than extremely
> small ranges.
>
> For information, it looks like xrange() was added on 26 Oct 1993,
> which pre-dates Python 1.0.
>

No chance, or so I thought, but to save others from looking it up 
https://hg.python.org/cpython/annotate/89e1e5d9ccbf/Python/bltinmodule.c

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From Cecil at decebal.nl  Sat May  2 18:29:17 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 03 May 2015 00:29:17 +0200
Subject: Is this a good way to implement testing
Message-ID: <878ud6mx4y.fsf@Equus.decebal.nl>

Still on my journey to learn Python.

At the moment I define the test functionality in the following way:
    if __name__ == '__main__':
        keywords        = [
            'all',
            'factorial',
            'fibonacci',
            'happy',
            'lucky',
        ]
        keywords_msg    = [
            '--all',
            '--factorial',
            '--fibonacci',
            '--happy',
            '--lucky',
        ]
        (options,
         extraParams)   = getopt.getopt(sys.argv[1:], '', keywords)
        progname        = split(sys.argv[0])[1]

        if len(options) > 1 or len(extraParams) != 0:
            error   = '{0}: Wrong parameters ({1})'. \
                      format(progname, ' '.join(sys.argv[1:]))
            usage   = '    {0} {1}'.format(progname, ' | '.join(keywords_msg))
            print(error, file = sys.stderr)
            print(usage, file = sys.stderr)
            sys.exit(1)

        do_all = do_factorial = do_fibonacci = do_happy = do_lucky = False
        if len(options) == 0:
            do_all = True
        else:
            action = options[0][0]
            if   action == '--all':
                do_all          = True
            elif action == '--factorial':
                do_factorial    = True
            elif action == '--fibonacci':
                do_fibonacci    = True
            elif action == '--happy':
                do_happy        = True
            elif action == '--lucky':
                do_lucky        = True
            else:
                print >> sys.stderr, progname + ': Unhandled parameter ' + action
                sys.exit(1)

        if do_all or do_factorial:
        .
        .
        .

Is this an acceptable way of working?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From bc at freeuk.com  Sat May  2 18:33:39 2015
From: bc at freeuk.com (BartC)
Date: Sat, 02 May 2015 23:33:39 +0100
Subject: l = range(int(1E9))
In-Reply-To: <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <dhc1x.319346$qW.51334@fx20.am4>

On 02/05/2015 22:40, Jon Ribbens wrote:
> On 2015-05-02, BartC <bc at freeuk.com> wrote:
>> So do I, I think, if no-one is willing to admit that the original way of
>> implementing range() was a glaring mistake.
>
> I think the issue is that nobody else here thinks the "original way"
> of iterating was to use range(), for anything other than extremely
> small ranges.

What /is/ the way to iterate then?

I don't have much Python code lying around, but the first couple of 
files I looked at (not mine), one had this:

   for i in range(7,-1,-1):
     for j in range(8):

another had:

     for l in range(1,17):
       for i in range(1, self.bits[l] + 1):

     for i in range(256):

and so on. Plenty of examples. This was probably converted to Python 
from elsewhere, but why shouldn't it be able to run code trivially 
converted from somewhere else, without being unnecessarily slow? (What 
would be the 'Pythonic' way of writing a Jpeg decoder, as this was, anyway?)

I don't think the small size of the range is a mitigating factor, other 
than it won't run out of memory. Even a small loop can be executed 
millions of times.

> For information, it looks like xrange() was added on 26 Oct 1993,
> which pre-dates Python 1.0.

OK, so it's just an irritation then, as a workaround has been available 
for a long time. (For example, if you use xrange, it won't work on 3.x. 
If you use range, then it might be inefficient on 2.x.)

-- 
Bartc


From tjreedy at udel.edu  Sat May  2 18:41:46 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 02 May 2015 18:41:46 -0400
Subject: l = range(int(1E9))
In-Reply-To: <W061x.692563$396.195269@fx12.am4>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
Message-ID: <mi3jrf$sos$1@ger.gmane.org>

On 5/2/2015 11:26 AM, BartC wrote:

> When I first looked at Python 20 or so years ago this seemed to be the
> standard way of writing a for-loop:
>
> for i in range(N):
>     ....

As Mark said, the actual syntax was originally

for item in sequence: ...

where sequence technically meant an object with a .__getitem__(x) method 
that either returned an item or raised IndexError.  In 2.2, 'sequence' 
was replaced by the more general concept 'iterable'.

In python, 'for' is and has always been an abbreviation of 'for each' or 
'foreach', as used in other languages. C for loops are not 'for each' 
loops but are initialized while loops, and are so explained in K&R's 
book, page 56.

'''
for (initializer; condition; update;) body

is equivalent to

initializer
while (condition) {
     body
     update;
}
...
Whether to use while or for is largely a matter of taste.
'''

(I substituted 'initializer', 'condition', 'update' for 'expr1', 
'expr2', and 'expr3' as more meaningful.)  The proper translation of a C 
for loop to Python is the translation of the equivalent C while loop. 
Translating to a Python 'for range' loop is a bit of a mis-translation. 
  One should not be surprised if non-equivalent code behaves differently.

> I remember being completely astonished at the time that 'range' actually
> created a list of values from 0 to N-1.

The purpose and *documented* behavior of range was to produce an 
arithmetic sequence.  Map, filter, dict.keys, dict.values, dict.items, 
many others functions also returned lists.  In Python3, the ones I named 
above and some others now return 'less spacious' iterables.

> Python was already known to be slow yet it deliberately crippled itself
> by using just about the slowest method imaginable of executing a simple
> iterative loop? By first creating a list of a million objects!

No, you were also free to do the sensible thing when you did not need a 
concrete arithmetic sequence (list).

n = 0
while n < 1000000:
     pass
     n = n+1

> And sometimes you weren't even interested in the values but just wanted
> to execute something N times so it was a wasted effort.

# even simpler
n = 1000000
while n:
     pass
     n -= 1

Using for ... range() is a convenient abbreviation of either pattern. 
It works for small values of n, it did not work for large values.  When 
Python 1.0 was released in Feb 1992, memory for many system was limited 
to a megabyte or less. At that time, range(1000000) was an impossible 
object, so no one aware of memory constraints would try to create it.

-- 
Terry Jan Reedy



From python at mrabarnett.plus.com  Sat May  2 19:07:03 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 03 May 2015 00:07:03 +0100
Subject: Inner workings of this Python feature: Can a Python data structure
 reference itself?
In-Reply-To: <87a8xmn0yp.fsf@Equus.decebal.nl>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <mailman.26.1430597813.12865.python-list@python.org>
 <87a8xmn0yp.fsf@Equus.decebal.nl>
Message-ID: <55455897.1010807@mrabarnett.plus.com>

On 2015-05-02 22:06, Cecil Westerhof wrote:
> Op Saturday 2 May 2015 22:17 CEST schreef Tim Chase:
>
>> [dangit, had Control down when I hit <enter> and it sent
>> prematurely]
>>
>> On 2015-05-02 13:02, vasudevram wrote:
>>> http://jugad2.blogspot.in/2015/05/can-python-data-structure-reference.html
>>>
>>> https://docs.python.org/2/reference/datamodel.html
>>>
>>> and saw this excerpt:
>>>
>>> [ CPython implementation detail: CPython currently uses a
>>> reference-counting scheme with (optional) delayed
>>> detection of cyclically linked garbage, which collects
>>> most objects as soon as they become unreachable, but is
>>> not guaranteed to collect garbage containing circular
>>> references. ]
>>>
>>> Not sure whether it is relevant to the topic at hand,
>>> since, on the one hand, it uses the words "cyclically
>>> linked", but on the other, it says "garbage collection".
>>
>> The gotcha happens in a case where you do something like this:
>>
>> lst = []
>> lst.append(lst)  # create a cycle
>> del lst
>>
>> This creates a cycle, then makes it unreachable, but the list is
>> still referenced by itself, so the reference count never drops to
>> zero (where it would get GC'd), and thus that item lingers around in
>> memory.
>
> Maybe look at Java? If my memory is correct, the JVM gc reclaims those
> kind of things also.
>
Python _does_ have a secondary mechanism for cleaning up such reference
cycles using mark-and-sweep:

https://docs.python.org/3/reference/datamodel.html



From tjreedy at udel.edu  Sat May  2 19:17:21 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 02 May 2015 19:17:21 -0400
Subject: Inner workings of this Python feature: Can a Python data structure
 reference itself?
In-Reply-To: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
Message-ID: <mi3lu6$pi0$1@ger.gmane.org>

On 5/2/2015 4:02 PM, vasudevram wrote:
> Hi group,
>
> Please refer to this blog post about code showing that a Python data
> structure can be self-referential:
>
> http://jugad2.blogspot.in/2015/05/can-python-data-structure-reference.html
>
>  Gotten a couple of comments on it already, but interested in hearing
> thoughts of Python core dev team members or others who can comment on
> the internals of how this feature operates, why, etc.

Please correct the following:
   "g (a list) contains itself as a list item (of g)."
g is a dict, as you yourself later said.

"Case 2) But if the evaluation works in a different order, i.e. the 
globals() function is first called (before the variable g is created), 
then at this point its return value (the dict) should not contain the 
item with key 'g' (and value g), and it is this dict that should get 
assigned to the variable g. Hence when we print g, we should not see g 
again within it."

This seems like you are presenting this as a statement of fact, but you 
then admit it is false.  The lead in sentence should more carefully 
state that what follows are possible hypotheses.  one is true and the 
other (mostly) not.

The key point is the meaning of "the globals() function returns a dict 
representing the current global symbol table,"  "Global symbol table" is 
an abstraction.  In CPython, the implementation is a dict and globals 
returns that dict, not a copy.  Python generally does not copy objects 
unless requested.

Similarly, locals() returns a dict representing the current local symbol 
table. In a CPython class statement, the local symbol table is 
implemented with a dict, and locals() is that dict.  In a CPython def 
statement, the local symbol table is implemented as a C array (of 
pointers to PyObjects). Locals() is a dict (created just once) updated 
from local names in the code object and the objects in the array *at the 
time of the call*

 >>> def f(a):
	g = locals()
	print(id(g), g)
	g = locals()
	print(id(g), g)
	
 >>> f(3)
56288136 {'a': 3}
56288136 {'a': 3, 'g': {...}}

'Case 2" applies for the first locals() call, but only for the first.

I believe that there was a time when printing a recursive structure hit 
the recursion limit like your flatten did. But I will not reload 1.5 to 
check.

-- 
Terry Jan Reedy



From breamoreboy at yahoo.co.uk  Sat May  2 19:17:56 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 03 May 2015 00:17:56 +0100
Subject: Is this a good way to implement testing
In-Reply-To: <878ud6mx4y.fsf@Equus.decebal.nl>
References: <878ud6mx4y.fsf@Equus.decebal.nl>
Message-ID: <mi3lv8$ppg$1@ger.gmane.org>

On 02/05/2015 23:29, Cecil Westerhof wrote:
> Still on my journey to learn Python.
>
> At the moment I define the test functionality in the following way:
>      if __name__ == '__main__':
>          keywords        = [
>              'all',
>              'factorial',
>              'fibonacci',
>              'happy',
>              'lucky',
>          ]
>          keywords_msg    = [
>              '--all',
>              '--factorial',
>              '--fibonacci',
>              '--happy',
>              '--lucky',
>          ]
>          (options,
>           extraParams)   = getopt.getopt(sys.argv[1:], '', keywords)
>          progname        = split(sys.argv[0])[1]
>
>          if len(options) > 1 or len(extraParams) != 0:
>              error   = '{0}: Wrong parameters ({1})'. \
>                        format(progname, ' '.join(sys.argv[1:]))
>              usage   = '    {0} {1}'.format(progname, ' | '.join(keywords_msg))
>              print(error, file = sys.stderr)
>              print(usage, file = sys.stderr)
>              sys.exit(1)
>
>          do_all = do_factorial = do_fibonacci = do_happy = do_lucky = False
>          if len(options) == 0:
>              do_all = True
>          else:
>              action = options[0][0]
>              if   action == '--all':
>                  do_all          = True
>              elif action == '--factorial':
>                  do_factorial    = True
>              elif action == '--fibonacci':
>                  do_fibonacci    = True
>              elif action == '--happy':
>                  do_happy        = True
>              elif action == '--lucky':
>                  do_lucky        = True
>              else:
>                  print >> sys.stderr, progname + ': Unhandled parameter ' + action
>                  sys.exit(1)
>
>          if do_all or do_factorial:
>          .
>          .
>          .
>
> Is this an acceptable way of working?
>

For code like the above I prefer the third party docopt module 
https://github.com/docopt/docopt although you could also try 
https://docs.python.org/3/library/argparse.html#module-argparse

The standard library unit testing framework is here 
https://docs.python.org/3/library/unittest.html#module-unittest but also 
see https://wiki.python.org/moin/PythonTestingToolsTaxonomy

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From tjreedy at udel.edu  Sat May  2 19:26:45 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 02 May 2015 19:26:45 -0400
Subject: l = range(int(1E9))
In-Reply-To: <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <mi3mfp$sti$1@ger.gmane.org>

On 5/2/2015 5:40 PM, Jon Ribbens wrote:

> For information, it looks like xrange() was added on 26 Oct 1993,
> which pre-dates Python 1.0.

1.0 was released Feb 1991.  3.2 exactly 20 years later.


-- 
Terry Jan Reedy



From tjreedy at udel.edu  Sat May  2 19:51:58 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 02 May 2015 19:51:58 -0400
Subject: l = range(int(1E9))
In-Reply-To: <CALwzidnAGmimhH_Kztt6KA6x+WLnA6M-WjrZvEzGEj7Xxr0VmQ@mail.gmail.com>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <CALwzidnAGmimhH_Kztt6KA6x+WLnA6M-WjrZvEzGEj7Xxr0VmQ@mail.gmail.com>
Message-ID: <mi3nv3$l35$1@ger.gmane.org>

On 5/2/2015 5:31 PM, Ian Kelly wrote:

> Would it have been better if range() had been implemented as xrange()
> from the beginning? Sure, that would have been great. Except for one
> small detail: the iterator protocol didn't exist back then.

For loops originally used the getitem iterator protocol. xrange objects 
have a __getitem__ method, but not __iter__ or __next__.  As Mark 
pointed out, they were introduced in 1993.

Getitem iterators still work.  If iter(ob) is passed object ob without 
.__iter__, iter looks for ob.__getitem__ and if present, returns 
iterator(ob), where iterator is an internal protocol adapter class.  Its 
__next__ method calls ob.__getitem__ and converts IndexError to 
StopIteration.

In 3.4.3:

class xrange:
     def __init__(self, start, stop, step):
         if step < 1:
             raise ValueError('incomplete xrange require positive step')
         self.val = start
         self.stop = stop
         self.step = step
     def __getitem__(self, dummy):
         val = self.val
         self.val = val + self.step
         if val < self.stop:
             return val
         else:
             raise IndexError('')

it = iter(xrange(1, 10, 3))
print(type(it))
for i in it:
     print(i)
#
<class 'iterator'>
1
4
7

-- 
Terry Jan Reedy



From jon+usenet at unequivocal.co.uk  Sat May  2 20:01:32 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sun, 3 May 2015 00:01:32 +0000 (UTC)
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.36.1430609216.12865.python-list@python.org>
Message-ID: <slrnmkapcr.apd.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-02, Terry Reedy <tjreedy at udel.edu> wrote:
> On 5/2/2015 5:40 PM, Jon Ribbens wrote:
>> For information, it looks like xrange() was added on 26 Oct 1993,
>> which pre-dates Python 1.0.
>
> 1.0 was released Feb 1991.  3.2 exactly 20 years later.

No, you are thinking of 0.9, which was indeed February 1991.


From tjreedy at udel.edu  Sat May  2 20:06:20 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 02 May 2015 20:06:20 -0400
Subject: Is this a good way to implement testing
In-Reply-To: <878ud6mx4y.fsf@Equus.decebal.nl>
References: <878ud6mx4y.fsf@Equus.decebal.nl>
Message-ID: <mi3oq1$gg$1@ger.gmane.org>

On 5/2/2015 6:29 PM, Cecil Westerhof wrote:

> At the moment I define the test functionality in the following way:

Any automated testing is better than none.  For idlelib, I use unittest. 
  For an individual project with specialized needs, I use a custom test 
framework tuned to those needs.

>          else:
>              action = options[0][0]
>              if   action == '--all':
>                  do_all          = True
>              elif action == '--factorial':
>                  do_factorial    = True
>              elif action == '--fibonacci':
>                  do_fibonacci    = True
>              elif action == '--happy':
>                  do_happy        = True
>              elif action == '--lucky':
>                  do_lucky        = True
>              else:
>                  print >> sys.stderr, progname + ': Unhandled parameter ' + action
>                  sys.exit(1)

There is usually a way to factor out the duplication of repetitive code 
like this.  Often, a dict is somehow involved. I believe the following 
is equivalent to the above.

else:
   action = options[0][0]
   try:
     globals()['do_'+action[2:]] = True
   except KeyError:
     print >> sys.stderr, progname + ': Unhandled parameter ' + action
     sys.exit(1)

-- 
Terry Jan Reedy



From jon+usenet at unequivocal.co.uk  Sat May  2 20:13:02 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sun, 3 May 2015 00:13:02 +0000 (UTC)
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
 <dhc1x.319346$qW.51334@fx20.am4>
Message-ID: <slrnmkaq2d.apd.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-02, BartC <bc at freeuk.com> wrote:
> On 02/05/2015 22:40, Jon Ribbens wrote:
>> I think the issue is that nobody else here thinks the "original way"
>> of iterating was to use range(), for anything other than extremely
>> small ranges.
>
> What /is/ the way to iterate then?

Other people have already explained that to you several times.
If you want a literal 'for (i = 0; i < x; i++)' loop then it's
'for i in xrange(x)' (from Python 1.0 to Python 2.7), but most
of the time you simply wouldn't be writing code that works that
way.

> I don't have much Python code lying around, but the first couple of 
> files I looked at (not mine), one had this:
>
>    for i in range(7,-1,-1):
>      for j in range(8):
>
> another had:
>
>      for l in range(1,17):
>        for i in range(1, self.bits[l] + 1):
>
>      for i in range(256):
>
> and so on. Plenty of examples.

... of extremely small ranges, just as I said.

> I don't think the small size of the range is a mitigating factor, other 
> than it won't run out of memory. Even a small loop can be executed 
> millions of times.

So what? The overhead of creating a list of 17 integers is trivial
compared to executing the rest of the code. As the man said:
"premature optimization is the root of all evil".


From python.list at tim.thechases.com  Sat May  2 21:57:40 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Sat, 2 May 2015 20:57:40 -0500
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <87a8xmn0yp.fsf@Equus.decebal.nl>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <mailman.26.1430597813.12865.python-list@python.org>
 <87a8xmn0yp.fsf@Equus.decebal.nl>
Message-ID: <20150502205740.53527192@bigbox.christie.dr>

On 2015-05-02 23:06, Cecil Westerhof wrote:
> Op Saturday 2 May 2015 22:17 CEST schreef Tim Chase:
>> This creates a cycle, then makes it unreachable, but the list is
>> still referenced by itself, so the reference count never drops to
>> zero (where it would get GC'd), and thus that item lingers around
>> in memory.
> 
> Maybe look at Java? If my memory is correct, the JVM gc reclaims
> those kind of things also.

The two most common ways of doing garbage collection are
reference-counting and mark-and-sweep. They're not mutually
exclusive, and can be combined, but each has advantages to counter
disadvantages of the other.  In reference-counting, you have the
overhead of incrementing/decrementing a counter and the
above-mentioned unreachability issue.  With mark-and-sweep, you can
end up with pauses as the VM rejiggers memory.

It's also my understanding that Jython uses the JVM and thus gets the
JVM's generational mark-and-sweep memory management instead of the
reference-counting that the CPython implementation does.  The GC
technique isn't part of the Python spec (AFAIK), so it's up to the
individual implementation which one to use.  MRAB's post-with-link
regarding CPython's implementation suggests that CPython does
indeed provide optional mark-and-sweep style collection:

    """
    CPython implementation detail: CPython currently uses a
    reference-counting scheme with (optional) delayed detection
    of cyclically linked garbage, which collects most objects as
    soon as they become unreachable, but is not guaranteed to
    collect garbage containing circular references. See the
    documentation of the gc module for information on controlling
    the collection of cyclic garbage. Other implementations act
    differently and CPython may change. Do not depend on
    immediate finalization of objects when they become
    unreachable (so you should always close files explicitly).
    """

So it sounds like you have to request such a mark-and-sweep from
the gc module.

-tkc





From torriem at gmail.com  Sat May  2 22:07:31 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sat, 02 May 2015 20:07:31 -0600
Subject: l = range(int(1E9))
In-Reply-To: <dhc1x.319346$qW.51334@fx20.am4>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
 <dhc1x.319346$qW.51334@fx20.am4>
Message-ID: <554582E3.4030700@gmail.com>

On 05/02/2015 04:33 PM, BartC wrote:
> OK, so it's just an irritation then, as a workaround has been available 
> for a long time. (For example, if you use xrange, it won't work on 3.x. 
> If you use range, then it might be inefficient on 2.x.)

In both Python 2.7 and 3.3+, you can use the 3rd-party six module to
help with forward compatibility:

from six.moves import xrange

In Python 2.7, it's just the normal xrange (importing is a no op
basically), but in Python 3 it points to range.

Kind of wish parts of six were in the Python standard library.





From breamoreboy at yahoo.co.uk  Sat May  2 22:16:04 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 03 May 2015 03:16:04 +0100
Subject: l = range(int(1E9))
In-Reply-To: <554582E3.4030700@gmail.com>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
 <dhc1x.319346$qW.51334@fx20.am4> <554582E3.4030700@gmail.com>
Message-ID: <mi40d8$7ku$1@ger.gmane.org>

On 03/05/2015 03:07, Michael Torrie wrote:
> On 05/02/2015 04:33 PM, BartC wrote:
>> OK, so it's just an irritation then, as a workaround has been available
>> for a long time. (For example, if you use xrange, it won't work on 3.x.
>> If you use range, then it might be inefficient on 2.x.)
>
> In both Python 2.7 and 3.3+, you can use the 3rd-party six module to
> help with forward compatibility:
>
> from six.moves import xrange
>
> In Python 2.7, it's just the normal xrange (importing is a no op
> basically), but in Python 3 it points to range.
>
> Kind of wish parts of six were in the Python standard library.
>

Links to other useful third party modules are given here 
https://docs.python.org/3/howto/pyporting.html

I doubt that six will ever make the standard library as 2.7 only has 
another five years in official support.  By that time I suppose we'll to 
going through the porting pain all over again with the transition from 
Python 3 to Python 4.  Alright, alright, only joking :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From tjreedy at udel.edu  Sat May  2 23:18:37 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 02 May 2015 23:18:37 -0400
Subject: l = range(int(1E9))
In-Reply-To: <slrnmkapcr.apd.jon+usenet@frosty.unequivocal.co.uk>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.36.1430609216.12865.python-list@python.org>
 <slrnmkapcr.apd.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <mi442h$l1j$1@ger.gmane.org>

On 5/2/2015 8:01 PM, Jon Ribbens wrote:
> On 2015-05-02, Terry Reedy <tjreedy at udel.edu> wrote:
>> On 5/2/2015 5:40 PM, Jon Ribbens wrote:
>>> For information, it looks like xrange() was added on 26 Oct 1993,
>>> which pre-dates Python 1.0.
>>
>> 1.0 was released Feb 1991.  3.2 exactly 20 years later.
>
> No, you are thinking of 0.9, which was indeed February 1991.

'scuse me, I meant 1992


-- 
Terry Jan Reedy



From jon+usenet at unequivocal.co.uk  Sat May  2 23:40:07 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sun, 3 May 2015 03:40:07 +0000 (UTC)
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.36.1430609216.12865.python-list@python.org>
 <slrnmkapcr.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.42.1430623130.12865.python-list@python.org>
Message-ID: <slrnmkb66n.apd.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-03, Terry Reedy <tjreedy at udel.edu> wrote:
> On 5/2/2015 8:01 PM, Jon Ribbens wrote:
>> On 2015-05-02, Terry Reedy <tjreedy at udel.edu> wrote:
>>> On 5/2/2015 5:40 PM, Jon Ribbens wrote:
>>>> For information, it looks like xrange() was added on 26 Oct 1993,
>>>> which pre-dates Python 1.0.
>>>
>>> 1.0 was released Feb 1991.  3.2 exactly 20 years later.
>>
>> No, you are thinking of 0.9, which was indeed February 1991.
>
> 'scuse me, I meant 1992

No, it was January 1994.

http://python-history.blogspot.co.uk/2009/01/brief-timeline-of-python.html


From no.email at nospam.invalid  Sat May  2 23:58:59 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Sat, 02 May 2015 20:58:59 -0700
Subject: Is this a good way to implement testing
References: <878ud6mx4y.fsf@Equus.decebal.nl>
Message-ID: <87k2wq2tx8.fsf@jester.gateway.sonic.net>

Cecil Westerhof <Cecil at decebal.nl> writes:

> Still on my journey to learn Python.
>
> At the moment I define the test functionality in the following way:
>             action = options[0][0]
>             if   action == '--all': ...

Yecch, use an option parsing library for that, whichever one is
currently fashionable.  I think optparse is deprecated now but I still
use it because I'm used to it.

> Is this an acceptable way of working?

You should also use the currently fashionable unit testing library.  I
use unittest because yada yada but I think it's now considered old
school.


From ian.g.kelly at gmail.com  Sun May  3 00:29:40 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 2 May 2015 22:29:40 -0600
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <20150502205740.53527192@bigbox.christie.dr>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <mailman.26.1430597813.12865.python-list@python.org>
 <87a8xmn0yp.fsf@Equus.decebal.nl>
 <20150502205740.53527192@bigbox.christie.dr>
Message-ID: <CALwzid=2SxFbtvTxKeqgkECzvvKHMv4VydigfsmkCcNepDtn2Q@mail.gmail.com>

On Sat, May 2, 2015 at 7:57 PM, Tim Chase <python.list at tim.thechases.com> wrote:
> So it sounds like you have to request such a mark-and-sweep from
> the gc module.

You *can* request it. But as long as it hasn't been explicitly
disabled (by calling gc.disable()), the mark-and-sweep garbage
collection will also run automatically -- just not necessarily
immediately.


From ian.g.kelly at gmail.com  Sun May  3 00:43:01 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 2 May 2015 22:43:01 -0600
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <20150502151723.3a36e33b@bigbox.christie.dr>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <20150502151723.3a36e33b@bigbox.christie.dr>
Message-ID: <CALwzidnnxU7N0Kgx-eMM5Lq4th6xoRggXLQrsS7fnj9wJnEHOQ@mail.gmail.com>

On Sat, May 2, 2015 at 2:17 PM, Tim Chase <python.list at tim.thechases.com> wrote:
> If you know that you're creating such cyclical structures, it's best
> to manually unlink them before freeing them:
>
>   lst = []
>   lst.append(lst) # create the cycle
>   lst[:] = []   # break the cycle
>   # or lst.remove(lst) # though this takes more care
>   del lst

In general, this shouldn't be necessary. I believe that reference
cycles are guaranteed to be cleaned up in all major implementations of
Python, except that in CPython prior to version 3.4 reference cycles
containing objects with finalizers would not be collected. So the
better advice would be "don't use finalizers in reference cycles if
you need compatibility with Python 3.3 or earlier."


From ben+python at benfinney.id.au  Sun May  3 00:49:30 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 03 May 2015 14:49:30 +1000
Subject: Is this a good way to implement testing
References: <878ud6mx4y.fsf@Equus.decebal.nl>
 <87k2wq2tx8.fsf@jester.gateway.sonic.net>
Message-ID: <85y4l6e04l.fsf@benfinney.id.au>

Paul Rubin <no.email at nospam.invalid> writes:

> Cecil Westerhof <Cecil at decebal.nl> writes:
>
> >             action = options[0][0]
> >             if   action == '--all': ...
>
> Yecch, use an option parsing library for that, whichever one is
> currently fashionable. I think optparse is deprecated now but I still
> use it because I'm used to it.

Yes, ?optparse? has been superseded by the more capable ?argparse?
library <URL:https://docs.python.org/3/library/argparse.html>.

Migrating from ?optparse? to ?argparse? is quite straightforward, since
the latter was designed to be very similar.

> > Is this an acceptable way of working?
>
> You should also use the currently fashionable unit testing library. I
> use unittest because yada yada but I think it's now considered old
> school.

I disagree, using ?unittest? is still quite normal. It's also in the
standard library, unlike proposed replacements.

-- 
 \       ?Instead of having ?answers? on a math test, they should just |
  `\               call them ?impressions?, and if you got a different |
_o__)   ?impression?, so what, can't we all be brothers?? ?Jack Handey |
Ben Finney



From ian.g.kelly at gmail.com  Sun May  3 00:51:22 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 2 May 2015 22:51:22 -0600
Subject: l = range(int(1E9))
In-Reply-To: <mi3nv3$l35$1@ger.gmane.org>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <CALwzidnAGmimhH_Kztt6KA6x+WLnA6M-WjrZvEzGEj7Xxr0VmQ@mail.gmail.com>
 <mi3nv3$l35$1@ger.gmane.org>
Message-ID: <CALwzidm+BGM_A=fBDFbXRjMypOKe+-fPAki_aRJZ0TTgyOYFbA@mail.gmail.com>

On Sat, May 2, 2015 at 5:51 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 5/2/2015 5:31 PM, Ian Kelly wrote:
>
>> Would it have been better if range() had been implemented as xrange()
>> from the beginning? Sure, that would have been great. Except for one
>> small detail: the iterator protocol didn't exist back then.
>
>
> For loops originally used the getitem iterator protocol. xrange objects have
> a __getitem__ method, but not __iter__ or __next__.  As Mark pointed out,
> they were introduced in 1993.

I'm aware of getitem iterators; just didn't realize that xrange used
it or was that old.


From rosuav at gmail.com  Sun May  3 01:46:52 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 3 May 2015 15:46:52 +1000
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <CALwzidnnxU7N0Kgx-eMM5Lq4th6xoRggXLQrsS7fnj9wJnEHOQ@mail.gmail.com>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <20150502151723.3a36e33b@bigbox.christie.dr>
 <CALwzidnnxU7N0Kgx-eMM5Lq4th6xoRggXLQrsS7fnj9wJnEHOQ@mail.gmail.com>
Message-ID: <CAPTjJmqkHg_FGvnucFvoswd64tEdG8fnfKthXdgPK3=H1krYfA@mail.gmail.com>

On Sun, May 3, 2015 at 2:43 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Sat, May 2, 2015 at 2:17 PM, Tim Chase <python.list at tim.thechases.com> wrote:
>> If you know that you're creating such cyclical structures, it's best
>> to manually unlink them before freeing them:
>>
>>   lst = []
>>   lst.append(lst) # create the cycle
>>   lst[:] = []   # break the cycle
>>   # or lst.remove(lst) # though this takes more care
>>   del lst
>
> In general, this shouldn't be necessary. I believe that reference
> cycles are guaranteed to be cleaned up in all major implementations of
> Python, except that in CPython prior to version 3.4 reference cycles
> containing objects with finalizers would not be collected. So the
> better advice would be "don't use finalizers in reference cycles if
> you need compatibility with Python 3.3 or earlier."

It's not strictly necessary, but you can help things along by breaking
the cycle - if you're doing this kind of thing in a loop, breaking the
cycle will most likely give more consistent memory usage, whereas the
GC will potentially pick up a bunch of them all at once.

In simple applications, it's not a big deal either way. Just write
your code, and let Python worry about memory. But if it doesn't cost
you much to break the loop, you may as well do it  Sometimes it even
improves code clarity - showing that you're definitely done with this
thing now, even though in theory you might still be iterating over it.
Just don't warp your code around memory usage, because that's not what
Python's meant for. :)

ChrisA


From airween at gmail.com  Sun May  3 02:54:43 2015
From: airween at gmail.com (Ervin =?utf-8?Q?Heged=C3=BCs?=)
Date: Sun, 3 May 2015 08:54:43 +0200
Subject: Python xlrd
In-Reply-To: <4c202871-d102-4bb0-85c4-3868dac02d03@googlegroups.com>
References: <4c202871-d102-4bb0-85c4-3868dac02d03@googlegroups.com>
Message-ID: <20150503065443.GA4075@arxnet.hu>

hi,

On Sat, May 02, 2015 at 03:12:46PM -0700, lbertolotti via Python-list wrote:
> Ubuntu terminal gives me:
> 
> import xlrd
> 
> I get "ImportError: No module named xlrd"
> 
> Any ideas? I installed it by:
> 
> sudo apt-get install python-xlrd

what's the answers of your system for these questions?

dpkg -C		// check the broken packages
dpkg -l python-xlrd	// check the status of xlrd package



a.


-- 
I ? UTF-8


From Cecil at decebal.nl  Sun May  3 03:36:40 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 03 May 2015 09:36:40 +0200
Subject: Is this a good way to implement testing
References: <878ud6mx4y.fsf@Equus.decebal.nl>
Message-ID: <87383em7sn.fsf@Equus.decebal.nl>

Op Sunday 3 May 2015 00:29 CEST schreef Cecil Westerhof:

> Still on my journey to learn Python.
>
> At the moment I define the test functionality in the following way:
> if __name__ == '__main__':
> keywords        = [
> 'all',
> 'factorial',
> 'fibonacci',
> 'happy',
> 'lucky',
> ]
> keywords_msg    = [
> '--all',
> '--factorial',
> '--fibonacci',
> '--happy',
> '--lucky',
> ]
> (options,
> extraParams)   = getopt.getopt(sys.argv[1:], '', keywords)
> progname        = split(sys.argv[0])[1]
>
> if len(options) > 1 or len(extraParams) != 0:
> error   = '{0}: Wrong parameters ({1})'. \
> format(progname, ' '.join(sys.argv[1:]))
> usage   = '    {0} {1}'.format(progname, ' | '.join(keywords_msg))
> print(error, file = sys.stderr)
> print(usage, file = sys.stderr)
> sys.exit(1)
>
> do_all = do_factorial = do_fibonacci = do_happy = do_lucky = False
> if len(options) == 0:
> do_all = True
> else:
> action = options[0][0]
> if   action == '--all':
> do_all          = True
> elif action == '--factorial':
> do_factorial    = True
> elif action == '--fibonacci':
> do_fibonacci    = True
> elif action == '--happy':
> do_happy        = True
> elif action == '--lucky':
> do_lucky        = True
> else:
> print >> sys.stderr, progname + ': Unhandled parameter ' + action
> sys.exit(1)
>
> if do_all or do_factorial:
> .
> .
> .
>
> Is this an acceptable way of working?

Thanks for the tips. For most I have to read a ?little? first, so I
will not implement them immediately.
Another question. Is it acceptable to have it in the module itself, or
should I put it in something like test_<module>.py? The code for
testing is bigger as the code for the implementation, so I am leaning
to putting it in a separate file.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun May  3 04:11:44 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 03 May 2015 10:11:44 +0200
Subject: Converting 5.223701009526849e-05 to 5e-05
Message-ID: <87vbgakrlr.fsf@Equus.decebal.nl>

When I have a value like 5.223701009526849e-05 in most cases I am not
interested in all the digest after the dot. Is there a simple way to
convert it to a string like '5e-05'?
I could do something like:
    def format_small_number(n):
        abs_n = abs(n)
        assert (abs_n < 1) and (abs_n > 0)
        length = len(str(int(1 / abs_n)))
        if length <= 4:
            return '{0}'.format(int(n * 10 ** length) / 10.0 **  length)
        else:
            return '{0}E-{1}'.format(int(n * 10 ** length), length)

I use '5E-5' because I prefer that above '5e-5'.
I should take care of rounding: now 0.9999 becomes 0.9.

But I was wondering if there is a better way.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun May  3 04:22:50 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 03 May 2015 10:22:50 +0200
Subject: Messages with a time stamp
Message-ID: <87r3qykr39.fsf@Equus.decebal.nl>

For testing I want my messages time stamped like:
    02:06:32: Check that the non recursive variants give the same value from 1000 upto 100000 step 1000
    02:06:32: Currently at    1000
    02:06:33: Currently at   11000
    02:06:35: Currently at   21000
    02:06:42: Currently at   31000
    02:06:56: Currently at   41000
    02:07:18: Currently at   51000
    02:07:51: Currently at   61000
    02:08:43: Currently at   71000
    02:09:49: Currently at   81000
    02:11:13: Currently at   91000
    02:13:01: Calculating values OK

    02:13:01: Start with the time needed to calculate 100000 times
    02:13:01: Timing factorial_iterative         (985): 31
    02:13:32: Timing factorial_recursive         (985): 55
    02:14:28: Timing factorial_recursive_old     (985): 56
    02:15:24: Timing factorial_tail_recursion    (985): 35
    02:16:00: Timing factorial_tail_recursion_old(985): 40

    02:16:40: Start with the time needed to calculate 1 times
              No recursive, because without tail recursion you would run out of stack space
    02:16:40: Timing factorial_iterative         (100000): 3.7705
    02:16:44: Timing factorial_tail_recursion    (100000): 3.7692
    02:16:48: Timing factorial_tail_recursion_old(100000): 4.1537

And sometimes I do not want the time shown, to signify that the
message belongs to the previous message. And sometimes I want no
newline, because I want to print something behind it. For example the
time needed to calculate something:
    02:13:01: Timing factorial_iterative         (985): 31

For this I wrote:
    ### Have the possibility to give the stream instead of using stdout
    class TimedMessage:
        """
        For printing messages with time prepended before it
        Has the possibilty to keep time print blank for when several messages
        are send shortly after eachother.
        Also the possibilty to stay on the same line when things need to be appended
        """

        def give_msg(self, message, show_time = True, use_newline = True):
            """
            Prints the message to stdout
            Use show_time = False when you do not want time
            Use use_newline = False if you do not want a newline
            """

            if show_time:
                time = strftime(self._format)
            else:
                time = self._blank_time
            formatted_message = time + message
            if use_newline:
                print(formatted_message)
            else:
                sys.stdout.write(formatted_message)
                sys.stdout.flush()

        def __init__(self, format = '%H:%M:%S: '):
            self._format        = format
            self._blank_time    = ' ' * len(strftime(self._format))

Can I improve on this?

It is shared at:
    https://github.com/CecilWesterhof/PythonLibrary/blob/master/utilDecebal.py

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From breamoreboy at yahoo.co.uk  Sun May  3 04:38:22 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 03 May 2015 09:38:22 +0100
Subject: Is this a good way to implement testing
In-Reply-To: <87383em7sn.fsf@Equus.decebal.nl>
References: <878ud6mx4y.fsf@Equus.decebal.nl> <87383em7sn.fsf@Equus.decebal.nl>
Message-ID: <mi4mq3$dir$1@ger.gmane.org>

On 03/05/2015 08:36, Cecil Westerhof wrote:
>
> Thanks for the tips. For most I have to read a ?little? first, so I
> will not implement them immediately.
> Another question. Is it acceptable to have it in the module itself, or
> should I put it in something like test_<module>.py? The code for
> testing is bigger as the code for the implementation, so I am leaning
> to putting it in a separate file.
>

I'd go for the former if your implementation code base is measured in 
hundreds of lines of code, anything larger and I'd probably split the 
test code out.  That's just my own rule of thumb, I'm sure others will 
quote different figures, but what it ultimately gets down to is what are 
you comfortable with?  I ask because you should keep in mind that Python 
modules containing several classes and running into thousands of lines 
of code are fairly common.  Python is not Java, and Java isn't Python 
either :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From ben+python at benfinney.id.au  Sun May  3 04:40:38 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 03 May 2015 18:40:38 +1000
Subject: Converting 5.223701009526849e-05 to 5e-05
References: <87vbgakrlr.fsf@Equus.decebal.nl>
Message-ID: <85mw1mdpfd.fsf@benfinney.id.au>

Cecil Westerhof <Cecil at decebal.nl> writes:

> When I have a value like 5.223701009526849e-05 in most cases I am not
> interested in all the digest after the dot.

What type of value is it?

A ?float? value has many different textual representations, most of them
inaccurate. So talking about the digits of a ?float? value is only
partly meaningful; digits are a property of some chosen representation,
not intrinsic to the number.

A ?str? value can be converted in various ways, but is useless as a
number until you create a new number from the result.

Choosing a solution will rely on understanding that the textual
representation of a number is not itself a number; and vice versa, a
number value does not have a canonical text representation.

> Is there a simple way to convert it to a string like '5e-05'?

Assuming we're talking about a ?float? value::

    >>> foo = 5.223701009526849e-05
    >>> "{foo:5.1}".format(foo=foo)
    '5e-05'

See the ?str.format? documentation, especially the detailed
documentation for the ?format specification mini-language?
<URL:https://docs.python.org/3/library/string.html#format-specification-mini-language>
for how to specify exactly how you want values to be formatted as
text.

-- 
 \         ?The double standard that exempts religious activities from |
  `\       almost all standards of accountability should be dismantled |
_o__)                   once and for all.? ?Daniel Dennett, 2010-01-12 |
Ben Finney



From __peter__ at web.de  Sun May  3 04:45:29 2015
From: __peter__ at web.de (Peter Otten)
Date: Sun, 03 May 2015 10:45:29 +0200
Subject: Is this a good way to implement testing
References: <878ud6mx4y.fsf@Equus.decebal.nl> <87383em7sn.fsf@Equus.decebal.nl>
Message-ID: <mi4n7a$jb3$1@ger.gmane.org>

Cecil Westerhof wrote:

> Another question. Is it acceptable to have it in the module itself, or
> should I put it in something like test_<module>.py? The code for
> testing is bigger as the code for the implementation, so I am leaning
> to putting it in a separate file.

Definitely use an established testing framework instead of rolling your own, 
and definitely put it into a separate file -- by the time there is good 
coverage the test code is usually much bigger than the tested code.

Be aware that there is also doctest which scans docstrings for text 
resembling interactive Python sessions. Doctests are both tests and usage 
examples, so I think it's good to put a few of these into the module. Here's 
how it works:

$ cat factorial.py
def factorial(n):
    """Calculate the factorial 1 * 2 * ... * n.

    >>> factorial(0)
    1
    >>> factorial(1)
    1
    >>> factorial(10)
    3628800
    """
    return 1
$ python3 -m doctest factorial.py
**********************************************************************
File "/home/peter/clpy/factorial.py", line 8, in factorial.factorial
Failed example:
    factorial(10)
Expected:
    3628800
Got:
    1
**********************************************************************
1 items had failures:
   1 of   3 in factorial.factorial
***Test Failed*** 1 failures.
$ 




From ben+python at benfinney.id.au  Sun May  3 04:48:36 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 03 May 2015 18:48:36 +1000
Subject: Converting 5.223701009526849e-05 to 5e-05
References: <87vbgakrlr.fsf@Equus.decebal.nl> <85mw1mdpfd.fsf@benfinney.id.au>
Message-ID: <85iocadp23.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> Assuming we're talking about a ?float? value::
>
>     >>> foo = 5.223701009526849e-05
>     >>> "{foo:5.1}".format(foo=foo)
>     '5e-05'

That's not as clear as it could be. Better is to be explicit about
choosing ?exponential? format::

    >>> foo = 5.223701009526849e-05
    >>> "{foo:5.0e}".format(foo=foo)
    '5e-05'

-- 
 \      ?Science embraces facts and debates opinion; religion embraces |
  `\    opinion and debates the facts.? ?Tom Heehler, _The Well-Spoken |
_o__)                                                       Thesaurus_ |
Ben Finney



From ben+python at benfinney.id.au  Sun May  3 04:52:21 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sun, 03 May 2015 18:52:21 +1000
Subject: Is this a good way to implement testing
References: <878ud6mx4y.fsf@Equus.decebal.nl>
 <87383em7sn.fsf@Equus.decebal.nl> <mi4n7a$jb3$1@ger.gmane.org>
Message-ID: <85egmydovu.fsf@benfinney.id.au>

Peter Otten <__peter__ at web.de> writes:

> Be aware that there is also doctest which scans docstrings for text
> resembling interactive Python sessions. Doctests are both tests and
> usage examples, so I think it's good to put a few of these into the
> module.

Yes, it's definitely a good idea to put some examples into the
docstrings as doctests.

Be aware, though, that this is *not* a substitute for unit tests;
doctest is for testing your documentation, not testing your code.

You should not aim for extensive coverage in doctests. Heavy use of
doctests makes for bad tests *and* bad documentation.

Instead, write only those examples that the reader will find helpful to
understand normal usage; and use the ?doctest? module to test that your
documentation is still accurate.

Put all your other broad-coverage tests elsewhere (unit tests, behaviour
tests, etc.) and leave the docstrings readable.

-- 
 \          ?Generally speaking, the errors in religion are dangerous; |
  `\    those in philosophy only ridiculous.? ?David Hume, _A Treatise |
_o__)                                           of Human Nature_, 1739 |
Ben Finney



From motoom at xs4all.nl  Sun May  3 05:10:40 2015
From: motoom at xs4all.nl (Michiel Overtoom)
Date: Sun, 3 May 2015 11:10:40 +0200
Subject: Messages with a time stamp
In-Reply-To: <87r3qykr39.fsf@Equus.decebal.nl>
References: <87r3qykr39.fsf@Equus.decebal.nl>
Message-ID: <BF737230-5BA4-43B5-837B-8B885A04B429@xs4all.nl>


On May 3, 2015, at 10:22, Cecil Westerhof wrote:

> For testing I want my messages time stamped like:

For progress reporting, I often use the module below (eta.py), which also gives a projected time of completion:

import datetime, time, sys
etastart = 0

def eta(done, total, s, reportinterval=100, datetimeformat="%d %b %H:%M:%S"):
    global etastart
    if done == 0:
        etastart = datetime.datetime.now()
    if not done % reportinterval:
        noww = datetime.datetime.now()
        prtnow = noww.strftime(datetimeformat)
        if done:
            if not isinstance(etastart, datetime.datetime): raise RuntimeError("You should call eta() at least once with done=0")
            elapsed = noww - etastart
            secsperitem = float(elapsed.seconds) / done    
            totalsecs = secsperitem * total
            eta = etastart + datetime.timedelta(0, totalsecs)
            prteta = eta.strftime(datetimeformat)
            msg = "now %s, eta %s (%d/%d) %s" % (prtnow, prteta, done, total, s)
        else:
            msg = "now %s (%d/%d) %s" % (prtnow, done, total, s)
        sys.stderr.write(msg + "\n")
        
if __name__ == "__main__":
    for i in range(10):
        eta(i, 10, "idling for ten seconds", 1, "%H:%M:%S")
        time.sleep(1)



-- 
"You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes



From Cecil at decebal.nl  Sun May  3 05:22:18 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 03 May 2015 11:22:18 +0200
Subject: Converting 5.223701009526849e-05 to 5e-05
References: <87vbgakrlr.fsf@Equus.decebal.nl>
 <mailman.52.1430642455.12865.python-list@python.org>
Message-ID: <87mw1mkoc5.fsf@Equus.decebal.nl>

Op Sunday 3 May 2015 10:40 CEST schreef Ben Finney:

> Cecil Westerhof <Cecil at decebal.nl> writes:
>
>> When I have a value like 5.223701009526849e-05 in most cases I am
>> not interested in all the digest after the dot.
>
> What type of value is it?

If the absolute value is bigger as 0 and smaller as 1, it should be a
float. ;-)


> A ?float? value has many different textual representations, most of
> them inaccurate. So talking about the digits of a ?float? value is
> only partly meaningful; digits are a property of some chosen
> representation, not intrinsic to the number.
>
> A ?str? value can be converted in various ways, but is useless as a
> number until you create a new number from the result.
>
> Choosing a solution will rely on understanding that the textual
> representation of a number is not itself a number; and vice versa, a
> number value does not have a canonical text representation.

It is because I display things like:
    02:47:18: Increase memoize -> iterative 19
              (0.0004942629893776029 / 2.475001383572817e-05)

And that is way to specific.


>> Is there a simple way to convert it to a string like '5e-05'?
>
> Assuming we're talking about a ?float? value::
>
>>>> foo = 5.223701009526849e-05
>>>> "{foo:5.1}".format(foo=foo)
> '5e-05'
>
> See the ?str.format? documentation, especially the detailed
> documentation for the ?format specification mini-language?
> <URL:https://docs.python.org/3/library/string.html#format-specification-mini-language>
> for how to specify exactly how you want values to be formatted as
> text.

Very interesting documentation. I go for:
    '{foo:.3E}'.format(foo=foo)

Then it simplifies also big numbers and it works for int's also. (Not
needed now, but it never hurts to be prepared for the future.)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From __peter__ at web.de  Sun May  3 05:22:19 2015
From: __peter__ at web.de (Peter Otten)
Date: Sun, 03 May 2015 11:22:19 +0200
Subject: Is this a good way to implement testing
References: <878ud6mx4y.fsf@Equus.decebal.nl> <mi3oq1$gg$1@ger.gmane.org>
Message-ID: <mi4pcd$i04$1@ger.gmane.org>

Terry Reedy wrote:

> On 5/2/2015 6:29 PM, Cecil Westerhof wrote:
> 
>> At the moment I define the test functionality in the following way:
> 
> Any automated testing is better than none.  For idlelib, I use unittest.
>   For an individual project with specialized needs, I use a custom test
> framework tuned to those needs.
> 
>>          else:
>>              action = options[0][0]
>>              if   action == '--all':
>>                  do_all          = True
>>              elif action == '--factorial':
>>                  do_factorial    = True
>>              elif action == '--fibonacci':
>>                  do_fibonacci    = True
>>              elif action == '--happy':
>>                  do_happy        = True
>>              elif action == '--lucky':
>>                  do_lucky        = True
>>              else:
>>                  print >> sys.stderr, progname + ': Unhandled parameter '
>>                  + action sys.exit(1)
> 
> There is usually a way to factor out the duplication of repetitive code
> like this.  

> Often, a dict is somehow involved. 

That cannot be stressed enough. In Python if you test against multiple 
constant values you should always consider a dict-based approach.

Also, you should not shy away from using functions as variables. Instead of

do_lucky = True
...
if do_lucky:
    do_lucky_func()

you can often write something more direct

action = do_lucky_func
...
action_func()

> I believe the following
> is equivalent to the above.
> 
> else:
>    action = options[0][0]
>    try:
>      globals()['do_'+action[2:]] = True
>    except KeyError:
>      print >> sys.stderr, progname + ': Unhandled parameter ' + action
>      sys.exit(1)

To go a bit farther down that path of using naming conventions, dict instead 
of extensive if ... elif ... and functions instead of flags here's a 
complete implementation with argparse:

$ python3 choose_action.py -h
usage: choose_action.py [-h] [{all,fibonacci,factorial}]

positional arguments:
  {all,fibonacci,factorial}
                        Known actions: all, fibonacci, factorial

optional arguments:
  -h, --help            show this help message and exit
$ python3 choose_action.py all
fibonacci
factorial
$ python3 choose_action.py
factorial
fibonacci
$ python3 choose_action.py fibonacci
fibonacci
$ python3 choose_action.py non-existent
usage: choose_action.py [-h] [{all,factorial,fibonacci}]
choose_action.py: error: argument action: invalid choice: 'non-existent' 
(choose from 'all', 'factorial', 'fibonacci')


$ cat choose_action.py
import argparse


def do_factorial():
    print("factorial")
    # ...


def do_fibonacci():
    print("fibonacci")
    # ...


def do_all():
    for name, action in lookup.items():
        if name != "all":
            action()

lookup = {
    name[3:]: func
    for name, func in globals().items()
    if name.startswith("do_")
}

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "action",
        nargs="?",
        default="all",
        choices=lookup,
        help="Known actions: %(choices)s")
    args = parser.parse_args()

    lookup[args.action]()
$ 




From breamoreboy at yahoo.co.uk  Sun May  3 05:36:30 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 03 May 2015 10:36:30 +0100
Subject: Messages with a time stamp
In-Reply-To: <87r3qykr39.fsf@Equus.decebal.nl>
References: <87r3qykr39.fsf@Equus.decebal.nl>
Message-ID: <mi4q72$t6g$1@ger.gmane.org>

On 03/05/2015 09:22, Cecil Westerhof wrote:
> For testing I want my messages time stamped like:
>      02:06:32: Check that the non recursive variants give the same value from 1000 upto 100000 step 1000
>      02:06:32: Currently at    1000
>      02:06:33: Currently at   11000
>      02:06:35: Currently at   21000
>      02:06:42: Currently at   31000
>      02:06:56: Currently at   41000
>      02:07:18: Currently at   51000
>      02:07:51: Currently at   61000
>      02:08:43: Currently at   71000
>      02:09:49: Currently at   81000
>      02:11:13: Currently at   91000
>      02:13:01: Calculating values OK
>
>      02:13:01: Start with the time needed to calculate 100000 times
>      02:13:01: Timing factorial_iterative         (985): 31
>      02:13:32: Timing factorial_recursive         (985): 55
>      02:14:28: Timing factorial_recursive_old     (985): 56
>      02:15:24: Timing factorial_tail_recursion    (985): 35
>      02:16:00: Timing factorial_tail_recursion_old(985): 40
>
>      02:16:40: Start with the time needed to calculate 1 times
>                No recursive, because without tail recursion you would run out of stack space
>      02:16:40: Timing factorial_iterative         (100000): 3.7705
>      02:16:44: Timing factorial_tail_recursion    (100000): 3.7692
>      02:16:48: Timing factorial_tail_recursion_old(100000): 4.1537
>
> And sometimes I do not want the time shown, to signify that the
> message belongs to the previous message. And sometimes I want no
> newline, because I want to print something behind it. For example the
> time needed to calculate something:
>      02:13:01: Timing factorial_iterative         (985): 31
>
> For this I wrote:
>      ### Have the possibility to give the stream instead of using stdout
>      class TimedMessage:
>          """
>          For printing messages with time prepended before it
>          Has the possibilty to keep time print blank for when several messages
>          are send shortly after eachother.
>          Also the possibilty to stay on the same line when things need to be appended
>          """
>
>          def give_msg(self, message, show_time = True, use_newline = True):
>              """
>              Prints the message to stdout
>              Use show_time = False when you do not want time
>              Use use_newline = False if you do not want a newline
>              """
>
>              if show_time:
>                  time = strftime(self._format)
>              else:
>                  time = self._blank_time
>              formatted_message = time + message
>              if use_newline:
>                  print(formatted_message)
>              else:
>                  sys.stdout.write(formatted_message)
>                  sys.stdout.flush()
>
>          def __init__(self, format = '%H:%M:%S: '):
>              self._format        = format
>              self._blank_time    = ' ' * len(strftime(self._format))
>
> Can I improve on this?
>
> It is shared at:
>      https://github.com/CecilWesterhof/PythonLibrary/blob/master/utilDecebal.py
>

Rather than reinvent the wheel maybe you can pinch something from here 
https://docs.python.org/3/howto/logging-cookbook.html#logging-to-multiple-destinations

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From Cecil at decebal.nl  Sun May  3 05:49:44 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 03 May 2015 11:49:44 +0200
Subject: Is this a good way to implement testing
References: <878ud6mx4y.fsf@Equus.decebal.nl> <87383em7sn.fsf@Equus.decebal.nl>
 <mailman.53.1430642740.12865.python-list@python.org>
Message-ID: <87iocakn2f.fsf@Equus.decebal.nl>

Op Sunday 3 May 2015 10:45 CEST schreef Peter Otten:

> Cecil Westerhof wrote:
>
>> Another question. Is it acceptable to have it in the module itself,
>> or should I put it in something like test_<module>.py? The code for
>> testing is bigger as the code for the implementation, so I am
>> leaning to putting it in a separate file.
>
> Definitely use an established testing framework instead of rolling
> your own, and definitely put it into a separate file -- by the time
> there is good coverage the test code is usually much bigger than the
> tested code.

Yep, the module already has 370 lines of testing code and only 225 of
working code. And I just started.


> Be aware that there is also doctest which scans docstrings for text
> resembling interactive Python sessions. Doctests are both tests and
> usage examples, so I think it's good to put a few of these into the
> module. Here's how it works:
>
> $ cat factorial.py
> def factorial(n):
> """Calculate the factorial 1 * 2 * ... * n.
>
>>>> factorial(0)
> 1
>>>> factorial(1)
> 1
>>>> factorial(10)
> 3628800 """ return 1 $ python3 -m doctest factorial.py
> **********************************************************************
> File "/home/peter/clpy/factorial.py", line 8, in factorial.factorial
> Failed example: factorial(10) Expected: 3628800 Got: 1
> **********************************************************************
> 1 items had failures: 1 of 3 in factorial.factorial ***Test
> Failed*** 1 failures. $

That looks very promising. But I use the test to verify the
correctness and show the performance. Is that also possible? Or should
I split those out.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From rosuav at gmail.com  Sun May  3 05:51:44 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 3 May 2015 19:51:44 +1000
Subject: Converting 5.223701009526849e-05 to 5e-05
In-Reply-To: <87mw1mkoc5.fsf@Equus.decebal.nl>
References: <87vbgakrlr.fsf@Equus.decebal.nl>
 <mailman.52.1430642455.12865.python-list@python.org>
 <87mw1mkoc5.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmqPePq7=j0H11P=0CUs5m0C1u4R6LgH-oqBqk7e2WZCQQ@mail.gmail.com>

On Sun, May 3, 2015 at 7:22 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> Op Sunday 3 May 2015 10:40 CEST schreef Ben Finney:
>
>> Cecil Westerhof <Cecil at decebal.nl> writes:
>>
>>> When I have a value like 5.223701009526849e-05 in most cases I am
>>> not interested in all the digest after the dot.
>>
>> What type of value is it?
>
> If the absolute value is bigger as 0 and smaller as 1, it should be a
> float. ;-)

Or maybe a fractions.Fraction, or a decimal.Decimal, or a complex, or
maybe a RXSTRING or a Gmp.mpf! There's more than one way to store a
number...

ChrisA


From Cecil at decebal.nl  Sun May  3 06:02:21 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 03 May 2015 12:02:21 +0200
Subject: Converting 5.223701009526849e-05 to 5e-05
References: <87vbgakrlr.fsf@Equus.decebal.nl>
 <mailman.52.1430642455.12865.python-list@python.org>
 <87mw1mkoc5.fsf@Equus.decebal.nl>
 <mailman.59.1430646707.12865.python-list@python.org>
Message-ID: <87egmykmhe.fsf@Equus.decebal.nl>

Op Sunday 3 May 2015 11:51 CEST schreef Chris Angelico:

> On Sun, May 3, 2015 at 7:22 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> Op Sunday 3 May 2015 10:40 CEST schreef Ben Finney:
>>
>>> Cecil Westerhof <Cecil at decebal.nl> writes:
>>>
>>>> When I have a value like 5.223701009526849e-05 in most cases I am
>>>> not interested in all the digest after the dot.
>>>
>>> What type of value is it?
>>
>> If the absolute value is bigger as 0 and smaller as 1, it should be
>> a float. ;-)
>
> Or maybe a fractions.Fraction, or a decimal.Decimal, or a complex,
> or maybe a RXSTRING or a Gmp.mpf! There's more than one way to store
> a number...

Oops. :'-(
I still have to learn a lot.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun May  3 06:08:25 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 03 May 2015 12:08:25 +0200
Subject: Messages with a time stamp
References: <87r3qykr39.fsf@Equus.decebal.nl>
 <mailman.58.1430645806.12865.python-list@python.org>
Message-ID: <87a8xmkm7a.fsf@Equus.decebal.nl>

Op Sunday 3 May 2015 11:36 CEST schreef Mark Lawrence:

> Rather than reinvent the wheel maybe you can pinch something from
> here
> https://docs.python.org/3/howto/logging-cookbook.html#logging-to-multiple-destinations

That looks very promising. The only problem could be that it seems not
to have the possibility to do:
    02:16:40: Start with the time needed to calculate 1 times
              No recursive, because without tail recursion you would run out of stack space
    02:16:40: Timing factorial_iterative         (100000): 3.7705

But I could write a patch for that. Then I would be contributing
besides bugging this list. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From breamoreboy at yahoo.co.uk  Sun May  3 06:21:36 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 03 May 2015 11:21:36 +0100
Subject: Is this a good way to implement testing
In-Reply-To: <87iocakn2f.fsf@Equus.decebal.nl>
References: <878ud6mx4y.fsf@Equus.decebal.nl> <87383em7sn.fsf@Equus.decebal.nl>
 <mailman.53.1430642740.12865.python-list@python.org>
 <87iocakn2f.fsf@Equus.decebal.nl>
Message-ID: <mi4srk$31b$1@ger.gmane.org>

On 03/05/2015 10:49, Cecil Westerhof wrote:
> Op Sunday 3 May 2015 10:45 CEST schreef Peter Otten:
>
>> Cecil Westerhof wrote:
>>
>>> Another question. Is it acceptable to have it in the module itself,
>>> or should I put it in something like test_<module>.py? The code for
>>> testing is bigger as the code for the implementation, so I am
>>> leaning to putting it in a separate file.
>>
>> Definitely use an established testing framework instead of rolling
>> your own, and definitely put it into a separate file -- by the time
>> there is good coverage the test code is usually much bigger than the
>> tested code.
>
> Yep, the module already has 370 lines of testing code and only 225 of
> working code. And I just started.
>
>
>> Be aware that there is also doctest which scans docstrings for text
>> resembling interactive Python sessions. Doctests are both tests and
>> usage examples, so I think it's good to put a few of these into the
>> module. Here's how it works:
>>
>> $ cat factorial.py
>> def factorial(n):
>> """Calculate the factorial 1 * 2 * ... * n.
>>
>>>>> factorial(0)
>> 1
>>>>> factorial(1)
>> 1
>>>>> factorial(10)
>> 3628800 """ return 1 $ python3 -m doctest factorial.py
>> **********************************************************************
>> File "/home/peter/clpy/factorial.py", line 8, in factorial.factorial
>> Failed example: factorial(10) Expected: 3628800 Got: 1
>> **********************************************************************
>> 1 items had failures: 1 of 3 in factorial.factorial ***Test
>> Failed*** 1 failures. $
>
> That looks very promising. But I use the test to verify the
> correctness and show the performance. Is that also possible? Or should
> I split those out.
>

Get it working correctly and if it's fast enough for your needs then job 
done.  If and only if you actually have a performance issue profile your 
code to find the bottlenecks, as gut instinct about Python performance 
is wrong 99.99% of the time.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From steve+comp.lang.python at pearwood.info  Sun May  3 06:32:14 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 03 May 2015 20:32:14 +1000
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
Message-ID: <5545f930$0$12976$c3e8da3$5496439d@news.astraweb.com>

On Sun, 3 May 2015 02:17 am, BartC wrote:

> But for looping over a simple integer range, then using 'range' to
> denote the range (and build a list as it used to do), was how it was
> done. And earlier on people would have been porting coding code to
> Python at which point a straightforward 'for i=a to b' loop suddenly
> acquired a substantial overhead it didn't have before!


The oldest version of Python that is documented on the python.org site is
1.4, and it had xrange:

https://docs.python.org/release/1.4/lib/node26.html

The oldest version of Python I have is Python 0.9.1, which has range but not
xrange:

steve at runes:~/personal/python/Interpreters/python-0.9.1$ ./python0.9.1
>>> range
<built-in function 'builtin.range'>
>>> xrange
Unhandled exception: undefined name: xrange
Stack backtrace (innermost last):
  File "<stdin>", line 1


but then Python 0.9 was missing a lot of features, such as double quoted
strings, and various operators:

>>> "a"
Parsing error: file <stdin>, line 1:
"a"
 ^
Unhandled exception: run-time error: syntax error
>>> 2**3
Parsing error: file <stdin>, line 1:
2**3
   ^
Unhandled exception: run-time error: syntax error


It's unfair to consider anything before version 1.0: it is obvious that
pre-1.0 Python was still incomplete and a work in progress.

If you check the versions here:

http://legacy.python.org/download/releases/src/

you will see that xrange was available in 1.1, but (probably) not in 1.0. So
xrange has been available since almost the start. By version 1.4, or
possibly 1.5, the general advice given as I remember it was that if the
number of loops was small, range was faster than xrange, but if it was
large, xrange used less memory.

On the basis of two software development principles:

(1) release early, release often;

(2) avoid premature optimization;

I don't think there is any problem with range being released before the need
for xrange was proven. Python, especially in the early days, was considered
more of a scripting language than a general purpose language, and scripting
languages often lack a C-style for-loop, using a foreach loop instead. E.g.
I believe the canonical way to loop in bash is something like:

    for $i in `seq start stop` do ... 

(by memory).


If we were creating Python from scratch now, we would never have had an
eager range() that returns a list. But back in the early 1990s, that wasn't
as obvious as it is in hindsight.



-- 
Steven



From Cecil at decebal.nl  Sun May  3 06:50:37 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 03 May 2015 12:50:37 +0200
Subject: Is this a good way to implement testing
References: <878ud6mx4y.fsf@Equus.decebal.nl> <87383em7sn.fsf@Equus.decebal.nl>
 <mailman.53.1430642740.12865.python-list@python.org>
 <87iocakn2f.fsf@Equus.decebal.nl>
 <mailman.60.1430648512.12865.python-list@python.org>
Message-ID: <87618akk8y.fsf@Equus.decebal.nl>

Op Sunday 3 May 2015 12:21 CEST schreef Mark Lawrence:

>> That looks very promising. But I use the test to verify the
>> correctness and show the performance. Is that also possible? Or
>> should I split those out.
>>
>
> Get it working correctly and if it's fast enough for your needs then
> job done. If and only if you actually have a performance issue
> profile your code to find the bottlenecks, as gut instinct about
> Python performance is wrong 99.99% of the time.

I'll split them out. One is to verify the correctness, the other is to
show the difference between implementations. For example: even with
memoization the recursive version of fibonacci is much slower as the
iterative version.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From vasudevram at gmail.com  Sun May  3 06:52:31 2015
From: vasudevram at gmail.com (vasudevram)
Date: Sun, 3 May 2015 03:52:31 -0700 (PDT)
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
Message-ID: <96759e19-8150-4fcf-83bc-b945efb2409e@googlegroups.com>

On Sunday, May 3, 2015 at 1:32:14 AM UTC+5:30, vasudevram wrote:
> Hi group,
> 
> Please refer to this blog post about code showing that a Python data structure can be self-referential:
> 
> http://jugad2.blogspot.in/2015/05/can-python-data-structure-reference.html
> 
> Gotten a couple of comments on it already, but interested in hearing thoughts of Python core dev team members or others who can comment on the internals of how this feature operates, why, etc.
> 
> Thanks
> Vasudev Ram
> Site: www.dancingbison.com
> Python posts: jugad2.blogspot.com/search/label/python


Thanks again to all who answered. I will reply separately today to individual answers that need a reply.

- Vasudev



From steve+comp.lang.python at pearwood.info  Sun May  3 06:56:36 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 03 May 2015 20:56:36 +1000
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <5543041d$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <rkb1x.386614$JH2.12955@fx11.am4>
Message-ID: <5545fee5$0$13007$c3e8da3$5496439d@news.astraweb.com>

On Sun, 3 May 2015 07:28 am, Tony the Tiger wrote:

> On Fri, 01 May 2015 14:42:04 +1000, Steven D'Aprano wrote:
> 
>> use "l" as a variable name, as it looks too much like 1
> 
> If you use a better font, they are very different. Besides, a variable
> name cannot start with a digit (nor can it be a single digit), so it's a
> given that it's an 'l'.

How do you know its a variable?

x+1

Is that 1 the constant or l the variable?

Yes, "use a better font" is good advice, if you can -- you don't always have
control over the machine you are working on, and the fonts which you have
access to -- but even with a better font, they still look two similar for
comfort. It's just common sense[1] to avoid easily confusable names
whenever possible:

advice = 23
advise = 23
makenewhttpfactorywidgetcreator = True
makenewhttpfactoryuidgetcreator = False

regardless of how awesome your font is. Unless you're spelling out each word
letter by letter, sometimes you will misread words.

Another one is rn versus m, especially at small text sizes, or if you're
silly or unlucky enough to be using Arial.




[1] So rare it's a superpower:

http://i0.wp.com/lolzombie.com/wp-content/uploads/2010/10/8eqf.jpeg




-- 
Steven



From ben.usenet at bsb.me.uk  Sun May  3 06:56:42 2015
From: ben.usenet at bsb.me.uk (Ben Bacarisse)
Date: Sun, 03 May 2015 11:56:42 +0100
Subject: code blocks
References: <mailman.49.1430633046.12865.python-list@python.org>
Message-ID: <87a8xm3p5h.fsf@bsb.me.uk>

"Dr. John Q. Hacker" <zondervanz at gmail.com> writes:

> I'm thinking how interesting it would be to add code blocks to Python,
> so that arbitrary strings of code can be passed around. It would open
> up some interesting possibilities for self-modifying code and generic
> programming.
>
> Since Python has already a plethora of ambiguous string designations,
> one of them could be set aside specificially for code blocks:
>
> """for i in n: print i"""
>
> For any variables, like "n", it would access the scope in which it was
> running. When you tried to print a triple-double-quoted code block,
> perhaps it could invoke the code.
>
> My suggestion would be to use triple double-quoted strings. 
>
> You probably already know that Ruby has code blocks.

What Ruby calls code blocks, Python calls lambdas:

  Ruby:  [1,2,3].map { |x| x*x }
  Python: map(lambda x: x*x, [1,2,3])

In some cases, the lambda is implicit.  For example, the above can be
written as

  [x*x for x in [1,2,3]]

You can use strings (if you really must), using eval() at the
appropriate place eval() but that opens up all sorts of cans of worms.

-- 
Ben.


From rosuav at gmail.com  Sun May  3 06:57:25 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 3 May 2015 20:57:25 +1000
Subject: Converting 5.223701009526849e-05 to 5e-05
In-Reply-To: <87egmykmhe.fsf@Equus.decebal.nl>
References: <87vbgakrlr.fsf@Equus.decebal.nl>
 <mailman.52.1430642455.12865.python-list@python.org>
 <87mw1mkoc5.fsf@Equus.decebal.nl>
 <mailman.59.1430646707.12865.python-list@python.org>
 <87egmykmhe.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmohy+HpV+gFhcQ83QomyABki3kr6VoTO3vKDZhpc_KhQA@mail.gmail.com>

On Sun, May 3, 2015 at 8:02 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>> If the absolute value is bigger as 0 and smaller as 1, it should be
>>> a float. ;-)
>>
>> Or maybe a fractions.Fraction, or a decimal.Decimal, or a complex,
>> or maybe a RXSTRING or a Gmp.mpf! There's more than one way to store
>> a number...
>
> Oops. :'-(
> I still have to learn a lot.

We all do :) Not all of those are Python types, incidentally. The
first three are, although 'complex' is a bit of a cheat (a complex
number is stored as two floats, so a complex with an imag of 0 is
virtually identical to a straight float); RXSTRING is the one and only
data type in REXX, and is (as the name suggests) stored as a string;
and Gmp.mpf is Pike's data type for a floating-point value stored
using the GNU Multiprecision library (gmp), and is thus capable of
arbitrary precision storage just like Python's own integer type. And
of course, the old fogeys among us know a bunch more ways to store
floating point values - not to mention all the ways of storing *fixed*
point values (the simplest being to just store an integer with the
number of hundredths of whatever it is you have - eg storing a dollar
amount as an integral number of cents). You'd be amazed how many
different ways there are of doing the same thing!

ChrisA


From rosuav at gmail.com  Sun May  3 06:59:54 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 3 May 2015 20:59:54 +1000
Subject: l = range(int(1E9))
In-Reply-To: <5545f930$0$12976$c3e8da3$5496439d@news.astraweb.com>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <5545f930$0$12976$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmoonktAHuXmv4nhrw9b6Xg+AF-esz=Yrz2sGpHoGm-H0Q@mail.gmail.com>

On Sun, May 3, 2015 at 8:32 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> scripting
> languages often lack a C-style for-loop, using a foreach loop instead. E.g.
> I believe the canonical way to loop in bash is something like:
>
>     for $i in `seq start stop` do ...
>
> (by memory).

Newer versions of bash have grown an alternative syntax for that
common case, but if you need to support older forms, or other shells,
then yes, it's something like that. Maybe without the dollar sign. (I
can never remember the exact syntax, and I know bash is really finicky
about where you put newlines and where you don't, but it looks
something like what you had.)

ChrisA


From steve+comp.lang.python at pearwood.info  Sun May  3 07:05:40 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 03 May 2015 21:05:40 +1000
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <55460105$0$13008$c3e8da3$5496439d@news.astraweb.com>

On Sun, 3 May 2015 07:40 am, Jon Ribbens wrote:

> On 2015-05-02, BartC <bc at freeuk.com> wrote:
>> So do I, I think, if no-one is willing to admit that the original way of
>> implementing range() was a glaring mistake.
> 
> I think the issue is that nobody else here thinks the "original way"
> of iterating was to use range(), for anything other than extremely
> small ranges.
> 
> For information, it looks like xrange() was added on 26 Oct 1993,
> which pre-dates Python 1.0.

Ah yes, you are right: my searching failed to find xrange in Python 1.0.1,
but it was actually introduced in 1.0.0. This is from the Misc/NEWS file:

* New function xrange() creates a "range object".  Its arguments are
the same as those of range(), and when used in a for loop a range
objects also behaves identical.  The advantage of xrange() over
range() is that its representation (if the range contains many
elements) is much more compact than that of range().  The disadvantage
is that the result cannot be used to initialize a list object or for
the "Python idiom" [RED, GREEN, BLUE] = range(3).  On some modern
architectures, benchmarks have shown that "for i in range(...): ..."
actually executes *faster* than "for i in xrange(...): ...", but on
memory starved machines like PCs running DOS range(100000) may be just
too big to be represented at all...





-- 
Steven



From steve+comp.lang.python at pearwood.info  Sun May  3 07:10:39 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 03 May 2015 21:10:39 +1000
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <mailman.34.1430608657.12865.python-list@python.org>
Message-ID: <5546022f$0$13008$c3e8da3$5496439d@news.astraweb.com>

On Sun, 3 May 2015 09:17 am, Terry Reedy wrote:

> I believe that there was a time when printing a recursive structure hit
> the recursion limit like your flatten did. But I will not reload 1.5 to
> check.

No, that was already fixed by 1.5:

[steve at ando ~]$ python1.5
Python 1.5.2 (#1, Aug 27 2012, 09:09:18)  [GCC 4.1.2 20080704 (Red Hat
4.1.2-52)] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> L = []
>>> L.append(L)
>>> print L
[[...]]


I'm not sure when it was fixed -- possibly 1.4?



-- 
Steven



From steve+comp.lang.python at pearwood.info  Sun May  3 07:15:40 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 03 May 2015 21:15:40 +1000
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
 <dhc1x.319346$qW.51334@fx20.am4>
Message-ID: <5546035d$0$13001$c3e8da3$5496439d@news.astraweb.com>

On Sun, 3 May 2015 08:33 am, BartC wrote:

> OK, so it's just an irritation then, as a workaround has been available
> for a long time. (For example, if you use xrange, it won't work on 3.x.
> If you use range, then it might be inefficient on 2.x.)


That is trivially easy to deal with. Put this at the top of your module:

try:
    xrange
except NameError:
    xrange = range


and then use xrange throughout the module. Or if you prefer:

try:
    range = xrange
except NameError:
    pass

and just use range.


-- 
Steven



From steve+comp.lang.python at pearwood.info  Sun May  3 07:16:56 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 03 May 2015 21:16:56 +1000
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
 <dhc1x.319346$qW.51334@fx20.am4> <554582E3.4030700@gmail.com>
 <mailman.40.1430619379.12865.python-list@python.org>
Message-ID: <554603aa$0$12981$c3e8da3$5496439d@news.astraweb.com>

On Sun, 3 May 2015 12:16 pm, Mark Lawrence wrote:

> I doubt that six will ever make the standard library as 2.7 only has
> another five years in official support. ?By that time I suppose we'll to
> going through the porting pain all over again with the transition from
> Python 3 to Python 4. ?Alright, alright, only joking


Guido has said that Python 4 will just be an incremental update from 3.x.


-- 
Steven



From steve+comp.lang.python at pearwood.info  Sun May  3 07:21:58 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 03 May 2015 21:21:58 +1000
Subject: l = range(int(1E9))
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <CALwzidnAGmimhH_Kztt6KA6x+WLnA6M-WjrZvEzGEj7Xxr0VmQ@mail.gmail.com>
 <mi3nv3$l35$1@ger.gmane.org>
 <mailman.46.1430628725.12865.python-list@python.org>
Message-ID: <554604d6$0$12981$c3e8da3$5496439d@news.astraweb.com>

On Sun, 3 May 2015 02:51 pm, Ian Kelly wrote:

> On Sat, May 2, 2015 at 5:51 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> On 5/2/2015 5:31 PM, Ian Kelly wrote:
>>
>>> Would it have been better if range() had been implemented as xrange()
>>> from the beginning? Sure, that would have been great. Except for one
>>> small detail: the iterator protocol didn't exist back then.
>>
>>
>> For loops originally used the getitem iterator protocol. xrange objects
>> have
>> a __getitem__ method, but not __iter__ or __next__.  As Mark pointed out,
>> they were introduced in 1993.
> 
> I'm aware of getitem iterators; just didn't realize that xrange used
> it or was that old.


Yep, xrange objects are *not* iterators:

py> r = xrange(100)
py> iter(r) is r
False
py> next(r)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: xrange object is not an iterator



-- 
Steven



From rosuav at gmail.com  Sun May  3 07:29:21 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 3 May 2015 21:29:21 +1000
Subject: l = range(int(1E9))
In-Reply-To: <5546035d$0$13001$c3e8da3$5496439d@news.astraweb.com>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
 <dhc1x.319346$qW.51334@fx20.am4>
 <5546035d$0$13001$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmpBCmP0sUUo+X7ef=ACYmZZBGd8_+-_wZjMz=ssR1qa6w@mail.gmail.com>

On Sun, May 3, 2015 at 9:15 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> Or if you prefer:
>
> try:
>     range = xrange
> except NameError:
>     pass
>
> and just use range.

I prefer this idiom, on the basis that code should be written for the
more recent version, and have minimal code to support older versions.
Then when you drop support for the older versions, all you have to do
is delete a bit at the top. The same principle applies to the
__future__ import system - you can declare that your Python 2.5 code
is able to use the 'with' statement, but you can't (and shouldn't)
make your 2.6+ code use 'with' as a name.

ChrisA


From rosuav at gmail.com  Sun May  3 07:30:57 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 3 May 2015 21:30:57 +1000
Subject: l = range(int(1E9))
In-Reply-To: <554603aa$0$12981$c3e8da3$5496439d@news.astraweb.com>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
 <dhc1x.319346$qW.51334@fx20.am4> <554582E3.4030700@gmail.com>
 <mailman.40.1430619379.12865.python-list@python.org>
 <554603aa$0$12981$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmpyv-2zGqKc=Vkb6MsxNqarYbvZBHaT5g408ya8Xqqs9A@mail.gmail.com>

On Sun, May 3, 2015 at 9:16 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Sun, 3 May 2015 12:16 pm, Mark Lawrence wrote:
>
>> I doubt that six will ever make the standard library as 2.7 only has
>> another five years in official support.  By that time I suppose we'll to
>> going through the porting pain all over again with the transition from
>> Python 3 to Python 4.  Alright, alright, only joking
>
>
> Guido has said that Python 4 will just be an incremental update from 3.x.

There is still, as I understand it, the possibility that Python 4.0
will ditch some things that will have been deprecated for a long time
by 3.9. But yes, it'll be no more upheavalous to go from 3.9 to 4.0
than from 3.8 to 3.9.

ChrisA


From davea at davea.name  Sun May  3 07:34:27 2015
From: davea at davea.name (Dave Angel)
Date: Sun, 03 May 2015 07:34:27 -0400
Subject: Converting 5.223701009526849e-05 to 5e-05
In-Reply-To: <87mw1mkoc5.fsf@Equus.decebal.nl>
References: <87vbgakrlr.fsf@Equus.decebal.nl>
 <mailman.52.1430642455.12865.python-list@python.org>
 <87mw1mkoc5.fsf@Equus.decebal.nl>
Message-ID: <554607C3.9000904@davea.name>

On 05/03/2015 05:22 AM, Cecil Westerhof wrote:
> Op Sunday 3 May 2015 10:40 CEST schreef Ben Finney:
>
>> Cecil Westerhof <Cecil at decebal.nl> writes:
>>
>>> When I have a value like 5.223701009526849e-05 in most cases I am
>>> not interested in all the digest after the dot.
>>
>> What type of value is it?
>
> If the absolute value is bigger as 0 and smaller as 1, it should be a
> float. ;-)
>

or for many uses, more likely a decimal.Decimal(), where many of the 
problems Ben mentions are moot.


-- 
DaveA


From vasudevram at gmail.com  Sun May  3 08:59:54 2015
From: vasudevram at gmail.com (vasudevram)
Date: Sun, 3 May 2015 05:59:54 -0700 (PDT)
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <mailman.34.1430608657.12865.python-list@python.org>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <mailman.34.1430608657.12865.python-list@python.org>
Message-ID: <69353f16-7c04-469d-b3ac-05fbe75667d2@googlegroups.com>

On Sunday, May 3, 2015 at 4:48:11 AM UTC+5:30, Terry Reedy wrote:
> On 5/2/2015 4:02 PM, vasudevram wrote:
> > Hi group,
> >
> > Please refer to this blog post about code showing that a Python data
> > structure can be self-referential:
> >
> > http://jugad2.blogspot.in/2015/05/can-python-data-structure-reference.html
> >
> >  Gotten a couple of comments on it already, but interested in hearing
> > thoughts of Python core dev team members or others who can comment on
> > the internals of how this feature operates, why, etc.
> 
> Please correct the following:
>    "g (a list) contains itself as a list item (of g)."
> g is a dict, as you yourself later said.
> 
> "Case 2) But if the evaluation works in a different order, i.e. the 
> globals() function is first called (before the variable g is created), 
> then at this point its return value (the dict) should not contain the 
> item with key 'g' (and value g), and it is this dict that should get 
> assigned to the variable g. Hence when we print g, we should not see g 
> again within it."
> 
> This seems like you are presenting this as a statement of fact, but you 
> then admit it is false.  The lead in sentence should more carefully 
> state that what follows are possible hypotheses.  one is true and the 
> other (mostly) not.
> 
> The key point is the meaning of "the globals() function returns a dict 
> representing the current global symbol table,"  "Global symbol table" is 
> an abstraction.  In CPython, the implementation is a dict and globals 
> returns that dict, not a copy.  Python generally does not copy objects 
> unless requested.
> 
> Similarly, locals() returns a dict representing the current local symbol 
> table. In a CPython class statement, the local symbol table is 
> implemented with a dict, and locals() is that dict.  In a CPython def 
> statement, the local symbol table is implemented as a C array (of 
> pointers to PyObjects). Locals() is a dict (created just once) updated 
> from local names in the code object and the objects in the array *at the 
> time of the call*
> 
>  >>> def f(a):
> 	g = locals()
> 	print(id(g), g)
> 	g = locals()
> 	print(id(g), g)
> 	
>  >>> f(3)
> 56288136 {'a': 3}
> 56288136 {'a': 3, 'g': {...}}
> 
> 'Case 2" applies for the first locals() call, but only for the first.
> 
> I believe that there was a time when printing a recursive structure hit 
> the recursion limit like your flatten did. But I will not reload 1.5 to 
> check.
> 
> -- 
> Terry Jan Reedy

Terry Reedy:

Thanks for the detailed answer. I have corrected the list vs. dict mistake in a comment to my original post on my blog. Don't want to edit the post itself since some readers will get it twice via feed readers.

Re. statement of fact vs. hypotheses. While I'm not sure of your exact meaning in that paragraph, I understand the concept, and yes, I was not clear enough in phrasing that part. It should have read like something along these lines:

Observations -> One or more hypotheses -> deductions -> one or more alternative conclusions.

I mixed that up a bit.

Thanks.
- Vasudev


From vasudevram at gmail.com  Sun May  3 09:05:47 2015
From: vasudevram at gmail.com (vasudevram)
Date: Sun, 3 May 2015 06:05:47 -0700 (PDT)
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <69353f16-7c04-469d-b3ac-05fbe75667d2@googlegroups.com>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <mailman.34.1430608657.12865.python-list@python.org>
 <69353f16-7c04-469d-b3ac-05fbe75667d2@googlegroups.com>
Message-ID: <667d6137-a01a-42d0-ab47-4ec3930d82c7@googlegroups.com>

On Sunday, May 3, 2015 at 6:30:16 PM UTC+5:30, vasudevram wrote:
> On Sunday, May 3, 2015 at 4:48:11 AM UTC+5:30, Terry Reedy wrote:
> > On 5/2/2015 4:02 PM, vasudevram wrote:
> > > Hi group,
> > >
> > > Please refer to this blog post about code showing that a Python data
> > > structure can be self-referential:
> > >
> > > http://jugad2.blogspot.in/2015/05/can-python-data-structure-reference.html
> > >
> > >  Gotten a couple of comments on it already, but interested in hearing
> > > thoughts of Python core dev team members or others who can comment on
> > > the internals of how this feature operates, why, etc.
> > 
> > Please correct the following:
> >    "g (a list) contains itself as a list item (of g)."
> > g is a dict, as you yourself later said.
> > 
> > "Case 2) But if the evaluation works in a different order, i.e. the 
> > globals() function is first called (before the variable g is created), 
> > then at this point its return value (the dict) should not contain the 
> > item with key 'g' (and value g), and it is this dict that should get 
> > assigned to the variable g. Hence when we print g, we should not see g 
> > again within it."
> > 
> > This seems like you are presenting this as a statement of fact, but you 
> > then admit it is false.  The lead in sentence should more carefully 
> > state that what follows are possible hypotheses.  one is true and the 
> > other (mostly) not.
> > 
> > The key point is the meaning of "the globals() function returns a dict 
> > representing the current global symbol table,"  "Global symbol table" is 
> > an abstraction.  In CPython, the implementation is a dict and globals 
> > returns that dict, not a copy.  Python generally does not copy objects 
> > unless requested.
> > 
> > Similarly, locals() returns a dict representing the current local symbol 
> > table. In a CPython class statement, the local symbol table is 
> > implemented with a dict, and locals() is that dict.  In a CPython def 
> > statement, the local symbol table is implemented as a C array (of 
> > pointers to PyObjects). Locals() is a dict (created just once) updated 
> > from local names in the code object and the objects in the array *at the 
> > time of the call*
> > 
> >  >>> def f(a):
> > 	g = locals()
> > 	print(id(g), g)
> > 	g = locals()
> > 	print(id(g), g)
> > 	
> >  >>> f(3)
> > 56288136 {'a': 3}
> > 56288136 {'a': 3, 'g': {...}}
> > 
> > 'Case 2" applies for the first locals() call, but only for the first.
> > 
> > I believe that there was a time when printing a recursive structure hit 
> > the recursion limit like your flatten did. But I will not reload 1.5 to 
> > check.
> > 
> > -- 
> > Terry Jan Reedy
> 
> Terry Reedy:
> 
> Thanks for the detailed answer. I have corrected the list vs. dict mistake in a comment to my original post on my blog. Don't want to edit the post itself since some readers will get it twice via feed readers.
> 
> Re. statement of fact vs. hypotheses. While I'm not sure of your exact meaning in that paragraph, I understand the concept, and yes, I was not clear enough in phrasing that part. It should have read like something along these lines:
> 
> Observations -> One or more hypotheses -> deductions -> one or more alternative conclusions.
> 
> I mixed that up a bit.
> 
> Thanks.
> - Vasudev

I may have needed to put the "experiments" step in there as well :)

See:

http://en.wikipedia.org/wiki/Scientific_method



From rosuav at gmail.com  Sun May  3 09:08:14 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 3 May 2015 23:08:14 +1000
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <69353f16-7c04-469d-b3ac-05fbe75667d2@googlegroups.com>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <mailman.34.1430608657.12865.python-list@python.org>
 <69353f16-7c04-469d-b3ac-05fbe75667d2@googlegroups.com>
Message-ID: <CAPTjJmrVf4FsjcsnAvQTOB6Xu2f=tLs+i3CTyL3UQ9hEN9JtbA@mail.gmail.com>

On Sun, May 3, 2015 at 10:59 PM, vasudevram <vasudevram at gmail.com> wrote:
> Re. statement of fact vs. hypotheses. While I'm not sure of your exact meaning in that paragraph, I understand the concept, and yes, I was not clear enough in phrasing that part. It should have read like something along these lines:
>
> Observations -> One or more hypotheses -> deductions -> one or more alternative conclusions.

I just took a glance at your blog post, and it's reasonably clear now.
(Though I tend to try to avoid making substantive edits to blog posts
once they've been published and read; a codicil, or at very least an
italicized comment saying "(Edit: Actually, blah blah blah.)", is
clearer. Of course, trivial edits like typos can be corrected - nobody
would be bothered by those unexpectedly changing.) Posts to the
newsgroup/mailing list basically can't be edited, so the normal way to
acknowledge an error or correction is simply a follow-up post.

ChrisA


From jon+usenet at unequivocal.co.uk  Sun May  3 10:40:05 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sun, 3 May 2015 14:40:05 +0000 (UTC)
Subject: Unicode surrogate pairs (Python 3.4)
Message-ID: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>

If I have a string containing surrogate pairs like this in Python 3.4:

  "\udb40\udd9d"

How do I convert it into the proper form:

  "\U000E019D"

? The answer appears not to be "unicodedata.normalize".


From rosuav at gmail.com  Sun May  3 11:05:31 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 4 May 2015 01:05:31 +1000
Subject: Unicode surrogate pairs (Python 3.4)
In-Reply-To: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>
References: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <CAPTjJmo9OJAwmJxvFfBSz9wR8agV7obDxZ-_-m3shsc67EcF+g@mail.gmail.com>

On Mon, May 4, 2015 at 12:40 AM, Jon Ribbens
<jon+usenet at unequivocal.co.uk> wrote:
> If I have a string containing surrogate pairs like this in Python 3.4:
>
>   "\udb40\udd9d"
>
> How do I convert it into the proper form:
>
>   "\U000E019D"
>
> ? The answer appears not to be "unicodedata.normalize".

No, it's not, because Unicode normalization is a very specific thing.
You're looking for a fix for some kind of encoding issue; Unicode
normalization translates between combining characters and combined
characters.

You shouldn't even actually _have_ those in your string in the first
place. How did you construct/receive that data? Ideally, catch it at
that point, and deal with it there. But if you absolutely have to
convert the surrogates, it ought to be possible to do a sloppy UCS-2
conversion to bytes, then a proper UTF-16 decode on the result.

ChrisA


From Cecil at decebal.nl  Sun May  3 11:12:21 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 03 May 2015 17:12:21 +0200
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl>
Message-ID: <871tixlmp6.fsf@Equus.decebal.nl>

Op Saturday 2 May 2015 16:20 CEST schreef Cecil Westerhof:

> I am throwing the cat among the pigeons. ;-)
>
> In another thread I mentioned that I liked to have tail recursion in
> Python. To be clear not automatic, but asked for.
>
> Looking at the replies I did hit a nerve. But I still want to
> continue.
>
> Some things are better expressed recursively for the people reading
> the code. But there are two problems with that:
> - You can get out of stack space
> - It is less efficient
>
> Most of the time the first problem is the most important.
>
> When I write factorial (I know it is already written, but I use it
> as an example to show a point), the recursive variant can not be
> called with 1.000 without tail recursion. So for functions that
> could go very deep, tail recursion would be a blessing.
>
> By the way: I think that even if the recursion does not go further
> as 500, it is still a good idea to use tail recursion. Why use stack
> space when it is not necessary?

I pushed the example to GitHub:
    https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun May  3 11:21:32 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 03 May 2015 17:21:32 +0200
Subject: Best way to use globally format
Message-ID: <87wq0pk7pf.fsf@Equus.decebal.nl>

I have a file where I used a lot of {0}, {1} and {2}. Most but not all
are changed to {0:.3E}, {1:.3E} and {2:.3E}. But when I want to change
the format I come in dependency hell.

I could do something like:
    format = ':.3E'
    fmt0   = '{0' + format + '}
    fmt1   = '{1' + format + '}
    fmt2   = '{2' + format + '}

and replace occurrences of:
    'before {0} after'
with:
    'before ' + fmt0 + ' after'

But that does not really make me happy. Is there a better way?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From jon+usenet at unequivocal.co.uk  Sun May  3 11:32:14 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sun, 3 May 2015 15:32:14 +0000 (UTC)
Subject: Unicode surrogate pairs (Python 3.4)
References: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.67.1430665534.12865.python-list@python.org>
Message-ID: <slrnmkcftt.230.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-03, Chris Angelico <rosuav at gmail.com> wrote:
> On Mon, May 4, 2015 at 12:40 AM, Jon Ribbens
><jon+usenet at unequivocal.co.uk> wrote:
>> If I have a string containing surrogate pairs like this in Python 3.4:
>>
>>   "\udb40\udd9d"
>>
>> How do I convert it into the proper form:
>>
>>   "\U000E019D"
>>
>> ? The answer appears not to be "unicodedata.normalize".
>
> No, it's not, because Unicode normalization is a very specific thing.
> You're looking for a fix for some kind of encoding issue; Unicode
> normalization translates between combining characters and combined
> characters.
>
> You shouldn't even actually _have_ those in your string in the first
> place. How did you construct/receive that data? Ideally, catch it at
> that point, and deal with it there.

That would, unfortunately, be "tell the Unicode Consortium to format
their documents differently", which seems unlikely to happen. I'm
trying to read in: http://www.unicode.org/Public/idna/6.3.0/IdnaTest.txt

> But if you absolutely have to convert the surrogates, it ought to be
> possible to do a sloppy UCS-2 conversion to bytes, then a proper
> UTF-16 decode on the result.

Python doesn't appear to have UCS-2 support, so I guess what you're
saying is that I have to write my own surrogate-decoder? This seems
a little surprising.


From marko at pacujo.net  Sun May  3 11:35:17 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sun, 03 May 2015 18:35:17 +0300
Subject: Unicode surrogate pairs (Python 3.4)
References: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.67.1430665534.12865.python-list@python.org>
 <slrnmkcftt.230.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <87fv7disi2.fsf@elektro.pacujo.net>

Jon Ribbens <jon+usenet at unequivocal.co.uk>:

> Python doesn't appear to have UCS-2 support, so I guess what you're
> saying is that I have to write my own surrogate-decoder? This seems a
> little surprising.

Try UTF-16.


Marko


From vasudevram at gmail.com  Sun May  3 11:46:09 2015
From: vasudevram at gmail.com (vasudevram)
Date: Sun, 3 May 2015 08:46:09 -0700 (PDT)
Subject: Inner workings of this Python feature: Can a Python data
 structure reference itself?
In-Reply-To: <mailman.66.1430658498.12865.python-list@python.org>
References: <387ac520-9da3-411a-a3b8-b326e1e8a2c0@googlegroups.com>
 <mailman.34.1430608657.12865.python-list@python.org>
 <69353f16-7c04-469d-b3ac-05fbe75667d2@googlegroups.com>
 <mailman.66.1430658498.12865.python-list@python.org>
Message-ID: <68c2b223-783d-4892-9e23-9f5163ab65da@googlegroups.com>

On Sunday, May 3, 2015 at 6:38:28 PM UTC+5:30, Chris Angelico wrote:
> On Sun, May 3, 2015 at 10:59 PM, vasudevram <vasudevram at gmail.com> wrote:
> > Re. statement of fact vs. hypotheses. While I'm not sure of your exact meaning in that paragraph, I understand the concept, and yes, I was not clear enough in phrasing that part. It should have read like something along these lines:
> >
> > Observations -> One or more hypotheses -> deductions -> one or more alternative conclusions.
> 
> I just took a glance at your blog post, and it's reasonably clear now.
> (Though I tend to try to avoid making substantive edits to blog posts
> once they've been published and read; a codicil, or at very least an
> italicized comment saying "(Edit: Actually, blah blah blah.)", is
> clearer. Of course, trivial edits like typos can be corrected - nobody
> would be bothered by those unexpectedly changing.) Posts to the
> newsgroup/mailing list basically can't be edited, so the normal way to
> acknowledge an error or correction is simply a follow-up post.
> 
> ChrisA

Good points, thanks.


From rosuav at gmail.com  Sun May  3 11:48:47 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 4 May 2015 01:48:47 +1000
Subject: Unicode surrogate pairs (Python 3.4)
In-Reply-To: <slrnmkcftt.230.jon+usenet@frosty.unequivocal.co.uk>
References: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.67.1430665534.12865.python-list@python.org>
 <slrnmkcftt.230.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <CAPTjJmqTGim2XEoqKNwcMmt9b+9KEVyKf8trvfjn4+ZhM-zXoQ@mail.gmail.com>

On Mon, May 4, 2015 at 1:32 AM, Jon Ribbens
<jon+usenet at unequivocal.co.uk> wrote:
>> You shouldn't even actually _have_ those in your string in the first
>> place. How did you construct/receive that data? Ideally, catch it at
>> that point, and deal with it there.
>
> That would, unfortunately, be "tell the Unicode Consortium to format
> their documents differently", which seems unlikely to happen. I'm
> trying to read in: http://www.unicode.org/Public/idna/6.3.0/IdnaTest.txt

Ah, so what you _actually_ have is "\\udb40\\udd9d" - the backslashes
are in your input. I'm not sure what the best way to deal with that
is... it's a bit of a mess. You may find yourself needing to do
something manually, unless there's a way to ask Python to encode to
pseudo-UCS-2 that allows surrogates. Some languages may have sloppy
conversions available, but Python's seems to be quite strict (which is
correct). Is there an errors handler that can do this?

ChrisA


From python at mrabarnett.plus.com  Sun May  3 11:53:47 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 03 May 2015 16:53:47 +0100
Subject: Unicode surrogate pairs (Python 3.4)
In-Reply-To: <slrnmkcftt.230.jon+usenet@frosty.unequivocal.co.uk>
References: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.67.1430665534.12865.python-list@python.org>
 <slrnmkcftt.230.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <5546448B.2090307@mrabarnett.plus.com>

On 2015-05-03 16:32, Jon Ribbens wrote:
> On 2015-05-03, Chris Angelico <rosuav at gmail.com> wrote:
>> On Mon, May 4, 2015 at 12:40 AM, Jon Ribbens
>><jon+usenet at unequivocal.co.uk> wrote:
>>> If I have a string containing surrogate pairs like this in Python 3.4:
>>>
>>>   "\udb40\udd9d"
>>>
>>> How do I convert it into the proper form:
>>>
>>>   "\U000E019D"
>>>
>>> ? The answer appears not to be "unicodedata.normalize".
>>
>> No, it's not, because Unicode normalization is a very specific thing.
>> You're looking for a fix for some kind of encoding issue; Unicode
>> normalization translates between combining characters and combined
>> characters.
>>
>> You shouldn't even actually _have_ those in your string in the first
>> place. How did you construct/receive that data? Ideally, catch it at
>> that point, and deal with it there.
>
> That would, unfortunately, be "tell the Unicode Consortium to format
> their documents differently", which seems unlikely to happen. I'm
> trying to read in: http://www.unicode.org/Public/idna/6.3.0/IdnaTest.txt
>
That document looks like it's encoded in UTF-8.

>> But if you absolutely have to convert the surrogates, it ought to be
>> possible to do a sloppy UCS-2 conversion to bytes, then a proper
>> UTF-16 decode on the result.
>
> Python doesn't appear to have UCS-2 support, so I guess what you're
> saying is that I have to write my own surrogate-decoder? This seems
> a little surprising.
>



From versesane at gmail.com  Sun May  3 12:01:57 2015
From: versesane at gmail.com (Ankur Gupta)
Date: Sun, 3 May 2015 09:01:57 -0700 (PDT)
Subject: ImportPython Newsletter
Message-ID: <e5b402f5-b249-41ce-99fd-14a018c9f181@googlegroups.com>

Hey Guys,

Just like to draw attention to ImportPython a weekly Python newsletter. This is the 30th issue of the newsletter http://importpython.com/newsletter/no/30/. Check out a listing of all Python Books here http://importpython.com/books/

Thanks,
Ankur


From __peter__ at web.de  Sun May  3 12:16:41 2015
From: __peter__ at web.de (Peter Otten)
Date: Sun, 03 May 2015 18:16:41 +0200
Subject: Best way to use globally format
References: <87wq0pk7pf.fsf@Equus.decebal.nl>
Message-ID: <mi5hl9$i5p$1@ger.gmane.org>

Cecil Westerhof wrote:

> I have a file where I used a lot of {0}, {1} and {2}. Most but not all
> are changed to {0:.3E}, {1:.3E} and {2:.3E}. But when I want to change
> the format I come in dependency hell.
> 
> I could do something like:
>     format = ':.3E'
>     fmt0   = '{0' + format + '}
>     fmt1   = '{1' + format + '}
>     fmt2   = '{2' + format + '}
> 
> and replace occurrences of:
>     'before {0} after'
> with:
>     'before ' + fmt0 + ' after'
> 
> But that does not really make me happy. Is there a better way?
 
There's limited support for nesting {...}:

>>> "{0:{fmt}}, {1:{fmt}}, {2:{fmt}}".format(1.12345789, 2., 3., fmt=".3E")
'1.123E+00, 2.000E+00, 3.000E+00'
>>> "{0:{fmt}}, {1:{fmt}}, {2:{fmt}}".format(1.12345789, 2., 3., fmt="6.2")
'   1.1,    2.0,    3.0'
>>> "{0:{fmt}}, {1:{fmt}}, {2:{fmt}}".format(1.12345789, 2., 3., fmt="06.2")
'0001.1, 0002.0, 0003.0'

Converting the numbers to string first may be clearer though:

>>> formatted_numbers = [format(x, "010.2") for x in [1.12345789, 2., 3.]]
>>> "{0}, {1}, {2}".format(*formatted_numbers)
'00000001.1, 00000002.0, 00000003.0'




From jon+usenet at unequivocal.co.uk  Sun May  3 12:26:27 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sun, 3 May 2015 16:26:27 +0000 (UTC)
Subject: Unicode surrogate pairs (Python 3.4)
References: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.67.1430665534.12865.python-list@python.org>
 <slrnmkcftt.230.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.69.1430668429.12865.python-list@python.org>
Message-ID: <slrnmkcj3j.230.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-03, MRAB <python at mrabarnett.plus.com> wrote:
> On 2015-05-03 16:32, Jon Ribbens wrote:
>> That would, unfortunately, be "tell the Unicode Consortium to format
>> their documents differently", which seems unlikely to happen. I'm
>> trying to read in: http://www.unicode.org/Public/idna/6.3.0/IdnaTest.txt
>>
> That document looks like it's encoded in UTF-8.

It is. But it also, for reasons best known to the Unicode Consortium,
contains strings of the form \uXXXX which need to be parsed into the
appropriate character, and some of *those* are then surrogate pairs,
which need to be further converted.


From jon+usenet at unequivocal.co.uk  Sun May  3 12:30:40 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sun, 3 May 2015 16:30:40 +0000 (UTC)
Subject: Unicode surrogate pairs (Python 3.4)
References: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.67.1430665534.12865.python-list@python.org>
 <slrnmkcftt.230.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.68.1430668130.12865.python-list@python.org>
Message-ID: <slrnmkcjbf.230.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-03, Chris Angelico <rosuav at gmail.com> wrote:
> On Mon, May 4, 2015 at 1:32 AM, Jon Ribbens
><jon+usenet at unequivocal.co.uk> wrote:
>> That would, unfortunately, be "tell the Unicode Consortium to format
>> their documents differently", which seems unlikely to happen. I'm
>> trying to read in: http://www.unicode.org/Public/idna/6.3.0/IdnaTest.txt
>
> Ah, so what you _actually_ have is "\\udb40\\udd9d" - the backslashes
> are in your input.

Well, they were, but I already wrote code to convert them into the
strings I showed in my original post.

> I'm not sure what the best way to deal with that is... it's a bit of
> a mess. You may find yourself needing to do something manually,
> unless there's a way to ask Python to encode to pseudo-UCS-2 that
> allows surrogates. Some languages may have sloppy conversions
> available, but Python's seems to be quite strict (which is correct).
> Is there an errors handler that can do this?

I did some experimentation, and it looks like the answer is:

  "\udb40\udd9d".encode("utf16", "surrogatepass").decode("utf16")

Thanks for your help!


From ben.usenet at bsb.me.uk  Sun May  3 12:32:51 2015
From: ben.usenet at bsb.me.uk (Ben Bacarisse)
Date: Sun, 03 May 2015 17:32:51 +0100
Subject: Best way to use globally format
References: <87wq0pk7pf.fsf@Equus.decebal.nl>
Message-ID: <87a8xlzkng.fsf@bsb.me.uk>

Cecil Westerhof <Cecil at decebal.nl> writes:

> I have a file where I used a lot of {0}, {1} and {2}. Most but not all
> are changed to {0:.3E}, {1:.3E} and {2:.3E}. But when I want to change
> the format I come in dependency hell.
>
> I could do something like:
>     format = ':.3E'
>     fmt0   = '{0' + format + '}
>     fmt1   = '{1' + format + '}
>     fmt2   = '{2' + format + '}
>
> and replace occurrences of:
>     'before {0} after'
> with:
>     'before ' + fmt0 + ' after'
>
> But that does not really make me happy. Is there a better way?

'Better' is often a bit tricky.  You could always factor the formatted
printing into a function, but you will have considered that.  You could
pass the format to used to the format function:

  format_for_numbers = '.3E'
  ...
  'x = {0:{nfmt}}, y = {1:{nfmt}}'.format(3.4, 4.5, nfmt=format_for_numbers);

or you could do that with just the width (if that is the variable part):

  'x = {0:.{nwd}E}, y = {1:.{nwd}E}'.format(3.4, 4.5, nwd=4);
  
-- 
Ben.


From rosuav at gmail.com  Sun May  3 12:47:36 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 4 May 2015 02:47:36 +1000
Subject: Unicode surrogate pairs (Python 3.4)
In-Reply-To: <slrnmkcjbf.230.jon+usenet@frosty.unequivocal.co.uk>
References: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.67.1430665534.12865.python-list@python.org>
 <slrnmkcftt.230.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.68.1430668130.12865.python-list@python.org>
 <slrnmkcjbf.230.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <CAPTjJmoxfGRzXT1pdktQ51TtnE5jNyNjMrAaH2k2Pn4eZTCzSQ@mail.gmail.com>

On Mon, May 4, 2015 at 2:30 AM, Jon Ribbens
<jon+usenet at unequivocal.co.uk> wrote:
> I did some experimentation, and it looks like the answer is:
>
>   "\udb40\udd9d".encode("utf16", "surrogatepass").decode("utf16")
>
> Thanks for your help!

Ha! That's the one. I went poking around but couldn't find the name
for it. That's exactly the sloppy encoding that I was talking about.

ChrisA


From mal at europython.eu  Sun May  3 13:07:56 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Sun, 03 May 2015 19:07:56 +0200
Subject: EuroPython 2015: Financial Aid Program launched
Message-ID: <554655EC.8030208@europython.eu>

We are happy to announce a program for people in need of financial aid
to attend EuroPython.

You can find all the details on our financial aid page:


                      *** Financial Aid Program ***

             https://ep2015.europython.eu/en/financial-aid/


In short, we will be giving out grants in three categories:

 * Ticket discounts

 * Free tickets

 * Travel costs and accommodation

Anyone who wants to attend EuroPython 2015 can apply, including people
who have already purchased tickets. We want to make the event
affordable for as many people as possible.

Looking for financial aid sponsors
----------------------------------

We are still looking for sponsors to increase the budget we have
available for financial aid. If your company would like to sign up as
financial aid sponsor, please contact the sponsors team:

    https://ep2015.europython.eu/en/sponsor/


Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From python at mrabarnett.plus.com  Sun May  3 13:09:20 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 03 May 2015 18:09:20 +0100
Subject: Unicode surrogate pairs (Python 3.4)
In-Reply-To: <slrnmkcj3j.230.jon+usenet@frosty.unequivocal.co.uk>
References: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.67.1430665534.12865.python-list@python.org>
 <slrnmkcftt.230.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.69.1430668429.12865.python-list@python.org>
 <slrnmkcj3j.230.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <55465640.8070600@mrabarnett.plus.com>

On 2015-05-03 17:26, Jon Ribbens wrote:
> On 2015-05-03, MRAB <python at mrabarnett.plus.com> wrote:
>> On 2015-05-03 16:32, Jon Ribbens wrote:
>>> That would, unfortunately, be "tell the Unicode Consortium to format
>>> their documents differently", which seems unlikely to happen. I'm
>>> trying to read in: http://www.unicode.org/Public/idna/6.3.0/IdnaTest.txt
>>>
>> That document looks like it's encoded in UTF-8.
>
> It is. But it also, for reasons best known to the Unicode Consortium,
> contains strings of the form \uXXXX which need to be parsed into the
> appropriate character, and some of *those* are then surrogate pairs,
> which need to be further converted.
>
Ah, so it's r"\udb40\udd9d". :-)

There's also a mistake in this bit:

"""
# Note that according to the \uXXXX escaping convention, a supplemental 
character (> 0x10FFFF) is represented
# by a sequence of two surrogate characters: the first between D800 and 
DBFF, and the second between DC00 and DFFF.
"""



From lucas.bertolotti at yahoo.com  Sun May  3 13:33:25 2015
From: lucas.bertolotti at yahoo.com (lbertolotti)
Date: Sun, 3 May 2015 10:33:25 -0700 (PDT)
Subject: Python xlrd
In-Reply-To: <mailman.50.1430636094.12865.python-list@python.org>
References: <4c202871-d102-4bb0-85c4-3868dac02d03@googlegroups.com>
 <mailman.50.1430636094.12865.python-list@python.org>
Message-ID: <3af767be-8693-41ce-8c7e-5f4ff757c35f@googlegroups.com>

lucas at lucas-K55VD:~$ dpkg -l python-xlrd
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  python-xlrd    0.9.2-1      all          extract data from Microsoft Excel


From tjreedy at udel.edu  Sun May  3 14:00:51 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 03 May 2015 14:00:51 -0400
Subject: ImportPython Newsletter
In-Reply-To: <e5b402f5-b249-41ce-99fd-14a018c9f181@googlegroups.com>
References: <e5b402f5-b249-41ce-99fd-14a018c9f181@googlegroups.com>
Message-ID: <mi5nok$f6p$1@ger.gmane.org>

On 5/3/2015 12:01 PM, Ankur Gupta wrote:
> Hey Guys,
>
> Just like to draw attention to ImportPython a weekly Python
> newsletter. This is the 30th issue of the newsletter
> http://importpython.com/newsletter/no/30/.

Nice, but when I tried to subscribe,
"Unable to reach server"

-- 
Terry Jan Reedy



From dan at tombstonezero.net  Sun May  3 14:14:56 2015
From: dan at tombstonezero.net (Dan Sommers)
Date: Sun, 3 May 2015 18:14:56 +0000 (UTC)
Subject: Python xlrd
References: <4c202871-d102-4bb0-85c4-3868dac02d03@googlegroups.com>
 <mailman.50.1430636094.12865.python-list@python.org>
 <3af767be-8693-41ce-8c7e-5f4ff757c35f@googlegroups.com>
Message-ID: <mi5oj0$8pb$1@dont-email.me>

On Sun, 03 May 2015 10:33:25 -0700, lbertolotti wrote:

> lucas at lucas-K55VD:~$ dpkg -l python-xlrd
> Desired=Unknown/Install/Remove/Purge/Hold
> | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
> |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
> ||/ Name           Version      Architecture Description
> +++-==============-============-============-=================================
> ii  python-xlrd    0.9.2-1      all          extract data from Microsoft Excel

Make sure that the python versions match.  My system has both
python-xlrd and python3-xlrd.

HTH,
Dan


From breamoreboy at yahoo.co.uk  Sun May  3 14:36:25 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 03 May 2015 19:36:25 +0100
Subject: l = range(int(1E9))
In-Reply-To: <CAPTjJmpyv-2zGqKc=Vkb6MsxNqarYbvZBHaT5g408ya8Xqqs9A@mail.gmail.com>
References: <87k2wtvbx1.fsf@Equus.decebal.nl>
 <slrnmk4nmp.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.136.1430414427.3680.python-list@python.org>
 <W061x.692563$396.195269@fx12.am4>
 <mailman.12.1430581268.12865.python-list@python.org>
 <_M61x.477467$Ek.357048@fx07.am4>
 <mailman.15.1430584786.12865.python-list@python.org>
 <bN81x.386497$G53.218869@fx13.am4>
 <mailman.23.1430594149.12865.python-list@python.org>
 <CV91x.402510$zE7.111881@fx40.am4>
 <slrnmkah47.apd.jon+usenet@frosty.unequivocal.co.uk>
 <dhc1x.319346$qW.51334@fx20.am4> <554582E3.4030700@gmail.com>
 <mailman.40.1430619379.12865.python-list@python.org>
 <554603aa$0$12981$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmpyv-2zGqKc=Vkb6MsxNqarYbvZBHaT5g408ya8Xqqs9A@mail.gmail.com>
Message-ID: <mi5pre$em9$1@ger.gmane.org>

On 03/05/2015 12:30, Chris Angelico wrote:
> On Sun, May 3, 2015 at 9:16 PM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>> On Sun, 3 May 2015 12:16 pm, Mark Lawrence wrote:
>>
>>> I doubt that six will ever make the standard library as 2.7 only has
>>> another five years in official support.  By that time I suppose we'll to
>>> going through the porting pain all over again with the transition from
>>> Python 3 to Python 4.  Alright, alright, only joking
>>
>>
>> Guido has said that Python 4 will just be an incremental update from 3.x.
>
> There is still, as I understand it, the possibility that Python 4.0
> will ditch some things that will have been deprecated for a long time
> by 3.9. But yes, it'll be no more upheavalous to go from 3.9 to 4.0
> than from 3.8 to 3.9.
>
> ChrisA
>

https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_GET_SIZE

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From jon+usenet at unequivocal.co.uk  Sun May  3 15:20:04 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sun, 3 May 2015 19:20:04 +0000 (UTC)
Subject: Unicode surrogate pairs (Python 3.4)
References: <slrnmkccs4.apd.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.67.1430665534.12865.python-list@python.org>
 <slrnmkcftt.230.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.69.1430668429.12865.python-list@python.org>
 <slrnmkcj3j.230.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.73.1430672962.12865.python-list@python.org>
Message-ID: <slrnmkct94.230.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-03, MRAB <python at mrabarnett.plus.com> wrote:
> There's also a mistake in this bit:
>
> """
> # Note that according to the \uXXXX escaping convention, a supplemental 
> character (> 0x10FFFF) is represented
> # by a sequence of two surrogate characters: the first between D800 and 
> DBFF, and the second between DC00 and DFFF.
> """

Do you mean that it should say "(> 0xFFFF)" ? Far be it from me to
correct the Unicode Consortium on the subject of Unicode ;-)


From mal at europython.eu  Mon May  4 03:39:19 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Mon, 04 May 2015 09:39:19 +0200
Subject: EuroPython 2015: Vote for the talks you want to see
Message-ID: <55472227.3070007@europython.eu>

Having received over 300 great proposals for talks, trainings,
helpdesks and posters, we now call out to all attendees to vote for
what you want to see on the conference schedule.

You can search for topics and communicate your personal priorities by
casting your vote for each submission on our talk voting page:


    *** ?Attendees: This is your chance to shape the conference ! ***

             https://ep2015.europython.eu/en/talk-voting/

Talk voting will be open until Friday, May 15.

The program workgroup (WG) will then use the talk voting results as
basis for their talk selection and announce the schedule late in May.

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From Cecil at decebal.nl  Mon May  4 04:58:40 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 10:58:40 +0200
Subject: Cannot update OpenSSL for Python3
Message-ID: <87k2wok9bz.fsf@Equus.decebal.nl>

But when I do:
    import urllib3.contrib.pyopenssl
I get:
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.4/site-packages/urllib3/contrib/pyopenssl.py", line 55, in <module>
        import OpenSSL.SSL
      File "/usr/lib64/python3.4/site-packages/OpenSSL/__init__.py", line 8, in <module>
        from OpenSSL import rand, crypto, SSL
      File "/usr/lib64/python3.4/site-packages/OpenSSL/rand.py", line 9, in <module>
        from six import integer_types as _integer_types
    ImportError: No module named 'six'

When I then give:
    pip3 install -U OpenSSL
It goes wrong:
    Could not fetch URL https://pypi.python.org/simple/OpenSSL/: 404 Client Error: Not Found

I checked and even
    https://pypi.python.org/simple/
does not exist. Anyone an idea what is happening here?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From arekfu at gmail.com  Mon May  4 05:50:35 2015
From: arekfu at gmail.com (arekfu at gmail.com)
Date: Mon, 4 May 2015 02:50:35 -0700 (PDT)
Subject: when does newlines get set in universal newlines mode?
Message-ID: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>

Hi all,

I have a text file with Windows-style line terminators (\r\n) which I open in universal newlines mode (Python 2.7). I would expect the newlines attribute to be set after the first call to the readline() method, but apparently this is not the case:

>>> f=open('test_crlf', 'rU')
>>> f.newlines
>>> f.readline()
'foo\n'
>>> f.newlines
>>> f.readline()
'bar\n'
>>> f.newlines
'\r\n'
On the other hand, the newlines attribute gets set after the first call to readline() on a file with Unix-style line endings.

Is this a bug or a feature?

Thanks in advance,
Davide


From rosuav at gmail.com  Mon May  4 06:10:30 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 4 May 2015 20:10:30 +1000
Subject: Cannot update OpenSSL for Python3
In-Reply-To: <87k2wok9bz.fsf@Equus.decebal.nl>
References: <87k2wok9bz.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmpysqW0kq7kE7ieA0QmCFTyHQHcwsY54ZJ9e-Yjn70EHw@mail.gmail.com>

On Mon, May 4, 2015 at 6:58 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> When I then give:
>     pip3 install -U OpenSSL
> It goes wrong:
>     Could not fetch URL https://pypi.python.org/simple/OpenSSL/: 404 Client Error: Not Found
>
> I checked and even
>     https://pypi.python.org/simple/
> does not exist. Anyone an idea what is happening here?

I think what you want is called pyOpenSSL, not just OpenSSL:

https://pypi.python.org/pypi/pyOpenSSL
https://pypi.python.org/simple/pyopenssl/

Not sure why /simple/ doesn't work, but you're not normally meant to
grab that page manually - it's for script work. You could raise a
tracker issue about that if you like, but it may not be considered
important.

Does 'pip3 install -U pyOpenSSL' work?

ChrisA


From Cecil at decebal.nl  Mon May  4 07:31:16 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 13:31:16 +0200
Subject: Why do I get SyntaxError: invalid syntax
Message-ID: <87d22gk29n.fsf@Equus.decebal.nl>

While copying pasting code to test, the following works:
    from itertools  import islice
    from os         import rename
    from os.path    import expanduser, split
    from tempfile   import NamedTemporaryFile

    real_file = (expanduser('~/Twitter/testing.txt'))
    (filepath,
     file)      = split(real_file)
    with NamedTemporaryFile(mode = 'w', prefix = file + '_', dir = filepath, delete = False) as tf:
        tempfile = tf.name
        with open(real_file, 'r') as f:
            for line in islice(f, 1, None):
                tf.write(line)

    rename(tempfile, real_file)

But first I used:
    from itertools  import islice
    from os         import rename
    from os.path    import expanduser, split
    from tempfile   import NamedTemporaryFile

    real_file = (expanduser('~/Twitter/testing.txt'))
    (filepath,
     file)      = split(real_file)
    with NamedTemporaryFile(mode = 'w', prefix = file + '_', dir = filepath, delete = False) as tf:
        tempfile = tf.name
        with open(real_file, 'r') as f:
            for line in islice(f, 1, None):
                tf.write(line)
    rename(tempfile, real_file)

But that gave:
      File "<stdin>", line 6
        rename(tempfile, real_file)
             ^
    SyntaxError: invalid syntax

Why?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Mon May  4 07:32:51 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 13:32:51 +0200
Subject: Cannot update OpenSSL for Python3
References: <87k2wok9bz.fsf@Equus.decebal.nl>
 <mailman.80.1430734238.12865.python-list@python.org>
Message-ID: <878ud4k270.fsf@Equus.decebal.nl>

Op Monday 4 May 2015 12:10 CEST schreef Chris Angelico:

> On Mon, May 4, 2015 at 6:58 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> When I then give: pip3 install -U OpenSSL It goes wrong: Could not
>> fetch URL https://pypi.python.org/simple/OpenSSL/: 404 Client
>> Error: Not Found
>>
>> I checked and even
>> https://pypi.python.org/simple/
>> does not exist. Anyone an idea what is happening here?
>
> I think what you want is called pyOpenSSL, not just OpenSSL:
>
> https://pypi.python.org/pypi/pyOpenSSL
> https://pypi.python.org/simple/pyopenssl/
>
> Not sure why /simple/ doesn't work, but you're not normally meant to
> grab that page manually - it's for script work. You could raise a
> tracker issue about that if you like, but it may not be considered
> important.
>
> Does 'pip3 install -U pyOpenSSL' work?
Not really, because that gives:
    Requirement already up-to-date: pyOpenSSL in /usr/lib64/python3.4/site-packages
    Cleaning up...

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From __peter__ at web.de  Mon May  4 08:01:05 2015
From: __peter__ at web.de (Peter Otten)
Date: Mon, 04 May 2015 14:01:05 +0200
Subject: when does newlines get set in universal newlines mode?
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
Message-ID: <mi7n22$mbc$1@ger.gmane.org>

arekfu at gmail.com wrote:

> Hi all,
> 
> I have a text file with Windows-style line terminators (\r\n) which I open
> in universal newlines mode (Python 2.7). I would expect the newlines
> attribute to be set after the first call to the readline() method, but
> apparently this is not the case:
> 
>>>> f=open('test_crlf', 'rU')
>>>> f.newlines
>>>> f.readline()
> 'foo\n'
>>>> f.newlines
>>>> f.readline()
> 'bar\n'
>>>> f.newlines
> '\r\n'
> On the other hand, the newlines attribute gets set after the first call to
> readline() on a file with Unix-style line endings.
> 
> Is this a bug or a feature?

According to

https://docs.python.org/2.7/library/functions.html#open

"""
If Python is built without universal newlines support a mode with 'U' is the 
same as normal text mode. Note that file objects so opened also have an 
attribute called newlines which has a value of None (if no newlines have yet 
been seen), '\n', '\r', '\r\n', or a tuple containing all the newline types 
seen.
"""

I tried:

>>> with open("tmp.txt", "wb") as f: f.write("alpha\r\nbeta\rgamma\n")
... 
>>> f = open("tmp.txt", "rU")
>>> f.newlines
>>> f.readline()
'alpha\n'
>>> f.newlines 
# expected: '\r\n'
>>> f.readline()
'beta\n'
>>> f.newlines
'\r\n' # expected: ('\r', '\r\n')
>>> f.readline()
'gamma\n'
>>> f.newlines
('\r', '\n', '\r\n')

I believe this is a bug.



From rosuav at gmail.com  Mon May  4 08:07:41 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 4 May 2015 22:07:41 +1000
Subject: Why do I get SyntaxError: invalid syntax
In-Reply-To: <87d22gk29n.fsf@Equus.decebal.nl>
References: <87d22gk29n.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmpn_ym3y5-OyyhraL=-xkRbwftQxe2WBXZ2o27KBDCsLw@mail.gmail.com>

On Mon, May 4, 2015 at 9:31 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> While copying pasting code to test, the following works:
> [chomp]
> But first I used:
>     with NamedTemporaryFile(mode = 'w', prefix = file + '_', dir = filepath, delete = False) as tf:
>         tempfile = tf.name
>         with open(real_file, 'r') as f:
>             for line in islice(f, 1, None):
>                 tf.write(line)
>     rename(tempfile, real_file)
>
> But that gave:
>       File "<stdin>", line 6
>         rename(tempfile, real_file)
>              ^
>     SyntaxError: invalid syntax
>
> Why?

To clarify: When you say "to test", you mean the interactive
interpreter, right? If so, you need to end blocks of text with blank
lines (and not have any blank lines in between). It's because the
parser has to know when to run stuff; when you run a script, it parses
the whole thing and then runs it, but interactively, it has to work
piece-meal.

ChrisA


From rosuav at gmail.com  Mon May  4 08:13:31 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 4 May 2015 22:13:31 +1000
Subject: when does newlines get set in universal newlines mode?
In-Reply-To: <mi7n22$mbc$1@ger.gmane.org>
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
 <mi7n22$mbc$1@ger.gmane.org>
Message-ID: <CAPTjJmp_5uZE1Zm2DVOG1CHDGYr9i1jLAOVaRaOVf8eXkE=btw@mail.gmail.com>

On Mon, May 4, 2015 at 10:01 PM, Peter Otten <__peter__ at web.de> wrote:
> I tried:
>
>>>> with open("tmp.txt", "wb") as f: f.write("alpha\r\nbeta\rgamma\n")
> ...
>>>> f = open("tmp.txt", "rU")
>>>> f.newlines
>>>> f.readline()
> 'alpha\n'
>>>> f.newlines
> # expected: '\r\n'
>>>> f.readline()
> 'beta\n'
>>>> f.newlines
> '\r\n' # expected: ('\r', '\r\n')
>>>> f.readline()
> 'gamma\n'
>>>> f.newlines
> ('\r', '\n', '\r\n')
>
> I believe this is a bug.

I'm not sure it is, actually; imagine the text is coming in one
character at a time (eg from a pipe), and it's seen "alpha\r". It
knows that this is a line, so it emits it; but until the next
character is read, it can't know whether it's going to be \r or \r\n.
What should it do? Read another character, which might block? Put "\r"
into .newlines, which might be wrong? Once it sees the \n, it knows
that it was \r\n (or rather, it assumes that files do not have lines
of text terminated by \r followed by blank lines terminated by \n -
because that would be stupid).

It may be worth documenting this limitation, but it's not something
that can easily be fixed without removing support for \r newlines -
although that might be an option, given that non-OSX Macs are
basically history now.

ChrisA


From rosuav at gmail.com  Mon May  4 08:14:59 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 4 May 2015 22:14:59 +1000
Subject: Cannot update OpenSSL for Python3
In-Reply-To: <878ud4k270.fsf@Equus.decebal.nl>
References: <87k2wok9bz.fsf@Equus.decebal.nl>
 <mailman.80.1430734238.12865.python-list@python.org>
 <878ud4k270.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmoMrkS6kuPMwn=NELceJCh0KqWcTT6=B4zCuH0j12kueQ@mail.gmail.com>

On Mon, May 4, 2015 at 9:32 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> Does 'pip3 install -U pyOpenSSL' work?
> Not really, because that gives:
>     Requirement already up-to-date: pyOpenSSL in /usr/lib64/python3.4/site-packages
>     Cleaning up...

I don't know why it wasn't automatically installed, but 'six' is a
listed dependency of pyOpenSSL. What happens if you try to install
six?

ChrisA


From pythonistaforhire at gmail.com  Mon May  4 08:43:23 2015
From: pythonistaforhire at gmail.com (Alex McFerron)
Date: Mon, 4 May 2015 05:43:23 -0700
Subject: ImportPython Newsletter
In-Reply-To: <mi5nok$f6p$1@ger.gmane.org>
References: <e5b402f5-b249-41ce-99fd-14a018c9f181@googlegroups.com>
 <mi5nok$f6p$1@ger.gmane.org>
Message-ID: <059CCA62-F32B-49D4-B9F7-5ECF362BB143@gmail.com>

I had the same problem just now

Sent from my iPhone

> On May 3, 2015, at 11:00 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> 
>> On 5/3/2015 12:01 PM, Ankur Gupta wrote:
>> Hey Guys,
>> 
>> Just like to draw attention to ImportPython a weekly Python
>> newsletter. This is the 30th issue of the newsletter
>> http://importpython.com/newsletter/no/30/.
> 
> Nice, but when I tried to subscribe,
> "Unable to reach server"
> 
> -- 
> Terry Jan Reedy
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From briansgoins at yahoo.com  Mon May  4 09:08:09 2015
From: briansgoins at yahoo.com (briansgoins)
Date: Mon, 4 May 2015 06:08:09 -0700 (PDT)
Subject: http://premiumnaturalgarciniacambogiahelp.com/new-slim-5-garcinia-cambogia/
Message-ID: <1430744889561-5094018.post@n6.nabble.com>

New Slim 5 Garcinia Cambogia You can find plenty of Weight Loss tips one sees
around in company, infomercials, TV and publications. The stark reality is
that many of these tips actually work and a few really do not. To be honest
with you, the fat diets, weight products and pills recommendations or goods
are primarily those who don't function. Those who work so are fairly simple
to accomplish and are in reality easy to discover. They're largely
recommendations on DIETS and WORKOUTS. But I must contact your attention to
the recognition that the Weight Loss supplement marketplace is saturated in
the reality most situations is complicated to like the item being offered as
well as misleading tips and plenty of silly lies.

http://premiumnaturalgarciniacambogiahelp.com/new-slim-5-garcinia-cambogia/






--
View this message in context: http://python.6.x6.nabble.com/http-premiumnaturalgarciniacambogiahelp-com-new-slim-5-garcinia-cambogia-tp5094018.html
Sent from the Python - python-list mailing list archive at Nabble.com.


From Cecil at decebal.nl  Mon May  4 09:13:09 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 15:13:09 +0200
Subject: Cannot update OpenSSL for Python3
References: <87k2wok9bz.fsf@Equus.decebal.nl>
 <mailman.80.1430734238.12865.python-list@python.org>
 <878ud4k270.fsf@Equus.decebal.nl>
 <mailman.84.1430741703.12865.python-list@python.org>
Message-ID: <874mnsjxju.fsf@Equus.decebal.nl>

Op Monday 4 May 2015 14:14 CEST schreef Chris Angelico:

> On Mon, May 4, 2015 at 9:32 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>> Does 'pip3 install -U pyOpenSSL' work?
>> Not really, because that gives: Requirement already up-to-date:
>> pyOpenSSL in /usr/lib64/python3.4/site-packages Cleaning up...
>
> I don't know why it wasn't automatically installed, but 'six' is a
> listed dependency of pyOpenSSL. What happens if you try to install
> six?

That gets installed. And then I get:
    ImportError: No module named 'cryptography'

So I try to install that. This gives:
    Command /usr/bin/python3 -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-_7jexj87-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/cryptography
Storing debug log for failure in /root/.pip/pip.log

In the log I see:
    c/_cffi_backend.c:2:20: fatal error: Python.h: No such file or directory
     #include <Python.h>
                        ^

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Mon May  4 09:16:24 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 15:16:24 +0200
Subject: Why do I get SyntaxError: invalid syntax
References: <87d22gk29n.fsf@Equus.decebal.nl>
 <mailman.82.1430741264.12865.python-list@python.org>
Message-ID: <87zj5kiitz.fsf@Equus.decebal.nl>

Op Monday 4 May 2015 14:07 CEST schreef Chris Angelico:

> On Mon, May 4, 2015 at 9:31 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> While copying pasting code to test, the following works: [chomp]
>> But first I used: with NamedTemporaryFile(mode = 'w', prefix = file
>> + '_', dir = filepath, delete = False) as tf: tempfile = tf.name
>> with open(real_file, 'r') as f: for line in islice(f, 1, None):
>> tf.write(line) rename(tempfile, real_file)
>>
>> But that gave:
>> File "<stdin>", line 6
>> rename(tempfile, real_file)
>> ^
>> SyntaxError: invalid syntax
>>
>> Why?
>
> To clarify: When you say "to test", you mean the interactive
> interpreter, right?

Yes, that is what I mend. Should have been clearer.


> If so, you need to end blocks of text with blank
> lines (and not have any blank lines in between). It's because the
> parser has to know when to run stuff; when you run a script, it
> parses the whole thing and then runs it, but interactively, it has
> to work piece-meal.

OK, thanks: I understand it now.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From arekfu at gmail.com  Mon May  4 09:35:46 2015
From: arekfu at gmail.com (Davide Mancusi)
Date: Mon, 4 May 2015 06:35:46 -0700 (PDT)
Subject: when does newlines get set in universal newlines mode?
In-Reply-To: <mailman.83.1430741619.12865.python-list@python.org>
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
 <mi7n22$mbc$1@ger.gmane.org>
 <mailman.83.1430741619.12865.python-list@python.org>
Message-ID: <b41e75a7-d890-4ff8-b83b-2e8e223137b5@googlegroups.com>

>> I believe this is a bug.
>
> I'm not sure it is, actually; imagine the text is coming in one
> character at a time (eg from a pipe), and it's seen "alpha\r". It
> knows that this is a line, so it emits it; but until the next
> character is read, it can't know whether it's going to be \r or \r\n.
> What should it do? Read another character, which might block? Put "\r"
> into .newlines, which might be wrong? Once it sees the \n, it knows
> that it was \r\n (or rather, it assumes that files do not have lines
> of text terminated by \r followed by blank lines terminated by \n -
> because that would be stupid).

I think this is a good point. However, I will probably submit a bug
report anyway and let the devs make their decisions. It is at least a
documentation bug.

Cheers,
Davide


From rosuav at gmail.com  Mon May  4 10:18:53 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 5 May 2015 00:18:53 +1000
Subject: Cannot update OpenSSL for Python3
In-Reply-To: <874mnsjxju.fsf@Equus.decebal.nl>
References: <87k2wok9bz.fsf@Equus.decebal.nl>
 <mailman.80.1430734238.12865.python-list@python.org>
 <878ud4k270.fsf@Equus.decebal.nl>
 <mailman.84.1430741703.12865.python-list@python.org>
 <874mnsjxju.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmoxNY6SYsLBUU8zUArH5cFppgdi1bm0Oh+6Uq2+Ko1LhQ@mail.gmail.com>

On Mon, May 4, 2015 at 11:13 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> That gets installed. And then I get:
>     ImportError: No module named 'cryptography'
>
> So I try to install that. This gives:
>     Command /usr/bin/python3 -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-_7jexj87-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/cryptography
> Storing debug log for failure in /root/.pip/pip.log
>
> In the log I see:
>     c/_cffi_backend.c:2:20: fatal error: Python.h: No such file or directory
>      #include <Python.h>

Okay, that one's easy enough to deal with!

You have something that needs to build a C extension. To do that, you
need to have the Python headers installed. How did you install Python?
On Debian/Ubuntu family Linuxes, that's probably "apt-get install
python3" - so getting the headers would be "apt-get install
python3-dev". Give that a try, and then retry the pip install.

ChrisA


From Cecil at decebal.nl  Mon May  4 11:11:51 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 17:11:51 +0200
Subject: Cannot update OpenSSL for Python3
References: <87k2wok9bz.fsf@Equus.decebal.nl>
 <mailman.80.1430734238.12865.python-list@python.org>
 <878ud4k270.fsf@Equus.decebal.nl>
 <mailman.84.1430741703.12865.python-list@python.org>
 <874mnsjxju.fsf@Equus.decebal.nl>
 <mailman.89.1430749136.12865.python-list@python.org>
Message-ID: <87vbg8idhk.fsf@Equus.decebal.nl>

Op Monday 4 May 2015 16:18 CEST schreef Chris Angelico:

> On Mon, May 4, 2015 at 11:13 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> That gets installed. And then I get:
>> ImportError: No module named 'cryptography'
>>
>> So I try to install that. This gives: Command /usr/bin/python3 -c
>> "import setuptools,
>> tokenize;__file__='/tmp/pip_build_root/cryptography/setup.py';exec(compile(getattr(tokenize,
>> 'open', open)(__file__).read().replace('\r\n', '\n'), __file__,
>> 'exec'))" install --record
>> /tmp/pip-_7jexj87-record/install-record.txt
>> --single-version-externally-managed --compile failed with error
>> code 1 in /tmp/pip_build_root/cryptography Storing debug log for
>> failure in /root/.pip/pip.log
>>
>> In the log I see: c/_cffi_backend.c:2:20: fatal error: Python.h: No
>> such file or directory #include <Python.h>
>
> Okay, that one's easy enough to deal with!
>
> You have something that needs to build a C extension. To do that,
> you need to have the Python headers installed. How did you install
> Python? On Debian/Ubuntu family Linuxes, that's probably "apt-get
> install python3" - so getting the headers would be "apt-get install
> python3-dev". Give that a try, and then retry the pip install.

I should have thought about that myself. :-(

Now I get:
    c/../_cffi1/ffi_obj.c:489:5: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
         PyObject *u = PyUnicode_DecodeLatin1(PyBytes_AS_STRING(res),
         ^
    cc1: some warnings being treated as errors

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From __peter__ at web.de  Mon May  4 11:17:21 2015
From: __peter__ at web.de (Peter Otten)
Date: Mon, 04 May 2015 17:17:21 +0200
Subject: when does newlines get set in universal newlines mode?
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
 <mi7n22$mbc$1@ger.gmane.org>
 <CAPTjJmp_5uZE1Zm2DVOG1CHDGYr9i1jLAOVaRaOVf8eXkE=btw@mail.gmail.com>
Message-ID: <mi82i2$tlo$1@ger.gmane.org>

Chris Angelico wrote:

> On Mon, May 4, 2015 at 10:01 PM, Peter Otten <__peter__ at web.de> wrote:
>> I tried:
>>
>>>>> with open("tmp.txt", "wb") as f: f.write("alpha\r\nbeta\rgamma\n")
>> ...
>>>>> f = open("tmp.txt", "rU")
>>>>> f.newlines
>>>>> f.readline()
>> 'alpha\n'
>>>>> f.newlines
>> # expected: '\r\n'
>>>>> f.readline()
>> 'beta\n'
>>>>> f.newlines
>> '\r\n' # expected: ('\r', '\r\n')
>>>>> f.readline()
>> 'gamma\n'
>>>>> f.newlines
>> ('\r', '\n', '\r\n')
>>
>> I believe this is a bug.
> 
> I'm not sure it is, actually; imagine the text is coming in one
> character at a time (eg from a pipe), and it's seen "alpha\r". It
> knows that this is a line, so it emits it; but until the next
> character is read, it can't know whether it's going to be \r or \r\n.
> What should it do? Read another character, which might block? Put "\r"
> into .newlines, which might be wrong? Once it sees the \n, it knows
> that it was \r\n (or rather, it assumes that files do not have lines
> of text terminated by \r followed by blank lines terminated by \n -
> because that would be stupid).
> 
> It may be worth documenting this limitation, but it's not something
> that can easily be fixed without removing support for \r newlines -
> although that might be an option, given that non-OSX Macs are
> basically history now.

OK, you convinced me. Then I tried:

>>> with open("tmp.txt", "wb") as f: f.write("0\r\n3\r5\n7")
... 
>>> assert len(open("tmp.txt", "rb").read()) == 8
>>> f = open("tmp.txt", "rU")
>>> f.readline()
'0\n'
>>> f.newlines
>>> f.tell()
3
>>> f.newlines
'\r\n'

Hm, so tell() moves the file pointer? Is that sane?



From Cecil at decebal.nl  Mon May  4 11:20:05 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 17:20:05 +0200
Subject: Bitten by my C/Java experience
Message-ID: <87r3qwid3u.fsf@Equus.decebal.nl>

Potential dangerous bug introduced by programming in Python as if it
was C/Java. :-(
I used:
    ++tries
that has to be:
    tries += 1

Are there other things I have to be careful on? That does not work as
in C/Java, but is correct syntax.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From rosuav at gmail.com  Mon May  4 11:26:45 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 5 May 2015 01:26:45 +1000
Subject: when does newlines get set in universal newlines mode?
In-Reply-To: <mi82i2$tlo$1@ger.gmane.org>
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
 <mi7n22$mbc$1@ger.gmane.org>
 <CAPTjJmp_5uZE1Zm2DVOG1CHDGYr9i1jLAOVaRaOVf8eXkE=btw@mail.gmail.com>
 <mi82i2$tlo$1@ger.gmane.org>
Message-ID: <CAPTjJmrkAO6o-Cm46-KmbC1Fm5b7O-WbWWwvOGHJk4wf=XNwWw@mail.gmail.com>

On Tue, May 5, 2015 at 1:17 AM, Peter Otten <__peter__ at web.de> wrote:
> OK, you convinced me. Then I tried:
>
>>>> with open("tmp.txt", "wb") as f: f.write("0\r\n3\r5\n7")
> ...
>>>> assert len(open("tmp.txt", "rb").read()) == 8
>>>> f = open("tmp.txt", "rU")
>>>> f.readline()
> '0\n'
>>>> f.newlines
>>>> f.tell()
> 3
>>>> f.newlines
> '\r\n'
>
> Hm, so tell() moves the file pointer? Is that sane?

... wow. Okay! That's a bit weird.

It's possible that something's being done with internal buffering
(after all, it's horribly inefficient to *actually* read text one byte
at a time, even if that's what's happening conceptually), and that
tell() causes some checks to be done. But that really is rather
strange. I'd be interested to know what happens if another process
writes to a pipe "0\r", then sleeps while the readline() and tell()
happen, and then writes a "\n" - what will that do to newlines?

By the way, it's as well to clarify, with all these examples, what
Python version you're using. There may be significant differences.

ChrisA


From ian.g.kelly at gmail.com  Mon May  4 11:33:50 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 4 May 2015 09:33:50 -0600
Subject: when does newlines get set in universal newlines mode?
In-Reply-To: <mi82i2$tlo$1@ger.gmane.org>
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
 <mi7n22$mbc$1@ger.gmane.org>
 <CAPTjJmp_5uZE1Zm2DVOG1CHDGYr9i1jLAOVaRaOVf8eXkE=btw@mail.gmail.com>
 <mi82i2$tlo$1@ger.gmane.org>
Message-ID: <CALwzid=3SLsm6X4NwC4euhgouVE2j-Q4tEZFYk3Fjsbrv6jrbA@mail.gmail.com>

On Mon, May 4, 2015 at 9:17 AM, Peter Otten <__peter__ at web.de> wrote:
> OK, you convinced me. Then I tried:
>
>>>> with open("tmp.txt", "wb") as f: f.write("0\r\n3\r5\n7")
> ...
>>>> assert len(open("tmp.txt", "rb").read()) == 8
>>>> f = open("tmp.txt", "rU")
>>>> f.readline()
> '0\n'
>>>> f.newlines
>>>> f.tell()
> 3
>>>> f.newlines
> '\r\n'
>
> Hm, so tell() moves the file pointer? Is that sane?

If I call readline() followed by tell(), I expect the result to be the
position of the start of the next line. Maybe this is considered safe
because tell() on a pipe raises an exception?


From rosuav at gmail.com  Mon May  4 12:03:37 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 5 May 2015 02:03:37 +1000
Subject: Cannot update OpenSSL for Python3
In-Reply-To: <87vbg8idhk.fsf@Equus.decebal.nl>
References: <87k2wok9bz.fsf@Equus.decebal.nl>
 <mailman.80.1430734238.12865.python-list@python.org>
 <878ud4k270.fsf@Equus.decebal.nl>
 <mailman.84.1430741703.12865.python-list@python.org>
 <874mnsjxju.fsf@Equus.decebal.nl>
 <mailman.89.1430749136.12865.python-list@python.org>
 <87vbg8idhk.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmpYfXm+mG=0uN7-qEG9vf+paTrQSwb_QrbVFxP0mWfk4w@mail.gmail.com>

On Tue, May 5, 2015 at 1:11 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> Now I get:
>     c/../_cffi1/ffi_obj.c:489:5: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
>          PyObject *u = PyUnicode_DecodeLatin1(PyBytes_AS_STRING(res),
>          ^
>     cc1: some warnings being treated as errors

Interesting. I'm not sure why yours is complaining about that; mine
doesn't. (Possibly because I'm running Python 3.5, and stuff may have
been changed.) In any case, this would be a reasonable thing to make a
bug report about. In the meantime, you can simply override that
warning-equals-error parameter:

http://stackoverflow.com/questions/25587039/error-compiling-rpy2-on-python3-4-due-to-werror-declaration-after-statement

ChrisA


From toby at tobiah.org  Mon May  4 13:18:47 2015
From: toby at tobiah.org (Tobiah)
Date: Mon, 04 May 2015 10:18:47 -0700
Subject: Bitten by my C/Java experience
References: <87r3qwid3u.fsf@Equus.decebal.nl>
Message-ID: <mi89lm$daj$1@speranza.aioe.org>

On 05/04/2015 08:20 AM, Cecil Westerhof wrote:
> Potential dangerous bug introduced by programming in Python as if it
> was C/Java. :-(
> I used:
>      ++tries
> that has to be:
>      tries += 1
>
> Are there other things I have to be careful on? That does not work as
> in C/Java, but is correct syntax.
>

One surprise for the new user is an otherwise handy rule of scope.
A variable in a function will by default access any global variables of
the same name *unless* it is assigned to in the function.

def glob():
         print "global:", foo

def loc():
         foo = 2
         print "local:", foo

def alt():
         global foo
         foo = 1
         print "altered:", foo

foo = 3

glob()
print "Original:", foo

loc()
print "Original:", foo

alt()
print "Original:", foo

################# Output ##################

global: 3
Original: 3
local: 2
Original: 3
altered: 1
Original: 1


From Cecil at decebal.nl  Mon May  4 13:19:00 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 19:19:00 +0200
Subject: Cannot update OpenSSL for Python3
References: <87k2wok9bz.fsf@Equus.decebal.nl>
 <mailman.80.1430734238.12865.python-list@python.org>
 <878ud4k270.fsf@Equus.decebal.nl>
 <mailman.84.1430741703.12865.python-list@python.org>
 <874mnsjxju.fsf@Equus.decebal.nl>
 <mailman.89.1430749136.12865.python-list@python.org>
 <87vbg8idhk.fsf@Equus.decebal.nl>
 <mailman.94.1430755420.12865.python-list@python.org>
Message-ID: <87mw1ki7ln.fsf@Equus.decebal.nl>

Op Monday 4 May 2015 18:03 CEST schreef Chris Angelico:

> On Tue, May 5, 2015 at 1:11 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> Now I get: c/../_cffi1/ffi_obj.c:489:5: error: ISO C90 forbids
>> mixed declarations and code [-Werror=declaration-after-statement]
>> PyObject *u = PyUnicode_DecodeLatin1(PyBytes_AS_STRING(res), ^ cc1:
>> some warnings being treated as errors
>
> Interesting. I'm not sure why yours is complaining about that; mine
> doesn't. (Possibly because I'm running Python 3.5, and stuff may
> have been changed.) In any case, this would be a reasonable thing to
> make a bug report about. In the meantime, you can simply override
> that warning-equals-error parameter:
>
> http://stackoverflow.com/questions/25587039/error-compiling-rpy2-on-python3-4-due-to-werror-declaration-after-statement

It looks like I am encircled by Gremlins:
    >>> import urllib3.contrib.pyopenssl
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.4/site-packages/urllib3/contrib/pyopenssl.py", line 58, in <module>
        from socket import _fileobject, timeout
    ImportError: cannot import name '_fileobject'

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From irmen.NOSPAM at xs4all.nl  Mon May  4 13:32:55 2015
From: irmen.NOSPAM at xs4all.nl (Irmen de Jong)
Date: Mon, 04 May 2015 19:32:55 +0200
Subject: Bitten by my C/Java experience
In-Reply-To: <87r3qwid3u.fsf@Equus.decebal.nl>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
Message-ID: <5547ad48$0$2864$e4fe514c@news.xs4all.nl>

On 4-5-2015 17:20, Cecil Westerhof wrote:
> Potential dangerous bug introduced by programming in Python as if it
> was C/Java. :-(
> I used:
>     ++tries
> that has to be:
>     tries += 1
> 
> Are there other things I have to be careful on? That does not work as
> in C/Java, but is correct syntax.
> 

That is a broad question, but one thing that comes to mind is the current (python 3)
behavior of integer division. It gives the exact result and doesn't truncate to integers:


>>> 5/4
1.25


To be prepared for the future you should probably use python's time machine and enable
this behavior for 2.x as well by from __future__ import division.


Another thing is that functions are first class citizens in Python. Java will give a
compiler error if you forget to call them and leave out the parentheses (I think Java 8
allows it though). Python will accept passing them on as a function object just fine. If
you do this by mistake you will probably get an exception a tiny bit down the line, at
runtime. But it is syntactically correct so your code will compile without error.


-irmen



From tjreedy at udel.edu  Mon May  4 13:38:43 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 04 May 2015 13:38:43 -0400
Subject: when does newlines get set in universal newlines mode?
In-Reply-To: <b41e75a7-d890-4ff8-b83b-2e8e223137b5@googlegroups.com>
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
 <mi7n22$mbc$1@ger.gmane.org>
 <mailman.83.1430741619.12865.python-list@python.org>
 <b41e75a7-d890-4ff8-b83b-2e8e223137b5@googlegroups.com>
Message-ID: <mi8ar4$7rj$1@ger.gmane.org>

On 5/4/2015 9:35 AM, Davide Mancusi wrote:
>>> I believe this is a bug.
>>
>> I'm not sure it is, actually; imagine the text is coming in one
>> character at a time (eg from a pipe), and it's seen "alpha\r". It
>> knows that this is a line, so it emits it; but until the next
>> character is read, it can't know whether it's going to be \r or \r\n.
>> What should it do? Read another character, which might block? Put "\r"
>> into .newlines, which might be wrong? Once it sees the \n, it knows
>> that it was \r\n (or rather, it assumes that files do not have lines
>> of text terminated by \r followed by blank lines terminated by \n -
>> because that would be stupid).
>
> I think this is a good point. However, I will probably submit a bug
> report anyway and let the devs make their decisions. It is at least a
> documentation bug.

Be sure to report the exact python binary you are using, as reported 
when you start the interactive interpreter or Idle shell.

-- 
Terry Jan Reedy



From rosuav at gmail.com  Mon May  4 13:40:50 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 5 May 2015 03:40:50 +1000
Subject: Bitten by my C/Java experience
In-Reply-To: <5547ad48$0$2864$e4fe514c@news.xs4all.nl>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <5547ad48$0$2864$e4fe514c@news.xs4all.nl>
Message-ID: <CAPTjJmpBO_z5tCqgNviFA1HnY8bw=8c0G5Tprw9roVTGmu3a1w@mail.gmail.com>

On Tue, May 5, 2015 at 3:32 AM, Irmen de Jong <irmen.NOSPAM at xs4all.nl> wrote:
> That is a broad question, but one thing that comes to mind is the current (python 3)
> behavior of integer division. It gives the exact result and doesn't truncate to integers:
>
>
>>>> 5/4
> 1.25

Using the word "exact" around non-integer values can be a little
ambiguous, since floats are often inexact. But yes, int/int -> float,
and yes, it WILL bite C programmers.

ChrisA


From ian.g.kelly at gmail.com  Mon May  4 13:43:33 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 4 May 2015 11:43:33 -0600
Subject: Bitten by my C/Java experience
In-Reply-To: <87r3qwid3u.fsf@Equus.decebal.nl>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
Message-ID: <CALwzidnnb1bjxxLJ8XvqikVBkHFBXwvgH8MPRXD3nK_O7FwwNA@mail.gmail.com>

On Mon, May 4, 2015 at 9:20 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> Potential dangerous bug introduced by programming in Python as if it
> was C/Java. :-(
> I used:
>     ++tries
> that has to be:
>     tries += 1
>
> Are there other things I have to be careful on? That does not work as
> in C/Java, but is correct syntax.

Some other gotchas that aren't necessarily related to C/Java but can
be surprising nonetheless:

*    () is a zero-element tuple, and (a, b) is a two-element tuple,
but (a) is not a one-element tuple. Tuples are created by commas, not
parentheses, so use (a,) instead.

*    Default function arguments are created at definition time, not at
call time. So if you do something like:

    def foo(a, b=[]):
        b.append(a)
        print(b)

The b list will be the same list on each call and will retain all
changes from previous calls.

*    super() doesn't do what you might expect in multiple inheritance
situations, particularly if you're coming from Java where you never
have to deal with multiple inheritance. It binds to the next class in
the method resolution order, *not* necessarily the immediate
superclass. This also means that the particular class bound to can
vary depending on the specific class of the object.

*    [[None] * 8] * 8 doesn't create a 2-dimensional array of None. It
creates one list containing None 8 times, and then it creates a second
list containing the first list 8 times, *not* a list of 8 distinct
lists.

*    If some_tuple is a tuple containing a list, then some_tuple[0] +=
['foo'] will concatenate the list *but* will also raise a TypeError
when it tries to reassign the list back to the tuple.


From rosuav at gmail.com  Mon May  4 13:49:37 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 5 May 2015 03:49:37 +1000
Subject: Cannot update OpenSSL for Python3
In-Reply-To: <87mw1ki7ln.fsf@Equus.decebal.nl>
References: <87k2wok9bz.fsf@Equus.decebal.nl>
 <mailman.80.1430734238.12865.python-list@python.org>
 <878ud4k270.fsf@Equus.decebal.nl>
 <mailman.84.1430741703.12865.python-list@python.org>
 <874mnsjxju.fsf@Equus.decebal.nl>
 <mailman.89.1430749136.12865.python-list@python.org>
 <87vbg8idhk.fsf@Equus.decebal.nl>
 <mailman.94.1430755420.12865.python-list@python.org>
 <87mw1ki7ln.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmoT9RL_LE0jB21N4GTvO0EekUcYubY=Obc7V4=44brFNA@mail.gmail.com>

On Tue, May 5, 2015 at 3:19 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> It looks like I am encircled by Gremlins:
>     >>> import urllib3.contrib.pyopenssl
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>       File "/usr/lib/python3.4/site-packages/urllib3/contrib/pyopenssl.py", line 58, in <module>
>         from socket import _fileobject, timeout
>     ImportError: cannot import name '_fileobject'

This is looking like a pyopenssl bug - I can't import that name
either, and given that it has the leading underscore, it's probably
not an official part of the socket module's API.

ChrisA


From breamoreboy at yahoo.co.uk  Mon May  4 13:59:17 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 04 May 2015 18:59:17 +0100
Subject: Bitten by my C/Java experience
In-Reply-To: <87r3qwid3u.fsf@Equus.decebal.nl>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
Message-ID: <mi8c1r$3h6$1@ger.gmane.org>

On 04/05/2015 16:20, Cecil Westerhof wrote:
> Potential dangerous bug introduced by programming in Python as if it
> was C/Java. :-(
> I used:
>      ++tries
> that has to be:
>      tries += 1
>
> Are there other things I have to be careful on? That does not work as
> in C/Java, but is correct syntax.
>

Not dangerous at all, your test code picks it up.  I'd also guess, but 
don't actually know, that one of the various linter tools could be 
configured to find this problem.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Mon May  4 14:04:18 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 04 May 2015 19:04:18 +0100
Subject: Cannot update OpenSSL for Python3
In-Reply-To: <87vbg8idhk.fsf@Equus.decebal.nl>
References: <87k2wok9bz.fsf@Equus.decebal.nl>
 <mailman.80.1430734238.12865.python-list@python.org>
 <878ud4k270.fsf@Equus.decebal.nl>
 <mailman.84.1430741703.12865.python-list@python.org>
 <874mnsjxju.fsf@Equus.decebal.nl>
 <mailman.89.1430749136.12865.python-list@python.org>
 <87vbg8idhk.fsf@Equus.decebal.nl>
Message-ID: <mi8cb7$8c9$1@ger.gmane.org>

On 04/05/2015 16:11, Cecil Westerhof wrote:
> Op Monday 4 May 2015 16:18 CEST schreef Chris Angelico:
>
>> On Mon, May 4, 2015 at 11:13 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>> That gets installed. And then I get:
>>> ImportError: No module named 'cryptography'
>>>
>>> So I try to install that. This gives: Command /usr/bin/python3 -c
>>> "import setuptools,
>>> tokenize;__file__='/tmp/pip_build_root/cryptography/setup.py';exec(compile(getattr(tokenize,
>>> 'open', open)(__file__).read().replace('\r\n', '\n'), __file__,
>>> 'exec'))" install --record
>>> /tmp/pip-_7jexj87-record/install-record.txt
>>> --single-version-externally-managed --compile failed with error
>>> code 1 in /tmp/pip_build_root/cryptography Storing debug log for
>>> failure in /root/.pip/pip.log
>>>
>>> In the log I see: c/_cffi_backend.c:2:20: fatal error: Python.h: No
>>> such file or directory #include <Python.h>
>>
>> Okay, that one's easy enough to deal with!
>>
>> You have something that needs to build a C extension. To do that,
>> you need to have the Python headers installed. How did you install
>> Python? On Debian/Ubuntu family Linuxes, that's probably "apt-get
>> install python3" - so getting the headers would be "apt-get install
>> python3-dev". Give that a try, and then retry the pip install.
>
> I should have thought about that myself. :-(
>

An alternative is to switch to Windows and do away with this archaic 
concept of users having to build code :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From info at wingware.com  Mon May  4 14:19:29 2015
From: info at wingware.com (Wingware)
Date: Mon, 04 May 2015 14:19:29 -0400
Subject: ANN: Wing IDE 5.1.4 released
Message-ID: <5547B831.6070609@wingware.com>

Hi,

Wingware has released version 5.1.4 of Wing IDE, our cross-platform 
integrated development environment for the Python programming language.

Wing IDE features a professional code editor with vi, emacs, visual 
studio, and other key bindings, auto-completion, call tips, 
context-sensitive auto-editing, goto-definition, find uses, refactoring, 
a powerful debugger, version control, unit testing, search, project 
management, and many other features.

This release includes the following improvements:

     Find Symbol in Project dialog
     Support Django 1.8
     Support debugging Django with auto-reload enabled
     Basic support for Python 3.5 alpha
     Change Case operations to Source menu
     Fix alignment of monospaced characters on OS X and Linux
     Improve pytest support
     Fix several code analysis problems
     About 40 other bug fixes and improvements

For details see http://wingware.com/news/2015-05-01 and 
http://wingware.com/pub/wingide/5.1.4/CHANGELOG.txt

What's New in Wing 5.1:

Wing IDE 5.1 adds multi-process and child process debugging, syntax 
highlighting in the shells, support for pytest, Find Symbol in Project, 
persistent time-stamped unit test results, auto-conversion of indents on 
paste, an XCode keyboard personality, support for Flask, Django 1.7 and 
1.8, Python 3.5 and recent Google App Engine versions, improved 
auto-completion for PyQt, recursive snippet invocation, and many other 
minor features and improvements.

Free trial: http://wingware.com/wingide/trial
Downloads: http://wingware.com/downloads
Feature list: http://wingware.com/wingide/features
Sales: http://wingware.com/store/purchase
Upgrades: https://wingware.com/store/upgrade

Questions?  Don't hesitate to email us at support at wingware.com.

Thanks,

-- 

Stephan Deibel
Wingware | Python IDE

The Intelligent Development Environment for Python Programmers

wingware.com




From Cecil at decebal.nl  Mon May  4 14:59:33 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 20:59:33 +0200
Subject: Why from en to two times with sending email
Message-ID: <87ioc8i2y2.fsf@Equus.decebal.nl>

I want to change an old Bash script to Python. When I look at:
    https://docs.python.org/2/library/email-examples.html

Then from and to have to be used two times? Why is that?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From irmen.NOSPAM at xs4all.nl  Mon May  4 15:02:27 2015
From: irmen.NOSPAM at xs4all.nl (Irmen de Jong)
Date: Mon, 04 May 2015 21:02:27 +0200
Subject: Cannot update OpenSSL for Python3
In-Reply-To: <87mw1ki7ln.fsf@Equus.decebal.nl>
References: <87k2wok9bz.fsf@Equus.decebal.nl>	<mailman.80.1430734238.12865.python-list@python.org>	<878ud4k270.fsf@Equus.decebal.nl>	<mailman.84.1430741703.12865.python-list@python.org>	<874mnsjxju.fsf@Equus.decebal.nl>	<mailman.89.1430749136.12865.python-list@python.org>	<87vbg8idhk.fsf@Equus.decebal.nl>	<mailman.94.1430755420.12865.python-list@python.org>
 <87mw1ki7ln.fsf@Equus.decebal.nl>
Message-ID: <5547c244$0$2913$e4fe514c@news.xs4all.nl>

On 4-5-2015 19:19, Cecil Westerhof wrote:

> It looks like I am encircled by Gremlins:
>     >>> import urllib3.contrib.pyopenssl
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>       File "/usr/lib/python3.4/site-packages/urllib3/contrib/pyopenssl.py", line 58, in <module>
>         from socket import _fileobject, timeout
>     ImportError: cannot import name '_fileobject'
> 

Looks to me as if you have installed a Python 2 version of urllib3? pyopenssl? and are
trying to run that under python 3.

(socket module in python 2 does have a _fileobject, whereas in python 3 it no longer has
it.  Checked in CPython on Windows.)


Irmen



From ian.g.kelly at gmail.com  Mon May  4 15:39:46 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 4 May 2015 13:39:46 -0600
Subject: Bitten by my C/Java experience
In-Reply-To: <mi8c1r$3h6$1@ger.gmane.org>
References: <87r3qwid3u.fsf@Equus.decebal.nl> <mi8c1r$3h6$1@ger.gmane.org>
Message-ID: <CALwzidkaY2CcNPMSPMmWp72W0LeCCuWLQGHeHbPVcSW1hCD2kg@mail.gmail.com>

On Mon, May 4, 2015 at 11:59 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> On 04/05/2015 16:20, Cecil Westerhof wrote:
>>
>> Potential dangerous bug introduced by programming in Python as if it
>> was C/Java. :-(
>> I used:
>>      ++tries
>> that has to be:
>>      tries += 1
>>
>> Are there other things I have to be careful on? That does not work as
>> in C/Java, but is correct syntax.
>>
>
> Not dangerous at all, your test code picks it up.  I'd also guess, but don't
> actually know, that one of the various linter tools could be configured to
> find this problem.

pylint reports it as an error.


From ian.g.kelly at gmail.com  Mon May  4 15:48:42 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 4 May 2015 13:48:42 -0600
Subject: Why from en to two times with sending email
In-Reply-To: <87ioc8i2y2.fsf@Equus.decebal.nl>
References: <87ioc8i2y2.fsf@Equus.decebal.nl>
Message-ID: <CALwzidnKefVec203i8ktQcDM-_=M2qxHxRx4VkTRnBeN51fmug@mail.gmail.com>

On Mon, May 4, 2015 at 12:59 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> I want to change an old Bash script to Python. When I look at:
>     https://docs.python.org/2/library/email-examples.html
>
> Then from and to have to be used two times? Why is that?

Once to construct the message headers, and once to instruct the SMTP
server where to send the message. These are not required to agree; for
instance, bcc recipients need to be supplied to the server but aren't
included in the headers.


From Cecil at decebal.nl  Mon May  4 15:52:46 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 21:52:46 +0200
Subject: Cannot update OpenSSL for Python3
References: <87k2wok9bz.fsf@Equus.decebal.nl>
 <mailman.80.1430734238.12865.python-list@python.org>
 <878ud4k270.fsf@Equus.decebal.nl>
 <mailman.84.1430741703.12865.python-list@python.org>
 <874mnsjxju.fsf@Equus.decebal.nl>
 <mailman.89.1430749136.12865.python-list@python.org>
 <87vbg8idhk.fsf@Equus.decebal.nl>
 <mailman.94.1430755420.12865.python-list@python.org>
 <87mw1ki7ln.fsf@Equus.decebal.nl>
 <5547c244$0$2913$e4fe514c@news.xs4all.nl>
Message-ID: <87bni0i0hd.fsf@Equus.decebal.nl>

Op Monday 4 May 2015 21:02 CEST schreef Irmen de Jong:

> On 4-5-2015 19:19, Cecil Westerhof wrote:
>
>> It looks like I am encircled by Gremlins:
>>>>> import urllib3.contrib.pyopenssl
>> Traceback (most recent call last): File "<stdin>", line 1, in
>> <module> File
>> "/usr/lib/python3.4/site-packages/urllib3/contrib/pyopenssl.py",
>> line 58, in <module> from socket import _fileobject, timeout
>> ImportError: cannot import name '_fileobject'
>>
>
> Looks to me as if you have installed a Python 2 version of urllib3?
> pyopenssl? and are trying to run that under python 3.
>
> (socket module in python 2 does have a _fileobject, whereas in
> python 3 it no longer has it. Checked in CPython on Windows.)

I did an uninstall and installed it again:
    pip3 install urllib3
    Downloading/unpacking urllib3
      Downloading urllib3-1.10.4.tar.gz (138kB): 138kB downloaded
      Running setup.py (path:/tmp/pip_build_root/urllib3/setup.py) egg_info for package urllib3

        warning: no previously-included files matching '*' found under directory 'docs/_build'
    Installing collected packages: urllib3
      Running setup.py install for urllib3

        warning: no previously-included files matching '*' found under directory 'docs/_build'
    Successfully installed urllib3
    Cleaning up...

But I keep getting the error. Only 2 lines earlier:
    >>> import urllib3.contrib.pyopenssl
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.4/site-packages/urllib3/contrib/pyopenssl.py", line 56, in <module>
        from socket import _fileobject, timeout
    ImportError: cannot import name '_fileobject'

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Mon May  4 15:54:31 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 21:54:31 +0200
Subject: Cannot update OpenSSL for Python3
References: <87k2wok9bz.fsf@Equus.decebal.nl>
 <mailman.80.1430734238.12865.python-list@python.org>
 <878ud4k270.fsf@Equus.decebal.nl>
 <mailman.84.1430741703.12865.python-list@python.org>
 <874mnsjxju.fsf@Equus.decebal.nl>
 <mailman.89.1430749136.12865.python-list@python.org>
 <87vbg8idhk.fsf@Equus.decebal.nl>
 <mailman.100.1430762705.12865.python-list@python.org>
Message-ID: <877fsoi0eg.fsf@Equus.decebal.nl>

Op Monday 4 May 2015 20:04 CEST schreef Mark Lawrence:

> An alternative is to switch to Windows and do away with this archaic
> concept of users having to build code :)

Well, maybe I get rid of some problems. But the ones I get back ?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From tjreedy at udel.edu  Mon May  4 16:06:29 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 04 May 2015 16:06:29 -0400
Subject: Bitten by my C/Java experience
In-Reply-To: <CALwzidnnb1bjxxLJ8XvqikVBkHFBXwvgH8MPRXD3nK_O7FwwNA@mail.gmail.com>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <CALwzidnnb1bjxxLJ8XvqikVBkHFBXwvgH8MPRXD3nK_O7FwwNA@mail.gmail.com>
Message-ID: <mi8jg6$14c$1@ger.gmane.org>

On 5/4/2015 1:43 PM, Ian Kelly wrote:

> *    () is a zero-element tuple, and (a, b) is a two-element tuple,
> but (a) is not a one-element tuple. Tuples are created by commas, not
> parentheses, so use (a,) instead.

Which means that a, and a,b (or a,b,) are 1 and 2 element tuples 
respectively. Except for empty tuples, parentheses are optional unless 
needed to fence off the tuple from surrounding code (which happens to be 
most of the time, but not always).  A trailing comma is prohibited for 
zero element tuples, required for one element tuples, and optional for 
multiple element tuples.

-- 
Terry Jan Reedy



From Cecil at decebal.nl  Mon May  4 16:12:08 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 22:12:08 +0200
Subject: Updating FaceBook
Message-ID: <87383chzl3.fsf@Equus.decebal.nl>

Using Python to update Twitter is reasonable straight forward.
I do:
    from libturpial.api.core    import Core
    from libturpial.exceptions  import StatusDuplicated

I fill an account_id and a message and I do:
    Core().update_status(account_id, message)

And my message is posted.

It looks like there is not something like that for FaceBook.  I found:
    https://github.com/pythonforfacebook/facebook-sdk
but that looks not easy to implement. Is there an easier way?

In the past you could send an email, but that seems not to work
anymore.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Mon May  4 16:13:24 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 22:13:24 +0200
Subject: Why from en to two times with sending email
References: <87ioc8i2y2.fsf@Equus.decebal.nl>
 <mailman.103.1430768965.12865.python-list@python.org>
Message-ID: <87y4l4gkyj.fsf@Equus.decebal.nl>

Op Monday 4 May 2015 21:48 CEST schreef Ian Kelly:

> On Mon, May 4, 2015 at 12:59 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> I want to change an old Bash script to Python. When I look at:
>> https://docs.python.org/2/library/email-examples.html
>>
>> Then from and to have to be used two times? Why is that?
>
> Once to construct the message headers, and once to instruct the SMTP
> server where to send the message. These are not required to agree;
> for instance, bcc recipients need to be supplied to the server but
> aren't included in the headers.

OK, thanks.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Mon May  4 16:28:09 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 04 May 2015 22:28:09 +0200
Subject: Bitten by my C/Java experience
References: <87r3qwid3u.fsf@Equus.decebal.nl> <mi8c1r$3h6$1@ger.gmane.org>
 <mailman.102.1430768429.12865.python-list@python.org>
Message-ID: <87twvsgk9y.fsf@Equus.decebal.nl>

Op Monday 4 May 2015 21:39 CEST schreef Ian Kelly:

> On Mon, May 4, 2015 at 11:59 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>> On 04/05/2015 16:20, Cecil Westerhof wrote:
>>>
>>> Potential dangerous bug introduced by programming in Python as if
>>> it was C/Java. :-( I used: ++tries that has to be: tries += 1
>>>
>>> Are there other things I have to be careful on? That does not work
>>> as in C/Java, but is correct syntax.
>>>
>>
>> Not dangerous at all, your test code picks it up. I'd also guess,
>> but don't actually know, that one of the various linter tools could
>> be configured to find this problem.
>
> pylint reports it as an error.

I installed it. Get a lot of messages. Mostly convention. For example:
    Unnecessary parens after 'print' keyword

And:
    Invalid variable name "f"
for:
    with open(real_file, 'r') as f:

But still something to add to my toolbox.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From breamoreboy at yahoo.co.uk  Mon May  4 16:38:22 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 04 May 2015 21:38:22 +0100
Subject: Cannot update OpenSSL for Python3
In-Reply-To: <87k2wok9bz.fsf@Equus.decebal.nl>
References: <87k2wok9bz.fsf@Equus.decebal.nl>
Message-ID: <mi8lc3$voa$1@ger.gmane.org>

On 04/05/2015 09:58, Cecil Westerhof wrote:
> But when I do:
>      import urllib3.contrib.pyopenssl
> I get:
>      Traceback (most recent call last):
>        File "<stdin>", line 1, in <module>
>        File "/usr/lib/python3.4/site-packages/urllib3/contrib/pyopenssl.py", line 55, in <module>
>          import OpenSSL.SSL
>        File "/usr/lib64/python3.4/site-packages/OpenSSL/__init__.py", line 8, in <module>
>          from OpenSSL import rand, crypto, SSL
>        File "/usr/lib64/python3.4/site-packages/OpenSSL/rand.py", line 9, in <module>
>          from six import integer_types as _integer_types
>      ImportError: No module named 'six'
>
> When I then give:
>      pip3 install -U OpenSSL
> It goes wrong:
>      Could not fetch URL https://pypi.python.org/simple/OpenSSL/: 404 Client Error: Not Found
>
> I checked and even
>      https://pypi.python.org/simple/
> does not exist. Anyone an idea what is happening here?
>

Showing my complete ignorance of *nix, what is the difference betweeen 
"/usr/lib/python3.4/..." and "/usr/lib64/python3.4/..."?  Simply 32 
versus 64 bit, which can or can't be mixed, or what?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From amc96 at cam.ac.uk  Mon May  4 16:57:09 2015
From: amc96 at cam.ac.uk (Andrew Cooper)
Date: Mon, 04 May 2015 21:57:09 +0100
Subject: Bitten by my C/Java experience
In-Reply-To: <mailman.97.1430761463.12865.python-list@python.org>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <mailman.97.1430761463.12865.python-list@python.org>
Message-ID: <mi8mf6$k1$1@flame.srcf.net>

On 04/05/2015 18:43, Ian Kelly wrote:
> 
> Some other gotchas that aren't necessarily related to C/Java but can
> be surprising nonetheless:
> 
> *    () is a zero-element tuple, and (a, b) is a two-element tuple,
> but (a) is not a one-element tuple. Tuples are created by commas, not
> parentheses, so use (a,) instead.

* {} is an empty set(), not dict().

Particularly subtle when combined with **kwargs

$ python3
Python 3.4.0 (default, Apr 11 2014, 13:05:11)
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def foo(**kwargs):
...   return { (k, kwargs[k]) for k in kwargs }
...
>>> foo()
set()
>>> foo(a=1)
{('a', 1)}
>>>

~Andrew


From random832 at fastmail.us  Mon May  4 17:16:03 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Mon, 04 May 2015 17:16:03 -0400
Subject: Bitten by my C/Java experience
In-Reply-To: <mi8mf6$k1$1@flame.srcf.net>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <mailman.97.1430761463.12865.python-list@python.org>
 <mi8mf6$k1$1@flame.srcf.net>
Message-ID: <1430774163.2262973.262615509.1FFCAC09@webmail.messagingengine.com>

On Mon, May 4, 2015, at 16:57, Andrew Cooper wrote:
> * {} is an empty set(), not dict().

You've got it backwards.

> Particularly subtle when combined with **kwargs

The function in your example below _always_ returns a set, and kwargs is
always a dict. There's no subtlety outside of the repr output. The fact
that the empty set (as a result of empty kwargs) repr's as set() is a
consequence of the fact that {} is a dict.

> 
> $ python3
> Python 3.4.0 (default, Apr 11 2014, 13:05:11)
> [GCC 4.8.2] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> def foo(**kwargs):
> ...   return { (k, kwargs[k]) for k in kwargs }
> ...
> >>> foo()
> set()
> >>> foo(a=1)
> {('a', 1)}
> >>>
> 
> ~Andrew
> -- 
> https://mail.python.org/mailman/listinfo/python-list


-- 
Random832


From python.list at tim.thechases.com  Mon May  4 17:26:30 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Mon, 4 May 2015 16:26:30 -0500
Subject: Bitten by my C/Java experience
In-Reply-To: <mi8mf6$k1$1@flame.srcf.net>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <mailman.97.1430761463.12865.python-list@python.org>
 <mi8mf6$k1$1@flame.srcf.net>
Message-ID: <20150504162630.4a850fad@bigbox.christie.dr>

On 2015-05-04 21:57, Andrew Cooper wrote:
> On 04/05/2015 18:43, Ian Kelly wrote:
> > 
> > Some other gotchas that aren't necessarily related to C/Java but
> > can be surprising nonetheless:
> > 
> > *    () is a zero-element tuple, and (a, b) is a two-element
> > tuple, but (a) is not a one-element tuple. Tuples are created by
> > commas, not parentheses, so use (a,) instead.
> 
> * {} is an empty set(), not dict().
> 
> Particularly subtle when combined with **kwargs
> 
> $ python3
> Python 3.4.0 (default, Apr 11 2014, 13:05:11)
> [GCC 4.8.2] on linux
> Type "help", "copyright", "credits" or "license" for more
> information.
> >>> def foo(**kwargs):
> ...   return { (k, kwargs[k]) for k in kwargs }
> ...
> >>> foo()
> set()
> >>> foo(a=1)
> {('a', 1)}
> >>>

It's a dict:

Python 3.2.3 (default, Feb 20 2013, 14:44:27) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> type({})
<class 'dict'>


What you're seeing is that your generator creates single-element
tuples in a set constructor (note that your last item isn't
"{'a': 1}". Try instead

>>> def foo(**kwargs):
...     return {k: kwargs[k] for k in kwargs}
... 
>>> foo()
{}
>>> foo(a=42)
{'a': 42}

Note the colons, indicating that it's a dict.  You're using the
dict() syntax:

  dict((k,v) for k,v in some_iter())

-tkc








From irmen.NOSPAM at xs4all.nl  Mon May  4 17:59:05 2015
From: irmen.NOSPAM at xs4all.nl (Irmen de Jong)
Date: Mon, 04 May 2015 23:59:05 +0200
Subject: Cannot update OpenSSL for Python3
In-Reply-To: <87bni0i0hd.fsf@Equus.decebal.nl>
References: <87k2wok9bz.fsf@Equus.decebal.nl>	<mailman.80.1430734238.12865.python-list@python.org>	<878ud4k270.fsf@Equus.decebal.nl>	<mailman.84.1430741703.12865.python-list@python.org>	<874mnsjxju.fsf@Equus.decebal.nl>	<mailman.89.1430749136.12865.python-list@python.org>	<87vbg8idhk.fsf@Equus.decebal.nl>	<mailman.94.1430755420.12865.python-list@python.org>	<87mw1ki7ln.fsf@Equus.decebal.nl>	<5547c244$0$2913$e4fe514c@news.xs4all.nl>
 <87bni0i0hd.fsf@Equus.decebal.nl>
Message-ID: <5547ebab$0$2827$e4fe514c@news2.news.xs4all.nl>

On 4-5-2015 21:52, Cecil Westerhof wrote:

> But I keep getting the error. Only 2 lines earlier:
>     >>> import urllib3.contrib.pyopenssl
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>       File "/usr/lib/python3.4/site-packages/urllib3/contrib/pyopenssl.py", line 56, in <module>
>         from socket import _fileobject, timeout
>     ImportError: cannot import name '_fileobject'
> 

Right. This seems to be an issue with the "contrib" module pyopenssl that is provided as
a courtesy with urllib3. The latter is 100% python 3 compatible from what I read in
their docs.

Looking at that contrib module however:
https://github.com/shazow/urllib3/blob/master/urllib3/contrib/pyopenssl.py

In the first few lines in the module docstring it states it is for Python 2.
I guess you won't be able to use this urllib3 contrib module with python 3. Maybe you
can contact its author to ask for a fix?


Irmen



From bc at freeuk.com  Mon May  4 18:02:57 2015
From: bc at freeuk.com (BartC)
Date: Mon, 04 May 2015 23:02:57 +0100
Subject: Bitten by my C/Java experience
In-Reply-To: <87r3qwid3u.fsf@Equus.decebal.nl>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
Message-ID: <n0S1x.370451$O36.280928@fx02.am4>

On 04/05/2015 16:20, Cecil Westerhof wrote:
> Potential dangerous bug introduced by programming in Python as if it
> was C/Java. :-(
> I used:
>      ++tries
> that has to be:
>      tries += 1

I think I've come across that. It doesn't mind ++ so people are likely 
to be assume that increment works as in other languages.

I guess it just means +(+(a)).

But in that case, what meaning does:

a

or

a+b

have in Python? If they were function calls: a() or (a+b)(), then that's 
clear enough. But a+b doesn't do anything!

(I think I would have picked up "++" and "--" as special tokens even if 
increment/decrement ops weren't supported. Just because they would 
likely cause errors through misunderstanding.)

-- 
Bartc


From davea at davea.name  Mon May  4 19:17:26 2015
From: davea at davea.name (Dave Angel)
Date: Mon, 04 May 2015 19:17:26 -0400
Subject: Bitten by my C/Java experience
In-Reply-To: <87twvsgk9y.fsf@Equus.decebal.nl>
References: <87r3qwid3u.fsf@Equus.decebal.nl> <mi8c1r$3h6$1@ger.gmane.org>
 <mailman.102.1430768429.12865.python-list@python.org>
 <87twvsgk9y.fsf@Equus.decebal.nl>
Message-ID: <5547FE06.4090808@davea.name>

On 05/04/2015 04:28 PM, Cecil Westerhof wrote:
> Op Monday 4 May 2015 21:39 CEST schreef Ian Kelly:
>
>> On Mon, May 4, 2015 at 11:59 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>>> On 04/05/2015 16:20, Cecil Westerhof wrote:
>>>>
>>>> Potential dangerous bug introduced by programming in Python as if
>>>> it was C/Java. :-( I used: ++tries that has to be: tries += 1
>>>>
>>>> Are there other things I have to be careful on? That does not work
>>>> as in C/Java, but is correct syntax.
>>>>
>>>
>>> Not dangerous at all, your test code picks it up. I'd also guess,
>>> but don't actually know, that one of the various linter tools could
>>> be configured to find this problem.
>>
>> pylint reports it as an error.
>
> I installed it. Get a lot of messages. Mostly convention. For example:
>      Unnecessary parens after 'print' keyword

Sounds like it's configured for Python 2.x.  There's probably a setting 
to tell it to use Python3 rules.

>
> And:
>      Invalid variable name "f"
> for:
>      with open(real_file, 'r') as f:

Sounds like a bad wording.  Nothing invalid about it, though it is a bit 
short.  There are certain one letter variables which are so common as to 
be expected, but others should be avoided.


>
> But still something to add to my toolbox.
>


-- 
DaveA


From jia.j.chen at foxmail.com  Mon May  4 22:41:23 2015
From: jia.j.chen at foxmail.com (=?utf-8?B?SmlhIENIRU4=?=)
Date: Tue, 5 May 2015 10:41:23 +0800
Subject: urllib2.urlopen error "socket.error: [Errno 104] Connection reset by
 peer"
Message-ID: <tencent_2B2ADFEE54D112343708D73B@qq.com>

Hi There,
 
I have the error below when trying to download the html content of a webpage. I can open this webpage in a browser without any problem. I am using Ubuntu 14.04. Could you give me come clues about what is happening and how to solve the issue? Thanks.
 
$python 
 
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
 
[GCC 4.8.2] on linux2
 
Type "help", "copyright", "credits" or "license" for more information.
 
>>> import urllib2
 
request = urllib2.Request('http://guggenheiminvestments.com/products/etf/gsy/holdings')
 
response = urllib2.urlopen(request)
 
>>> >>> Traceback (most recent call last):
 
File "<stdin>", line 1, in <module>
 
File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
 
return _opener.open(url, data, timeout)
 
File "/usr/lib/python2.7/urllib2.py", line 404, in open
 
response = self._open(req, data)
 
File "/usr/lib/python2.7/urllib2.py", line 422, in _open
 
'_open', req)
 
File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
 
result = func(*args)
 
File "/usr/lib/python2.7/urllib2.py", line 1214, in http_open
 
return self.do_open(httplib.HTTPConnection, req)
 
File "/usr/lib/python2.7/urllib2.py", line 1187, in do_open
 
r = h.getresponse(buffering=True)
 
File "/usr/lib/python2.7/httplib.py", line 1045, in getresponse
 
response.begin()
 
File "/usr/lib/python2.7/httplib.py", line 409, in begin
 
version, status, reason = self._read_status()
 
File "/usr/lib/python2.7/httplib.py", line 365, in _read_status
 
line = self.fp.readline(_MAXLINE + 1)
 
File "/usr/lib/python2.7/socket.py", line 476, in readline
 
data = self._sock.recv(self._rbufsize)
 
socket.error: [Errno 104] Connection reset by peer
 
>>>

Best,

Jia CHEN
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150505/1a72a2f3/attachment.html>

From steve+comp.lang.python at pearwood.info  Tue May  5 04:19:59 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 05 May 2015 18:19:59 +1000
Subject: Bitten by my C/Java experience
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
Message-ID: <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>

On Tuesday 05 May 2015 08:02, BartC wrote:

> On 04/05/2015 16:20, Cecil Westerhof wrote:
>> Potential dangerous bug introduced by programming in Python as if it
>> was C/Java. :-(
>> I used:
>>      ++tries
>> that has to be:
>>      tries += 1
> 
> I think I've come across that. It doesn't mind ++ so people are likely
> to be assume that increment works as in other languages.
> 
> I guess it just means +(+(a)).

Correct.


> But in that case, what meaning does:
> 
> a
> 
> or
> 
> a+b
> 
> have in Python? If they were function calls: a() or (a+b)(), then that's
> clear enough. But a+b doesn't do anything!

Not so.

The first one just does a name lookup and then throws the result away. The 
second one looks up names a and b, then adds them together, throwing away 
the result.

Here is one use for the first idiom:

try:
    bin
except NameError:
    # No built-in bin function, perhaps our Python is too old?
    def bin(num):
        ...


Here's a good use for the second:


def calculate(x):
    x + 0  # Fails if x is not a number.
    ...


That's an example of duck-typing. It allows any argument which supports 
addition with integers, e.g. x could be a number, or some kind of array or 
vector which supports addition with a scalar.


> (I think I would have picked up "++" and "--" as special tokens even if
> increment/decrement ops weren't supported. Just because they would
> likely cause errors through misunderstanding.)

Just because C made a mistake, doesn't mean other languages have to 
slavishly follow it.



-- 
Steven



From steve+comp.lang.python at pearwood.info  Tue May  5 04:31:30 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 05 May 2015 18:31:30 +1000
Subject: when does newlines get set in universal newlines mode?
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
 <mi7n22$mbc$1@ger.gmane.org>
 <mailman.83.1430741619.12865.python-list@python.org>
Message-ID: <55487fe3$0$12919$c3e8da3$5496439d@news.astraweb.com>

On Monday 04 May 2015 22:13, Chris Angelico wrote:

> It may be worth documenting this limitation, but it's not something
> that can easily be fixed without removing support for \r newlines -
> although that might be an option, given that non-OSX Macs are
> basically history now.

Non-OSX Macs are history, but the text files they created are not.



-- 
Steve



From rosuav at gmail.com  Tue May  5 04:41:12 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 5 May 2015 18:41:12 +1000
Subject: when does newlines get set in universal newlines mode?
In-Reply-To: <55487fe3$0$12919$c3e8da3$5496439d@news.astraweb.com>
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
 <mi7n22$mbc$1@ger.gmane.org>
 <mailman.83.1430741619.12865.python-list@python.org>
 <55487fe3$0$12919$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmoVi42eFB6aw=DmbNSg6iDcdqTiya99Yh_0i6i8koxsbA@mail.gmail.com>

On Tue, May 5, 2015 at 6:31 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Monday 04 May 2015 22:13, Chris Angelico wrote:
>
>> It may be worth documenting this limitation, but it's not something
>> that can easily be fixed without removing support for \r newlines -
>> although that might be an option, given that non-OSX Macs are
>> basically history now.
>
> Non-OSX Macs are history, but the text files they created are not.

True. Like I said, "might be". I could imagine, for instance, a caveat
being put on the newlines attribute saying that "when reading a file
delimited by \r, this may erroneously imply that \r\n is being used,
until a second line has been read". But that's probably more
complexity than it's worth.

ChrisA


From Cecil at decebal.nl  Tue May  5 04:52:54 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 05 May 2015 10:52:54 +0200
Subject: Step further with filebasedMessages
Message-ID: <87oalzh0d5.fsf@Equus.decebal.nl>

I now defined get_message_slice:
    ### Add step
    def get_message_slice(message_filename, start, end):
        """
        Get a slice of messages, where 0 is the first message
        Works with negative indexes
        The values can be ascending and descending
        """

        message_list    = []
        real_file       = expanduser(message_filename)
        nr_of_messages  = get_nr_of_messages(real_file)
        if start < 0:
            start += nr_of_messages
        if end < 0:
            end += nr_of_messages
        assert (start >= 0) and (start < nr_of_messages)
        assert (end   >= 0) and (end   < nr_of_messages)
        if start > end:
            tmp             = start
            start           = end
            end             = tmp
            need_reverse    = True
        else:
            need_reverse    = False
        with open(real_file, 'r') as f:
            for message in islice(f, start, end + 1):
                message_list.append(message.rstrip())
        if need_reverse:
            message_list.reverse()
        return message_list

Is that a good way?

I also had:
    def get_indexed_message(message_filename, index):
        """
        Get index message from a file, where 0 gets the first message
        A negative index gets messages indexed from the end of the file
        Use get_nr_of_messages to get the number of messages in the file
        """

        real_file       = expanduser(message_filename)
        nr_of_messages  = get_nr_of_messages(real_file)
        if index < 0:
            index += nr_of_messages
        assert (index >= 0) and (index < nr_of_messages)
        with open(real_file, 'r') as f:
            [line] = islice(f, index, index + 1)
            return line.rstrip()

But changed it to:
    def get_indexed_message(message_filename, index):
        """
        Get index message from a file, where 0 gets the first message
        A negative index gets messages indexed from the end of the file
        Use get_nr_of_messages to get the number of messages in the file
        """

        return get_message_slice(message_filename, index, index)[0]

Is that acceptable? I am a proponent of DRY.
Or should I at least keep the assert in it?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From rosuav at gmail.com  Tue May  5 05:20:19 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 5 May 2015 19:20:19 +1000
Subject: Step further with filebasedMessages
In-Reply-To: <87oalzh0d5.fsf@Equus.decebal.nl>
References: <87oalzh0d5.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmqsG9gG5-K5npc=i_sR7X6C3wZq994o+WpNk91JPo5MKw@mail.gmail.com>

On Tue, May 5, 2015 at 6:52 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> I now defined get_message_slice:

You're doing a lot of work involving flat-file storage of sequential
data. There are two possibilities:

1) Your files are small, so you shouldn't concern yourself with
details at all - just do whatever looks reasonable, nothing will
matter; or
2) Your files are bigger than that, performance might be a problem
(especially when your Big Oh starts looking bad), and you should move
to a database.

Maybe even with small files, a database would be cleaner. You can grab
whichever rows you want based on their IDs, and the database will do
the work for you. Grab SQLite3 or PostgreSQL, give it a whirl - you
may find that it does everything you need, right out of the box.

ChrisA


From djhon9813 at gmail.com  Tue May  5 05:23:40 2015
From: djhon9813 at gmail.com (david jhon)
Date: Tue, 5 May 2015 14:23:40 +0500
Subject: Exception in thread Thread-4:
Message-ID: <CANPUzQ0MUXpv+ctMzZ6AeAvm6L4gatc7eTq=PmvU=opy+Np6mw@mail.gmail.com>

Hello everyone,

I am initializing lock and threading related variables in __init__() method
of the class as follows:

from threading import Timer, Lock

class miTestClass(EventMixin):
    def __init__(self, t, r, bw):
         self.statMonitorLock = Lock() #to lock the multi access threads
         self.statMonitorLock.acquire()
         statMonitorTimer = Timer(10.0, self._collectFlowStats()) #timer to
collect stats
         statMonitorTimer.start()

but I am getting following error:

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 1082, in run
    self.function(*self.args, **self.kwargs)
TypeError: 'NoneType' object is not callable

I am new to python and after reading various links, I am not able to
resolve this error.
I hope anybody here could help me get it fixed. Many thanks in advance!

Regards,
David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150505/fc7bc365/attachment.html>

From arekfu at gmail.com  Tue May  5 05:23:53 2015
From: arekfu at gmail.com (Davide Mancusi)
Date: Tue, 5 May 2015 02:23:53 -0700 (PDT)
Subject: when does newlines get set in universal newlines mode?
In-Reply-To: <mailman.112.1430815275.12865.python-list@python.org>
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
 <mi7n22$mbc$1@ger.gmane.org>
 <mailman.83.1430741619.12865.python-list@python.org>
 <55487fe3$0$12919$c3e8da3$5496439d@news.astraweb.com>
 <mailman.112.1430815275.12865.python-list@python.org>
Message-ID: <5728620b-6076-4752-8025-93ed460afba0@googlegroups.com>

I just opened a bug report:

http://bugs.python.org/issue24126

We'll see what they say.


From rosuav at gmail.com  Tue May  5 05:28:00 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 5 May 2015 19:28:00 +1000
Subject: when does newlines get set in universal newlines mode?
In-Reply-To: <5728620b-6076-4752-8025-93ed460afba0@googlegroups.com>
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
 <mi7n22$mbc$1@ger.gmane.org>
 <mailman.83.1430741619.12865.python-list@python.org>
 <55487fe3$0$12919$c3e8da3$5496439d@news.astraweb.com>
 <mailman.112.1430815275.12865.python-list@python.org>
 <5728620b-6076-4752-8025-93ed460afba0@googlegroups.com>
Message-ID: <CAPTjJmo=VbHWhggq-WFCmHz-sdNS59Z4P54yPS+igd4Rwab0LA@mail.gmail.com>

On Tue, May 5, 2015 at 7:23 PM, Davide Mancusi <arekfu at gmail.com> wrote:
> I just opened a bug report:
>
> http://bugs.python.org/issue24126
>
> We'll see what they say.

Cool. I suggest posting in the tracker thread the exact Python
version(s) you've tested this with, in case it matters.

ChrisA


From flebber.crue at gmail.com  Tue May  5 05:28:26 2015
From: flebber.crue at gmail.com (Sayth Renshaw)
Date: Tue, 5 May 2015 02:28:26 -0700 (PDT)
Subject: Is it normal to cry when given XML?
Message-ID: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>

Hi

Just checking if the reaction to cry when given XML is normal. 

I thought maybe I am approaching it all wrong, using lxml largely or some xquery to club it into submission. 

See the usual goal is just to take the entire XML and push it into a database. or in future experiment with Mongo or Hdf5 .

See its never basic xml, usually comes from some database with a walk of tables and strange relationships.

Am I doing it wrong is there a simple way I am missing? 

Sayth


From rosuav at gmail.com  Tue May  5 05:44:38 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 5 May 2015 19:44:38 +1000
Subject: Is it normal to cry when given XML?
In-Reply-To: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
Message-ID: <CAPTjJmqLhdXk7f+QBfArCpbTuLTKN4M_zNh89e_ZWbhA_gxEnQ@mail.gmail.com>

On Tue, May 5, 2015 at 7:28 PM, Sayth Renshaw <flebber.crue at gmail.com> wrote:
> Hi
>
> Just checking if the reaction to cry when given XML is normal.

It's not unsurprising, especially with bad XML structures.

> I thought maybe I am approaching it all wrong, using lxml largely or some xquery to club it into submission.
>
> See the usual goal is just to take the entire XML and push it into a database. or in future experiment with Mongo or Hdf5 .
>
> See its never basic xml, usually comes from some database with a walk of tables and strange relationships.
>
> Am I doing it wrong is there a simple way I am missing?

Generally, I work with XML only as a transport layer; and most of the
time, it's for a document structure that would be better served by
JSON anyway. (This may mean that I have an unfairly negative view of
XML, but it's extremely common.) My usual technique is to parse it
into something native (usually a dictionary - and probably the same
structure that the other end constructed the XML from), then work with
that. For example, querying the ePond API [1] gives back a pile of XML
data, so I might have a single function that performs a synchronous
HTTP query, takes the response body, parses it using a fairly generic
XML parser like lxml, then digs three levels into the resulting tree
to pull out the bit that actually matters, leaving behind all the
framing and stuff.

The less time you spend with actual XML, the better. XML is not the answer.

ChrisA

[1] A completely fictional web site, of course, and in no way implying
that I have had a frustrating time with a well-known online
sales/auction company.


From rosuav at gmail.com  Tue May  5 05:52:10 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 5 May 2015 19:52:10 +1000
Subject: Exception in thread Thread-4:
In-Reply-To: <CANPUzQ0MUXpv+ctMzZ6AeAvm6L4gatc7eTq=PmvU=opy+Np6mw@mail.gmail.com>
References: <CANPUzQ0MUXpv+ctMzZ6AeAvm6L4gatc7eTq=PmvU=opy+Np6mw@mail.gmail.com>
Message-ID: <CAPTjJmrpufXsGOKEn_TC0+7MZVFTKbg6gekEqX_LpwqKQnTJrQ@mail.gmail.com>

On Tue, May 5, 2015 at 7:23 PM, david jhon <djhon9813 at gmail.com> wrote:
> from threading import Timer, Lock
>
> class miTestClass(EventMixin):
>     def __init__(self, t, r, bw):
>          self.statMonitorLock = Lock() #to lock the multi access threads
>          self.statMonitorLock.acquire()
>          statMonitorTimer = Timer(10.0, self._collectFlowStats()) #timer to
> collect stats
>          statMonitorTimer.start()
>
> but I am getting following error:
>
> Exception in thread Thread-4:
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
>     self.run()
>   File "/usr/lib/python2.7/threading.py", line 1082, in run
>     self.function(*self.args, **self.kwargs)
> TypeError: 'NoneType' object is not callable

The Timer() class will call a function when it's ready. To use that,
you want to pass it a function. Try putting these two lines of code
into your __init__ function:

print(self._collectFlowStats)
print(self._collectFlowStats())

(Aside: This is something I like to call "IIDPIO debugging": If In
Doubt, Print It Out. You sometimes have more sophisticated debugging
techniques available, but you hardly ever are unable to basic 'print',
in some form or another. It's incredibly useful.)

The first one will print out something like this:

<bound method miTestClass._collectFlowStats of <__main__.miTestClass
object at 0x12345678>>

The second will actually call that function (which may or may not do
anything visible), and then print out:

None

If you change your _collectFlowStats function a bit, you can see even
more of what's happening. Something like this:

def _collectFlowStats(self):
    print("_collectFlowStats has been called. Returning 42...")
    return 42

Then you'll see that it gets called, and does its print, and then 42
gets printed out at the end.

In your case, simply removing the parentheses from
self._collectFlowStats should do what you want - the Timer constructor
will be passed a function (in this case, a bound method, but same
same), and that function won't be called until the timer is up.

Hope that helps!

ChrisA


From steve+comp.lang.python at pearwood.info  Tue May  5 06:04:29 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 05 May 2015 20:04:29 +1000
Subject: Is it normal to cry when given XML?
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
Message-ID: <554895ae$0$2765$c3e8da3$76491128@news.astraweb.com>

On Tuesday 05 May 2015 19:28, Sayth Renshaw wrote:

> Just checking if the reaction to cry when given XML is normal.


Cry? When people give me XML, sometimes I lose control of my bladder.


-- 
Steve



From Cecil at decebal.nl  Tue May  5 06:14:27 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 05 May 2015 12:14:27 +0200
Subject: Step further with filebasedMessages
References: <87oalzh0d5.fsf@Equus.decebal.nl>
 <mailman.113.1430817623.12865.python-list@python.org>
Message-ID: <87k2wngwl8.fsf@Equus.decebal.nl>

Op Tuesday 5 May 2015 11:20 CEST schreef Chris Angelico:

> On Tue, May 5, 2015 at 6:52 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> I now defined get_message_slice:
>
> You're doing a lot of work involving flat-file storage of sequential
> data. There are two possibilities:
>
> 1) Your files are small, so you shouldn't concern yourself with
> details at all - just do whatever looks reasonable, nothing will
> matter;

In my case the files are very small. Biggest is 150 lines. But if you
publish your code, you never know in which situation it will be used.

The suggestion implement a slice came from this newsgroup. ;-) And I
found it a good idea.

It is also to get reacquainted with Python.

> or 2) Your files are bigger than that, performance might be
> a problem (especially when your Big Oh starts looking bad), and you
> should move to a database.
>
> Maybe even with small files, a database would be cleaner. You can
> grab whichever rows you want based on their IDs, and the database
> will do the work for you. Grab SQLite3 or PostgreSQL, give it a
> whirl - you may find that it does everything you need, right out of
> the box.

Is a next step. I want to use PostgreSQL with SQLAlchemy.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From awilliam at whitemice.org  Tue May  5 06:20:28 2015
From: awilliam at whitemice.org (Adam Tauno Williams)
Date: Tue, 05 May 2015 06:20:28 -0400
Subject: Is it normal to cry when given XML?
In-Reply-To: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
Message-ID: <1430821228.2635.7.camel@whitemice.org>

On Tue, 2015-05-05 at 02:28 -0700, Sayth Renshaw wrote:
> Just checking if the reaction to cry when given XML is normal. 

No, not at all.  I leap for ecstatic joy when given, as all to rarely
happens, XML.  Rather than someone's turdy text [which includes JSON]
file.  I wish all 1,200+ of my vendors and suppliers would provide their
data in XML rather than the random swirl of bizarre crap I receive.

> I thought maybe I am approaching it all wrong, using lxml largely or 
> some xquery to club it into submission. 

I do most of my processing with LXML, XSLT, and XPath.  Fast, efficient,
reliable, works great.  And it is easy to abstract these tools to
automate what I need to do over and over again.

> See the usual goal is just to take the entire XML and push it into a
> database. 

I do a lot of pushing into an SQL database, no problem.  XSLT does 99.4%
of the work.

> See its never basic xml, 

"basic xml"? 

> usually comes from some database with a walk of tables and strange relationships.

No problem.

> Am I doing it wrong is there a simple way I am missing? 

I suspect so.  Data is easily transformed into

<ResultSet table="xyz">
  <row>
    <field1 dataType="string" isNull="false"
isPrimaryKey="true">abc</field1>
    <field2 dataType="string" isNull="true" isPrimaryKey="false"/>
    <field3 dataType="string" isNull="false"
isPrimaryKey="false">stanley</field1>
  </row>
 ...
</ResultSet>

And then that data can be processes as SQL insert/update/delete/upsert
operations while being swung around on the end of a stick while you
dance.

<https://sourceforge.net/p/coils/code/ci/default/tree/coils/logic/workflow/actions/sql/upsert.py>

XML is the ultimate file "format".  It solves problems, much better than
everyone trying to re-engineer their needs INTO a format [like JSON].





-- 
Adam Tauno Williams <mailto:awilliam at whitemice.org> GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA



From robin at reportlab.com  Tue May  5 06:36:49 2015
From: robin at reportlab.com (Robin Becker)
Date: Tue, 05 May 2015 11:36:49 +0100
Subject: GAE environment differences
In-Reply-To: <mi24gq$p4u$1@ger.gmane.org>
References: <55435578.50003@chamonix.reportlab.co.uk>
 <CAPTjJmqUeu7AXm6MAiBPwabFv+GRgFOV7B1BUpeLLO7GhiXRvw@mail.gmail.com>
 <554379D3.1080202@chamonix.reportlab.co.uk> <mi24gq$p4u$1@ger.gmane.org>
Message-ID: <55489D41.4000609@chamonix.reportlab.co.uk>

On 02/05/2015 10:14, Kev Dwyer wrote:
> Robin Becker wrote:
>
............
>>> ```
>> the user suggests that even though claims are made that you can use a
>> filesystem, but stuff like pwd is missing. Apparently the user module has
>> no meaning, but there is a users module? I guess I'll need to keep
>> patching reportlab when GAE users find these glitches.
>
> For what it's worth, we use reportlab on GAE to generate a simple pdf
> and the above error is the only one that I've encountered.  For us it was
> enough to trap the ImportError.
>
> Thanks for all your work on reportlab,
>
> Kev
>
the original reporter suggested originally that a KeyError was raised, but I 
think that maybe some difference between development environments and the live GAE.
-- 
Robin Becker



From steve+comp.lang.python at pearwood.info  Tue May  5 06:41:51 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 05 May 2015 20:41:51 +1000
Subject: Step further with filebasedMessages
References: <87oalzh0d5.fsf@Equus.decebal.nl>
Message-ID: <55489e80$0$12913$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 05 May 2015 18:52, Cecil Westerhof wrote:

> I now defined get_message_slice:
>     ### Add step
>     def get_message_slice(message_filename, start, end):
>         """
>         Get a slice of messages, where 0 is the first message
>         Works with negative indexes
>         The values can be ascending and descending
>         """

What's a message in this context? Just a line of text?


>         message_list    = []
>         real_file       = expanduser(message_filename)
>         nr_of_messages  = get_nr_of_messages(real_file)
>         if start < 0:
>             start += nr_of_messages
>         if end < 0:
>             end += nr_of_messages
>         assert (start >= 0) and (start < nr_of_messages)
>         assert (end   >= 0) and (end   < nr_of_messages)

You can write that as:

    assert 0 <= start < nr_of_messages

except you probably shouldn't, because that's not a good use for assert: 
start and end are user-supplied parameters, not internal invariants.

You might find this useful for understanding when to use assert:

http://import-that.dreamwidth.org/676.html



>         if start > end:
>             tmp             = start
>             start           = end
>             end             = tmp
>             need_reverse    = True

You can swap two variables like this:

    start, end = end, start

The language guarantees that the right hand side will be evaluated before 
the assignments are done, so it will automatically do the right thing.


>         else:
>             need_reverse    = False

Your behaviour when start and end are in opposite order does not match the 
standard slicing behaviour:

py> "abcdef"[3:5]
'de'
py> "abcdef"[5:3]
''

That doesn't mean your behaviour is wrong, but it will surprise anyone who 
expects your slicing to be like the slicing they are used to.


>         with open(real_file, 'r') as f:
>             for message in islice(f, start, end + 1):
>                 message_list.append(message.rstrip())
>         if need_reverse:
>             message_list.reverse()
>         return message_list
> 
> Is that a good way?


I think what I would do is:

def get_message_slice(message_filename, start=0, end=None, step=1):
    real_file = expanduser(message_filename)
    with open(real_file, 'r') as f:
        messages = f.readlines()
    return messages[start:end:step]


until such time that I could prove that I needed something more 
sophisticated. Then, and only then, would I consider your approach, except 
using a slice object:

# Untested.
def get_message_slice(message_filename, start=0, end=None, step=1):
    real_file = expanduser(message_filename)
    messages = []
    # FIXME: I assume this is expensive. Can we avoid it?
    nr_of_messages = get_nr_of_messages(real_file)
    the_slice = slice(start, end, step)
    # Calculate the indexes in the given slice, e.g.
    # start=1, stop=7, step=2 gives [1,3,5].
    indices = range(*(the_slice.indices(nr_of_messages)))
    with open(real_file, 'r') as f:
        for i, message in enumerate(f):
            if i in indices:
                messages.append(message)
    return messages


There is still room for optimization: e.g. if the slice is empty, don't 
bother iterating over the file. I leave that to you.



-- 
Steve




From arekfu at gmail.com  Tue May  5 06:58:13 2015
From: arekfu at gmail.com (Davide Mancusi)
Date: Tue, 5 May 2015 03:58:13 -0700 (PDT)
Subject: when does newlines get set in universal newlines mode?
In-Reply-To: <mailman.115.1430818087.12865.python-list@python.org>
References: <3c45772b-77e0-4c17-8b3d-aa246c4b511c@googlegroups.com>
 <mi7n22$mbc$1@ger.gmane.org>
 <mailman.83.1430741619.12865.python-list@python.org>
 <55487fe3$0$12919$c3e8da3$5496439d@news.astraweb.com>
 <mailman.112.1430815275.12865.python-list@python.org>
 <5728620b-6076-4752-8025-93ed460afba0@googlegroups.com>
 <mailman.115.1430818087.12865.python-list@python.org>
Message-ID: <6ba588a6-2a82-4111-9462-17cd98116f97@googlegroups.com>

> Cool. I suggest posting in the tracker thread the exact Python
> version(s) you've tested this with, in case it matters.

Done. Good point.


From __peter__ at web.de  Tue May  5 07:08:16 2015
From: __peter__ at web.de (Peter Otten)
Date: Tue, 05 May 2015 13:08:16 +0200
Subject: Step further with filebasedMessages
References: <87oalzh0d5.fsf@Equus.decebal.nl>
Message-ID: <mia8b1$dv3$1@ger.gmane.org>

Cecil Westerhof wrote:

> I now defined get_message_slice:
>     ### Add step
>     def get_message_slice(message_filename, start, end):

Intervals are usually half-open in Python. I recommend that you follow that 
convention.

>         """
>         Get a slice of messages, where 0 is the first message
>         Works with negative indexes
>         The values can be ascending and descending
>         """
> 
>         message_list    = []
>         real_file       = expanduser(message_filename)
>         nr_of_messages  = get_nr_of_messages(real_file)
>         if start < 0:
>             start += nr_of_messages
>         if end < 0:
>             end += nr_of_messages
>         assert (start >= 0) and (start < nr_of_messages)

You should raise an exception. While asserts are rarely switched off in 
Python you still have to be prepeared, and an IndexError would be a better 
fit anyway.

>         assert (end   >= 0) and (end   < nr_of_messages)
>         if start > end:
>             tmp             = start
>             start           = end
>             end             = tmp
>             need_reverse    = True
>         else:
>             need_reverse    = False
>         with open(real_file, 'r') as f:
>             for message in islice(f, start, end + 1):
>                 message_list.append(message.rstrip())
>         if need_reverse:
>             message_list.reverse()
>         return message_list
> 
> Is that a good way?
> 
> I also had:
>     def get_indexed_message(message_filename, index):
>         """
>         Get index message from a file, where 0 gets the first message
>         A negative index gets messages indexed from the end of the file
>         Use get_nr_of_messages to get the number of messages in the file
>         """
> 
>         real_file       = expanduser(message_filename)
>         nr_of_messages  = get_nr_of_messages(real_file)
>         if index < 0:
>             index += nr_of_messages
>         assert (index >= 0) and (index < nr_of_messages)
>         with open(real_file, 'r') as f:
>             [line] = islice(f, index, index + 1)
>             return line.rstrip()
> 
> But changed it to:
>     def get_indexed_message(message_filename, index):
>         """
>         Get index message from a file, where 0 gets the first message
>         A negative index gets messages indexed from the end of the file
>         Use get_nr_of_messages to get the number of messages in the file
>         """
> 
>         return get_message_slice(message_filename, index, index)[0]
> 
> Is that acceptable? 

Yes.

> I am a proponent of DRY.

But note that you are implementing parts of the slicing logic that Python's 
sequence already has. Consider becoming a pronent of DRWTOGAD*.

> Or should I at least keep the assert in it?
 
No.

I see you have a tendency to overengineer. Here's
how I would approach case (1) in Chris' answer, where memory is not a 
concern:

import os

def read_messages(filename):
    with open(os.path.expanduser(filename)) as f:
        return [line.rstrip() for line in f]

# get_messages_slice(filename, start, end)
print(read_messages(filename)[start:end+1])

# get_indexed_message(filename, index)
print(read_messages(filename)[index])

Should you later decide that a database is a better fit you can change 
read_messages() to return a class that transparently accesses that database. 
Again, most of the work is already done:

class Messages(collections.Sequence):
    def __init__(self, filename):
        self.filename = filename)
    def __getitem__(self, index):
        # read record(s) from db
    def __len__(self): 
        # return num-records in db

def read_messages(filename):
    return Messages(filename)

By the way, where do you plan to use your functions? And where do the 
indices you feed them come from?

(*) Don't repeat what those other guys already did. Yeah sorry, I have a 
soft spot for lame jokes...



From Cecil at decebal.nl  Tue May  5 07:27:54 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 05 May 2015 13:27:54 +0200
Subject: Step further with filebasedMessages
References: <87oalzh0d5.fsf@Equus.decebal.nl>
 <55489e80$0$12913$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87fv7bgt6t.fsf@Equus.decebal.nl>

Op Tuesday 5 May 2015 12:41 CEST schreef Steven D'Aprano:

> On Tuesday 05 May 2015 18:52, Cecil Westerhof wrote:
>
>> I now defined get_message_slice:
>> ### Add step
>> def get_message_slice(message_filename, start, end):
>> """
>> Get a slice of messages, where 0 is the first message
>> Works with negative indexes
>> The values can be ascending and descending
>> """
>
> What's a message in this context? Just a line of text?

Yes. In this case it is. Iuse it to post on Twitter. (OmgangMetTijd)
It was done with an old PHP script. But I switched it to Python, just
to get some needed exercise in Python. But I found it very easy to
extend the functionality.

In my twitter application I use '^' to have the possibility to create
newlines.

>
>
>> message_list    = []
>> real_file       = expanduser(message_filename)
>> nr_of_messages  = get_nr_of_messages(real_file)
>> if start < 0:
>> start += nr_of_messages
>> if end < 0:
>> end += nr_of_messages
>> assert (start >= 0) and (start < nr_of_messages)
>> assert (end   >= 0) and (end   < nr_of_messages)
>
> You can write that as:
>
> assert 0 <= start < nr_of_messages

I was thought that my way is more clear.


> except you probably shouldn't, because that's not a good use for
> assert: start and end are user-supplied parameters, not internal
> invariants.
>
> You might find this useful for understanding when to use assert:
>
> http://import-that.dreamwidth.org/676.html

I will read it.


>> if start > end:
>> tmp             = start
>> start           = end
>> end             = tmp
>> need_reverse    = True
>
> You can swap two variables like this:
>
> start, end = end, start
>
> The language guarantees that the right hand side will be evaluated
> before the assignments are done, so it will automatically do the
> right thing.

That is a lot clearer. Thanks.


> Your behaviour when start and end are in opposite order does not
> match the standard slicing behaviour:
>
> py> "abcdef"[3:5]
> 'de'
> py> "abcdef"[5:3]
> ''
>
> That doesn't mean your behaviour is wrong, but it will surprise
> anyone who expects your slicing to be like the slicing they are used
> to.

Good point: something to think about.


> I think what I would do is:
>
> def get_message_slice(message_filename, start=0, end=None, step=1):
> real_file = expanduser(message_filename)
> with open(real_file, 'r') as f:
> messages = f.readlines()
> return messages[start:end:step]

The idea is that this is expensive for large files.


> until such time that I could prove that I needed something more
> sophisticated. Then, and only then, would I consider your approach,
> except using a slice object:

Because I publish it, I should take reasonable care of the
possibilities of its use.


> # Untested.
> def get_message_slice(message_filename, start=0, end=None, step=1):
> real_file = expanduser(message_filename)
> messages = []
> # FIXME: I assume this is expensive. Can we avoid it?
> nr_of_messages = get_nr_of_messages(real_file)

If I want to give the possibility to use negative values also, I need
the value.


> the_slice = slice(start, end, step)
> # Calculate the indexes in the given slice, e.g.
> # start=1, stop=7, step=2 gives [1,3,5].
> indices = range(*(the_slice.indices(nr_of_messages)))
> with open(real_file, 'r') as f:
> for i, message in enumerate(f):
> if i in indices:
> messages.append(message)
> return messages

I will look into it.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From djhon9813 at gmail.com  Tue May  5 07:36:37 2015
From: djhon9813 at gmail.com (david jhon)
Date: Tue, 5 May 2015 16:36:37 +0500
Subject: Exception in thread Thread-4:
In-Reply-To: <CAPTjJmrpufXsGOKEn_TC0+7MZVFTKbg6gekEqX_LpwqKQnTJrQ@mail.gmail.com>
References: <CANPUzQ0MUXpv+ctMzZ6AeAvm6L4gatc7eTq=PmvU=opy+Np6mw@mail.gmail.com>
 <CAPTjJmrpufXsGOKEn_TC0+7MZVFTKbg6gekEqX_LpwqKQnTJrQ@mail.gmail.com>
Message-ID: <CANPUzQ3SuaRL9Xonv85QrqceOCaE_Wx08sahrW5Up7J1AXsT5g@mail.gmail.com>

Hi Chris,

Thanks a lot for such a comprehensive reply, I got it fixed now. Thanks
again :)

On Tue, May 5, 2015 at 2:52 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Tue, May 5, 2015 at 7:23 PM, david jhon <djhon9813 at gmail.com> wrote:
> > from threading import Timer, Lock
> >
> > class miTestClass(EventMixin):
> >     def __init__(self, t, r, bw):
> >          self.statMonitorLock = Lock() #to lock the multi access threads
> >          self.statMonitorLock.acquire()
> >          statMonitorTimer = Timer(10.0, self._collectFlowStats()) #timer
> to
> > collect stats
> >          statMonitorTimer.start()
> >
> > but I am getting following error:
> >
> > Exception in thread Thread-4:
> > Traceback (most recent call last):
> >   File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
> >     self.run()
> >   File "/usr/lib/python2.7/threading.py", line 1082, in run
> >     self.function(*self.args, **self.kwargs)
> > TypeError: 'NoneType' object is not callable
>
> The Timer() class will call a function when it's ready. To use that,
> you want to pass it a function. Try putting these two lines of code
> into your __init__ function:
>
> print(self._collectFlowStats)
> print(self._collectFlowStats())
>
> (Aside: This is something I like to call "IIDPIO debugging": If In
> Doubt, Print It Out. You sometimes have more sophisticated debugging
> techniques available, but you hardly ever are unable to basic 'print',
> in some form or another. It's incredibly useful.)
>
> The first one will print out something like this:
>
> <bound method miTestClass._collectFlowStats of <__main__.miTestClass
> object at 0x12345678>>
>
> The second will actually call that function (which may or may not do
> anything visible), and then print out:
>
> None
>
> If you change your _collectFlowStats function a bit, you can see even
> more of what's happening. Something like this:
>
> def _collectFlowStats(self):
>     print("_collectFlowStats has been called. Returning 42...")
>     return 42
>
> Then you'll see that it gets called, and does its print, and then 42
> gets printed out at the end.
>
> In your case, simply removing the parentheses from
> self._collectFlowStats should do what you want - the Timer constructor
> will be passed a function (in this case, a bound method, but same
> same), and that function won't be called until the timer is up.
>
> Hope that helps!
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150505/622ed0e9/attachment.html>

From flebber.crue at gmail.com  Tue May  5 08:03:35 2015
From: flebber.crue at gmail.com (Sayth Renshaw)
Date: Tue, 5 May 2015 05:03:35 -0700 (PDT)
Subject: Is it normal to cry when given XML?
In-Reply-To: <mailman.122.1430822460.12865.python-list@python.org>
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
 <mailman.122.1430822460.12865.python-list@python.org>
Message-ID: <070bb8f5-3a15-49d7-a60f-cfd059e87b8c@googlegroups.com>

Adam I am glad to hear it in someways because it's something I have never heard it. For a person relatively new to XML most articles and tutorials demonstrate getting it out to a more "manageable" format.

I had been using xbase to inspect the data and query but really ask I want to do was push it to a db and then query it as is more normal for me. 

I will check out your link. 


From jtim.arnold at gmail.com  Tue May  5 09:05:54 2015
From: jtim.arnold at gmail.com (Tim)
Date: Tue, 5 May 2015 06:05:54 -0700 (PDT)
Subject: Is it normal to cry when given XML?
In-Reply-To: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
Message-ID: <7136ec23-2a14-4305-8d41-2b8304d028e6@googlegroups.com>

On Tuesday, May 5, 2015 at 5:28:37 AM UTC-4, Sayth Renshaw wrote:
> Hi
> 
> Just checking if the reaction to cry when given XML is normal. 
> 
> Sayth

Hi Sayth,
My experience in general is just like what Chris said. Except when dealing with DocBook XML which is probably not what you have. But I cannot say enough good things about lxml, which is my favorite 3rd-party package ever. Its support for xpath really makes it easy to traverse and select elements in a document tree.  Plus the ability to drop subtrees, move elements around in a live tree, etc. Great library.

Using lxml to parse and restructure the xml to a dictionary isn't too hard, but of course it depends on the xml you have to deal with. Sometimes weeping is just part of the job.

good luck,
--Tim


From bc at freeuk.com  Tue May  5 09:20:08 2015
From: bc at freeuk.com (BartC)
Date: Tue, 05 May 2015 14:20:08 +0100
Subject: Bitten by my C/Java experience
In-Reply-To: <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
 <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>
Message-ID: <es32x.478654$Qj7.158587@fx36.am4>

On 05/05/2015 09:19, Steven D'Aprano wrote:
> On Tuesday 05 May 2015 08:02, BartC wrote:

>> (I think I would have picked up "++" and "--" as special tokens even if
>> increment/decrement ops weren't supported. Just because they would
>> likely cause errors through misunderstanding.)
>
> Just because C made a mistake, doesn't mean other languages have to
> slavishly follow it.

I would have thought there was more rapport between the two languages. 
Python is often implemented in C and extensions are often implemented in 
C, suggesting there are quite a few people familiar with both, sometimes 
in areas that are critical (ie. creating code that will affect thousands 
of Python apps).

So why pretend that ++ and -- don't exist? After all Python borrows "=", 
"==" and "!=" from C.

(Writing a==b instead of a=b is less likely in Python than in a language 
where a=b is an equality test rather than assignment. But I've used just 
such a language where mistakenly writing a=b (which happens when 
switching between languages) caused difficult-to-find bugs.

Until I disallowed standalone expressions as statements, then these 
things are picked up, and they are invariably unintended errors. Where 
it is actually necessary to evaluate an expression and throw away the 
result, then a simple prefix can be used.)

-- 
Bartc






From ian.g.kelly at gmail.com  Tue May  5 09:57:00 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 5 May 2015 07:57:00 -0600
Subject: Step further with filebasedMessages
In-Reply-To: <87fv7bgt6t.fsf@Equus.decebal.nl>
References: <87oalzh0d5.fsf@Equus.decebal.nl>
 <55489e80$0$12913$c3e8da3$5496439d@news.astraweb.com>
 <87fv7bgt6t.fsf@Equus.decebal.nl>
Message-ID: <CALwzid=WbXaoEhow1PRanr4Y4ZFWcrn2Aw9nN6W5fXKEgWpQ1A@mail.gmail.com>

On May 5, 2015 5:46 AM, "Cecil Westerhof" <Cecil at decebal.nl> wrote:
>
> Op Tuesday 5 May 2015 12:41 CEST schreef Steven D'Aprano:
>
> > # Untested.
> > def get_message_slice(message_filename, start=0, end=None, step=1):
> > real_file = expanduser(message_filename)
> > messages = []
> > # FIXME: I assume this is expensive. Can we avoid it?
> > nr_of_messages = get_nr_of_messages(real_file)
>
> If I want to give the possibility to use negative values also, I need
> the value.

You could make this call only if one of the boundaries is actually
negative. Then callers that provide positive values don't need to pay the
cost of that case.

Alternatively, consider that it's common for slices of iterators to
disallow negative indices altogether, and question whether you really need
that.

> > the_slice = slice(start, end, step)
> > # Calculate the indexes in the given slice, e.g.
> > # start=1, stop=7, step=2 gives [1,3,5].
> > indices = range(*(the_slice.indices(nr_of_messages)))
> > with open(real_file, 'r') as f:
> > for i, message in enumerate(f):
> > if i in indices:
> > messages.append(message)
> > return messages

I approve of using slice.indices instead of calculating the indices
manually, but otherwise, the islice approach feels cleaner to me. This
reads like a reimplementation of that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150505/f28addae/attachment.html>

From Cecil at decebal.nl  Tue May  5 10:51:56 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 05 May 2015 16:51:56 +0200
Subject: Step further with filebasedMessages
References: <87oalzh0d5.fsf@Equus.decebal.nl>
 <55489e80$0$12913$c3e8da3$5496439d@news.astraweb.com>
 <87fv7bgt6t.fsf@Equus.decebal.nl>
 <mailman.125.1430834230.12865.python-list@python.org>
Message-ID: <87bnhzgjqr.fsf@Equus.decebal.nl>

Op Tuesday 5 May 2015 15:57 CEST schreef Ian Kelly:

> On May 5, 2015 5:46 AM, "Cecil Westerhof" <Cecil at decebal.nl> wrote:
>>
>> Op Tuesday 5 May 2015 12:41 CEST schreef Steven D'Aprano:
>>
>>> # Untested. def get_message_slice(message_filename, start=0,
>>> end=None, step=1): real_file = expanduser(message_filename)
>>> messages = [] # FIXME: I assume this is expensive. Can we avoid
>>> it? nr_of_messages = get_nr_of_messages(real_file)
>>
>> If I want to give the possibility to use negative values also, I
>> need the value.
>
> You could make this call only if one of the boundaries is actually
> negative. Then callers that provide positive values don't need to
> pay the cost of that case.

You have a point there. I will think about it.


> Alternatively, consider that it's common for slices of iterators to
> disallow negative indices altogether, and question whether you
> really need that.

It was an idea I got from this newsgroup. :-D And I liked it,
otherwise I would not have implemented it.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From p.f.moore at gmail.com  Tue May  5 11:22:59 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 5 May 2015 08:22:59 -0700 (PDT)
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
Message-ID: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>

I'm working my way through the asyncio documentation. I have got to the "Tasks and coroutines" section, but I'm frankly confused as to the difference between the various things described in that section: coroutines, tasks, and futures.

I think can understand a coroutine. Correct me if I'm wrong, but it's roughly "something that you can run which can suspend itself".

But I don't understand what a Future is. The document just says it's almost the same as a concurrent.futures.Future, which is described as something that "encapsulates the asynchronous execution of a callable". Which doesn't help a lot. In concurrent.futures, you don't create Futures, you get them back from submit(), but in the asyncio docs it looks like you can create them by hand (example "Future with run_until_complete"). And there's nothing that says what a Future is, just what it's like... :-(

A Task is a subclass of Future, but the documentation doesn't say what it *is*, but rather that it "schedules the execution of a coroutine". But that doesn't make sense to me - objects don't do things, they *are* things. I thought the event loop did the scheduling?

Reading between the lines, it seems that the event loop schedules Tasks (which makes sense) and that Tasks somehow wrap up coroutines - but I don't see *why* you need to wrap a task in a coroutine rather than just scheduling coroutines. And I don't see where Futures fit in - why not just wrap a coroutine in a Future, if it needs to be wrapped up at all?

I concede that I've not read the rest of the asyncio documentation in much detail yet, and I'm skipping everything to do with IO (I want to understand the "async" bit for now, not so much the "IO" side). But I don't really want to dive into the details while I am this hazy on the basic concepts.

Can anyone clarify for me?

Thanks,
Paul


From Cecil at decebal.nl  Tue May  5 11:25:05 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 05 May 2015 17:25:05 +0200
Subject: Step further with filebasedMessages
References: <87oalzh0d5.fsf@Equus.decebal.nl>
 <mailman.123.1430824107.12865.python-list@python.org>
Message-ID: <877fsngi7i.fsf@Equus.decebal.nl>

Op Tuesday 5 May 2015 13:08 CEST schreef Peter Otten:

> Cecil Westerhof wrote:
>
>> I now defined get_message_slice:
>> ### Add step
>> def get_message_slice(message_filename, start, end):
>
> Intervals are usually half-open in Python. I recommend that you
> follow that convention.

I will change it.


>> """
>> Get a slice of messages, where 0 is the first message
>> Works with negative indexes
>> The values can be ascending and descending
>> """
>>
>> message_list    = []
>> real_file       = expanduser(message_filename)
>> nr_of_messages  = get_nr_of_messages(real_file)
>> if start < 0:
>> start += nr_of_messages
>> if end < 0:
>> end += nr_of_messages
>> assert (start >= 0) and (start < nr_of_messages)
>
> You should raise an exception. While asserts are rarely switched off
> in Python you still have to be prepeared, and an IndexError would be
> a better fit anyway.

OK.


> Should you later decide that a database is a better fit you can
> change read_messages() to return a class that transparently accesses
> that database. Again, most of the work is already done:
>
> class Messages(collections.Sequence):
> def __init__(self, filename):
> self.filename = filename)
> def __getitem__(self, index):
> # read record(s) from db
> def __len__(self): 
> # return num-records in db
>
> def read_messages(filename):
> return Messages(filename)

Another thing I have to look into. :-D


> By the way, where do you plan to use your functions? And where do
> the indices you feed them come from?

In my case from:
    get_random_message
and:
    dequeue_message

Both also in:
    https://github.com/CecilWesterhof/PythonLibrary/blob/master/filebasedMessages.py

I should write some documentation with those functions. ;-)


I have a file with quotes and a file with tips. I want to place random
messages from those two (without them being repeated to soon) on my
Twitter page. This I do with ?get_random_message?. I also want to put
the first message of another file and remove it from the file. For
this I use ?dequeue_message?.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From p.f.moore at gmail.com  Tue May  5 11:47:26 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 5 May 2015 08:47:26 -0700 (PDT)
Subject: Throw the cat among the pigeons
In-Reply-To: <871tixlmp6.fsf@Equus.decebal.nl>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
Message-ID: <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>

On Sunday, 3 May 2015 16:23:59 UTC+1, Cecil Westerhof  wrote:
> > By the way: I think that even if the recursion does not go further
> > as 500, it is still a good idea to use tail recursion. Why use stack
> > space when it is not necessary?
> 
> I pushed the example to GitHub:
>     https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py

You already know this, as your code shows, but tail call recursion elimination is only possible when you have a *direct* tail call (one with the result of the tail call returned immediately to the caller). Even the standard trivial factorial example doesn't have a direct tail call, without rewriting to use an accumulator variable. Which is a non-intuitive transformation to anyone who's not familiar with recursive functional languages and their idioms.

If you're rewriting your recursive function *anyway*, it's not that much harder in many (most?) cases to rewrite it iteratively.

An example of a function that naturally uses direct tail call recursion, but which doesn't have a simple iterative rewrite, would be more compelling. Not particularly compelling (to me, anyway) even so, but still better than factorial or fibonnaci examples.

Paul


From zachary.ware+pylist at gmail.com  Tue May  5 12:11:03 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Tue, 5 May 2015 11:11:03 -0500
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
Message-ID: <CAKJDb-PQkUVCAtC-zbCgbkhCzdq+uMXfXdain1-S0VKQP5w=bw@mail.gmail.com>

On Tue, May 5, 2015 at 10:22 AM, Paul  Moore <p.f.moore at gmail.com> wrote:
> I'm working my way through the asyncio documentation. I have got to the "Tasks and coroutines" section, but I'm frankly confused as to the difference between the various things described in that section: coroutines, tasks, and futures.

I've been using (and, in part at least, understanding :)) asyncio for
work for the past few months, so I can at least give you my
impressions based on use.

> I think can understand a coroutine. Correct me if I'm wrong, but it's roughly "something that you can run which can suspend itself".

That's basically how I understand it.

> But I don't understand what a Future is. The document just says it's almost the same as a concurrent.futures.Future, which is described as something that "encapsulates the asynchronous execution of a callable". Which doesn't help a lot. In concurrent.futures, you don't create Futures, you get them back from submit(), but in the asyncio docs it looks like you can create them by hand (example "Future with run_until_complete"). And there's nothing that says what a Future is, just what it's like... :-(

My understanding is that Futures are somewhat like 'non-executable
coroutines', if that makes any sense whatsoever.  Futures are used as
something you can pass around when you need to start execution of the
"job" in one method and finish it in another.  For instance, talking
to a remote network entity, and you want to just do "yield from
server.get(something)".  Your 'get' method can make the request,
create a Future, stick it somewhere that the method monitoring
incoming traffic can find it (along with some identifying metadata),
and then do 'return (yield from future)' to wait for the Future to be
fulfilled (by the listener) and return its result.  The listener then
matches up incoming requests with Futures, and calls
Future.set_result() or Future.set_exception() to fulfill the Future.
Once one of those methods has been called on the Future (and control
has been passed back to the scheduler), your server.get method
unblocks from its '(yield from future)' call, and either raises the
exception or returns the result that was set on the Future.

> A Task is a subclass of Future, but the documentation doesn't say what it *is*, but rather that it "schedules the execution of a coroutine". But that doesn't make sense to me - objects don't do things, they *are* things. I thought the event loop did the scheduling?
>
> Reading between the lines, it seems that the event loop schedules Tasks (which makes sense) and that Tasks somehow wrap up coroutines - but I don't see *why* you need to wrap a task in a coroutine rather than just scheduling coroutines. And I don't see where Futures fit in - why not just wrap a coroutine in a Future, if it needs to be wrapped up at all?

You kind of mixed things up in this paragraph (you said "Tasks somehow
wrap up coroutines", then "wrap a task in a coroutine"; the first is
more correct, I believe).  As I understand it, Tasks are
specializations of Futures that take care of the
set_result/set_exception based on the execution of the coroutine.  I'm
not clear on how that is all implemented (Are all coroutines wrapped
in Tasks?  Just those wrapped in asyncio.async()? No idea.), but I use
it something like this:

   # save a reference to this task so it can be cancelled elsewhere if need be
   self.current_task = asyncio.async(some_coroutine())
   # now the coroutine is scheduled, but we still have control
   # we can do anything else we need to here, including scheduling
other coroutines
   return (yield from self.current_task)

> I concede that I've not read the rest of the asyncio documentation in much detail yet, and I'm skipping everything to do with IO (I want to understand the "async" bit for now, not so much the "IO" side). But I don't really want to dive into the details while I am this hazy on the basic concepts.
>
> Can anyone clarify for me?

I hope I've done some good on that front, but I'm still a bit hazy
myself.  I found it worked best to just take examples or otherwise
working code, and poke and prod at it until it breaks and you can
figure out how you broke it.  Just try to avoid mixing asyncio and
threads for as long as you can :).

-- 
Zach


From Cecil at decebal.nl  Tue May  5 12:18:41 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 05 May 2015 18:18:41 +0200
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
Message-ID: <87zj5jf15q.fsf@Equus.decebal.nl>

Op Tuesday 5 May 2015 17:47 CEST schreef Paul Moore:

> On Sunday, 3 May 2015 16:23:59 UTC+1, Cecil Westerhof  wrote:
>>> By the way: I think that even if the recursion does not go further
>>> as 500, it is still a good idea to use tail recursion. Why use
>>> stack space when it is not necessary?
>>
>> I pushed the example to GitHub:
>> https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py
>
> You already know this, as your code shows, but tail call recursion
> elimination is only possible when you have a *direct* tail call (one
> with the result of the tail call returned immediately to the
> caller). Even the standard trivial factorial example doesn't have a
> direct tail call, without rewriting to use an accumulator variable.
> Which is a non-intuitive transformation to anyone who's not familiar
> with recursive functional languages and their idioms.
>
> If you're rewriting your recursive function *anyway*, it's not that
> much harder in many (most?) cases to rewrite it iteratively.
>
> An example of a function that naturally uses direct tail call
> recursion, but which doesn't have a simple iterative rewrite, would
> be more compelling. Not particularly compelling (to me, anyway) even
> so, but still better than factorial or fibonnaci examples.

Well, I did not write many tail recursive functions. But what surprised
me was that for large values the ?tail recursive? version was more
efficient as the iterative version. And that was with myself
implementing the tail recursion. I expect the code to be more
efficient when the compiler implements the tail recursion.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From rustompmody at gmail.com  Tue May  5 12:24:10 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 5 May 2015 09:24:10 -0700 (PDT)
Subject: Bitten by my C/Java experience
In-Reply-To: <es32x.478654$Qj7.158587@fx36.am4>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
 <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>
 <es32x.478654$Qj7.158587@fx36.am4>
Message-ID: <1a74804c-958f-44ad-a354-e6ac1510a8b6@googlegroups.com>

On Tuesday, May 5, 2015 at 6:50:29 PM UTC+5:30, BartC wrote:
> On 05/05/2015 09:19, Steven D'Aprano wrote:
> > On Tuesday 05 May 2015 08:02, BartC wrote:
> 
> >> (I think I would have picked up "++" and "--" as special tokens even if
> >> increment/decrement ops weren't supported. Just because they would
> >> likely cause errors through misunderstanding.)
> >
> > Just because C made a mistake, doesn't mean other languages have to
> > slavishly follow it.
> 
> I would have thought there was more rapport between the two languages. 
> Python is often implemented in C and extensions are often implemented in 
> C, suggesting there are quite a few people familiar with both, sometimes 
> in areas that are critical (ie. creating code that will affect thousands 
> of Python apps).

There are mistakes and there are mistakes.
Like there are crimes and crimes, some get community service, some get death sentence

> 
> So why pretend that ++ and -- don't exist? After all Python borrows "=", 
> "==" and "!=" from C.
> 
> (Writing a==b instead of a=b is less likely in Python than in a language 
> where a=b is an equality test rather than assignment. But I've used just 
> such a language where mistakenly writing a=b (which happens when 
> switching between languages) caused difficult-to-find bugs.

Yeah I happen to me in that minuscule minority that regards '= denotes 
assignment' a bigger mistake than ++

Interestingly K&R admitted to the fact that the precedences of C have errors;
in particular x&y == 0x1  groups wrong and so invariably requires brackets.
Unfortunately no such admission for the choice of the most ubiquitous operator =.


From torriem at gmail.com  Tue May  5 12:54:19 2015
From: torriem at gmail.com (Michael Torrie)
Date: Tue, 05 May 2015 10:54:19 -0600
Subject: Is it normal to cry when given XML?
In-Reply-To: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
Message-ID: <5548F5BB.8050409@gmail.com>

On 05/05/2015 03:28 AM, Sayth Renshaw wrote:
> Hi
> 
> Just checking if the reaction to cry when given XML is normal.

I'd say it is normal.  XML is like violence.  If it doesn't solve your
problems, you're not using enough of it[1].

[1] Can anyone tell me who originated this line?


From invalid at invalid.invalid  Tue May  5 12:58:10 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Tue, 5 May 2015 16:58:10 +0000 (UTC)
Subject: Is it normal to cry when given XML?
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
Message-ID: <miasr2$b0t$1@reader1.panix.com>

On 2015-05-05, Sayth Renshaw <flebber.crue at gmail.com> wrote:

> Just checking if the reaction to cry when given XML is normal.

It depends on whether or not you're allowed to swear and throw things.

-- 
Grant Edwards               grant.b.edwards        Yow! I just heard the
                                  at               SEVENTIES were over!!  And
                              gmail.com            I was just getting in touch
                                                   with my LEISURE SUIT!!


From worship.brother at gmail.com  Tue May  5 12:59:03 2015
From: worship.brother at gmail.com (worship.brother at gmail.com)
Date: Tue, 5 May 2015 09:59:03 -0700 (PDT)
Subject: CHALLENGE HELP - GOOGLE DEVELOPER DAY
Message-ID: <c82211b3-4e58-4827-a2c1-64b824193b32@googlegroups.com>

Good afternoon everyone.

I'm with the following exercise of the option is a modification of a google developer day exercise.

SOMEONE HELP ME IN THIS CHALLENGE?

Archaeologists have found a scroll with the following texts:

txtA = '' 'cncdbm pjcjzct vdbbxdtw rfqsr mkt gvhkcsvw qcxr kmk pnhc zwwdsd pgjjhr lxzscps lmbjjx LGH mdcqlbwx ppzpvfbv ??vdszkb NJV nvmfhshh ztvkmv dfbnht xfpj nlbcwrvv dcrzslrf WZB krdb zndtfmf fwwm vmqzmg cpcnpnww nkjk ncrmzr jfmcl hxr vcj wcptgbc gvbfxtbv wcjlrs psjc lrljqdct ltjwn vmhp wnlmw rkvg rtm djsv rwjls NGC zcjjttpt kbg tdjzcwj pmgtzlng zvtnbs svztn pctgq qnghqcch mclvp qdht cbk lvqckrl qmwknt CQW jfxzx tkljpkfc mjhskxjh lfjvrhr whshcrk kvtgzpl gxglr mpqxtbzb zbrpktxj knhmmtks MHJ xdlsm wcc sbrkdjmh gpn vwddsdl mqfsrwt khhmfng qkkjgg QDR qlpt njq xlc jfhhv kgzsb lqncvrh JGW xhjk krprtxf dtrsfb rwtzbhs qbvvz nqbh pdfqlsv hhrrx PVF mdvcfqkt bsb gcnkszfk rkzlhbnx njfqrzxj gtvrwp SMR qddg gqjjcbjw khlbtg cdswmbgb lmbfvsv nfhvn jbnplx dmp hmnf bzrznlh gkh vqdmjdsb ktbcx fgvsxzp gmv xftg lxtf dcxsrkl tzwvbkfq tpvdrlf khzzw srtfttv lmknq jvkwr bplzzpjw rgtrj gcmdfzdx ksptr jzdfqq xmtzz pkqxrsw gvwg fqppvgk hsc hdj gthkq trxl gkhp rbr skxpc lbkw bbdqpvn llshm bhrfvh gjw zgnlpgwl pdhqn ttjtx vfb djftp kkrr ptmnzqkg tmjlqw pcvpwjb ggd jntktdz Vth WTSP mlfmddsn phn hrdqrds phbq vkffrqvt zxljnd LQG blnw brxk skwkh vqbq dlbnhz xpbddsjw gscvghmj pxcpvkmm jsfpllns kpmmgxb pljpvcn wknwbcq wthwv bdjxr wdc lqmtsnrn wbcjprr htldnxcn QWL fvjdqd cwmb bfjmw chv blx xblfrb gjbbg njqw mkmnwtzv dgsxl tgtclv xxcfpp brzxg lbrnrsf pfcpt wmvjxdsw jmstqwx zjcrkzm ndtbsqsh jqbcddqh ggvf kcr mfhllb lbbkssx gsrxcb rwtxljp cwkc xwxjhbc pfcgz zbdbsq wzfsf gqhdfchv cgp kwvnrfm ptmzl bkjcm dbl rskfms mqdgvskt hbtjpvm swff vjq dwlrgxtm kqzrlxz vmpdxkv bwfgzmjn zzrr MLSM kknhkq jhtxc gjvnj mcld vftnnd pgbkhc kzzsbq xqbs hlkxtc JXG vttn zmnc kwhcrd bfjbs lxkmnzx pbmp Int ZTM fkzql bjgllxv bghqpnl hphvdvl lwlqdh xldsnqds tsmrhhtm gnksf lzhr RQM jvcmkbxv nsx cxpplqbq jzrr tnt wpfl lnkfqjt tplwbr hzrv mxqhpg xxzdlwzx cgrtr jbvqmb cmsqdjx qcrx ljxh jbnw lgf tzd cltdd jtmdmt djbgqqk tgsffh hbff jjqn wtlsx qlmhxrrf sklfrc dwsk rgpgqz zhzvm brprszrc hqlgx tdbsgf fkmrtn frskk qjvg jlhpgh rxrmqp nmc dxpx lljs kszjq hlxx nbkvsrf dggshkxz
nlvgr zldk tvphkg hlnls wlsxsvf mmksm lzgfnkmg tbw nrpzqfr gfc sxcdqtlt rmmhtmbp jkk fdh jcw nwjjkrdq kztrcqxt lphf frs dcsbhwp tgnq LMQ tpppxf vmd tnbqgv xxtlt ljdz hdslhv dkzxctcn qsfctn tdsbdhv dxmkk ghfntj dckqls knlwvk mrddntg fwqwfxs tzqcvz sbnfjs kcxl dsgnhsf KGJ mfm vjmmf wptc rtb dtblv FRP wwmngbk dxss txz fzlxll lbqjrp hhcxnnhn skhcwxw lllkssb dgwd lzq czhhtclr bpngkf krh bzccxd rqkr mdz PHK dfctpgr GKT bdnfbh wlj lfhkmb crbhzfs qzz jbmlsfkx htkmdbx mxjccgv FPJ ttl jbw PMNT khqr jhztz dvmj kwchwcv xggzgkln DXZ fdsfsc CgPN pphslgf jxsd fsp spzzbc trvpx WJG nmdzgmnr qslm znngdwb zqwb dcgxxcqm kcdwcpdm ngqzs SBST ltqxkt tjgmjzh nmmqnzfn ngsbphqb gsqfgjn jtwhxdx PQC jsfkd ccx dfj wrgf kslnx zgqxqpvm rflzw jgl rsd fshm zlmtqs qfbrdg dknttlb xvqnd gtrpzhwb ngwrbrfk skkfq bzfqkpjj RZP trzz fgmcd ptg WZG mdxlzvkg rfqdh lvwht vkvqhk pdnqm gwpcph qbpsv vhxdj VXF tvpjwzb tjpdp frtvg xrqhp zjqcbxf fpbwbzb whnjkv dqfjwtv mjwrncd xvgkv xrsrjv zdrfjwc lmbtkhch srwg LVQ bvrjztfm vwmb rpxwnbrw cmvjrf cdxsgqgs mtrfm wrzct pznlvfk cgcdvbpt cnpqtw sfwsnkkw qjlmkkmv wrsctd lwdbkws qhvszv jzzpb jlgz ftdqspbf vvsvkq znktlpx vcvccs kswfwbzv hwfbd grplms dfvt pmmd nvb tgjpq hgsx qgsvbgws gzrq kmbgdwm tgmwmsc lsmphpk pwpznr mhcskk bmzqtdx pgwd jplzc cbkgf zbmclc tzhqvbt wkq hljqwkz vgclcmdx ggzb vsng tjw ckxtmqx wffrgzp wbht rsqb gqwxpncw mkszj mhlrmd pcl jqjgd rdrphff ftzkqg dgrhmgn ZGL plkxf hzdb mmj plphnnv jvvc tlthnhrh ngkgnln NFV rtxct ppsp rcfxhhh mbzkdw smdlrm cstjtb rhhmzvp tqbs szbmqd gbn fjmt fcppm qdgqls gxltm mdrgqdht vxpbxrdj twxcfxk qzj wtfh vglkdghk xtdzrz rjldhzld mbd fgrfb hffjd hcr vghjwkvl pfkjshg rrflt zflwbn xffjdlfs bbzvs wdxmfr fvntg twjhgcc zwvwrnn gcnzl ftfpd wfqxrnzf mbccsd szltzjm kpbslq wxxchz szzh TGQ jxnng cmrgdh pdxjxpxr bslbmwm mdkc nqjflf VSRP gbprtv mkfsfcwd ZKF pqsq chbb bmt smtkrxjx nfkltv cvhxd zwwx bzqcnzwp wcpn jmqkbclx '' '. split ()

txtB = '''pwbfdmtc jms gswg wvsscb ffq lbrhbn lcxc hcr thc mghts vkgfc nrvfgs dsrdq tcmfz scqskgsl twgzh whts dqt twtksl lcrdlc dpzrl hwlqvc xcfstz rfkvbr bzmvqp qxrs jlmwtcs nmjkkmpg kbcbg shdf qxpm qbm hlcnqnw jwhvvrtr kccw njxtbh hbtn lqmxbx krnn hcv ptqtwp xgnfggb bjdd nfgkxsw kgzcf bgncx rbsfrrcf vjwsjpw jbtcbqm xhhg kfpqcpx bfxlg qddzdv rvfqp hphjhns xhk npdd gsxm ffkbj gwdxkhr ddqmr jnzznp jzsgkb lcgsgjvh xvsbdw klzsxz xpjkjxc gth dtrmkn qzcsksd vsdrhj vtlxg kdtxsj sgs chnz bdllcsdl trggnlpd gwbvj stnhs vqbhj tdhps sgkk mxnswm ghm sqhfcnlk lpwqpn gcgg mjxh prmqclss zfn gplktxj vkjnkkv fzzx vdslwsdk fxt pnbqqbk ksfgcvw hxfq xxd rvqzhmm ctvfgxzv nrzdkx nsxmr bnvkmhcl srvc nczkp zbgsxg nmpx vrqq xfmnsjc zszmrfjv cbwjfldn fgn mzpp crjnct cmh cwh cvdk cslq tvr gggck pfs thfdcpxt fcffvg bwxr bstbzwsx ghhq ldzzlkg zmfkxvms lfn zzzfchrk lktdlrlx wzjcvj nbbkqjt lthk wnxsmnx dzftkjr fqfqcjtd dzsvqbnx zhprp cqlphsk cjvrwg fkhr mxrg tdbxnwrg sdcptlln tjsh vbmd vlgfskcx xtkdp ttjcc qtlmgsh kbndtscg nfw kctx rcltszw hbjr zld npqqm bpcqrhq szzw vbxj ghhmdq nhfptrsg vpkgwcd rkkxqk jhxpngr qnkfd wvcsmb bgnvwqln gfbrqn rjjp dvfqjqsf rspxz cvrjxjq gtps fbg rdp pbzqnc nssd rrpzcwp drfrgjx wvcpw fst frp lmz gbb brzbhlns qsjgzzzp jwvbhl pchqk vhpbdwd mwtlm wbdxk rsfpxl kqt psr bcdktps mrgnflhr fvxsmsbx rqprpvj clnr hdqzjc bndwhjwp fbbkvdhr mlgtjw ntk pxv nsnv jnwp vtksnpbb lpwl rslljk hsd rzfmdp xlbnkbw ptfhnlc jjc dpptqzc jgrt jgxn bgg hslbhksz lhld jfdjq mmttm hwjbnqwv dcxggwhc dcnhhltm fttbf xjnjhgh tchctgjn fvjkmj wkfxqzkx knbhrwgs xmszj smnrwmlv cdbdwsjf grtkzrwh rwmbvt zssswpc cdvr klhtb bhkwfwxm bdjzlg nnw hnw fkm dzxpk fmvx kwfj vbf bgp frfbhk kvqwc skddwrtg brgkfqnf xmwth wrmv rzmjrbfb pkj hckr sfbvz vtfbq fmzf tkhnb srd slkbcmj ppq kxgdbxhh grwpg hxhjznc ttmgnwb lljfz ftkgv fsjmrvcx dljps mtgnc bkwfwfnj npfvr qlgpv hmqhxfpb vvwtkrf nfchzb phmhxkck ngrngr lvd dgbwpk txlttnpb ppldgl wsmngb xtxsblgt xxtctgsj pbvtkm pmcmrmvf phcxvpf wtbfv mvclz dvsl tmzxrrg gbjz dtlsp klmjxg fxh svtlgdl vvlhntpt zgtkjdm lbjrnmt fbbhqvg dwqnsgj bjjcsvms tlpzlxj bcw rtvmzn kjtqpxhw zkvkdxz dcx zqmsnl rvqw kgsh gbwdh wslrbz pfnpqh mgj kgmq hpzmp kpr jgz bksx lvsbxzv qgzf qcgpc pvf xlt znjntxpj stgwsc vxgcfc cbvwhf tbxwpk nbjbkrz rgc wps rjlfpch bxqhw nckdtf bsncq cnsmqxwn mzmlgpp vnxr qgjs vpkpbsn tgmw lbcxxgsf nnmr wbpdssgm nmddl zcbpcbpt twrkx sdqxsnw lntr rzv hgjjksxf qpnnjwl hbcv fkwkbd ncv plr lmpfkk dpcj jzjbjgp bdttl wrrdnmjz mxqxqdxc shztl gdzj rntpnh rjrlrfk rncp qlrhnww rdzhzx qnnxhm gtd lqklxr gpgpqtrc hfhp hxl bnr fpvxzwmx pfrxglb xmchrvwx wbnxl vjxgbs vddhjkq wndwxs mqndvm hvbncjw pbmlw hzjwqn nfgxqmb pfvnpwj xbwknvmr xtm cnxck qnmtrvx kmhj hdfrtd gqz srlml ckx pwlhnpgf rkln tvq vjgrlfs vpvwnjtg wbswcvbh dzcjppjm slt zvxhgq xhcvvc rjd xhqdvhmp nqlnsk hxmjpmnv sjwwc hbjvpw dpmdnz sxpb qznnxl nwnlmbx vdb hgkkwd znsxfqs kqwjtrcg vhbnd rpgtkzz fmt nmzhrrqn qbqbvpsm kqwxr gvp xvrvsdf pxwt vkdns dpf jwnwz mxpwc xdvs drrlpnr xvpztf pxzm jtg fvfgnzx qndpq dmzwnfgm jzknzgk clbpzcpd xhxsqp zbfck btzjd jwbt gwtll kqj wlsdx sdvnw mqpvxk kjdkt frgwz mpqnqr lpj gvc hcdp zpvrdnc ckvmtbvf bddvc mptrq xrzwj lzlbc pvgkrhd wlkdtjz pslzhzhc qmrr crkxcs jtxhfvr qzd fwrgdmjt cmg xvhcb zmllbxs mxg plzxjqlk cwnf mqt hlsssh lvmptxcd zdbsvmll wshnn xzrz xsnhn jhg jtkqhh kcsb bgsfnz mfxmqjn glzb qtwhllw nfkjfn xgw mvssxl hpb vjhlfgld cgfwq qdvjskx ntnhcl ckm rqrsw dpff krrkl mcs xnk jpnx llw ljhqlbhs njdm gph nwmm bcclbzz wjfktwv mgthn kltqfx hqntlps bdr dqtswd vqmkgkb pmznqzh mwgf nndtsx xfrmgqqj mvkfdhh qxp pvpcmx mhnhb slw clvtxn nfpnlr tsssrk rnvdjpc ptkp hrwx zgblvhlj lqrdrz bhtlqhvv mlpkx jsl vlj kbmfjgs
ktzb wrnn ztbcph lxccgcxh bkrhjtsl cbmhp hwswwqg rnwqq srhnz fkvl kcnr qbxwpg hnss gjdn rnxhwgd jgngwzc kfvg nwkjt rhjtsvv txk szkpmn nnzbqwgs pjjzqkvx bkw dfcbw rffn qph kckksgp nzn tpqnm znzppsg tvcgnrb zgdsp tqlqrf vjqqxsp pwj pgft cvl cvr cnhgxsd lkd qlw vwtbh mfxs gbgw'''.split()


These scrolls are in the ancient and mysterious language Googlon. After many years of study, linguists already know some features that language. First, Googlon letters are sorted into two groups: the letters f, t, s, j and c are called "letters fatecsjc type", while the remaining are known as "type letters zerg." Linguists have found that the prepositions in Googlon are the words with number of pairs letters and ending a zerg type letter, but where there is the letter x. Therefore, it is easy to see that there are 166 prepositions in the text A. 

QUESTION A) and the text B, there are many prepositions? Another interesting fact discovered by linguists is that in Googlon, verbs are always 5 letter words starting in font zerg. Also, if a verb ends with a letter fatecsjc type, the verb is in first person. Thus, reading the text A, you can identify 69 verbs in the text, of which 19 are in the first person. 

QUESTION B) You Text B, as there are verbs? 

QUESTION C) How many B Text verbs are in first person? 

A professor of FATEC SJC use texts A and B to teach Googlon students in Algorithms discipline. To help students understand the text, the teacher must create a vocabulary list for each text, ie, an ordered list (without repetitions) of the words that appear in each of the texts. These lists should be sorted and can not occur repeats the words. In Googlon as well as in our alphabet, words are sorted lexicographically, but the problem is that in Googlon, the order of letters in the alphabet is different from ours. Alphabet Googlon, in order, is: tshjmpnzwlrcbxkqvdgf. Thus, when making these lists, the teacher must respect the alphabetical order Googlon.

The teacher prepared the list (ordered) vocabulary for the text A:

listaA = '''ttjtx ttl tsmrhhtm tjpdp tjw tjgmjzh tmjlqw tpppxf tplwbr tpvdrlf tnt tnbqgv tzhqvbt tzwvbkfq tzqcvz tzd twjhgcc twxcfxk tlthnhrh trzz trxl trvpx tbw txz tkljpkfc tqbs tvphkg tvpjwzb tdsbdhv tdjzcwj tdbsgf tgtclv tgsffh tgjpq tgmwmsc tgnq tgq smtkrxjx smdlrm spzzbc szzh szltzjm szbmqd swff srtfttv srwg sbst sbnfjs sbrkdjmh sxcdqtlt skhcwxw skwkh sklfrc skxpc skkfq svztn sfwsnkkw htldnxcn htkmdbx hsc hhrrx hhcxnnhn hmnf hphvdvl hzrv hzdb hwfbd hljqwkz hlnls hlxx hlkxtc hrdqrds hcr hbtjpvm hbff hxr hqlgx hdslhv hdj hgsx hffjd jtmdmt jtwhxdx jsfpllns jsfkd jhtxc jhztz jjqn jmstqwx jmqkbclx jplzc jntktdz jzzpb jzrr jzdfqq jlhpgh jlgz jcw jbmlsfkx jbnplx jbnw jbw jbvqmb jxsd jxnng jkk jqjgd jqbcddqh jvcmkbxv jvkwr jvvc jgw jgl jfhhv jfmcl jfxzx mtrfm mhj mhlrmd mhcskk mjhskxjh mjwrncd mmj mmksm mpqxtbzb mlsm mlfmddsn mrddntg mclvp mcld mbzkdw mbccsd mbd mxjccgv mxqhpg mkt mkszj mkmnwtzv mkfsfcwd mqdgvskt mqfsrwt mdz mdrgqdht mdcqlbwx mdxlzvkg mdkc mdvcfqkt mfhllb mfm ptmnzqkg ptmzl ptg psjc phn phbq phk pjcjzct pmmd pmnt pmgtzlng ppsp pphslgf ppzpvfbv pnhc pznlvfk pwpznr pljpvcn plphnnv plkxf pctgq pcl pcvpwjb pbmp pxcpvkmm pkqxrsw pqsq pqc pvf pdhqn pdnqm pdxjxpxr pdfqlsv pgjjhr pgwd pgbkhc pfcpt pfcgz pfkjshg nsx njq njqw njv njfqrzxj nmmqnzfn nmc nmdzgmnr nwjjkrdq nlbcwrvv nlvgr nrpzqfr ncrmzr nbkvsrf nkjk nqjflf nqbh nvmfhshh nvb ndtbsqsh ngsbphqb ngwrbrfk ngc ngkgnln ngqzs nfhvn nfkltv nfv ztm ztvkmv zhzvm zjcrkzm zjqcbxf zmnc znngdwb znktlpx zndtfmf zzrr zwwx zwwdsd zwvwrnn zlmtqs zldk zcjjttpt zbmclc zbrpktxj zbdbsq zxljnd zkf zqwb zvtnbs zdrfjwc zgnlpgwl zgl zgqxqpvm zflwbn wtsp wthwv wtlsx wtfh whshcrk whnjkv wjg wmvjxdsw wptc wpfl wnlmw wzb wzg wzfsf wwmngbk wlsxsvf wlj wrsctd wrzct wrgf wcjlrs wcptgbc wcpn wcc wbht wbcjprr wxxchz wknwbcq wkq wdc wdxmfr wfqxrnzf wffrgzp ltjwn ltqxkt lsmphpk ljxh ljdz lmbtkhch lmbjjx lmbfvsv lmknq lmq lphf lnt lnkfqjt lzhr lzq lzgfnkmg lwlqdh lwdbkws llshm lljs lllkssb lrljqdct lbrnrsf lbbkssx lbkw lbqjrp lxtf lxzscps lxkmnzx lqmtsnrn lqncvrh lqg lvwht lvq lvqckrl lgh lgf lfhkmb lfjvrhr rtm rtb rtxct rskfms rsqb rsd rhhmzvp rjldhzld rmmhtmbp rpxwnbrw rzp rwtzbhs rwtxljp rwjls rrflt rcfxhhh rbr rxrmqp rkzlhbnx rkvg rqm rqkr rdrphff rgtrj rgpgqz rflzw rfqsr rfqdh cstjtb chbb chv cmsqdjx cmrgdh cmvjrf cmgp cpcnpnww cnpqtw
cncdbm czhhtclr cwmb cwkc cltdd crbhzfs ccx cbk cbkgf cxpplqbq ckxtmqx cqw cvhxd cdswmbgb cdxsgqgs cgp cgpn cgrtr cgcdvbpt bslbmwm bsb bhrfvh bjgllxv bmt bmzqtdx bpngkf bplzzpjw bzrznlh bzccxd bzqcnzwp bzfqkpjj bwfgzmjn blnw blx brprszrc brzxg brxk bbzvs bbdqpvn bkjcm bvrjztfm bdjxr bdnfbh bghqpnl bfjmw bfjbs xtdzrz xhjk xmtzz xpbddsjw xwxjhbc xlc xldsnqds xrsrjv xrqhp xblfrb xxtlt xxzdlwzx xxcfpp xqbs xvqnd xvgkv xdlsm xgj xggzgkln xftg xfpj xffjdlfs ktbcx ksptr kszjq kswfwbzv kslnx khhmfng khzzw khlbtg khqr kmbgdwm kmk kpmmgxb kpbslq knhmmtks knlwvk kztrcqxt kzzsbq kwhcrd kwchwcv kwvnrfm krh krprtxf krdb kcr kcxl kcdwcpdm kbg kknhkq kkrr kqzrlxz kvtgzpl kgj kgzsb qslm qsfctn qhvszv qjlmkkmv qjvg qmwknt qnghqcch qzj qzz qwl qlmhxrrf qlpt qcrx qcxr qbpsv qbvvz qkkjgg qdht qdr qddg qdgqls qgsvbgws qfbrdg vttn vth vsng vsrp vhxdj vjmmf vjq vmhp vmpdxkv vmqzmg vmd vwmb vwddsdl vcj vcvccs vxpbxrdj vxf vkvqhk vkffrqvt vqbq vqdmjdsb vvsvkq vdszkb vdbbxdtw vghjwkvl vglkdghk vgclcmdx vftnnd vfb dtrsfb dtblv dsgnhsf djsv djbgqqk djftp dmp dpl dwsk dwlrgxtm dlbnhz dcsbhwp dcrzslrf dcxsrkl dckqls dcgxxcqm dxss dxmkk dxpx dxz dknttlb dkzxctcn dqfjwtv dvmj dgsxl dgwd dgrhmgn dggshkxz dfj dfctpgr dfbnht dfvt gthkq gtrpzhwb gtvrwp gsrxcb gscvghmj gsqfgjn ghfntj gjw gjbbg gjvnj gmv gpn gnksf gzrq gwpcph grplms gcmdfzdx gcnzl gcnkszfk gbprtv gbn gxltm gxglr gkt gkh gkhp gqhdfchv gqjjcbjw gqwxpncw gvhkcsvw gvwg gvbfxtbv ggzb ggvf ggd gfc ftzkqg ftdqspbf ftfpd fshm fsp fjmt fpj fpbwbzb fzlxll fwwm fwqwfxs frtvg frs frskk frp fcppm fkmrtn fkzql fqppvgk fvjdqd fvntg fdsfsc fdh fgmcd fgrfb fgvsxzp'''.split()

QUESTION D) How would the vocabulary list Text B? But as Googlons write numbers? Well Googlon in the figures are also word data in the base 20 where each letter is a digit, digits are ordered from least significant to most significant (the inverse of our system). That is, the first position is the drive position the second valley 20, the third valley 400, and so on. The values ??of the letters are given in the order that they appear in the alphabet Googlon (which is different from our order, as we have seen above). That is, the first letter of the alphabet is Googlon the digit 0, the second is the digit 1, and so on. For example, cncdbm word has the numerical value of 14860531. The Googlons consider a nice number if it satisfies these two properties: ? The sum of the digits is even not occur digit repeated ? When we consider the text as a list of numbers (ie is interpreting every word as a number using the convention explained above), we note that there are 153 beautiful numbers. QUESTION E) And the Text B, how many beautiful numbers are there?


From kashifrana84 at gmail.com  Tue May  5 13:09:30 2015
From: kashifrana84 at gmail.com (Kashif Rana)
Date: Tue, 5 May 2015 10:09:30 -0700 (PDT)
Subject: Writing list of dictionaries to CSV
Message-ID: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>

Hello Experts

When I am writing list of dictionaries to CSV file, the key 'schedule' has value 'Mar 2012' becomes Mar-12. I really do not have clue why thats happening. Below is the code. 

dic_1 = {'action': 'permit',
 'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169',
 'from': 'DMZ Web',
 'id': '1000',
 'log': 'Enable, session-init',
 'name': 'Test Rule Temporary ',
 'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL',
 'src-address': 'sparkeregap1, sparkeregap2',
 'to': 'Trust'}

dic_2 {'action': 'permit',
 'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net',
 'from': 'DMZ Web',
 'id': '4000',
 'log': 'Enable, session-init',
 'schedule': 'Mar 2012',
 'service': 'SMTP',
 'src-address': 'sparkeregap1, sparkeregap2',
 'to': 'Trust'}
 
 my_list = 
 [{'to': 'Trust', 'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL', 'from': 'DMZ Web', 'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169', 'name': 'Test Rule Temporary ', 'action': 'permit', 'id': '1000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable, session-init'}, {'to': 'Trust', 'from': 'DMZ Web', 'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net', 'service': 'SMTP', 'schedule': 'Mar 2012', 'action': 'permit', 'id': '4000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable, session-init'}]

 pol_keys = ['id', 'name', 'from', 'to', 'src-address', 'dst-address', 'service', 'action', 'nat_status', 'nat_type', 'nat_src_ip', 'nat_dst_ip', 'nat_dst_port', 'log', 'schedule']
 
with open('test.csv', 'wb') as f:
	w = csv.DictWriter(f, pol_keys)
	w.writeheader()
	w.writerows(my_list)


From davea at davea.name  Tue May  5 13:10:39 2015
From: davea at davea.name (Dave Angel)
Date: Tue, 05 May 2015 13:10:39 -0400
Subject: Step further with filebasedMessages
In-Reply-To: <877fsngi7i.fsf@Equus.decebal.nl>
References: <87oalzh0d5.fsf@Equus.decebal.nl>
 <mailman.123.1430824107.12865.python-list@python.org>
 <877fsngi7i.fsf@Equus.decebal.nl>
Message-ID: <5548F98F.7050202@davea.name>

On 05/05/2015 11:25 AM, Cecil Westerhof wrote:

>
> I have a file with quotes and a file with tips. I want to place random
> messages from those two (without them being repeated to soon) on my
> Twitter page. This I do with ?get_random_message?. I also want to put
> the first message of another file and remove it from the file. For
> this I use ?dequeue_message?.
>

Removing lines from the start of a file is an n-squared operation. 
Sometiomes it pays to reverse the file once, and just remove from the 
end.  Truncating a file doesn't require the whole thing to be rewritten, 
nor risk losing the file if the make-new-file-rename-delete-old isn't 
quite done right.

Alternatively, you could overwrite the line, or more especially the 
linefeed before it.  Then you always do two readline() calls, using the 
second one's result.

Various other games might include storing an offset at the begin of 
file, so you start by reading that, doing a seek to the place you want, 
and then reading the new line from there.


Not recommending any of these, just bringing up alternatives.

-- 
DaveA


From tjreedy at udel.edu  Tue May  5 13:15:37 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 05 May 2015 13:15:37 -0400
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
Message-ID: <miatrr$e03$1@ger.gmane.org>

On 5/5/2015 11:22 AM, Paul Moore wrote:
> I'm working my way through the asyncio documentation. I have got to
> the "Tasks and coroutines" section, but I'm frankly confused as to
> the difference between the various things described in that section:
> coroutines, tasks, and futures.
>
> I think can understand a coroutine. Correct me if I'm wrong, but it's
> roughly "something that you can run which can suspend itself".

The simple answer is that a coroutine within asyncio is a generator used 
as a (semi)coroutine rather than merely as an iterator.  IE, the send, 
throw, and close methods are used.  I believe PEP 492 will change that 
definition (for Python) to any object with with those methods.

However, if an asyncio coroutine is the result of the coroutine 
decorator, the situation is more complex, as the result is either a 
generator function, a coro function that can wrap any function, or a 
debug wrapper.  Details in asyncio/coroutine.py.

> I concede that I've not read the rest of the asyncio documentation in
> much detail yet, and I'm skipping everything to do with IO (I want to
> understand the "async" bit for now, not so much the "IO" side). But I
> don't really want to dive into the details while I am this hazy on
> the basic concepts.

You might try reading the Python code for the task and future classes.
asyncio/futures.py, asyncio/tasks.py

-- 
Terry Jan Reedy



From python.list at tim.thechases.com  Tue May  5 13:34:22 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Tue, 5 May 2015 12:34:22 -0500
Subject: Writing list of dictionaries to CSV
In-Reply-To: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
Message-ID: <20150505123422.5528885f@bigbox.christie.dr>

On 2015-05-05 10:09, Kashif Rana wrote:
> When I am writing list of dictionaries to CSV file, the key
> 'schedule' has value 'Mar 2012' becomes Mar-12. 

How are you making this determination?  Are you looking at the raw
CSV output, or are you looking at the CSV file loaded into a
spreadsheet like Excel?

I suspect that it's a matter of the file being what you want, but
your spreadsheet mangling/reformatting it visually.

-tkc





From marko at pacujo.net  Tue May  5 13:45:27 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 05 May 2015 20:45:27 +0300
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
Message-ID: <878ud27waw.fsf@elektro.pacujo.net>

Paul  Moore <p.f.moore at gmail.com>:

> But I don't understand what a Future is.

A future stands for a function that is scheduled to execute in the
background.

Personally, I have never found futures a very useful idiom in any
language (Scheme, Java, Python). Or more to the point, concurrency and
the notion of a function don't gel well in my mind.

See also: <URL: http://en.wikipedia.org/wiki/Futures_and_promises>

> A Task is a subclass of Future, but the documentation doesn't say what
> it *is*,

A task is almost synonymous with a future. It's something that needs to
get done.

--

I think asyncio is a worthwhile step up from the old asyncore module.
Its main advantage is portability between operating systems. Also, it
has one big advantage over threads: events can be multiplexed using
asyncio.wait(return_when=FIRST_COMPLETED).

However, I happen to have the privilege of working with linux so
select.epoll() gives me all the power I need to deal with asynchrony.
The asyncio model has the same basic flaw as threads: it makes the
developer think of concurrency in terms of a bunch of linear streaks of
execution. I much prefer to look at concurrency through states and
events (what happened? what next?).

My preferred model is known as "Callback Hell" (qv.). Asyncio seeks to
offer a salvation from it. However, I am at home in the Callback Hell;
equipped with Finite State Machines, the inherent complexities of the
Reality are managed in the most natural, flexible manner.


Marko


From ian.g.kelly at gmail.com  Tue May  5 13:46:24 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 5 May 2015 11:46:24 -0600
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
Message-ID: <CALwzidkO_k5SbTsMv4EH8mHoC_YUNBKov_prHfzxU8AJFEj3Jw@mail.gmail.com>

On Tue, May 5, 2015 at 9:22 AM, Paul  Moore <p.f.moore at gmail.com> wrote:
> I'm working my way through the asyncio documentation. I have got to the "Tasks and coroutines" section, but I'm frankly confused as to the difference between the various things described in that section: coroutines, tasks, and futures.
>
> I think can understand a coroutine. Correct me if I'm wrong, but it's roughly "something that you can run which can suspend itself".
>
> But I don't understand what a Future is. The document just says it's almost the same as a concurrent.futures.Future, which is described as something that "encapsulates the asynchronous execution of a callable". Which doesn't help a lot. In concurrent.futures, you don't create Futures, you get them back from submit(), but in the asyncio docs it looks like you can create them by hand (example "Future with run_until_complete"). And there's nothing that says what a Future is, just what it's like... :-(

Fundamentally, a future is a placeholder for something that isn't
available yet. You can use it to set a callback to be called when that
thing is available, and once it's available you can get that thing
from it.

> A Task is a subclass of Future, but the documentation doesn't say what it *is*, but rather that it "schedules the execution of a coroutine". But that doesn't make sense to me - objects don't do things, they *are* things. I thought the event loop did the scheduling?

In asyncio, a Task is a a Future that serves as a placeholder for the
result of a coroutine. The event loop manages callbacks, and that's
all it does. An event that it's been told to listen for occurs, and
the event loop calls the callback associated with that event. The Task
manages a coroutine's interaction with the event loop; when the
coroutine yields a future, the Task instructs the event loop to listen
for the completion of that future, setting a callback that will resume
the coroutine.

> Reading between the lines, it seems that the event loop schedules Tasks (which makes sense) and that Tasks somehow wrap up coroutines - but I don't see *why* you need to wrap a task in a coroutine rather than just scheduling coroutines. And I don't see where Futures fit in - why not just wrap a coroutine in a Future, if it needs to be wrapped up at all?

The coroutines themselves are not that interesting of an interface;
all you can do with them is resume them. The asynchronous execution
done by asyncio is all based on futures. Because a coroutine can
easily be wrapped in a Task, this allows for coroutines to be used
anywhere a future is expected.

I don't know if I've done a good job explaining, but I hope this helps.


From vs at it.uu.se  Tue May  5 13:54:06 2015
From: vs at it.uu.se (Virgil Stokes)
Date: Tue, 5 May 2015 19:54:06 +0200
Subject: An improved version of the main snippet
Message-ID: <554903BE.7060101@it.uu.se>

I have attached what I believe to be an improved version of my main snippet for 
testing.

--V :-)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: YahooForexData_A3.py
Type: text/x-java
Size: 3419 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150505/cce445c8/attachment.java>

From skip.montanaro at gmail.com  Tue May  5 13:55:34 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 5 May 2015 12:55:34 -0500
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <CALwzidkO_k5SbTsMv4EH8mHoC_YUNBKov_prHfzxU8AJFEj3Jw@mail.gmail.com>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <CALwzidkO_k5SbTsMv4EH8mHoC_YUNBKov_prHfzxU8AJFEj3Jw@mail.gmail.com>
Message-ID: <CANc-5UwXV8v4cepU7zwhRY3wF5=xOxE2rBS8iGPwVWkvJ9HDXg@mail.gmail.com>

Paul> ... I'm frankly confused ...

You and me both. I'm pretty sure I understand what a Future is, and
until the long discussion about PEP 492 (?) started up, I thought I
understood what a coroutine was from my days in school many years ago.
Now I'm not so sure.

Calling Dave Beazley... Calling Dave Beazley... We need a keynote...

Skip


From p.f.moore at gmail.com  Tue May  5 13:55:41 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 5 May 2015 10:55:41 -0700 (PDT)
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <mailman.126.1430842286.12865.python-list@python.org>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <mailman.126.1430842286.12865.python-list@python.org>
Message-ID: <9eb4c2c4-fee6-403c-895a-6ff4db4fe392@googlegroups.com>

On Tuesday, 5 May 2015 17:11:39 UTC+1, Zachary Ware  wrote:
>On Tue, May 5, 2015 at 10:22 AM, Paul  Moore wrote:
>> I'm working my way through the asyncio documentation. I have got to the
>> "Tasks and coroutines" section, but I'm frankly confused as to the
>> difference between the various things described in that section:
>> coroutines, tasks, and futures.
> 
> I've been using (and, in part at least, understanding :)) asyncio for
> work for the past few months, so I can at least give you my
> impressions based on use.

Thanks for taking the time.
 
>> I think can understand a coroutine. Correct me if I'm wrong, but it's
>> roughly "something that you can run which can suspend itself".
> 
> That's basically how I understand it.

Cool, I got that right at least :-)

>> But I don't understand what a Future is.
[...]
> 
> My understanding is that Futures are somewhat like 'non-executable
> coroutines', if that makes any sense whatsoever.  Futures are used as
> something you can pass around when you need to start execution of the
> "job" in one method and finish it in another.  For instance, talking
> to a remote network entity, and you want to just do "yield from
> server.get(something)".  Your 'get' method can make the request,
> create a Future, stick it somewhere that the method monitoring
> incoming traffic can find it (along with some identifying metadata),
> and then do 'return (yield from future)' to wait for the Future to be
> fulfilled (by the listener) and return its result.  The listener then
> matches up incoming requests with Futures, and calls
> Future.set_result() or Future.set_exception() to fulfill the Future.
> Once one of those methods has been called on the Future (and control
> has been passed back to the scheduler), your server.get method
> unblocks from its '(yield from future)' call, and either raises the
> exception or returns the result that was set on the Future.

Hmm. My head hurts. I sort of think I see what you might mean, but I have no idea how that translates to "almost the same as a concurrent.futures.Future, which in my experience is nothing like that.

A c.f.Future is (in my naive view) a "promise" of a result, which you can later wait to be fulfilled. Creating one directly (as opposed to via submit) doesn't make sense because then there's nothing that's promised to provide a result.

What you described sounds more like a "container that can signal when it's been filled", which I can sort of see as related to the Future API, but not really as related to a c.f.Future.

>> Reading between the lines, it seems that the event loop schedules Tasks
>> (which makes sense) and that Tasks somehow wrap up coroutines - but I
>> don't see *why* you need to wrap a task in a coroutine rather than just
>> scheduling coroutines. And I don't see where Futures fit in - why not
>> just wrap a coroutine in a Future, if it needs to be wrapped up at all?
> 
> You kind of mixed things up in this paragraph (you said "Tasks somehow
> wrap up coroutines", then "wrap a task in a coroutine"; the first is
> more correct, I believe).

Whoops, sorry, my mistake - the first was what I meant, the second was a typo.

> As I understand it, Tasks are
> specializations of Futures that take care of the
> set_result/set_exception based on the execution of the coroutine.

That sounds far more like a concurrent.futures.Future. Except that (AIUI) tasks get scheduled, and you can list all tasks for an event loop, and things like that. So they seem like a little bit more than a c.f.Future, which would sort of match with the idea that an asyncio.Future was equivalent to a c.f.Future, except that it's not (as above).

There's also things like asyncio.async - which makes no sense to me at all (I can see how the examples *use* it, in much the same way as I can see how the examples work, in general, but I can't understand its role, so I'm not sure I'd be able to design code that used it from scratch :-()

>> Can anyone clarify for me?
> 
> I hope I've done some good on that front, but I'm still a bit hazy
> myself.  I found it worked best to just take examples or otherwise
> working code, and poke and prod at it until it breaks and you can
> figure out how you broke it.  Just try to avoid mixing asyncio and
> threads for as long as you can :).

Thanks for the insights. I think I've learned some things, but to be honest I'm still confused. Maybe it's because of the angle I'm coming at it from. I don't typically write network code directly[1], so the examples in the asyncio docs are mostly irrelevant at best for me. I do find that I use threads a reasonable amount, and I'd like to know if asyncio would be a worthwhile alternative. So I need to understand the concepts and try to apply them to my situations. Hence the comment above about designing asyncio code from scratch.

Paul

[1] When I do write network code, I typically want to use a high level library like requests. So in an asyncio context, I'm looking at how to use an existing, synchronous library, in async code. In twisted, I'd use defer_to_thread, and I think run_in_executor is the equivalent asyncio concept. The docs say that is a coroutine, so I probably need to wrap that in a task to schedule it, but that's where a proper understanding of the concepts becomes crucial...


From p.f.moore at gmail.com  Tue May  5 14:03:00 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 5 May 2015 11:03:00 -0700 (PDT)
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <mailman.130.1430848034.12865.python-list@python.org>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <mailman.130.1430848034.12865.python-list@python.org>
Message-ID: <68ed8c75-0d2f-4a87-bf55-d11a60d73cef@googlegroups.com>

On Tuesday, 5 May 2015 18:48:09 UTC+1, Ian  wrote:
> Fundamentally, a future is a placeholder for something that isn't
> available yet. You can use it to set a callback to be called when that
> thing is available, and once it's available you can get that thing
> from it.

OK, that makes a lot of sense. There seems to be two distinct strands within the asyncio world, the callback model and the task/coroutine model. AIUI, coroutines/tasks are supposed to let you avoid callbacks.

So I guess in that model, a Future isn't something you should use directly in task-based code, because it works via callbacks. And yet tasks are futures, so it's not that simple :-)

> In asyncio, a Task is a a Future that serves as a placeholder for the
> result of a coroutine. The event loop manages callbacks, and that's
> all it does.

Um, I thought it scheduled tasks?

> An event that it's been told to listen for occurs, and
> the event loop calls the callback associated with that event. The Task
> manages a coroutine's interaction with the event loop; when the
> coroutine yields a future, the Task instructs the event loop to listen
> for the completion of that future, setting a callback that will resume
> the coroutine.

OK, that (to an extent) explains how the callback and task worlds link together. That's useful.

> The coroutines themselves are not that interesting of an interface;
> all you can do with them is resume them. The asynchronous execution
> done by asyncio is all based on futures. Because a coroutine can
> easily be wrapped in a Task, this allows for coroutines to be used
> anywhere a future is expected.

And yet, futures are callback-based whereas tasks are suspension-point (yield from) based. I get a sense of what you're saying, but it's not very clear yet :-)

> I don't know if I've done a good job explaining, but I hope this helps.

It's all helping. It's a shame the docs don't do a better job of explaining the underlying model, to be honest, but I do feel like I'm slowly getting closer to an understanding.

It's no wonder terminology is such a hot topic in the discussion of the new async PEP on python-dev :-(

Paul


From python at mrabarnett.plus.com  Tue May  5 14:11:09 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 05 May 2015 19:11:09 +0100
Subject: Writing list of dictionaries to CSV
In-Reply-To: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
Message-ID: <554907BD.4040804@mrabarnett.plus.com>

On 2015-05-05 18:09, Kashif Rana wrote:
> Hello Experts
>
> When I am writing list of dictionaries to CSV file, the key 'schedule' has value 'Mar 2012' becomes Mar-12. I really do not have clue why thats happening. Below is the code.
>
> dic_1 = {'action': 'permit',
>   'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169',
>   'from': 'DMZ Web',
>   'id': '1000',
>   'log': 'Enable, session-init',
>   'name': 'Test Rule Temporary ',
>   'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL',
>   'src-address': 'sparkeregap1, sparkeregap2',
>   'to': 'Trust'}
>
Missing '=' in the next line.

> dic_2 {'action': 'permit',
>   'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net',
>   'from': 'DMZ Web',
>   'id': '4000',
>   'log': 'Enable, session-init',
>   'schedule': 'Mar 2012',
>   'service': 'SMTP',
>   'src-address': 'sparkeregap1, sparkeregap2',
>   'to': 'Trust'}
>
>   my_list =
>   [{'to': 'Trust', 'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL', 'from': 'DMZ Web', 'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169', 'name': 'Test Rule Temporary ', 'action': 'permit', 'id': '1000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable, session-init'}, {'to': 'Trust', 'from': 'DMZ Web', 'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net', 'service': 'SMTP', 'schedule': 'Mar 2012', 'action': 'permit', 'id': '4000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable, session-init'}]
>
>   pol_keys = ['id', 'name', 'from', 'to', 'src-address', 'dst-address', 'service', 'action', 'nat_status', 'nat_type', 'nat_src_ip', 'nat_dst_ip', 'nat_dst_port', 'log', 'schedule']
>
> with open('test.csv', 'wb') as f:
> 	w = csv.DictWriter(f, pol_keys)
> 	w.writeheader()
> 	w.writerows(my_list)
>
It works OK for me.

(I'm assuming that you're reading the CSV file in a text editor, not
some other application that might be trying to be "clever" by
"interpreting" what it thinks looks a date as a date and then
displaying it differently...)



From p.f.moore at gmail.com  Tue May  5 14:19:34 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 5 May 2015 11:19:34 -0700 (PDT)
Subject: Stripping unencodable characters from a string
Message-ID: <24ef6c6d-a47a-4d8c-8651-c581e25161cb@googlegroups.com>

I want to write a string to an already-open file (sys.stdout, typically). However, I *don't* want encoding errors, and the string could be arbitrary Unicode (in theory). The best way I've found is

    data = data.encode(file.encoding, errors='replace').decode(file.encoding)
    file.write(data)

(I'd probably use backslashreplace rather than replace, but that's a minor point).

Is that the best way? The multiple re-encoding dance seems a bit clumsy, but it was the best I could think of.

Thanks,
Paul.


From coolshwetu at gmail.com  Tue May  5 14:25:29 2015
From: coolshwetu at gmail.com (shweta kaushik)
Date: Tue, 5 May 2015 23:55:29 +0530
Subject: Open a Url in new tab and tab switching in IE using python and
 selenium
In-Reply-To: <CAM--9s5FDhvCyxZi7p3izsWncuYyjAynuYJBR30LUR4HeU2ogA@mail.gmail.com>
References: <CAM--9s5FDhvCyxZi7p3izsWncuYyjAynuYJBR30LUR4HeU2ogA@mail.gmail.com>
Message-ID: <CAM--9s4jMm+HyCncHtwwhCAToVXryqT5U-E7SrEb7Ce7ocAkrQ@mail.gmail.com>

Hi All,

I am trying to open a new tab in IE10 and selenium 2.45. It is able to open
a new tab using pyrobot. But when i am trying to open url in new tab, it is
getting opened in first tab. Focus is not set to second tab and hence it is
not working and also switching of tab is not working. please provide
solution. I have provided my code below:
Code:

# Open first tab
IEDriverPath = "/../../../../../../../IEDriverServer.exe"
driver = webdriver.Ie(IEDriverPath, port=5555)
pyseldriver.get("https://www.google.com/")
time.sleep(5)
tab1 = pyseldriver.current_window_handle

#open another tab
obja = pyrobot.Robot()
obja.key_press(pyrobot.Keys.ctrl)
obja.key_press(pyrobot.Keys.t)
obja.key_release(pyrobot.Keys.ctrl)
obja.key_release(pyrobot.Keys.t)
time.sleep(2)
pyseldriver.switch_to_window(pyseldriver.window_handles[-1])
tab2 = pyseldriver.current_window_handle
pyseldriver.get("https://www.python.org/")
time.sleep(5)

#Switching to first tab and opening new url
pyseldriver.switch_to_window(tab1)
pyseldriver.get("https://www.yahoo.com/")
time.sleep(10)

#switching to second tab and opening new url
pyseldriver.switch_to_window(tab2)
pyseldriver.get("https://www.gmail.com/")
time.sleep(10)

But links are not opening in new tab and switching is also not happening.
All links are getting opened in first tab itself.

Regards,
Shweta
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150505/91db5384/attachment.html>

From skip.montanaro at gmail.com  Tue May  5 14:25:32 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 5 May 2015 13:25:32 -0500
Subject: Writing list of dictionaries to CSV
In-Reply-To: <554907BD.4040804@mrabarnett.plus.com>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <554907BD.4040804@mrabarnett.plus.com>
Message-ID: <CANc-5UzYmSvQMdUOTYf-g9jF4LjortNaQ9HxkkSsbP4BUXZheQ@mail.gmail.com>

On Tue, May 5, 2015 at 1:11 PM, MRAB <python at mrabarnett.plus.com> wrote:
> I'm assuming that you're reading the CSV file in a text editor, not
> some other application that might be trying to be "clever" by
> "interpreting" what it thinks looks a date as a date and then
> displaying it differently...

More likely, viewing the CSV file in Excel, Gnumeric, or some other
spreadsheet which interprets some inputs as dates and formats them
according to its default rules.

Skip


From Cecil at decebal.nl  Tue May  5 14:44:16 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 05 May 2015 20:44:16 +0200
Subject: Writing list of dictionaries to CSV
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
Message-ID: <87vbg6g8zj.fsf@Equus.decebal.nl>

Op Tuesday 5 May 2015 19:09 CEST schreef Kashif Rana:

> When I am writing list of dictionaries to CSV file, the key
> 'schedule' has value 'Mar 2012' becomes Mar-12. I really do not have
> clue why thats happening. Below is the code.
>
> dic_1 = {'action': 'permit', 'dst-address': 'maxprddb-scan-167,
> maxprddb-scan-168, maxprddb-scan-169', 'from': 'DMZ Web', 'id':
> '1000', 'log': 'Enable, session-init', 'name': 'Test Rule Temporary
> ', 'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL',
> 'src-address': 'sparkeregap1, sparkeregap2', 'to': 'Trust'}
>
> dic_2 {'action': 'permit',
> 'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net',
> 'from': 'DMZ Web',
> 'id': '4000',
> 'log': 'Enable, session-init',
> 'schedule': 'Mar 2012',
> 'service': 'SMTP',
> 'src-address': 'sparkeregap1, sparkeregap2',
> 'to': 'Trust'}
>
> my_list = [{'to': 'Trust', 'service': '1521,
> Oraccle-Maximo-Scan-1550, PING-ALL', 'from': 'DMZ Web',
> 'dst-address': 'maxprddb-scan-167, maxprddb-scan-168,
> maxprddb-scan-169', 'name': 'Test Rule Temporary ', 'action':
> 'permit', 'id': '1000', 'src-address': 'sparkeregap1, sparkeregap2',
> 'log': 'Enable, session-init'}, {'to': 'Trust', 'from': 'DMZ Web',
> 'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net',
> 'service': 'SMTP', 'schedule': 'Mar 2012', 'action': 'permit', 'id':
> '4000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable,
> session-init'}]
>
> pol_keys = ['id', 'name', 'from', 'to', 'src-address',
> 'dst-address', 'service', 'action', 'nat_status', 'nat_type',
> 'nat_src_ip', 'nat_dst_ip', 'nat_dst_port', 'log', 'schedule']
>
> with open('test.csv', 'wb') as f:
> 	w = csv.DictWriter(f, pol_keys)
> 	w.writeheader()
> 	w.writerows(my_list)

Well, I get:
    id,name,from,to,src-address,dst-address,service,action,nat_status,nat_type,nat_src_ip,nat_dst_ip,nat_dst_port,log,schedule
    1000,Test Rule Temporary ,DMZ Web,Trust,"sparkeregap1, sparkeregap2","maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169","1521, Oraccle-Maximo-Scan-1550, PING-ALL",permit,,,,,,"Enable, session-init",
    4000,,DMZ Web,Trust,"sparkeregap1, sparkeregap2","sparkcas01, sparkcas02, email.ab.spark.net",SMTP,permit,,,,,,"Enable, session-init",Mar 2012

So there is something different on your system.


By the way: next time make sure that we can copy paste your code:
- dic_2 does not have a '='
- get rid of the space before my_list
- make sure the '[' is at the same line as my_list (or use a \)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From davea at davea.name  Tue May  5 14:45:02 2015
From: davea at davea.name (Dave Angel)
Date: Tue, 05 May 2015 14:45:02 -0400
Subject: Throw the cat among the pigeons
In-Reply-To: <87zj5jf15q.fsf@Equus.decebal.nl>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl>
Message-ID: <55490FAE.8070501@davea.name>

On 05/05/2015 12:18 PM, Cecil Westerhof wrote:

>
> Well, I did not write many tail recursive functions. But what surprised
> me was that for large values the ?tail recursive? version was more
> efficient as the iterative version. And that was with myself
> implementing the tail recursion. I expect the code to be more
> efficient when the compiler implements the tail recursion.
>


You've said that repeatedly, so I finally took a look at your webpage
 
https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py

I didn't have your framework to call the code, so I just extracted some 
functions and did some testing.  I do see some differences, where the 
so-called tail_recursive functions are sometimes faster, but I did some 
investigating to try to determine why.


I came up with the conclusion that sometimes the multiply operation 
takes longer than other times.  And in particular, i can see more 
variation between the two following loops than between your two functions.


def factorial_iterative(x, simple=False):
     assert x >= 0
     result = 1
     j=2
     if not simple:
         for i in range(2, x + 1):
             result *= i
             j += 1
     else:
         for i in range(2, x + 1):
             result *= j
             j += 1
             pass

     return result

When the "simple" is True, the function takes noticeably and 
consistently longer.  For example, it might take 116 instead of 109 
seconds.  For the same counts, your code took 111.

I've looked at dis.dis(factorial_iterative), and can see no explicit 
reason for the difference.



-- 
DaveA


From davea at davea.name  Tue May  5 15:00:38 2015
From: davea at davea.name (Dave Angel)
Date: Tue, 05 May 2015 15:00:38 -0400
Subject: Stripping unencodable characters from a string
In-Reply-To: <24ef6c6d-a47a-4d8c-8651-c581e25161cb@googlegroups.com>
References: <24ef6c6d-a47a-4d8c-8651-c581e25161cb@googlegroups.com>
Message-ID: <55491356.3030500@davea.name>

On 05/05/2015 02:19 PM, Paul Moore wrote:

You need to specify that you're using Python 3.4 (or whichever) when 
starting a new thread.

> I want to write a string to an already-open file (sys.stdout, typically). However, I *don't* want encoding errors, and the string could be arbitrary Unicode (in theory). The best way I've found is
>
>      data = data.encode(file.encoding, errors='replace').decode(file.encoding)
>      file.write(data)
>
> (I'd probably use backslashreplace rather than replace, but that's a minor point).
>
> Is that the best way? The multiple re-encoding dance seems a bit clumsy, but it was the best I could think of.
>
> Thanks,
> Paul.
>

If you're going to take charge of the encoding of the file, why not just 
open the file in binary, and do it all with
     file.write(data.encode( myencoding, errors='replace') )

i can't see the benefit of two encodes and a decode just to write a 
string to the file.

Alternatively, there's probably a way to open the file using 
codecs.open(), and reassign it to sys.stdout.


-- 
DaveA


From p.f.moore at gmail.com  Tue May  5 15:24:56 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 5 May 2015 12:24:56 -0700 (PDT)
Subject: Stripping unencodable characters from a string
In-Reply-To: <mailman.137.1430852451.12865.python-list@python.org>
References: <24ef6c6d-a47a-4d8c-8651-c581e25161cb@googlegroups.com>
 <mailman.137.1430852451.12865.python-list@python.org>
Message-ID: <a75300bf-7a30-464a-84cb-eb3cde4ca40f@googlegroups.com>

On Tuesday, 5 May 2015 20:01:04 UTC+1, Dave Angel  wrote:
> On 05/05/2015 02:19 PM, Paul Moore wrote:
> 
> You need to specify that you're using Python 3.4 (or whichever) when 
> starting a new thread.

Sorry. 2.6, 2.7, and 3.3+. It's for use in a cross-version library.

> If you're going to take charge of the encoding of the file, why not just 
> open the file in binary, and do it all with
>      file.write(data.encode( myencoding, errors='replace') )

I don't have control of the encoding of the file. It's typically sys.stdout, which is already open. I can't replace sys.stdout (because the main program which calls my library code wouldn't like me messing with global state behind its back). And sys.stdout isn't open in binary mode.

> i can't see the benefit of two encodes and a decode just to write a 
> string to the file.

Nor can I - that's my point. But if all I have is an open text-mode file with the "strict" error mode, I have to incur one encode, and I have to make sure that no characters are passed to that encode which can't be encoded.

If there was a codec method to identify un-encodable characters, that might be an alternative (although it's quite possible that the encode/decode dance would be faster anyway, as it's mostly in C - not that performance is key here).

> Alternatively, there's probably a way to open the file using 
> codecs.open(), and reassign it to sys.stdout.

As I said, I have to work with the file (sys.stdout or whatever) that I'm given. I can't reopen or replace it.

Paul


From jon+usenet at unequivocal.co.uk  Tue May  5 15:33:36 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 5 May 2015 19:33:36 +0000 (UTC)
Subject: Stripping unencodable characters from a string
References: <24ef6c6d-a47a-4d8c-8651-c581e25161cb@googlegroups.com>
Message-ID: <slrnmki6qg.2fu.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-05, Paul Moore <p.f.moore at gmail.com> wrote:
> I want to write a string to an already-open file (sys.stdout,
> typically). However, I *don't* want encoding errors, and the string
> could be arbitrary Unicode (in theory). The best way I've found is
>
>     data = data.encode(file.encoding, errors='replace').decode(file.encoding)
>     file.write(data)
>
> (I'd probably use backslashreplace rather than replace, but that's a
> minor point).
>
> Is that the best way? The multiple re-encoding dance seems a bit
> clumsy, but it was the best I could think of.

Perhaps something like one of:

  file.buffer.write(data.encode(file.encoding, errors="replace"))

or:

  sys.stdout = io.TextIOWrapper(sys.stdout.detach(),
      encoding=sys.stdout.encoding, errors="replace")

(both of which could go wrong in various ways depending on your
circumstances).


From Cecil at decebal.nl  Tue May  5 15:42:52 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 05 May 2015 21:42:52 +0200
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl>
 <mailman.136.1430851516.12865.python-list@python.org>
Message-ID: <87r3qug69v.fsf@Equus.decebal.nl>

Op Tuesday 5 May 2015 20:45 CEST schreef Dave Angel:

> On 05/05/2015 12:18 PM, Cecil Westerhof wrote:
>
>>
>> Well, I did not write many tail recursive functions. But what
>> surprised me was that for large values the ?tail recursive? version
>> was more efficient as the iterative version. And that was with
>> myself implementing the tail recursion. I expect the code to be
>> more efficient when the compiler implements the tail recursion.
>>
>
>
> You've said that repeatedly, so I finally took a look at your
> webpage
>
> https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py
>
> I didn't have your framework to call the code, so I just extracted
> some functions and did some testing.

I definitely need to take care of documentation.

It can be called with:
    python3 mathDecebal.py --factorial

The problem is that it will do correctness and speed test. I have to
split those in two different things. And use a different file for
both.

Maybe make a directory test and put a correctness_<function>.py and a
speed_<function>.py.



> I do see some differences,
> where the so-called tail_recursive functions are sometimes faster,
> but I did some investigating to try to determine why.
>
>
> I came up with the conclusion that sometimes the multiply operation
> takes longer than other times.  And in particular, i can see more
> variation between the two following loops than between your two
> functions.
>
>
> def factorial_iterative(x, simple=False):
> assert x >= 0
> result = 1
> j=2
> if not simple:
> for i in range(2, x + 1):
> result *= i
> j += 1
> else:
> for i in range(2, x + 1):
> result *= j
> j += 1
> pass
>
> return result
>
> When the "simple" is True, the function takes noticeably and
> consistently longer.  For example, it might take 116 instead of 109
> seconds.  For the same counts, your code took 111.
>
> I've looked at dis.dis(factorial_iterative), and can see no explicit
> reason for the difference.

I would say that a variable that is filled by a range is different as
a normal variable. Do not ask me why. ;-)

Even if you (general not personal you) think that the tail recursion
is a waist of time, this is an interesting result I think.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From mmr15 at case.edu  Tue May  5 15:43:23 2015
From: mmr15 at case.edu (Matthew Ruffalo)
Date: Tue, 05 May 2015 15:43:23 -0400
Subject: Writing list of dictionaries to CSV
In-Reply-To: <CANc-5UzYmSvQMdUOTYf-g9jF4LjortNaQ9HxkkSsbP4BUXZheQ@mail.gmail.com>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <554907BD.4040804@mrabarnett.plus.com>
 <CANc-5UzYmSvQMdUOTYf-g9jF4LjortNaQ9HxkkSsbP4BUXZheQ@mail.gmail.com>
Message-ID: <55491D5B.3020800@case.edu>

On 2015-05-05 14:25, Skip Montanaro wrote:
> More likely, viewing the CSV file in Excel, Gnumeric, or some other
> spreadsheet which interprets some inputs as dates and formats them
> according to its default rules. Skip 

This is depressingly common, and I've even received CSV and plain text
data files that had some gene names(!) coerced to dates via a trip
through Excel. SEPT9 was one such gene, I believe.

MMR...


From siv.devops at gmail.com  Tue May  5 15:55:20 2015
From: siv.devops at gmail.com (pra devOPS)
Date: Tue, 5 May 2015 12:55:20 -0700
Subject: Json Comaprision
Message-ID: <CANvYX9V=+HJ9-Lhbbwyg7yPnmFqmwqDrdS_0YTd7X4VNmYSaDw@mail.gmail.com>

Hi All:


I wanted to compare two json files ignoring few of the keys in the json
files.

Can anybody suggest me few things?

Thanks,
Siva
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150505/e725efdb/attachment.html>

From marko at pacujo.net  Tue May  5 15:55:43 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 05 May 2015 22:55:43 +0300
Subject: Stripping unencodable characters from a string
References: <24ef6c6d-a47a-4d8c-8651-c581e25161cb@googlegroups.com>
 <mailman.137.1430852451.12865.python-list@python.org>
 <a75300bf-7a30-464a-84cb-eb3cde4ca40f@googlegroups.com>
Message-ID: <87zj5i6bpc.fsf@elektro.pacujo.net>

Paul  Moore <p.f.moore at gmail.com>:

> Nor can I - that's my point. But if all I have is an open text-mode
> file with the "strict" error mode, I have to incur one encode, and I
> have to make sure that no characters are passed to that encode which
> can't be encoded.

The file-like object you are given carries some baggage. IOW, it's not a
"file" in the sense you are thinking about it. It's some object that
accepts data with its write() method.

Now, Python file-like objects ostensibly implement a common interface.
However, as you are describing here, not all write() methods accept the
same arguments. Text file objects expect str objects while binary file
objects expect bytes objects. Maybe there are yet other file-like
objects that expect some other types of object as their arguments.

Bottom line: Python doesn't fulfill your expectation. Your library can't
operate on generic file-like objects because Python3 doesn't have
generic file-like objects. Your library must do something else. For
example, you could require a binary file object. The caller must then
possibly wrap their actual object inside a converter, which is
relatively trivial in Python.


Marko


From ian.g.kelly at gmail.com  Tue May  5 16:30:50 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 5 May 2015 14:30:50 -0600
Subject: Throw the cat among the pigeons
In-Reply-To: <55490FAE.8070501@davea.name>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl> <55490FAE.8070501@davea.name>
Message-ID: <CALwzidmuhD0HhAZKRGMbgEgVVHf05_3pw=+o2JyAwWdiOtcNNA@mail.gmail.com>

On Tue, May 5, 2015 at 12:45 PM, Dave Angel <davea at davea.name> wrote:
> When the "simple" is True, the function takes noticeably and consistently
> longer.  For example, it might take 116 instead of 109 seconds.  For the
> same counts, your code took 111.

I can't replicate this. What version of Python is it, and what value
of x are you testing with?

> I've looked at dis.dis(factorial_iterative), and can see no explicit reason
> for the difference.

My first thought is that maybe it's a result of the branch. Have you
tried swapping the branches, or reimplementing as separate functions
and comparing?


From tjreedy at udel.edu  Tue May  5 16:46:06 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 05 May 2015 16:46:06 -0400
Subject: Throw the cat among the pigeons
In-Reply-To: <87zj5jf15q.fsf@Equus.decebal.nl>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl>
Message-ID: <miba6g$19d$1@ger.gmane.org>

On 5/5/2015 12:18 PM, Cecil Westerhof wrote:
> Op Tuesday 5 May 2015 17:47 CEST schreef Paul Moore:
>
>> On Sunday, 3 May 2015 16:23:59 UTC+1, Cecil Westerhof  wrote:
>>>> By the way: I think that even if the recursion does not go further
>>>> as 500, it is still a good idea to use tail recursion. Why use
>>>> stack space when it is not necessary?
>>>
>>> I pushed the example to GitHub:
>>> https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py
>>
>> You already know this, as your code shows, but tail call recursion
>> elimination is only possible when you have a *direct* tail call (one

An 'indirect tail call' would not be a tail call.

>> with the result of the tail call returned immediately to the
>> caller). Even the standard trivial factorial example doesn't have a
>> direct tail call,  without rewriting to use an accumulator variable.
>> Which is a non-intuitive transformation to anyone who's not familiar
>> with recursive functional languages and their idioms.
>>
>> If you're rewriting your recursive function *anyway*, it's not that
>> much harder in many (most?) cases to rewrite it iteratively.

For count functions, the main change between tail recursion and while 
iteration is replacing 'if' with 'while' and converting the tail call to 
assignment.  (One may have to reverse the if-else first to put the tail 
call in the if branch.)

from math import factorial as fac
print(fac(0), fac(1), fac(2), fac(6))

def fac_rt(n, i=2, res=1):
     if i <= n:
         return fac_rt(n, i+1, res*i)
     else:
         return res

def fac_iw(n):
     i = 2
     res = 1
     while i <= n:
         i, res = i+1, res*i
     return res

for i in (0, 1, 2, 6):
     print(fac(i) == fac_rt(i) == fac_iw(i))

 >>>
1 1 2 720
True
True
True
True

For collection functions that process each item once, 'for item in 
collection: ...' is nearly always easier to write in the first place.

>> An example of a function that naturally uses direct tail call
>> recursion, but which doesn't have a simple iterative rewrite, would
>> be more compelling.

Simple, easily converted functions like the above, with one recursive 
call in one branch of an if-else, are the most common.  Something with 
multiple mutually exclusive tail calls, like the following

def f_rt1(*args):
   if nonbase1:
     return f(*revised-args1)
   elif base_condition:
     return base(args)
   else:
     return f(*revised-args2)

must be converted to if-else with all tail calls in the if branch.

def f_rt2(*args):
   if not base_condition:
     if nonbase1:
       return f(*revised-args1)
     else:
       return f(*revised-args2)
   else:
     return base(args)

Conversion would then be simple.  The complication is that the 
'base_condition' in the original version might not really be the base 
condition due to a dependence on nonbase1 being false.  This is a 
generic problem with reordering if-elif statements.

For non-linear (branching) recursion, in which multiple recursive calls 
may be made for one function call, the last recursive call may be a tail 
call.  An example is in-place quick sort.  Eliminating just the tail 
call may not be simple, but it also does not eliminate the possibility 
of blowing the call stack.  To do that, one must eliminate all recursive 
calls by adding explicit stacks.

> Well, I did not write many tail recursive functions. But what surprised
> me was that for large values the ?tail recursive? version was more
> efficient as the iterative version.

In your first thread, what you mislabelled 'tail recursive version' was 
an iterative while loop version while the 'iterative version' was an 
iterative for loop version.  In this thread, you just posted timings 
without code. I will not believe your claim until I see one file that I 
can run myself with an actual tail recursive function, as above, that 
beats the equivalent while or for loop version.

-- 
Terry Jan Reedy




From davea at davea.name  Tue May  5 17:00:05 2015
From: davea at davea.name (Dave Angel)
Date: Tue, 05 May 2015 17:00:05 -0400
Subject: Throw the cat among the pigeons
In-Reply-To: <CALwzidmuhD0HhAZKRGMbgEgVVHf05_3pw=+o2JyAwWdiOtcNNA@mail.gmail.com>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl> <55490FAE.8070501@davea.name>
 <CALwzidmuhD0HhAZKRGMbgEgVVHf05_3pw=+o2JyAwWdiOtcNNA@mail.gmail.com>
Message-ID: <55492F55.1020105@davea.name>

On 05/05/2015 04:30 PM, Ian Kelly wrote:
> On Tue, May 5, 2015 at 12:45 PM, Dave Angel <davea at davea.name> wrote:
>> When the "simple" is True, the function takes noticeably and consistently
>> longer.  For example, it might take 116 instead of 109 seconds.  For the
>> same counts, your code took 111.
>
> I can't replicate this. What version of Python is it, and what value
> of x are you testing with?
>
>> I've looked at dis.dis(factorial_iterative), and can see no explicit reason
>> for the difference.
>
> My first thought is that maybe it's a result of the branch. Have you
> tried swapping the branches, or reimplementing as separate functions
> and comparing?
>

Logic is quite simple:


def factorial_iterative(x, simple=False):
     assert x >= 0
     result = 1
     j=2
     if not simple:
         for i in range(2, x + 1):
             #print("range value is of type", type(i), "and value", i)
             #print("ordinary value is of type", type(j), "and value", j)
             result *= i
             j += 1
     else:
         for i in range(2, x + 1):
             result *= j
             j += 1

     return result

def loop(func, funcname, arg):
     start = time.time()
     for i in range(repeats):
         func(arg, True)
     print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))

     start = time.time()
     for i in range(repeats):
         func(arg)
     print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))

repeats = 1

and arg is 10**4
     loop(factorial_iterative,      "factorial_iterative      ", arg)

My actual program does the same thing with other versions of the 
function, including Cecil's factorial_tail_recursion, and my optimized 
version of that.


Python 3.4.0 (default, Apr 11 2014, 13:05:11)
[GCC 4.8.2] on linux

factorial_iterative      (100000) took   3.807
factorial_iterative      (100000) took   3.664

factorial_iterative      (200000) took   17.07
factorial_iterative      (200000) took    15.3

factorial_iterative      (300000) took   38.93
factorial_iterative      (300000) took   36.01


Note that I test them in the opposite order of where they appear in the 
function.  That's because I was originally using the simple flag to test 
an empty loop.  The empty loop is much quicker either way, so it's not 
the issue.  (But if it were, the for-range version is much quicker).

I think I'll take your last suggestion and write separate functions.



-- 
DaveA


From Cecil at decebal.nl  Tue May  5 17:12:49 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 05 May 2015 23:12:49 +0200
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl>
 <mailman.140.1430858780.12865.python-list@python.org>
Message-ID: <87mw1ig23y.fsf@Equus.decebal.nl>

Op Tuesday 5 May 2015 22:46 CEST schreef Terry Reedy:

>> Well, I did not write many tail recursive functions. But what
>> surprised me was that for large values the ?tail recursive? version
>> was more efficient as the iterative version.
>
> In your first thread, what you mislabelled 'tail recursive version'
> was an iterative while loop version

That is because Python has no tail recursion, so I needed to program
the tail recursion myself. Tail recursion would do under the hood what
I did there manually.


> while the 'iterative version'
> was an iterative for loop version. In this thread, you just posted
> timings without code. I will not believe your claim until I see one
> file that I can run myself with an actual tail recursive function,
> as above, that beats the equivalent while or for loop version.

https://github.com/CecilWesterhof/PythonLibrary/blob/master/mathDecebal.py

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From ian.g.kelly at gmail.com  Tue May  5 17:23:58 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 5 May 2015 15:23:58 -0600
Subject: Throw the cat among the pigeons
In-Reply-To: <55492F55.1020105@davea.name>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl> <55490FAE.8070501@davea.name>
 <CALwzidmuhD0HhAZKRGMbgEgVVHf05_3pw=+o2JyAwWdiOtcNNA@mail.gmail.com>
 <55492F55.1020105@davea.name>
Message-ID: <CALwzid=ZX69d3p=C96qksT0DRD+mdm5eeA9zUq0VtcW1vqT0uw@mail.gmail.com>

On Tue, May 5, 2015 at 3:00 PM, Dave Angel <davea at davea.name> wrote:
> def loop(func, funcname, arg):
>     start = time.time()
>     for i in range(repeats):
>         func(arg, True)
>     print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))
>
>     start = time.time()
>     for i in range(repeats):
>         func(arg)
>     print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))

Note that you're explicitly passing True in one case but leaving the
default in the other. I don't know whether that might be responsible
for the difference you're seeing.

Also, it's best to use the timeit module for timing code, e.g.:

>>> from timeit import Timer
>>> t1 = Timer("factorial_iterative(100000, False)", "from __main__ import factorial_iterative")
>>> t1.repeat(10, number=1)
[3.8517245299881324, 3.7571076710009947, 3.7780062559759244,
3.848508063936606, 3.7627131739864126, 3.8278848479967564,
3.776115525048226, 3.83024005102925, 3.8322679550619796,
3.8195601429324597]
>>> min(_), sum(_) / len(_)
(3.7571076710009947, 3.8084128216956743)
>>> t2 = Timer("factorial_iterative(100000, True)", "from __main__ import factorial_iterative")
>>> t2.repeat(10, number=1)
[3.8363616950809956, 3.753201302024536, 3.7838632150087506,
3.7670978900277987, 3.805312803015113, 3.7682680500438437,
3.856655619922094, 3.796431727008894, 3.8224815409630537,
3.765664782957174]
>>> min(_), sum(_) / len(_)
(3.753201302024536, 3.7955338626052253)

As you can see, in my testing the True case was actually marginally
(probably not significantly) faster in both the min and the average.


From ian.g.kelly at gmail.com  Tue May  5 17:39:36 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 5 May 2015 15:39:36 -0600
Subject: Throw the cat among the pigeons
In-Reply-To: <CALwzid=ZX69d3p=C96qksT0DRD+mdm5eeA9zUq0VtcW1vqT0uw@mail.gmail.com>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl> <55490FAE.8070501@davea.name>
 <CALwzidmuhD0HhAZKRGMbgEgVVHf05_3pw=+o2JyAwWdiOtcNNA@mail.gmail.com>
 <55492F55.1020105@davea.name>
 <CALwzid=ZX69d3p=C96qksT0DRD+mdm5eeA9zUq0VtcW1vqT0uw@mail.gmail.com>
Message-ID: <CALwzid=gtNo3sbg2BP_ycUh+X52M5bcjGMXvzsNgHyw3k-crDg@mail.gmail.com>

On Tue, May 5, 2015 at 3:23 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Tue, May 5, 2015 at 3:00 PM, Dave Angel <davea at davea.name> wrote:
>> def loop(func, funcname, arg):
>>     start = time.time()
>>     for i in range(repeats):
>>         func(arg, True)
>>     print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))
>>
>>     start = time.time()
>>     for i in range(repeats):
>>         func(arg)
>>     print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))
>
> Note that you're explicitly passing True in one case but leaving the
> default in the other. I don't know whether that might be responsible
> for the difference you're seeing.

I don't think that's the cause, but I do think that it has something
to do with the way the timing is being run. When I run your loop
function, I do observe the difference. If I reverse the order so that
the False case is tested first, I observe the opposite. That is, the
slower case is consistently the one that is timed *first* in the loop
function, regardless of which case that is.


From davea at davea.name  Tue May  5 17:55:01 2015
From: davea at davea.name (Dave Angel)
Date: Tue, 05 May 2015 17:55:01 -0400
Subject: Throw the cat among the pigeons
In-Reply-To: <CALwzid=gtNo3sbg2BP_ycUh+X52M5bcjGMXvzsNgHyw3k-crDg@mail.gmail.com>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl> <55490FAE.8070501@davea.name>
 <CALwzidmuhD0HhAZKRGMbgEgVVHf05_3pw=+o2JyAwWdiOtcNNA@mail.gmail.com>
 <55492F55.1020105@davea.name>
 <CALwzid=ZX69d3p=C96qksT0DRD+mdm5eeA9zUq0VtcW1vqT0uw@mail.gmail.com>
 <CALwzid=gtNo3sbg2BP_ycUh+X52M5bcjGMXvzsNgHyw3k-crDg@mail.gmail.com>
Message-ID: <55493C35.6080408@davea.name>

On 05/05/2015 05:39 PM, Ian Kelly wrote:
> On Tue, May 5, 2015 at 3:23 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> On Tue, May 5, 2015 at 3:00 PM, Dave Angel <davea at davea.name> wrote:
>>> def loop(func, funcname, arg):
>>>      start = time.time()
>>>      for i in range(repeats):
>>>          func(arg, True)
>>>      print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))
>>>
>>>      start = time.time()
>>>      for i in range(repeats):
>>>          func(arg)
>>>      print("{0}({1}) took {2:7.4}".format(funcname, arg, time.time()-start))
>>
>> Note that you're explicitly passing True in one case but leaving the
>> default in the other. I don't know whether that might be responsible
>> for the difference you're seeing.
>
> I don't think that's the cause, but I do think that it has something
> to do with the way the timing is being run. When I run your loop
> function, I do observe the difference. If I reverse the order so that
> the False case is tested first, I observe the opposite. That is, the
> slower case is consistently the one that is timed *first* in the loop
> function, regardless of which case that is.
>

I created two functions and called them with Timeit(), and the 
difference is now below 3%

And when I take your lead and double the loop() function so it runs each 
test twice, I get steadily decreasing numbers.

I think all of this has been noise caused by the caching of objects 
including function objects.  I was surprised by this, as the loops are 
small enough I'd figure the function object would be fully cached the 
first time it was called.

-- 
DaveA


From tjreedy at udel.edu  Tue May  5 18:38:41 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 05 May 2015 18:38:41 -0400
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <CALwzidkO_k5SbTsMv4EH8mHoC_YUNBKov_prHfzxU8AJFEj3Jw@mail.gmail.com>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <CALwzidkO_k5SbTsMv4EH8mHoC_YUNBKov_prHfzxU8AJFEj3Jw@mail.gmail.com>
Message-ID: <mibgpi$9ju$1@ger.gmane.org>

On 5/5/2015 1:46 PM, Ian Kelly wrote:
> On Tue, May 5, 2015 at 9:22 AM, Paul  Moore <p.f.moore at gmail.com> wrote:
>> I'm working my way through the asyncio documentation. I have got to the "Tasks and coroutines" section, but I'm frankly confused as to the difference between the various things described in that section: coroutines, tasks, and futures.
>>
>> I think can understand a coroutine. Correct me if I'm wrong, but it's roughly "something that you can run which can suspend itself".
>>
>> But I don't understand what a Future is. The document just says it's almost the same as a concurrent.futures.Future, which is described as something that "encapsulates the asynchronous execution of a callable". Which doesn't help a lot. In concurrent.futures, you don't create Futures, you get them back from submit(), but in the asyncio docs it looks like you can create them by hand (example "Future with run_until_complete"). And there's nothing that says what a Future is, just what it's like... :-(
>
> Fundamentally, a future is a placeholder for something that isn't
> available yet. You can use it to set a callback to be called when that
> thing is available, and once it's available you can get that thing
> from it.
>
>> A Task is a subclass of Future, but the documentation doesn't say what it *is*, but rather that it "schedules the execution of a coroutine". But that doesn't make sense to me - objects don't do things, they *are* things. I thought the event loop did the scheduling?
>
> In asyncio, a Task is a a Future that serves as a placeholder for the
> result of a coroutine. The event loop manages callbacks, and that's
> all it does. An event that it's been told to listen for occurs, and
> the event loop calls the callback associated with that event. The Task
> manages a coroutine's interaction with the event loop; when the
> coroutine yields a future, the Task instructs the event loop to listen
> for the completion of that future, setting a callback that will resume
> the coroutine.
>
>> Reading between the lines, it seems that the event loop schedules Tasks (which makes sense) and that Tasks somehow wrap up coroutines - but I don't see *why* you need to wrap a task in a coroutine rather than just scheduling coroutines. And I don't see where Futures fit in - why not just wrap a coroutine in a Future, if it needs to be wrapped up at all?
>
> The coroutines themselves are not that interesting of an interface;
> all you can do with them is resume them. The asynchronous execution
> done by asyncio is all based on futures. Because a coroutine can
> easily be wrapped in a Task, this allows for coroutines to be used
> anywhere a future is expected.
>
> I don't know if I've done a good job explaining, but I hope this helps.

It helps me.  Thanks.



-- 
Terry Jan Reedy



From emile at fenx.com  Tue May  5 19:04:07 2015
From: emile at fenx.com (Emile van Sebille)
Date: Tue, 05 May 2015 16:04:07 -0700
Subject: Is it normal to cry when given XML?
In-Reply-To: <5548F5BB.8050409@gmail.com>
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
 <5548F5BB.8050409@gmail.com>
Message-ID: <mibi9e$1sb$1@ger.gmane.org>

On 5/5/2015 9:54 AM, Michael Torrie wrote:
> On 05/05/2015 03:28 AM, Sayth Renshaw wrote:
>> Hi
>>
>> Just checking if the reaction to cry when given XML is normal.
>
> I'd say it is normal.  XML is like violence.  If it doesn't solve your
> problems, you're not using enough of it[1].
>
> [1] Can anyone tell me who originated this line?
>

I find references from 2006 attributing the quote to "Ted Neward" but 
can't confirm that's where it started.

Emile




From breamoreboy at yahoo.co.uk  Tue May  5 19:09:02 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 06 May 2015 00:09:02 +0100
Subject: PyPy Development: CFFI 1.0 beta 1
Message-ID: <mibiik$636$1@ger.gmane.org>

Read all about it http://morepypy.blogspot.co.uk/2015/05/cffi-10-beta-1.html

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From tjreedy at udel.edu  Tue May  5 19:22:55 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 05 May 2015 19:22:55 -0400
Subject: Throw the cat among the pigeons
In-Reply-To: <87mw1ig23y.fsf@Equus.decebal.nl>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl>
 <mailman.140.1430858780.12865.python-list@python.org>
 <87mw1ig23y.fsf@Equus.decebal.nl>
Message-ID: <mibjch$hrt$1@ger.gmane.org>

On 5/5/2015 5:12 PM, Cecil Westerhof wrote:
> Op Tuesday 5 May 2015 22:46 CEST schreef Terry Reedy:
>
>>> Well, I did not write many tail recursive functions. But what
>>> surprised me was that for large values the ?tail recursive? version
>>> was more efficient as the iterative version.
>>
>> In your first thread, what you mislabelled 'tail recursive version'
>> was an iterative while loop version
>
> That is because Python has no tail recursion,

Yes is does. Please stop using 'tail recursion' to refer to the absence 
of recursion or the process of removing recursion. I wrote a tail 
recursive function, and I believe you did too.

What Python does not have is automatic tail call optimization, or 
specifically, tail recursion *elimination*. Both possibilities, the 
general and specific, have been considered and rejected for excellent 
reasons. I hinted at some of them in my post.

See https://en.wikipedia.org/wiki/Tail_call
for 'tail recursion' and 'tail call elimination'

What I believe you showed is that a while loop can be faster than a more 
or less equivalent for loop that produces the same result by a slightly 
different method.  That is not surprising.  Relative timings for CPython 
vary a few percent between different combinations of Python version, C 
compiler and settings, operating system, and cpu and other hardware.

-- 
Terry Jan Reedy




From breamoreboy at yahoo.co.uk  Tue May  5 19:35:59 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 06 May 2015 00:35:59 +0100
Subject: Is it normal to cry when given XML?
In-Reply-To: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
Message-ID: <mibk55$t13$1@ger.gmane.org>

On 05/05/2015 10:28, Sayth Renshaw wrote:
> Hi
>
> Just checking if the reaction to cry when given XML is normal.
>
> I thought maybe I am approaching it all wrong, using lxml largely or some xquery to club it into submission.
>
> See the usual goal is just to take the entire XML and push it into a database. or in future experiment with Mongo or Hdf5 .
>
> See its never basic xml, usually comes from some database with a walk of tables and strange relationships.
>
> Am I doing it wrong is there a simple way I am missing?
>
> Sayth
>

When the two young lads from the consultants I was working with back in 
2000 found a bug with xml handling in the supplier's software I almost 
ended up in tears.  Why?  They didn't bother reporting it, they simply 
modified all their unit testing data to work around the problem, so when 
we got to integration testing nothing worked at all.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From bmanogna.17 at gmail.com  Tue May  5 19:37:06 2015
From: bmanogna.17 at gmail.com (bmanogna.17 at gmail.com)
Date: Tue, 5 May 2015 16:37:06 -0700 (PDT)
Subject: Beacon controller
Message-ID: <b7943bd5-e2e6-4fcd-8239-750d33b6cbec@googlegroups.com>

Hi,
I'm new to pyhton, can anyone suggest me how can I implement a DOS or DDOS attack in Beacon Controller.
Thanks,
Manogna.


From rosuav at gmail.com  Tue May  5 19:59:33 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 6 May 2015 09:59:33 +1000
Subject: CHALLENGE HELP - GOOGLE DEVELOPER DAY
In-Reply-To: <c82211b3-4e58-4827-a2c1-64b824193b32@googlegroups.com>
References: <c82211b3-4e58-4827-a2c1-64b824193b32@googlegroups.com>
Message-ID: <CAPTjJmpTw5eJHMAyDEC41Qkb22daNJdx84Ny-wEQ17Y2LGuoNw@mail.gmail.com>

On Wed, May 6, 2015 at 2:59 AM,  <worship.brother at gmail.com> wrote:
> Good afternoon everyone.
>
> I'm with the following exercise of the option is a modification of a google developer day exercise.
>
> SOMEONE HELP ME IN THIS CHALLENGE?

You haven't posted any of your own code, so you're just asking for
someone to do it for you. No, I will not.

ChrisA


From rosuav at gmail.com  Tue May  5 20:02:58 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 6 May 2015 10:02:58 +1000
Subject: Stripping unencodable characters from a string
In-Reply-To: <24ef6c6d-a47a-4d8c-8651-c581e25161cb@googlegroups.com>
References: <24ef6c6d-a47a-4d8c-8651-c581e25161cb@googlegroups.com>
Message-ID: <CAPTjJmrZuhzm+tCvYhmADMCPavKb6L8mLmrQPU_mKO71H=+v0A@mail.gmail.com>

On Wed, May 6, 2015 at 4:19 AM, Paul  Moore <p.f.moore at gmail.com> wrote:
> I want to write a string to an already-open file (sys.stdout, typically). However, I *don't* want encoding errors, and the string could be arbitrary Unicode (in theory). The best way I've found is
>
>     data = data.encode(file.encoding, errors='replace').decode(file.encoding)
>     file.write(data)
>
> (I'd probably use backslashreplace rather than replace, but that's a minor point).
>
> Is that the best way? The multiple re-encoding dance seems a bit clumsy, but it was the best I could think of.

The simplest solution would be to call ascii() on the string, which
will give you an ASCII-only representation (using backslash escapes).
If your goal is to write Unicode text to a log file in some safe way,
this is what I would be doing.

ChrisA


From rosuav at gmail.com  Tue May  5 20:07:19 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 6 May 2015 10:07:19 +1000
Subject: Is it normal to cry when given XML?
In-Reply-To: <mibk55$t13$1@ger.gmane.org>
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
 <mibk55$t13$1@ger.gmane.org>
Message-ID: <CAPTjJmo5pjfq+mT=ZLH=1QUNi+iZC0Hhfd1HS_tX-FGboZhgKA@mail.gmail.com>

On Wed, May 6, 2015 at 9:35 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> When the two young lads from the consultants I was working with back in 2000
> found a bug with xml handling in the supplier's software I almost ended up
> in tears.  Why?  They didn't bother reporting it, they simply modified all
> their unit testing data to work around the problem, so when we got to
> integration testing nothing worked at all.

See, that's not a problem with XML. That's not even a problem with
unit testing. That's a problem with the belief that the most important
thing is to have your tests pass. Sigh.

ChrisA


From rosuav at gmail.com  Tue May  5 20:08:11 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 6 May 2015 10:08:11 +1000
Subject: Beacon controller
In-Reply-To: <b7943bd5-e2e6-4fcd-8239-750d33b6cbec@googlegroups.com>
References: <b7943bd5-e2e6-4fcd-8239-750d33b6cbec@googlegroups.com>
Message-ID: <CAPTjJmoWtt0n8ba4tAznwe5t4gG_MYc3nvR2kyoPvtDdLgaNpw@mail.gmail.com>

On Wed, May 6, 2015 at 9:37 AM,  <bmanogna.17 at gmail.com> wrote:
> Hi,
> I'm new to pyhton, can anyone suggest me how can I implement a DOS or DDOS attack in Beacon Controller.
> Thanks,
> Manogna.

http://www.catb.org/esr/faqs/hacker-howto.html#I_want_to_crack_and_Im_an_idiot

ChrisA


From rustompmody at gmail.com  Tue May  5 20:19:31 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 5 May 2015 17:19:31 -0700 (PDT)
Subject: Is it normal to cry when given XML?
In-Reply-To: <mailman.153.1430870847.12865.python-list@python.org>
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
 <mibk55$t13$1@ger.gmane.org>
 <mailman.153.1430870847.12865.python-list@python.org>
Message-ID: <465c4e55-aa88-4df4-ae6c-2718c434de4d@googlegroups.com>

On Wednesday, May 6, 2015 at 5:38:12 AM UTC+5:30, Chris Angelico wrote:
> On Wed, May 6, 2015 at 9:35 AM, Mark Lawrence  wrote:
> > When the two young lads from the consultants I was working with back in 2000
> > found a bug with xml handling in the supplier's software I almost ended up
> > in tears.  Why?  They didn't bother reporting it, they simply modified all
> > their unit testing data to work around the problem, so when we got to
> > integration testing nothing worked at all.
> 
> See, that's not a problem with XML. That's not even a problem with
> unit testing. That's a problem with the belief that the most important
> thing is to have your tests pass. Sigh.
> 

I smell a PHB? in the wings...
-------
? Pointy-haired-boss


From rosuav at gmail.com  Tue May  5 20:29:38 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 6 May 2015 10:29:38 +1000
Subject: Is it normal to cry when given XML?
In-Reply-To: <465c4e55-aa88-4df4-ae6c-2718c434de4d@googlegroups.com>
References: <d429abb8-2b17-4235-9eea-27a9a6f1ec9f@googlegroups.com>
 <mibk55$t13$1@ger.gmane.org>
 <mailman.153.1430870847.12865.python-list@python.org>
 <465c4e55-aa88-4df4-ae6c-2718c434de4d@googlegroups.com>
Message-ID: <CAPTjJmoTf9vSXi+Ai4Zn_5A29RexDQmTy05CX06DJVO303AkHA@mail.gmail.com>

On Wed, May 6, 2015 at 10:19 AM, Rustom Mody <rustompmody at gmail.com> wrote:
> On Wednesday, May 6, 2015 at 5:38:12 AM UTC+5:30, Chris Angelico wrote:
>> On Wed, May 6, 2015 at 9:35 AM, Mark Lawrence  wrote:
>> > When the two young lads from the consultants I was working with back in 2000
>> > found a bug with xml handling in the supplier's software I almost ended up
>> > in tears.  Why?  They didn't bother reporting it, they simply modified all
>> > their unit testing data to work around the problem, so when we got to
>> > integration testing nothing worked at all.
>>
>> See, that's not a problem with XML. That's not even a problem with
>> unit testing. That's a problem with the belief that the most important
>> thing is to have your tests pass. Sigh.
>>
>
> I smell a PHB? in the wings...

It seems likely, yes. That's where the "c'mon guys, get those tests to
pass" impetus usually comes from. The geeks usually know that tests
are a tool, not a goal, although the more evil ones might still wield
that knowledge...

http://bofh.ntk.net/BOFH/1998/bastard98-35.php

ChrisA


From steve+comp.lang.python at pearwood.info  Tue May  5 21:27:01 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 06 May 2015 11:27:01 +1000
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl> <55490FAE.8070501@davea.name>
 <CALwzidmuhD0HhAZKRGMbgEgVVHf05_3pw=+o2JyAwWdiOtcNNA@mail.gmail.com>
 <55492F55.1020105@davea.name>
 <mailman.142.1430861087.12865.python-list@python.org>
Message-ID: <55496de5$0$12993$c3e8da3$5496439d@news.astraweb.com>

On Wed, 6 May 2015 07:23 am, Ian Kelly wrote:

> On Tue, May 5, 2015 at 3:00 PM, Dave Angel <davea at davea.name> wrote:
>> def loop(func, funcname, arg):
>>     start = time.time()
>>     for i in range(repeats):
>>         func(arg, True)
>>     print("{0}({1}) took {2:7.4}".format(funcname, arg,
>>     time.time()-start))
>>
>>     start = time.time()
>>     for i in range(repeats):
>>         func(arg)
>>     print("{0}({1}) took {2:7.4}".format(funcname, arg,
>>     time.time()-start))
> 
> Note that you're explicitly passing True in one case but leaving the
> default in the other. I don't know whether that might be responsible
> for the difference you're seeing.

It will be responsible for *some* difference, it is certainly faster to pass
one argument than two, but likely an entirely trivial amount compared to
the time to iterate some large number of times.


> Also, it's best to use the timeit module for timing code, e.g.:
> 
>>>> from timeit import Timer
>>>> t1 = Timer("factorial_iterative(100000, False)", "from __main__ import
>>>> factorial_iterative") t1.repeat(10, number=1)
> [3.8517245299881324, 3.7571076710009947, 3.7780062559759244,
> 3.848508063936606, 3.7627131739864126, 3.8278848479967564,
> 3.776115525048226, 3.83024005102925, 3.8322679550619796,
> 3.8195601429324597]
>>>> min(_), sum(_) / len(_)
> (3.7571076710009947, 3.8084128216956743)

Only the minimum is statistically useful.

The reason is that every timing measurement has an error, due to random
fluctuations of the OS, CPU pipelining, caches, etc, but that error is
always positive. The noise always makes the code take longer to run, not
faster!

So we can imagine every measurement is made up of the "true" value T, plus
an error, where T = the perfectly repeatable timing that the function would
take to run if we could somehow eliminate every possible source of noise
and error in the system. We can't, of course, but we would like to estimate
T as closely as possible.

Without loss of generality, suppose we collect five timings:

times = [T+?1, T+?2, T+?3, T+?4, T+?5]

where the epsilons ? are unknown errors due to noise. We don't know the
distribution of the errors, except that no epsilon can be less than zero.

We want an estimate for T, something which comes as close as possible to T.
It should be obvious that the following relationships hold:

mean(times) = T + mean([?1, ?2, ?3, ?4, ?5])

min(times) = T + min([?1, ?2, ?3, ?4, ?5])


in other words, both the mean of the timings and the minimum of the timings
are equivalent to the "true" timing, plus an error. It should also be
obvious that the mean of the epsilons must be larger than the smallest of
the errors (except in the miraculous case where all the epsilons are
exactly the same).

So the statistic with the minimum error is the minimum. Any other stat,
whether you use the mean, geometric mean, harmonic mean, mode, median, or
anything else, will have a larger error and be a worse estimate for
the "true" value T.

All because the errors are one-sided: they always make the timing take
longer, never less time.



-- 
Steven



From steve+comp.lang.python at pearwood.info  Tue May  5 22:19:19 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 06 May 2015 12:19:19 +1000
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl>
Message-ID: <55497a28$0$12995$c3e8da3$5496439d@news.astraweb.com>

On Wed, 6 May 2015 02:18 am, Cecil Westerhof wrote:

> Well, I did not write many tail recursive functions. But what surprised
> me was that for large values the ?tail recursive? version was more
> efficient as the iterative version. And that was with myself
> implementing the tail recursion. I expect the code to be more
> efficient when the compiler implements the tail recursion.


You cannot know that. Python is not C or Java, and your intuition as to what
will be fast and what will be slow will not be accurate in Python. Python
has its own fast paths, and slow paths, and they are not the same as C's or
Java's.

I have been using Python for over 15 years, and I would not want to guess
whether a hypothetical compiler-generated tail-recursion-elimination
function would be faster or slower than manually unrolling the recursion
into a while loop.

I am surprised that you find a manually unrolled version with a while loop
is faster than an iterative version. I assume the iterative version uses a
for-loop. In my experience, a for-loop is faster than a while loop.

But perhaps that depends on the exact version and platform.
 


-- 
Steven



From steve+comp.lang.python at pearwood.info  Tue May  5 22:57:30 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 06 May 2015 12:57:30 +1000
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl>
 <mailman.136.1430851516.12865.python-list@python.org>
 <87r3qug69v.fsf@Equus.decebal.nl>
Message-ID: <5549831b$0$12997$c3e8da3$5496439d@news.astraweb.com>

On Wed, 6 May 2015 05:42 am, Cecil Westerhof wrote:

> I would say that a variable that is filled by a range is different as
> a normal variable. Do not ask me why. ;-)


I would say that you are wrong. If I have understood you correctly, that
cannot possibly be the case in Python, all Python variables work the same
way[1], and none of them can remember where they came from.






[1] Well, technically local variables and global variables are implemented
differently, locals inside a function use a C array and globals use a dict,
but apart from a few implementation details like that, any two variables in
the same scope[2] operate identically.

[2] Anyone who raises the issue of exec() or import * inside a function body
in Python 2 will be slapped with a halibut.

-- 
Steven



From john_ladasky at sbcglobal.net  Tue May  5 23:54:15 2015
From: john_ladasky at sbcglobal.net (John Ladasky)
Date: Tue, 5 May 2015 20:54:15 -0700 (PDT)
Subject: CHALLENGE HELP - GOOGLE DEVELOPER DAY
In-Reply-To: <mailman.151.1430870381.12865.python-list@python.org>
References: <c82211b3-4e58-4827-a2c1-64b824193b32@googlegroups.com>
 <mailman.151.1430870381.12865.python-list@python.org>
Message-ID: <3c998f52-18b6-40ee-b859-b86e646138f6@googlegroups.com>

On Tuesday, May 5, 2015 at 4:59:51 PM UTC-7, Chris Angelico wrote:
> On Wed, May 6, 2015 at 2:59 AM,  <worship.bro... at gmail.com> wrote:
> > Good afternoon everyone.
> >
> > I'm with the following exercise of the option is a modification of a google developer day exercise.
> >
> > SOMEONE HELP ME IN THIS CHALLENGE?
> 
> You haven't posted any of your own code, so you're just asking for
> someone to do it for you. No, I will not.
> 
> ChrisA

+1.


From rosuav at gmail.com  Wed May  6 00:03:22 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 6 May 2015 14:03:22 +1000
Subject: Throw the cat among the pigeons
In-Reply-To: <5549831b$0$12997$c3e8da3$5496439d@news.astraweb.com>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl>
 <mailman.136.1430851516.12865.python-list@python.org>
 <87r3qug69v.fsf@Equus.decebal.nl>
 <5549831b$0$12997$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmrEN_sM4Ti6xOw7wHMVXuZhxrH1TU=5AU2Jb2vsoxZtNA@mail.gmail.com>

On Wed, May 6, 2015 at 12:57 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Wed, 6 May 2015 05:42 am, Cecil Westerhof wrote:
>
>> I would say that a variable that is filled by a range is different as
>> a normal variable. Do not ask me why. ;-)
>
>
> I would say that you are wrong. If I have understood you correctly, that
> cannot possibly be the case in Python, all Python variables work the same
> way[1], and none of them can remember where they came from.

My reading of the code was that one had two variables and the other
had just one. Could be the noise in the numbers came from cache
locality differences.

No difference between the variables, just a peculiarity of CPUs and
how they use memory.

ChrisA


From steve+comp.lang.python at pearwood.info  Wed May  6 00:05:23 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 06 May 2015 14:05:23 +1000
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl>
Message-ID: <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>

On Sun, 3 May 2015 12:20 am, Cecil Westerhof wrote:

[...]
> But to my surprise tail recursion could even be more efficient. I
> wrote two different versions of factorial with self implemented tail
> recursion. For bigger values both are more efficient. And I expect
> that if the tail recursion is done by the compiler instead of by hand,
> it will be a little faster still.


I decided to do some experimentation.

Firstly, when timing code, one should minimise the amount of "scaffolding"
used around the code you care about. Using a single function like this:

def factorial_iterative(x, flag):
    if flag:
        # first implementation
    else:
        # second implementation


is a bad idea, because you are not just timing the implementations, but also
the scaffolding that selects between them. Also, you increase the size of
the function, which increases the chance of cache misses and other
processor effects. So first step is to split the implementations into
separate functions. Here are my four implementations, the first two are
taken from your code:


def factorial_forloop1(n):
    assert n >= 0
    result = 1
    for i in range(2, n + 1):
        result *= i
    return result

def factorial_forloop2(n):
    # Version with a silly extra variable.
    assert n >= 0
    result = 1
    j=2
    for i in range(2, n + 1):
        result *= j
        j += 1
    return result

from operator import mul
try:
    reduce
except NameError:
    from functools import reduce

def factorial_reduce(n):
    assert n >= 0
    return reduce(mul, range(2, n+1), 1)

def factorial_while(n):
    assert n >= 0
    result = 1
    while n > 1:
        result *= n
        n -= 1
    return result



A quick test to verify that they return the same results:

py> factorial_while(10)
3628800
py> factorial_reduce(10)
3628800
py> factorial_forloop1(10)
3628800
py> factorial_forloop2(10)
3628800



There's no point in optimising code that does the wrong thing!

Now, let's do some timing tests. It is best to use a well-tested timing
framework rather than invent your own dodgy one, so I use the timeit
module. Read the comments in the module to see why rolling your own is a
bad idea.

I'll start with some relative small input sizes. Remember, the inputs may be
small, but the outputs will be very large.


from timeit import Timer
code = "fact(10); fact(20); fact(30)"
t1 = Timer(code, setup="from __main__ import factorial_while as fact")
t2 = Timer(code, setup="from __main__ import factorial_reduce as fact")
t3 = Timer(code, setup="from __main__ import factorial_forloop1 as fact")
t4 = Timer(code, setup="from __main__ import factorial_forloop2 as fact")


I'm in a bit of a hurry, so I may be a bit more slap-dash than normal.
Normally, I would pick a larger number of trials, and a larger number of
iterations per trial, but here I'm going to use just best of three trials,
each of 10000 iterations each:


for t in (t1, t2, t3, t4):
    print(min(t.repeat(repeat=3, number=10000)))


which prints:

0.22797810286283493  # while
0.17145151272416115  # reduce
0.16230526939034462  # for-loop
0.22819281555712223  # silly for-loop


(Comments added by me.) See my earlier post for why the minimum is the only
statistic worth looking at. These results show that:

- the version using while is significantly slower than the more
straightforward iterative versions using either reduce or a for loop.

- adding an extra, unnecessary, variable to the for-loop, and incrementing
it by hand, carries as much overhead as using a while loop;

- reduce is slightly slower than a pure Python for-loop (at least according
to this simple trial, on my computer -- your results may vary);

- the obvious winner is the straightforward iterative version with a
for-loop.



Now I'm going to test it with a larger input:

py> big = factorial_forloop1(50000)
py> sys.getsizeof(big)  # number of bytes
94460
py> len(str(big))  # number of digits
213237


Recreate the timer objects, and (again, because I'm in something of a hurry)
do a best-of-three with just 2 iterations per trial.

code = "fact(50000)"
t1 = Timer(code, setup="from __main__ import factorial_while as fact")
t2 = Timer(code, setup="from __main__ import factorial_reduce as fact")
t3 = Timer(code, setup="from __main__ import factorial_forloop1 as fact")
t4 = Timer(code, setup="from __main__ import factorial_forloop2 as fact")
for t in (t1, t2, t3, t4):
    print(min(t.repeat(repeat=3, number=2)))


which takes about two minutes on my computer, and prints:

8.604736926034093  # while loop
10.786483339965343  # reduce
10.91099695302546  # for loop
10.821452282369137  # silly version of the for loop

(Again, annotations are by me.)

These results are fascinating, and rather surprising. I think that they
demonstrate that, at this size of input argument, the time is dominated by
processing the BigNum int objects, not the iteration: whether we use
reduce, a straight-forward for-loop, or a for-loop with an extra variable
makes very little difference, of the order of 1%.

(I wouldn't read too much into the fact that the for-loop which does *more*
work, but manually adjusting a second variable, is slightly faster. Unless
that can be proven to be consistently the case, I expect that is just
random noise. I ran some quick trials, and did not replicate that result:

py> for t in (t3, t4, t3, t4):
...     print(min(t.repeat(repeat=3, number=1)))
...
5.282072028145194  # for-loop
5.415546240285039  # silly for-loop
5.358346642926335  # for-loop
5.328046130016446  # silly for-loop

Note that, as expected, doing two iterations per trial takes about twice as
long as one iteration per trial: 5 seconds versus 10.


What is surprising is that for very large input like this, the while loop is
significantly faster than reduce or either of the for-loops. I cannot
explain that result.

(Just goes to show that timing code in Python can surprise you when you
least expect it.)


Oh, and for the record, I'm using Python 3.3 on Linux:

py> sys.version
'3.3.0rc3 (default, Sep 27 2012, 18:44:58) \n[GCC 4.1.2 20080704 (Red Hat
4.1.2-52)]'

Results may vary on other platforms and versions.


-- 
Steven



From rustompmody at gmail.com  Wed May  6 00:47:17 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 5 May 2015 21:47:17 -0700 (PDT)
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <878ud27waw.fsf@elektro.pacujo.net>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
Message-ID: <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>

On Tuesday, May 5, 2015 at 11:15:42 PM UTC+5:30, Marko Rauhamaa wrote:
 
> Personally, I have never found futures a very useful idiom in any
> language (Scheme, Java, Python). Or more to the point, concurrency and
> the notion of a function don't gel well in my mind.

Interesting comment.

It strikes me that the FP crowd has stretched the notion of function beyond recognition
And the imperative/OO folks have distorted it beyond redemption.

And the middle road shown by Pascal has been overgrown with weeds for some 40 years...

If the classic Pascal (or Fortran or Basic) sibling balanced abstractions of function-for-value
procedure-for-effect were more in the collective consciousness rather than C's
travesty of function, things might not have been so messy.

Well... Dont feel right bashing C without some history...

C didn't start the mess of mixing procedure and function -- Lisp/Apl did.
Nor the confusion of = for assignment; Fortran did that.


From kashifrana84 at gmail.com  Wed May  6 01:32:28 2015
From: kashifrana84 at gmail.com (Kashif Rana)
Date: Tue, 5 May 2015 22:32:28 -0700 (PDT)
Subject: Writing list of dictionaries to CSV
In-Reply-To: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
Message-ID: <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>

Hello guys

thanks for the feedback. I think its problem with excel itself, showing wrong value. Because when I opened the csv file in text editor, I can see correct value but opening in excel showing wrong value. What I can do to see correct in excel as well.

Regards

On Tuesday, May 5, 2015 at 9:09:40 PM UTC+4, Kashif Rana wrote:
> Hello Experts
> 
> When I am writing list of dictionaries to CSV file, the key 'schedule' has value 'Mar 2012' becomes Mar-12. I really do not have clue why thats happening. Below is the code. 
> 
> dic_1 = {'action': 'permit',
>  'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169',
>  'from': 'DMZ Web',
>  'id': '1000',
>  'log': 'Enable, session-init',
>  'name': 'Test Rule Temporary ',
>  'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL',
>  'src-address': 'sparkeregap1, sparkeregap2',
>  'to': 'Trust'}
> 
> dic_2 {'action': 'permit',
>  'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net',
>  'from': 'DMZ Web',
>  'id': '4000',
>  'log': 'Enable, session-init',
>  'schedule': 'Mar 2012',
>  'service': 'SMTP',
>  'src-address': 'sparkeregap1, sparkeregap2',
>  'to': 'Trust'}
>  
>  my_list = 
>  [{'to': 'Trust', 'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL', 'from': 'DMZ Web', 'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169', 'name': 'Test Rule Temporary ', 'action': 'permit', 'id': '1000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable, session-init'}, {'to': 'Trust', 'from': 'DMZ Web', 'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net', 'service': 'SMTP', 'schedule': 'Mar 2012', 'action': 'permit', 'id': '4000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable, session-init'}]
> 
>  pol_keys = ['id', 'name', 'from', 'to', 'src-address', 'dst-address', 'service', 'action', 'nat_status', 'nat_type', 'nat_src_ip', 'nat_dst_ip', 'nat_dst_port', 'log', 'schedule']
>  
> with open('test.csv', 'wb') as f:
> 	w = csv.DictWriter(f, pol_keys)
> 	w.writeheader()
> 	w.writerows(my_list)


From steve+comp.lang.python at pearwood.info  Wed May  6 01:48:47 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 06 May 2015 15:48:47 +1000
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
 <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>
Message-ID: <5549ab43$0$11108$c3e8da3@news.astraweb.com>

On Wednesday 06 May 2015 14:47, Rustom Mody wrote:

> It strikes me that the FP crowd has stretched the notion of function
> beyond recognition And the imperative/OO folks have distorted it beyond
> redemption.

In what way?


> And the middle road shown by Pascal has been overgrown with weeds for some
> 40 years...

As much as I like Pascal, and am pleased to see someone defending it, I'm 
afraid I have no idea what you mean by this.


> If the classic Pascal (or Fortran or Basic) sibling balanced abstractions
> of function-for-value procedure-for-effect were more in the collective
> consciousness rather than C's travesty of function, things might not have
> been so messy.

I'm not really sure that having distinct procedures, as opposed to functions 
that you just ignore their return result, makes *such* a big difference. Can 
you explain what is the difference between these?

sort(stuff)  # A procedure.
sort(stuff)  # ignore the function return result

And why the first is so much better than the second?



> Well... Dont feel right bashing C without some history...
> 
> C didn't start the mess of mixing procedure and function -- Lisp/Apl did.
> Nor the confusion of = for assignment; Fortran did that.

Pardon, but = has been used for "assignment" for centuries, long before 
George Boole and Ada Lovelace even started working on computer theory. Just 
ask mathematicians:

"let y = 23"

Is that not a form of assignment?




-- 
Steve



From ian.g.kelly at gmail.com  Wed May  6 01:58:52 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 5 May 2015 23:58:52 -0600
Subject: Throw the cat among the pigeons
In-Reply-To: <55496de5$0$12993$c3e8da3$5496439d@news.astraweb.com>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl> <55490FAE.8070501@davea.name>
 <CALwzidmuhD0HhAZKRGMbgEgVVHf05_3pw=+o2JyAwWdiOtcNNA@mail.gmail.com>
 <55492F55.1020105@davea.name>
 <mailman.142.1430861087.12865.python-list@python.org>
 <55496de5$0$12993$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CALwzidnsfqAizAWtyrWgaNrC+-g7-LLMD_-13sSeRgmnz9KKrw@mail.gmail.com>

On Tue, May 5, 2015 at 7:27 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> Only the minimum is statistically useful.

I disagree. The minimum tells you how fast the code *can* run, under
optimal circumstances. The mean tells you how fast it *realistically*
runs, under typical load. Both can be useful to measure.


From steve+comp.lang.python at pearwood.info  Wed May  6 02:26:33 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 06 May 2015 16:26:33 +1000
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 06 May 2015 14:05, Steven D'Aprano wrote:

[...]

Here are those anomalous timing results again:

> code = "fact(50000)"
> t1 = Timer(code, setup="from __main__ import factorial_while as fact")
> t2 = Timer(code, setup="from __main__ import factorial_reduce as fact")
> t3 = Timer(code, setup="from __main__ import factorial_forloop1 as fact")
> t4 = Timer(code, setup="from __main__ import factorial_forloop2 as fact")
> for t in (t1, t2, t3, t4):
>     print(min(t.repeat(repeat=3, number=2)))
> 
> 
> which takes about two minutes on my computer, and prints:
> 
> 8.604736926034093  # while loop
> 10.786483339965343  # reduce
> 10.91099695302546  # for loop
> 10.821452282369137  # silly version of the for loop
[...]
> What is surprising is that for very large input like this, the while loop
> is significantly faster than reduce or either of the for-loops. I cannot
> explain that result.

I re-ran the results on a different computer, and got similar results:

7.364120149984956
9.26512472704053
9.141491871327162
9.16900822892785

The while loop is surprisingly faster for calculating fact(50000). A thought 
came to mind: the while loop is different from the other three versions, in 
that it performs the multiplications from n down to 2, rather than 2 up to 
n. Maybe that has something to do with it?

Introducing version five of the factorial:

def factorial_reduce2(n):
    assert n >= 0
    return reduce(mul, range(n, 1, -1), 1)

t5 = Timer(code, setup="from __main__ import factorial_reduce2 as fact")
for t in (t1, t2, t3, t4, t5):
    print(min(t.repeat(repeat=3, number=2)))



And results:

7.36792928725481  # while loop
9.271950023248792  # reduce, counting up
9.14769447594881  # for loop
9.154150342568755  # silly for loop
7.430811045691371  # reduce, counting down


My interpretation of this is that the difference has something to do with 
the cost of multiplications. Multiplying upwards seems to be more expensive 
than multiplying downwards, a result I never would have predicted, but 
that's what I'm seeing. I can only guess that it has something to do with 
the way multiplication is implemented, or perhaps the memory management 
involved, or something. Who the hell knows?


Just to be sure:

def factorial_forloop3(n):
    assert n >= 0
    result = 1
    for i in range(n, 1, -1):
        result *= i
    return result

t6 = Timer(code, setup="from __main__ import factorial_forloop3 as fact")
print(min(t6.repeat(repeat=3, number=2)))


which prints 7.36256698705256.


Lesson to be learned: what you think is important may not be, and what you 
think is irrelevant may be.



Historical fact: for a period of about 60 years, long after the trick to 
curing and preventing scurvy was known, the British navy suffered greatly 
from scurvy again (although not to the same degree as they had been in the 
17th and 18th centuries). The problem was due to confusion between *lemons* 
and *limes*. The navy started by using the term "lime" for both lemons and 
sweet limes from the Mediterranean, as well as West Indian limes: three 
different citrus fruit. To cut costs and encourage British commerce, the 
navy gradually stopping buying lemons from Spain and swapped over almost 
entirely to West Indian limes. Unfortunately limes, and especially West 
Indian limes, have significantly less Vitamin C than lemons, and the 
sailors' daily allowance was about 75% less effective on ships that used 
limes than for those that used Spanish lemons. Scurvy, which had practically 
be eradicated in the 1850s and 60s, returned, and was a significant factor 
in World War One. (Especially for the French and Russian armies.)

This simple confusion wasn't sorted out until the 1920s, long after Vitamin 
C was isolated and named. Apparently nobody noticed, or cared, that the two 
different fruit tasted different and looked different, or concluded that 
perhaps they had different medicinal properties.

But then, for many decades, vinegar and dilute sulphuric acid were 
recommended as cures for scurvy *solely* on the basis that they were acidic 
just like proven cures oranges, lemons and sauerkraut.



-- 
Steven



From rosuav at gmail.com  Wed May  6 02:34:33 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 6 May 2015 16:34:33 +1000
Subject: Writing list of dictionaries to CSV
In-Reply-To: <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
Message-ID: <CAPTjJmqDvge9TW_FvW+Q=2v8uz=Ap8iKBCcX_xXCZGCXpjq1wg@mail.gmail.com>

On Wed, May 6, 2015 at 3:32 PM, Kashif Rana <kashifrana84 at gmail.com> wrote:
> thanks for the feedback. I think its problem with excel itself, showing wrong value. Because when I opened the csv file in text editor, I can see correct value but opening in excel showing wrong value. What I can do to see correct in excel as well.
>

Simple fix: Ditch Excel.

Slightly more complicated fix: Open the file, then select everything,
and tell Excel that it's plain text. You can't say that in the CSV
file.

ChrisA


From palpandi111 at gmail.com  Wed May  6 02:37:02 2015
From: palpandi111 at gmail.com (Palpandi)
Date: Tue, 5 May 2015 23:37:02 -0700 (PDT)
Subject: Encrypt python files
Message-ID: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>

Hi,

What are the ways to encrypt python files?


From steve+comp.lang.python at pearwood.info  Wed May  6 02:47:22 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 06 May 2015 16:47:22 +1000
Subject: Encrypt python files
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
Message-ID: <5549b8fb$0$12906$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 06 May 2015 16:37, Palpandi wrote:

> Hi,
> 
> What are the ways to encrypt python files?

The same as the ways to encrypt any other file. Your encryption program 
shouldn't care whether you are encrypting text files, JPEGs, mp3 audio 
files, executable binary code, Python scripts, or a giant file of all ASCII 
nulls. It will just encrypt the file regardless of what kind of file it is.

What encryption program are you using?



-- 
Steve



From dpalao.python at gmail.com  Wed May  6 02:49:47 2015
From: dpalao.python at gmail.com (David Palao)
Date: Wed, 6 May 2015 08:49:47 +0200
Subject: Encrypt python files
In-Reply-To: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
Message-ID: <CAKUKWzmvMuC9jy2+s32-Faa720-e_59-qMVw_ZSxbcTt2CgHug@mail.gmail.com>

Hello,
I'm afraid your question is either not well defined (or not well
enough) or wrong for this list, at least as I understand it.
Could you please explain it better?

Best

2015-05-06 8:37 GMT+02:00 Palpandi <palpandi111 at gmail.com>:
> Hi,
>
> What are the ways to encrypt python files?
> --
> https://mail.python.org/mailman/listinfo/python-list


From Cecil at decebal.nl  Wed May  6 02:50:53 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 06 May 2015 08:50:53 +0200
Subject: Writing list of dictionaries to CSV
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
Message-ID: <87h9rqfbci.fsf@Equus.decebal.nl>

Op Wednesday 6 May 2015 07:32 CEST schreef Kashif Rana:

> thanks for the feedback. I think its problem with excel itself,
> showing wrong value. Because when I opened the csv file in text
> editor, I can see correct value but opening in excel showing wrong
> value. What I can do to see correct in excel as well.

A Python mailing list is not the right place to ask that. You can
better use a Microsoft (office) mailing list.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From steve+comp.lang.python at pearwood.info  Wed May  6 03:08:05 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 06 May 2015 17:08:05 +1000
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl> <55490FAE.8070501@davea.name>
 <CALwzidmuhD0HhAZKRGMbgEgVVHf05_3pw=+o2JyAwWdiOtcNNA@mail.gmail.com>
 <55492F55.1020105@davea.name>
 <mailman.142.1430861087.12865.python-list@python.org>
 <55496de5$0$12993$c3e8da3$5496439d@news.astraweb.com>
 <mailman.158.1430891975.12865.python-list@python.org>
Message-ID: <5549bde3$0$12911$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 06 May 2015 15:58, Ian Kelly wrote:

> On Tue, May 5, 2015 at 7:27 PM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>> Only the minimum is statistically useful.
> 
> I disagree. The minimum tells you how fast the code *can* run, under
> optimal circumstances. The mean tells you how fast it *realistically*
> runs, under typical load. Both can be useful to measure.

Er, not even close. Running code using timeit is in no way the same as 
running code "for real" under realistic circumstances. The fact that you are 
running the function or code snippet in isolation, in its own scope, via 
exec, rather than as part of some larger script or application, should be a 
hint. timeit itself has overhead, so you cannot measure the time taken by 
the operation alone, you can only measure the time taken by the operation 
within the timeit environment. We have no reason to think that the 
distribution of noise under timeit will be even vaguely similar to the noise 
when running in production.

The purpose of timeit is to compare individual algorithms, in as close as 
possible to an equal footing with as little noise as possible. If you want 
to profile code used in a realistic application, use a profiler, not timeit. 
And even that doesn't tell you how fast the code would be alone, because the 
profiler adds overhead.

Besides, "typical load" is a myth -- there is no such thing. A high-end 
Windows web server getting ten thousand hits a minute, a virtual machine 
starved for RAM, a Mac laptop, a Linux server idling away with a load of 0.1 
all day... any of those machines could run your code. How can you *possibly* 
say what is typical? The very idea is absurd.



-- 
Steve



From palpandi111 at gmail.com  Wed May  6 03:23:39 2015
From: palpandi111 at gmail.com (Palpandi)
Date: Wed, 6 May 2015 00:23:39 -0700 (PDT)
Subject: Encrypt python files
In-Reply-To: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
Message-ID: <72bd42d6-6926-44ce-bd6f-956ce40afe08@googlegroups.com>

On Wednesday, May 6, 2015 at 12:07:13 PM UTC+5:30, Palpandi wrote:
> Hi,
> 
> What are the ways to encrypt python files?

No, I just want to hide the scripts from others.


From ben+python at benfinney.id.au  Wed May  6 03:41:53 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 06 May 2015 17:41:53 +1000
Subject: DRM is self-defeating (was: Encrypt python files)
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
 <72bd42d6-6926-44ce-bd6f-956ce40afe08@googlegroups.com>
Message-ID: <85vbg6cfum.fsf_-_@benfinney.id.au>

Palpandi <palpandi111 at gmail.com> writes:

> On Wednesday, May 6, 2015 at 12:07:13 PM UTC+5:30, Palpandi wrote:

> > What are the ways to encrypt python files?
>
> No, I just want to hide the scripts from others.

Which others? You can hide the scripts from them by never showing those
others the scripts. If you don't trust a recipient, don't let them
receive the file.

I suspect you are asking ?how can I distribute Python modules to people
without those people being able to read them?? You can't, because
executing the file requires reading its contents.

Anyone who has a machine that can execute the file has a machine that
must, by necessity, read its contents.


This is the dilemma of those who think they want Digital Restrictions
Management (DRM): attempting to treat recipients as untrustworthy, while
still telling them they can use the files.

If you a person is in possession of the file you're trying to restrict,
on a machine they possess, with a key needed to unlock the content, then
that person is in possession of everything needed to unlock the content.

On the other hand, if one of those (e.g. the key to unlock the content)
is missing, then the file is useless for whatever you're saying the
recipient can do with it.

The only feasible solution is to distribute files only to those
recipients you want to have them, and can trust to do with them as you
ask. If you don't trust a recipient, don't hand the files to them.


On the other hand, if you're asking something else, you will need to be
much more explicit. You have not helped us understand what you want thus
far.

-- 
 \        ?This sentence contradicts itself ? no actually it doesn't.? |
  `\                                               ?Douglas Hofstadter |
_o__)                                                                  |
Ben Finney



From rosuav at gmail.com  Wed May  6 04:00:51 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 6 May 2015 18:00:51 +1000
Subject: DRM is self-defeating (was: Encrypt python files)
In-Reply-To: <85vbg6cfum.fsf_-_@benfinney.id.au>
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
 <72bd42d6-6926-44ce-bd6f-956ce40afe08@googlegroups.com>
 <85vbg6cfum.fsf_-_@benfinney.id.au>
Message-ID: <CAPTjJmo4G6ZvuWqK-7Lj5z7A0-HxJTyoijYtwh+rkpVbT2gcZw@mail.gmail.com>

On Wed, May 6, 2015 at 5:41 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> The only feasible solution is to distribute files only to those
> recipients you want to have them, and can trust to do with them as you
> ask. If you don't trust a recipient, don't hand the files to them.

in today's world, that basically gives two good options:

1) Distribute your code far and wide, let every man and his dog clone
your source code, and license it so they're allowed to; or
2) Host your code on a server that people don't get access to, and
have them interact with it remotely.

Either way works really well. You can have closed-source software that
people never directly see, yet benefit from (and potentially pay you),
or you can let people see everything they're running. The problem only
comes when you try to hybridize.

(Aside: Most people carry mobile phones they don't truly own/control,
so they're restricted by what the device manufacturer offers them.
Despite looking superficially like the first case, it's really more
like the second.)

ChrisA


From fomcl at yahoo.com  Wed May  6 04:24:36 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Wed, 6 May 2015 01:24:36 -0700
Subject: DRM is self-defeating (was: Encrypt python files)
Message-ID: <1430900676.98833.BPMail_high_carrier@web163803.mail.gq1.yahoo.com>


-----------------------------
On Wed, May 6, 2015 9:41 AM CEST Ben Finney wrote:

>Palpandi <palpandi111 at gmail.com> writes:
>
>> On Wednesday, May 6, 2015 at 12:07:13 PM UTC+5:30, Palpandi wrote:
>
>> > What are the ways to encrypt python files?
>>
>> No, I just want to hide the scripts from others.
>
>Which others? You can hide the scripts from them by never showing those
>others the scripts. If you don't trust a recipient, don't let them
>receive the file.

<snip>

>The only feasible solution is to distribute files only to those
>recipients you want to have them, and can trust to do with them as you
>ask. If you don't trust a recipient, don't hand the files to them.

Can tools like py2exe or cx_freeze also be used to obfuscate python source code?


From steve+comp.lang.python at pearwood.info  Wed May  6 05:04:23 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 06 May 2015 19:04:23 +1000
Subject: Encrypt python files
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
 <72bd42d6-6926-44ce-bd6f-956ce40afe08@googlegroups.com>
Message-ID: <5549d919$0$12911$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 06 May 2015 17:23, Palpandi wrote:

> On Wednesday, May 6, 2015 at 12:07:13 PM UTC+5:30, Palpandi wrote:
>> Hi,
>> 
>> What are the ways to encrypt python files?
> 
> No, I just want to hide the scripts from others.

Why, are you ashamed of your code?

Python is free, open source software. Hiding the code from others is not a 
priority for the developers of the language. Besides, you can't hide the 
code unless you only operate the application via a web service, or similar. 
As soon as you give people a copy of the code, whether it is binary code or 
source code, they have a copy of it and can look at it and work out how what 
it does.

If "hiding the code" was good for security, why are there so many viruses 
and spybots and worms and other malware for Windows?

No, as far as I am concerned, trying to hide the code is a waste of time 
with Python. But if you absolutely must, you can distribute the .pyc files 
instead of the .py files, and that will discourage casual tinkerers from 
poking around in the program. The .pyc file is compiled to byte-code rather 
than source code, so it's not readable without running it through a 
decompiler.


-- 
Steve



From rosuav at gmail.com  Wed May  6 05:25:39 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 6 May 2015 19:25:39 +1000
Subject: DRM is self-defeating (was: Encrypt python files)
In-Reply-To: <1430900676.98833.BPMail_high_carrier@web163803.mail.gq1.yahoo.com>
References: <1430900676.98833.BPMail_high_carrier@web163803.mail.gq1.yahoo.com>
Message-ID: <CAPTjJmp1WdZ0hcCGjFRkuMA+-VrK1NnArrjaZ1iiQA9cEf=39Q@mail.gmail.com>

On Wed, May 6, 2015 at 6:24 PM, Albert-Jan Roskam via Python-list
<python-list at python.org> wrote:
>>The only feasible solution is to distribute files only to those
>>recipients you want to have them, and can trust to do with them as you
>>ask. If you don't trust a recipient, don't hand the files to them.
>
> Can tools like py2exe or cx_freeze also be used to obfuscate python source code?

But barely. At very worst, you suppress your comments and variable
names, but all your actual functionality is all there still - it has
to be, or it won't run.

ChrisA


From fomcl at yahoo.com  Wed May  6 05:45:57 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Wed, 6 May 2015 02:45:57 -0700
Subject: Encrypt python files
Message-ID: <1430905557.27594.BPMail_high_carrier@web163805.mail.gq1.yahoo.com>


-----------------------------
On Wed, May 6, 2015 11:04 AM CEST Steven D'Aprano wrote:

>On Wednesday 06 May 2015 17:23, Palpandi wrote:
>
>> On Wednesday, May 6, 2015 at 12:07:13 PM UTC+5:30, Palpandi wrote:
>> Hi,
>> 
>> What are the ways to encrypt python files?
>> 
>> No, I just want to hide the scripts from others.
>
>Why, are you ashamed of your code?
>
>Python is free, open source software. Hiding the code from others is not a 
>priority for the developers of the language. Besides, you can't hide the 
>code unless you only operate the application via a web service, or similar. 
>As soon as you give people a copy of the code, whether it is binary code or 
>source code, they have a copy of it and can look at it and work out how what 
>it does.
>
>If "hiding the code" was good for security, why are there so many viruses 
>and spybots and worms and other malware for Windows?
>
>No, as far as I am concerned, trying to hide the code is a waste of time 
>with Python. But if you absolutely must, you can distribute the .pyc files 
>instead of the .py files, and that will discourage casual tinkerers from 
>poking around in the program. The .pyc file is compiled to byte-code rather 
>than source code, so it's not readable without running it through a 
>decompiler.

I used the marshal module before as a faster alternative to shelve (I marshalled a huge dictionary). I always understood marshal files require the *exact* same interpreter version (incl. built number). Do the same limitations apply for .pyc files? Isn't it the same format?

We have a VCS with an option to use private repos. I am always wary of users who want to use this feature. Why would you not want to share your code with your colleagues? Embarrassed about unreadable crappy code, perhaps?  




From greg.ewing at canterbury.ac.nz  Wed May  6 07:15:42 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Wed, 06 May 2015 23:15:42 +1200
Subject: Bitten by my C/Java experience
In-Reply-To: <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
 <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>
Message-ID: <cqubeuF86o6U1@mid.individual.net>

Steven D'Aprano wrote:

> The first one just does a name lookup and then throws the result away. The 
> second one looks up names a and b, then adds them together, throwing away 
> the result.

Actually, it's worse than that. :-) It's possible for a to
have an __add__ method with a side effect, although that
would be evil.

It's also possible for merely looking up a name to have
a side effect, if it's done in the right context -- but
that would be even more evil.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Wed May  6 07:19:58 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Wed, 06 May 2015 23:19:58 +1200
Subject: Bitten by my C/Java experience
In-Reply-To: <es32x.478654$Qj7.158587@fx36.am4>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
 <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>
 <es32x.478654$Qj7.158587@fx36.am4>
Message-ID: <cqubn0F8a5gU1@mid.individual.net>

BartC wrote:

> So why pretend that ++ and -- don't exist?

Probably because Python would gain very little from
having them.

Main uses of ++ in C are things like integer for
loops:

    for (i = 0; i < 10; i++) {...

and stepping through arrays:

    a[i++] = b[j++];

Python code usually operates at a higher level than
that.

-- 
Greg


From steve+comp.lang.python at pearwood.info  Wed May  6 07:50:17 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 06 May 2015 21:50:17 +1000
Subject: Encrypt python files
References: <mailman.166.1430905888.12865.python-list@python.org>
Message-ID: <5549fffb$0$12983$c3e8da3$5496439d@news.astraweb.com>

On Wed, 6 May 2015 07:45 pm, Albert-Jan Roskam wrote:

> I used the marshal module before as a faster alternative to shelve (I
> marshalled a huge dictionary). I always understood marshal files require
> the *exact* same interpreter version (incl. built number). Do the same
> limitations apply for .pyc files? Isn't it the same format?

marshal is currently up to version 4 of the (otherwise undocumented) file
format. Obviously that's much less than the number of minor versions of
Python, so, no, marshal does not change every release.

https://docs.python.org/3/library/marshal.html

.pyc files, on the other hand, have a magic number which changes for each
minor release of Python. I think that, in principle, it could change in a
point release (say, between 3.4.1 and 3.4.2) but in practice I don't
believe that has ever happened. I think it is safe to assume that pyc files
will be portable across any minor release (e.g. 3.4.x for any x) but not
across changes to the minor or major release.

I don't think that is an outright promise, but it is a strong convention.

And of course, other implementations may not even use .pyc files, if they
don't use the same sort of byte code. E.g. Jython compiles to JVM byte
code:

foo.py
foo$py.class


Nick Coghlan has a very good blog post about the uses of pyc only
distributed software:

http://www.curiousefficiency.org/posts/2011/04/benefits-and-limitations-of-pyc-only.html

And Ned Batchelder has a good discussion of their internals:

http://nedbatchelder.com/blog/200804/the_structure_of_pyc_files.html



-- 
Steven



From breamoreboy at yahoo.co.uk  Wed May  6 08:18:39 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 06 May 2015 13:18:39 +0100
Subject: Json Comaprision
In-Reply-To: <CANvYX9V=+HJ9-Lhbbwyg7yPnmFqmwqDrdS_0YTd7X4VNmYSaDw@mail.gmail.com>
References: <CANvYX9V=+HJ9-Lhbbwyg7yPnmFqmwqDrdS_0YTd7X4VNmYSaDw@mail.gmail.com>
Message-ID: <mid0r0$7l7$1@ger.gmane.org>

On 05/05/2015 20:55, pra devOPS wrote:
> Hi All:
>
>
> I wanted to compare two json files ignoring few of the keys in the json
> files.
>
> Can anybody suggest me few things?
>
> Thanks,
> Siva
>

https://docs.python.org/3/library/json.html#module-json

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From antoon.pardon at rece.vub.ac.be  Wed May  6 08:38:45 2015
From: antoon.pardon at rece.vub.ac.be (Antoon Pardon)
Date: Wed, 06 May 2015 14:38:45 +0200
Subject: Bitten by my C/Java experience
In-Reply-To: <1a74804c-958f-44ad-a354-e6ac1510a8b6@googlegroups.com>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
 <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>
 <es32x.478654$Qj7.158587@fx36.am4>
 <1a74804c-958f-44ad-a354-e6ac1510a8b6@googlegroups.com>
Message-ID: <554A0B55.3010505@rece.vub.ac.be>

Op 05-05-15 om 18:24 schreef Rustom Mody:

> Yeah I happen to me in that minuscule minority that regards '= denotes 
> assignment' a bigger mistake than ++

Nice to know I'm not alone. I Especially think it is a mistake, because
it is then used as a reason for not allowing something like

  if a = b - 1:

arguing it would lead to some difficult bugs.

Which in my mind is arguing backwards. Either you think an assigment
in a condition is useful or harmful. In the first case you then look
for an assignment token or assignment syntax that is not that likely
to lead to difficult to discover bugs instead of letting a possible
misleading token or syntax prevent you from implementing something
useful.

In the second case you just state why you think an assignment in a
condition is harmful. No need to hide behind awkward syntax.

-- 
Antoon Pardon



From bc at freeuk.com  Wed May  6 08:40:28 2015
From: bc at freeuk.com (BartC)
Date: Wed, 06 May 2015 13:40:28 +0100
Subject: Bitten by my C/Java experience
In-Reply-To: <cqubn0F8a5gU1@mid.individual.net>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
 <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>
 <es32x.478654$Qj7.158587@fx36.am4> <cqubn0F8a5gU1@mid.individual.net>
Message-ID: <3Zn2x.359727$Ox2.83653@fx23.am4>

On 06/05/2015 12:19, Gregory Ewing wrote:
> BartC wrote:
>
>> So why pretend that ++ and -- don't exist?
>
> Probably because Python would gain very little from
> having them.
>
> Main uses of ++ in C are things like integer for
> loops:
>
>     for (i = 0; i < 10; i++) {...
>
> and stepping through arrays:
>
>     a[i++] = b[j++];
>
> Python code usually operates at a higher level than
> that.

I think even in Python it is sometimes necessary to increment things (as 
the OP did).

But I had in mind not implementing ++ and --, but detecting them and 
issuing a warning, so forcing someone to type "+ +" or to use 
parentheses, which is unlikely to be much of an imposition as how often 
are two unary pluses going to be used together?


-- 
Bartc




From rustompmody at gmail.com  Wed May  6 09:03:54 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Wed, 6 May 2015 06:03:54 -0700 (PDT)
Subject: Bitten by my C/Java experience
In-Reply-To: <mailman.170.1430915935.12865.python-list@python.org>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
 <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>
 <es32x.478654$Qj7.158587@fx36.am4>
 <1a74804c-958f-44ad-a354-e6ac1510a8b6@googlegroups.com>
 <mailman.170.1430915935.12865.python-list@python.org>
Message-ID: <04032e3b-6b75-44e3-8e9e-a962885f04d9@googlegroups.com>

On Wednesday, May 6, 2015 at 6:09:08 PM UTC+5:30, Antoon Pardon wrote:
> Op 05-05-15 om 18:24 schreef Rustom Mody:
> 
> > Yeah I happen to me in that minuscule minority that regards '= denotes 
> > assignment' a bigger mistake than ++
> 
> Nice to know I'm not alone. I Especially think it is a mistake, because
> it is then used as a reason for not allowing something like
> 
>   if a = b - 1:
> 
> arguing it would lead to some difficult bugs.
> 
> Which in my mind is arguing backwards. Either you think an assigment
> in a condition is useful or harmful. In the first case you then look
> for an assignment token or assignment syntax that is not that likely
> to lead to difficult to discover bugs instead of letting a possible
> misleading token or syntax prevent you from implementing something
> useful.
> 
> In the second case you just state why you think an assignment in a
> condition is harmful. No need to hide behind awkward syntax.

Nice to know we agree though I am not sure I agree with your agreement :-)

In APL assignment is ? goto is ? [and of course equality and negated equality
are = and ? ]
But if you've seen hair-raising APL one-liners with multiple ? and even ? stuffed in...

No on second thoughts we probably agree... I am just in the second camp:
assignment in conditions is trouble; no need for syntax to argue that


From random832 at fastmail.us  Wed May  6 09:11:32 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 06 May 2015 09:11:32 -0400
Subject: Bitten by my C/Java experience
In-Reply-To: <n0S1x.370451$O36.280928@fx02.am4>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
Message-ID: <1430917892.2950498.263440081.24138BAB@webmail.messagingengine.com>

On Mon, May 4, 2015, at 18:02, BartC wrote:
> (I think I would have picked up "++" and "--" as special tokens even if 
> increment/decrement ops weren't supported. Just because they would 
> likely cause errors through misunderstanding.)

There's precedent for not doing this in C itself - even though "=+"
(from very early versions of C, also =-, =*, =&) no longer acts as an
add-in-place operator, it's not recognized as a special token to prevent
errors either.


From davea at davea.name  Wed May  6 09:12:05 2015
From: davea at davea.name (Dave Angel)
Date: Wed, 06 May 2015 09:12:05 -0400
Subject: Throw the cat among the pigeons
In-Reply-To: <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <554A1325.2060103@davea.name>

On 05/06/2015 02:26 AM, Steven D'Aprano wrote:
> On Wednesday 06 May 2015 14:05, Steven D'Aprano wrote:

> My interpretation of this is that the difference has something to do with
> the cost of multiplications. Multiplying upwards seems to be more expensive
> than multiplying downwards, a result I never would have predicted, but
> that's what I'm seeing. I can only guess that it has something to do with
> the way multiplication is implemented, or perhaps the memory management
> involved, or something. Who the hell knows?
>

I had guessed that the order of multiplication would make a big 
difference, once the product started getting bigger than the machine 
word size.

Reason I thought that is that if you multiply starting at the top value 
(and end with multiplying by 2) you're spending more of the time 
multiplying big-ints.

That's why I made sure that both Cecil's and my implementations were 
counting up, so that wouldn't be a distinction.

I'm still puzzled, as it seems your results imply that big-int*int is 
faster than int*int where the product is also int.

That could use some more testing, though.

I still say a cutoff of about 10% is where we should draw the line in an 
interpretive system.  Below that, you're frequently measuring noise and 
coincidence.

Remember the days when you knew how many cycles each assembly 
instruction took, and could simply add them up to compare algorithms?


-- 
DaveA


From rosuav at gmail.com  Wed May  6 09:16:48 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 6 May 2015 23:16:48 +1000
Subject: Bitten by my C/Java experience
In-Reply-To: <1430917892.2950498.263440081.24138BAB@webmail.messagingengine.com>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
 <1430917892.2950498.263440081.24138BAB@webmail.messagingengine.com>
Message-ID: <CAPTjJmoEADZFH7m1=o3kFmgBUiPBB_mK-7E_DnnYSWm_tLYs8w@mail.gmail.com>

On Wed, May 6, 2015 at 11:11 PM,  <random832 at fastmail.us> wrote:
> On Mon, May 4, 2015, at 18:02, BartC wrote:
>> (I think I would have picked up "++" and "--" as special tokens even if
>> increment/decrement ops weren't supported. Just because they would
>> likely cause errors through misunderstanding.)
>
> There's precedent for not doing this in C itself - even though "=+"
> (from very early versions of C, also =-, =*, =&) no longer acts as an
> add-in-place operator, it's not recognized as a special token to prevent
> errors either.

Given that the in-place operators changed to +=, -=, etc very early
on, I doubt there's anyone who is actually confused by them. And it'd
be extremely annoying to have to stop and think about parsing rules
when taking or dereferencing pointers:

/* This works */
x = &y;

/* Why shouldn't this? */
x=&y;

To the greatest extent possible, spaces around assignment operators
should be the domain of style guides, not syntax.

ChrisA


From rosuav at gmail.com  Wed May  6 09:55:56 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 6 May 2015 23:55:56 +1000
Subject: Throw the cat among the pigeons
In-Reply-To: <554A1325.2060103@davea.name>
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
 <554A1325.2060103@davea.name>
Message-ID: <CAPTjJmoD07MguM7dhTD23BXS_fL8mdpXr-zDDxJ+=xUP7ug6YQ@mail.gmail.com>

On Wed, May 6, 2015 at 11:12 PM, Dave Angel <davea at davea.name> wrote:
> I had guessed that the order of multiplication would make a big difference,
> once the product started getting bigger than the machine word size.
>
> Reason I thought that is that if you multiply starting at the top value (and
> end with multiplying by 2) you're spending more of the time multiplying
> big-ints.
>
> That's why I made sure that both Cecil's and my implementations were
> counting up, so that wouldn't be a distinction.
>
> I'm still puzzled, as it seems your results imply that big-int*int is faster
> than int*int where the product is also int.

Are you using Python 2 or Python 3 for your testing? In Py3, there's
no type difference, and barely no performance difference as you cross
the word-size boundary. (Bigger numbers are a bit slower to work with,
but not hugely.)

ChrisA


From davea at davea.name  Wed May  6 10:22:56 2015
From: davea at davea.name (Dave Angel)
Date: Wed, 06 May 2015 10:22:56 -0400
Subject: Throw the cat among the pigeons
In-Reply-To: <CAPTjJmoD07MguM7dhTD23BXS_fL8mdpXr-zDDxJ+=xUP7ug6YQ@mail.gmail.com>
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
 <554A1325.2060103@davea.name>
 <CAPTjJmoD07MguM7dhTD23BXS_fL8mdpXr-zDDxJ+=xUP7ug6YQ@mail.gmail.com>
Message-ID: <554A23C0.6060808@davea.name>

On 05/06/2015 09:55 AM, Chris Angelico wrote:
> On Wed, May 6, 2015 at 11:12 PM, Dave Angel <davea at davea.name> wrote:
>> I had guessed that the order of multiplication would make a big difference,
>> once the product started getting bigger than the machine word size.
>>
>> Reason I thought that is that if you multiply starting at the top value (and
>> end with multiplying by 2) you're spending more of the time multiplying
>> big-ints.
>>
>> That's why I made sure that both Cecil's and my implementations were
>> counting up, so that wouldn't be a distinction.
>>
>> I'm still puzzled, as it seems your results imply that big-int*int is faster
>> than int*int where the product is also int.
>
> Are you using Python 2 or Python 3 for your testing? In Py3, there's
> no type difference, and barely no performance difference as you cross
> the word-size boundary. (Bigger numbers are a bit slower to work with,
> but not hugely.)
>

Both Cecil and I are using 3.x  I'm using 3.4 in particular.  And I know 
int covers both big-int and int32.  that's why I called it big-int, 
rather than long.

I was, however, mistaken.  it's not that threshold that we're crossing 
here, but another one, for MUCH larger numbers.  factorial of 100000 and 
of 200000 have 456473 and 97350 digits, respectively.  In binary, that 
would be about 190k bytes and 404k bytes, respectively.

I was seeing factorial of 200000 taking about 4.5 times as long as 
factorial of 100000.  All the other increments seemed fairly proportional.

I'll bet the difference is something like the memory allocator using a 
different algorithm for blocks above 256k.  Or the cache logic hitting a 
threshold.

If it's caching, of course the threshold will differ wildly between 
machine architectures.

If it's the memory allocator, that could easily vary between Python 
versions as well.




-- 
DaveA


From no.email at nospam.invalid  Wed May  6 11:12:50 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Wed, 06 May 2015 08:12:50 -0700
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <8761853fkd.fsf@jester.gateway.sonic.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
> Multiplying upwards seems to be more expensive than multiplying
> downwards... I can only guess that it has something to do with the way
> multiplication is implemented, or perhaps the memory management
> involved, or something. Who the hell knows?

It seems pretty natural if multiplication uses the obvious
quadratic-time pencil and paper algorithm.  The cost of multiplying m*n
is basically w(m)*w(n) where w(x) is the width of x in machine words.
So for factorial where m is the counter and n is the running product,
w(m) is always 1 while w(n) is basically log2(n!).  From

    from math import log
    def xfac(seq):
        cost = logfac = 0.0
        for i in seq:
            logfac += log(i,2)
            cost += logfac
        return cost

    def upward(n): return xfac(xrange(1,n+1))
    def downward(n): return xfac(xrange(n,1,-1))

    print upward(40000),downward(40000)

I get: 10499542692.6 11652843833.5

A lower number for upward than downward.  The difference isn't as large
as your timings, but I think it still gives some explanation of the
effect.


From alain at universite-de-strasbourg.fr.invalid  Wed May  6 11:36:06 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Wed, 06 May 2015 17:36:06 +0200
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
 <8761853fkd.fsf@jester.gateway.sonic.net>
Message-ID: <87pp6doh09.fsf@universite-de-strasbourg.fr.invalid>

Paul Rubin <no.email at nospam.invalid> writes:

> Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
>> Multiplying upwards seems to be more expensive than multiplying
>> downwards... I can only guess that it has something to do with the way
>> multiplication is implemented, or perhaps the memory management
>> involved, or something. Who the hell knows?
>
> It seems pretty natural if multiplication uses the obvious
> quadratic-time pencil and paper algorithm.  The cost of multiplying m*n
> is basically w(m)*w(n) where w(x) is the width of x in machine words.
> So for factorial where m is the counter and n is the running product,
> w(m) is always 1 while w(n) is basically log2(n!).  From
>
>     from math import log
>     def xfac(seq):
>         cost = logfac = 0.0
>         for i in seq:
>             logfac += log(i,2)
>             cost += logfac
>         return cost
>
>     def upward(n): return xfac(xrange(1,n+1))
>     def downward(n): return xfac(xrange(n,1,-1))
>
>     print upward(40000),downward(40000)
>
> I get: 10499542692.6 11652843833.5
>
> A lower number for upward than downward.  The difference isn't as large
> as your timings, but I think it still gives some explanation of the
> effect.

Yes, plus the time for memory allocation. Since the code uses "r *=
...", space is reallocated when the result doesn't fit. The new size is
probably proportional to the current (insufficient) size. This means
that overall, you'll need fewer reallocations, because allocations are
made in bigger chunks.

-- Alain.


From ian.g.kelly at gmail.com  Wed May  6 11:47:36 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 6 May 2015 09:47:36 -0600
Subject: Throw the cat among the pigeons
In-Reply-To: <8761853fkd.fsf@jester.gateway.sonic.net>
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
 <8761853fkd.fsf@jester.gateway.sonic.net>
Message-ID: <CALwzid=Dy=+yERQRxgvt1gGZXzTctSfSQ4xLAKA-DJv8C--W-A@mail.gmail.com>

On Wed, May 6, 2015 at 9:12 AM, Paul Rubin <no.email at nospam.invalid> wrote:
> Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
>> Multiplying upwards seems to be more expensive than multiplying
>> downwards... I can only guess that it has something to do with the way
>> multiplication is implemented, or perhaps the memory management
>> involved, or something. Who the hell knows?
>
> It seems pretty natural if multiplication uses the obvious
> quadratic-time pencil and paper algorithm.  The cost of multiplying m*n
> is basically w(m)*w(n) where w(x) is the width of x in machine words.
> So for factorial where m is the counter and n is the running product,
> w(m) is always 1 while w(n) is basically log2(n!).  From
>
>     from math import log
>     def xfac(seq):
>         cost = logfac = 0.0
>         for i in seq:
>             logfac += log(i,2)
>             cost += logfac
>         return cost
>
>     def upward(n): return xfac(xrange(1,n+1))
>     def downward(n): return xfac(xrange(n,1,-1))
>
>     print upward(40000),downward(40000)
>
> I get: 10499542692.6 11652843833.5
>
> A lower number for upward than downward.  The difference isn't as large
> as your timings, but I think it still gives some explanation of the
> effect.

That was my initial thought as well, but the problem is that this
actually predicts the *opposite* of what is being reported: upward
should be less expensive, not more.


From steve+comp.lang.python at pearwood.info  Wed May  6 12:03:02 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 07 May 2015 02:03:02 +1000
Subject: Bitten by my C/Java experience
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
 <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>
 <es32x.478654$Qj7.158587@fx36.am4> <cqubn0F8a5gU1@mid.individual.net>
 <3Zn2x.359727$Ox2.83653@fx23.am4>
Message-ID: <554a3b37$0$12985$c3e8da3$5496439d@news.astraweb.com>

On Wed, 6 May 2015 10:40 pm, BartC wrote:

> But I had in mind not implementing ++ and --, but detecting them and
> issuing a warning, 

That's a job for a linter, not the compiler. The compiler should be as
flexible as possible in what it accepts:


a   ,    b=12+3     *          4,"hello"          .      upper   ()


is perfectly legal code. The compiler shouldn't force you to write good
looking code, apart from what is prohibited altogether.

Both + and - are unary prefix operators, so you can apply + and - to any
expression -- even an expression that already has a unary prefix operator:

py> -     --- +++ + - - + -- +++ --- 999
-999


Is that ugly, horrible code that nobody in their right mind would use in
production? Absolutely. But the compiler can and should accept it, and
linters (or human reviewers) should warn about it.



-- 
Steven



From ian.g.kelly at gmail.com  Wed May  6 12:17:45 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 6 May 2015 10:17:45 -0600
Subject: Throw the cat among the pigeons
In-Reply-To: <5549bde3$0$12911$c3e8da3$5496439d@news.astraweb.com>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl> <55490FAE.8070501@davea.name>
 <CALwzidmuhD0HhAZKRGMbgEgVVHf05_3pw=+o2JyAwWdiOtcNNA@mail.gmail.com>
 <55492F55.1020105@davea.name>
 <mailman.142.1430861087.12865.python-list@python.org>
 <55496de5$0$12993$c3e8da3$5496439d@news.astraweb.com>
 <mailman.158.1430891975.12865.python-list@python.org>
 <5549bde3$0$12911$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CALwzidnaOqS3oruM=uqiJO3SvhMSB_adA6nO7xbXfpN=oB6xNQ@mail.gmail.com>

On Wed, May 6, 2015 at 1:08 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Wednesday 06 May 2015 15:58, Ian Kelly wrote:
>
>> On Tue, May 5, 2015 at 7:27 PM, Steven D'Aprano
>> <steve+comp.lang.python at pearwood.info> wrote:
>>> Only the minimum is statistically useful.
>>
>> I disagree. The minimum tells you how fast the code *can* run, under
>> optimal circumstances. The mean tells you how fast it *realistically*
>> runs, under typical load. Both can be useful to measure.
>
> Er, not even close. Running code using timeit is in no way the same as
> running code "for real" under realistic circumstances. The fact that you are
> running the function or code snippet in isolation, in its own scope, via
> exec, rather than as part of some larger script or application, should be a
> hint. timeit itself has overhead, so you cannot measure the time taken by
> the operation alone, you can only measure the time taken by the operation
> within the timeit environment. We have no reason to think that the
> distribution of noise under timeit will be even vaguely similar to the noise
> when running in production.

You also can't be sure that the base time taken by the operation in
your development environment will be comparable to the time taken in
production; different system architectures may produce different
results, and what is faster on your workstation may be slower on a
server.

Also, different algorithms may react to load differently. For example,
an algorithm that goes to different parts of memory frequently may
start thrashing sooner than an algorithm with better spatial locality
if the system is paging a lot. I'll grant that just computing the
means on a workstation that is not under a controlled load is not the
best way to measure this -- but a difference in mean that is not
simply proportional to the difference in min is still potentially
useful information.

> The purpose of timeit is to compare individual algorithms, in as close as
> possible to an equal footing with as little noise as possible. If you want
> to profile code used in a realistic application, use a profiler, not timeit.
> And even that doesn't tell you how fast the code would be alone, because the
> profiler adds overhead.
>
> Besides, "typical load" is a myth -- there is no such thing. A high-end
> Windows web server getting ten thousand hits a minute, a virtual machine
> starved for RAM, a Mac laptop, a Linux server idling away with a load of 0.1
> all day... any of those machines could run your code. How can you *possibly*
> say what is typical? The very idea is absurd.

Agreed.


From gordon at panix.com  Wed May  6 12:27:49 2015
From: gordon at panix.com (John Gordon)
Date: Wed, 6 May 2015 16:27:49 +0000 (UTC)
Subject: Json Comaprision
References: <mailman.164.1430902487.12865.python-list@python.org>
Message-ID: <midfe5$ma7$1@reader1.panix.com>

In <mailman.164.1430902487.12865.python-list at python.org> pra devOPS <siv.devops at gmail.com> writes:

> I wanted to compare two json files ignoring few of the keys in the json
> files.

> Can anybody suggest me few things?

Load each json file into a python object, delete the keys you don't care
about, and compare the two objects.

-- 
John Gordon         Imagine what it must be like for a real medical doctor to
gordon at panix.com    watch 'House', or a real serial killer to watch 'Dexter'.



From no.email at nospam.invalid  Wed May  6 13:24:43 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Wed, 06 May 2015 10:24:43 -0700
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
 <8761853fkd.fsf@jester.gateway.sonic.net>
 <mailman.179.1430927606.12865.python-list@python.org>
Message-ID: <871tit39gk.fsf@jester.gateway.sonic.net>

Ian Kelly <ian.g.kelly at gmail.com> writes:
> That was my initial thought as well, but the problem is that this
> actually predicts the *opposite* of what is being reported: upward
> should be less expensive, not more.

Wait, what?  Hmm, you're right.  Needed coffee, will think about it more
later.


From python at mrabarnett.plus.com  Wed May  6 14:08:52 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 06 May 2015 19:08:52 +0100
Subject: Writing list of dictionaries to CSV
In-Reply-To: <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
Message-ID: <554A58B4.80700@mrabarnett.plus.com>

On 2015-05-06 06:32, Kashif Rana wrote:
> Hello guys
>
> thanks for the feedback. I think its problem with excel itself, showing wrong value. Because when I opened the csv file in text editor, I can see correct value but opening in excel showing wrong value. What I can do to see correct in excel as well.
>
> Regards
>
You could tell it to quote any value that's not a number:

	w = csv.DictWriter(f, pol_keys, quoting=csv.QUOTE_NONNUMERIC)

It looks like all of the values you have are strings, so they'll all be
quoted.

I would hope that Excel will then treat it as a string; it would be
stupid if it didn't! :-)

> On Tuesday, May 5, 2015 at 9:09:40 PM UTC+4, Kashif Rana wrote:
>> Hello Experts
>>
>> When I am writing list of dictionaries to CSV file, the key 'schedule' has value 'Mar 2012' becomes Mar-12. I really do not have clue why thats happening. Below is the code.
>>
>> dic_1 = {'action': 'permit',
>>  'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169',
>>  'from': 'DMZ Web',
>>  'id': '1000',
>>  'log': 'Enable, session-init',
>>  'name': 'Test Rule Temporary ',
>>  'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL',
>>  'src-address': 'sparkeregap1, sparkeregap2',
>>  'to': 'Trust'}
>>
>> dic_2 {'action': 'permit',
>>  'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net',
>>  'from': 'DMZ Web',
>>  'id': '4000',
>>  'log': 'Enable, session-init',
>>  'schedule': 'Mar 2012',
>>  'service': 'SMTP',
>>  'src-address': 'sparkeregap1, sparkeregap2',
>>  'to': 'Trust'}
>>
>>  my_list =
>>  [{'to': 'Trust', 'service': '1521, Oraccle-Maximo-Scan-1550, PING-ALL', 'from': 'DMZ Web', 'dst-address': 'maxprddb-scan-167, maxprddb-scan-168, maxprddb-scan-169', 'name': 'Test Rule Temporary ', 'action': 'permit', 'id': '1000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable, session-init'}, {'to': 'Trust', 'from': 'DMZ Web', 'dst-address': 'sparkcas01, sparkcas02, email.ab.spark.net', 'service': 'SMTP', 'schedule': 'Mar 2012', 'action': 'permit', 'id': '4000', 'src-address': 'sparkeregap1, sparkeregap2', 'log': 'Enable, session-init'}]
>>
>>  pol_keys = ['id', 'name', 'from', 'to', 'src-address', 'dst-address', 'service', 'action', 'nat_status', 'nat_type', 'nat_src_ip', 'nat_dst_ip', 'nat_dst_port', 'log', 'schedule']
>>
>> with open('test.csv', 'wb') as f:
>> 	w = csv.DictWriter(f, pol_keys)
>> 	w.writeheader()
>> 	w.writerows(my_list)
>



From python.list at tim.thechases.com  Wed May  6 14:22:55 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Wed, 6 May 2015 13:22:55 -0500
Subject: Writing list of dictionaries to CSV
In-Reply-To: <554A58B4.80700@mrabarnett.plus.com>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
 <554A58B4.80700@mrabarnett.plus.com>
Message-ID: <20150506132255.510c2eec@bigbox.christie.dr>

On 2015-05-06 19:08, MRAB wrote:
> You could tell it to quote any value that's not a number:
> 
> 	w = csv.DictWriter(f, pol_keys,
> quoting=csv.QUOTE_NONNUMERIC)
> 
> It looks like all of the values you have are strings, so they'll
> all be quoted.
> 
> I would hope that Excel will then treat it as a string; it would be
> stupid if it didn't! :-)

Sadly, Excel *is* that stupid based on the tests I tried just now. :-(

Regardless of whether "Mar 2015" is quoted or unquoted in the source
CSV file, Excel tries to outwit you and mangles the presentation.

-tkc





From ian.g.kelly at gmail.com  Wed May  6 14:27:31 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 6 May 2015 12:27:31 -0600
Subject: Writing list of dictionaries to CSV
In-Reply-To: <20150506132255.510c2eec@bigbox.christie.dr>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
 <554A58B4.80700@mrabarnett.plus.com>
 <20150506132255.510c2eec@bigbox.christie.dr>
Message-ID: <CALwzid=yoTxetHRuVDaDnCW9ASXVruF02gE2CZgsszJ6=3YxWQ@mail.gmail.com>

On Wed, May 6, 2015 at 12:22 PM, Tim Chase
<python.list at tim.thechases.com> wrote:
> On 2015-05-06 19:08, MRAB wrote:
>> You could tell it to quote any value that's not a number:
>>
>>       w = csv.DictWriter(f, pol_keys,
>> quoting=csv.QUOTE_NONNUMERIC)
>>
>> It looks like all of the values you have are strings, so they'll
>> all be quoted.
>>
>> I would hope that Excel will then treat it as a string; it would be
>> stupid if it didn't! :-)
>
> Sadly, Excel *is* that stupid based on the tests I tried just now. :-(
>
> Regardless of whether "Mar 2015" is quoted or unquoted in the source
> CSV file, Excel tries to outwit you and mangles the presentation.

Quoting a value in csv doesn't mean it's a string; it just means that
it's a single field.

You *can* force Excel to treat a value as a string by prefixing it
with an apostrophe, though.


From python.list at tim.thechases.com  Wed May  6 14:37:12 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Wed, 6 May 2015 13:37:12 -0500
Subject: Writing list of dictionaries to CSV
In-Reply-To: <CALwzid=yoTxetHRuVDaDnCW9ASXVruF02gE2CZgsszJ6=3YxWQ@mail.gmail.com>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
 <554A58B4.80700@mrabarnett.plus.com>
 <20150506132255.510c2eec@bigbox.christie.dr>
 <CALwzid=yoTxetHRuVDaDnCW9ASXVruF02gE2CZgsszJ6=3YxWQ@mail.gmail.com>
Message-ID: <20150506133712.5464d70c@bigbox.christie.dr>

On 2015-05-06 12:27, Ian Kelly wrote:
> On Wed, May 6, 2015 at 12:22 PM, Tim Chase
> <python.list at tim.thechases.com> wrote:
> > On 2015-05-06 19:08, MRAB wrote:
> >> You could tell it to quote any value that's not a number:
> >>
> >>       w = csv.DictWriter(f, pol_keys,
> >> quoting=csv.QUOTE_NONNUMERIC)
> >>
> >> It looks like all of the values you have are strings, so they'll
> >> all be quoted.
> >>
> >> I would hope that Excel will then treat it as a string; it would
> >> be stupid if it didn't! :-)
> >
> > Sadly, Excel *is* that stupid based on the tests I tried just
> > now. :-(
> >
> > Regardless of whether "Mar 2015" is quoted or unquoted in the
> > source CSV file, Excel tries to outwit you and mangles the
> > presentation.
> 
> Quoting a value in csv doesn't mean it's a string; it just means
> that it's a single field.
> 
> You *can* force Excel to treat a value as a string by prefixing it
> with an apostrophe, though.

Excel takes the apostrophe in the CSV file and puts it in the
content, rather than stripping it as an escape/formatting character:

  c:\temp> type test.csv
  "'Mar 2015",Mar 2015,3,2015
  "Apr 2015",Apr 2015,4,2015
  "2015-12",2015-12,12,2015
  c:\temp> start test.csv

A1 has the unformatted text, but includes the apostrophe in the
value.  B1, A2, and B2 get munged like the OP described to the form
"Apr-15".  The items in row #3 come through untouched.

At least on Excel 2003 on WinXP which is what I happen to have on
hand.

-tkc






From breamoreboy at yahoo.co.uk  Wed May  6 16:13:23 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 06 May 2015 21:13:23 +0100
Subject: Bitten by my C/Java experience
In-Reply-To: <554a3b37$0$12985$c3e8da3$5496439d@news.astraweb.com>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <n0S1x.370451$O36.280928@fx02.am4>
 <55487d30$0$2917$c3e8da3$76491128@news.astraweb.com>
 <es32x.478654$Qj7.158587@fx36.am4> <cqubn0F8a5gU1@mid.individual.net>
 <3Zn2x.359727$Ox2.83653@fx23.am4>
 <554a3b37$0$12985$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <midsl3$gse$1@ger.gmane.org>

On 06/05/2015 17:03, Steven D'Aprano wrote:
> On Wed, 6 May 2015 10:40 pm, BartC wrote:
>
>> But I had in mind not implementing ++ and --, but detecting them and
>> issuing a warning,
>
> That's a job for a linter, not the compiler. The compiler should be as
> flexible as possible in what it accepts:
>
>
> a   ,    b=12+3     *          4,"hello"          .      upper   ()
>
>
> is perfectly legal code. The compiler shouldn't force you to write good
> looking code, apart from what is prohibited altogether.
>
> Both + and - are unary prefix operators, so you can apply + and - to any
> expression -- even an expression that already has a unary prefix operator:
>
> py> -     --- +++ + - - + -- +++ --- 999
> -999
>
>
> Is that ugly, horrible code that nobody in their right mind would use in
> production? Absolutely. But the compiler can and should accept it, and
> linters (or human reviewers) should warn about it.
>

Linters were mentioned a day or two back.  Take a horse to water...

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From davea at davea.name  Wed May  6 16:23:08 2015
From: davea at davea.name (Dave Angel)
Date: Wed, 06 May 2015 16:23:08 -0400
Subject: Throw the cat among the pigeons
In-Reply-To: <87pp6doh09.fsf@universite-de-strasbourg.fr.invalid>
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
 <8761853fkd.fsf@jester.gateway.sonic.net>
 <87pp6doh09.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <554A782C.1070304@davea.name>

On 05/06/2015 11:36 AM, Alain Ketterlin wrote:
> Yes, plus the time for memory allocation. Since the code uses "r *=
> ...", space is reallocated when the result doesn't fit. The new size is
> probably proportional to the current (insufficient) size. This means
> that overall, you'll need fewer reallocations, because allocations are
> made in bigger chunks.

That sounds plausible, but  a=5; a*=4  does not update in place. It 
calculates and creates a new object.  Updating lists can work as you 
say, but an int is immutable.

It's an optimization that might be applied if the code generator were a 
lot smarter, (and if the ref count is exactly 1), but it would then be 
confusing to anyone who used id().

-- 
DaveA


From breamoreboy at yahoo.co.uk  Wed May  6 16:23:13 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 06 May 2015 21:23:13 +0100
Subject: Throw the cat among the pigeons
In-Reply-To: <CALwzidnaOqS3oruM=uqiJO3SvhMSB_adA6nO7xbXfpN=oB6xNQ@mail.gmail.com>
References: <87h9rvm576.fsf@Equus.decebal.nl> <871tixlmp6.fsf@Equus.decebal.nl>
 <750db03b-e393-43ff-9ccf-5cc050af7324@googlegroups.com>
 <87zj5jf15q.fsf@Equus.decebal.nl> <55490FAE.8070501@davea.name>
 <CALwzidmuhD0HhAZKRGMbgEgVVHf05_3pw=+o2JyAwWdiOtcNNA@mail.gmail.com>
 <55492F55.1020105@davea.name>
 <mailman.142.1430861087.12865.python-list@python.org>
 <55496de5$0$12993$c3e8da3$5496439d@news.astraweb.com>
 <mailman.158.1430891975.12865.python-list@python.org>
 <5549bde3$0$12911$c3e8da3$5496439d@news.astraweb.com>
 <CALwzidnaOqS3oruM=uqiJO3SvhMSB_adA6nO7xbXfpN=oB6xNQ@mail.gmail.com>
Message-ID: <midt7h$sqb$1@ger.gmane.org>

On 06/05/2015 17:17, Ian Kelly wrote:
> On Wed, May 6, 2015 at 1:08 AM, Steven D'Aprano
>>
>> Besides, "typical load" is a myth -- there is no such thing. A high-end
>> Windows web server getting ten thousand hits a minute, a virtual machine
>> starved for RAM, a Mac laptop, a Linux server idling away with a load of 0.1
>> all day... any of those machines could run your code. How can you *possibly*
>> say what is typical? The very idea is absurd.
>
> Agreed.
>

I must disagree with this, on the grounds that a "typical load" of old 
cobblers are frequently spouted on this list.  Practicality beats purity 
any day of the week.  But no, this '=' is misused, it should have been 
':='.  Who really cares, if you don't like the language, pick another 
one, there's thousands of them to choose from.  Ah, another chance for 
me to plug the virtues of CORAL 66 and CORAL 250.  Capability violation 
anybody?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From jenn.duerr at gmail.com  Wed May  6 16:27:15 2015
From: jenn.duerr at gmail.com (noydb)
Date: Wed, 6 May 2015 13:27:15 -0700 (PDT)
Subject: extracting zip item to in-memory
Message-ID: <95e1cce0-18ed-477a-a265-3256b7bcb25a@googlegroups.com>

I have a zip file containing several files and I want to extract out just the .xml file.  I have that code.  Curious if this xml file can be extracted into memory.  If so, how to?  I only need to move the file around, and maybe read some tags.

Thanks for any help!

python 2.7


From davea at davea.name  Wed May  6 16:38:21 2015
From: davea at davea.name (Dave Angel)
Date: Wed, 06 May 2015 16:38:21 -0400
Subject: extracting zip item to in-memory
In-Reply-To: <95e1cce0-18ed-477a-a265-3256b7bcb25a@googlegroups.com>
References: <95e1cce0-18ed-477a-a265-3256b7bcb25a@googlegroups.com>
Message-ID: <554A7BBD.6020907@davea.name>

On 05/06/2015 04:27 PM, noydb wrote:
> I have a zip file containing several files and I want to extract out just the .xml file.  I have that code.  Curious if this xml file can be extracted into memory.  If so, how to?  I only need to move the file around, and maybe read some tags.
>
> Thanks for any help!
>
> python 2.7
>

See https://docs.python.org/2.7/library/zipfile.html#zipfile.ZipFile.open

To open a particular member and get back a file-like object.

Once you have that, you can use the  read() method of that object.

Once you've coded this, if it doesn't work, post what you've got with a 
description of what doesn't work, and somebody here will be able to fix 
it up.


-- 
DaveA


From breamoreboy at yahoo.co.uk  Wed May  6 16:45:34 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 06 May 2015 21:45:34 +0100
Subject: extracting zip item to in-memory
In-Reply-To: <95e1cce0-18ed-477a-a265-3256b7bcb25a@googlegroups.com>
References: <95e1cce0-18ed-477a-a265-3256b7bcb25a@googlegroups.com>
Message-ID: <miduhf$ilq$1@ger.gmane.org>

On 06/05/2015 21:27, noydb wrote:
> I have a zip file containing several files and I want to extract out just the .xml file.  I have that code.  Curious if this xml file can be extracted into memory.  If so, how to?  I only need to move the file around, and maybe read some tags.
>
> Thanks for any help!
>
> python 2.7
>

https://docs.python.org/2/library/xml.html#module-xml

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From emile at fenx.com  Wed May  6 17:28:20 2015
From: emile at fenx.com (Emile van Sebille)
Date: Wed, 06 May 2015 14:28:20 -0700
Subject: Encrypt python files
In-Reply-To: <72bd42d6-6926-44ce-bd6f-956ce40afe08@googlegroups.com>
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
 <72bd42d6-6926-44ce-bd6f-956ce40afe08@googlegroups.com>
Message-ID: <mie11l$sbp$1@ger.gmane.org>

On 5/6/2015 12:23 AM, Palpandi wrote:
> On Wednesday, May 6, 2015 at 12:07:13 PM UTC+5:30, Palpandi wrote:
>> Hi,
>>
>> What are the ways to encrypt python files?
>
> No, I just want to hide the scripts from others.
>

See http://stackoverflow.com/questions/261638/how-do-i-protect-python-code

Emile



From zimmermann.code at gmail.com  Wed May  6 18:11:22 2015
From: zimmermann.code at gmail.com (Stefan Zimmermann)
Date: Wed, 6 May 2015 15:11:22 -0700 (PDT)
Subject: PEP idea: On Windows, subprocess should implicitly support .bat and
 .cmd scripts by using FindExecutable from win32 API
Message-ID: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>

Hi.

I don't like that subprocess.Popen(['command']) only works on Windows if there is a command.exe in %PATH%. As a Windows user you would normally expect that also command.bat and command.cmd can be run that way.

There are simple workarounds like Popen(..., shell=True) but that is a heavy overhead for .exe files.

Currently I use pywin32 and call Popen([win32api.FindExecutable('command')[1]]) as a workaround. This has zero overhead.

It should be default for Popen to call FindExecutable internally.

Was this discussed before?
Is it worth a PEP?
Or at least an issue?

Cheers,
Stefan


From denismfmcmahon at gmail.com  Wed May  6 19:27:16 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Wed, 6 May 2015 23:27:16 +0000 (UTC)
Subject: CHALLENGE HELP - GOOGLE DEVELOPER DAY
References: <c82211b3-4e58-4827-a2c1-64b824193b32@googlegroups.com>
Message-ID: <mie80k$meu$1@dont-email.me>

On Tue, 05 May 2015 09:59:03 -0700, worship.brother wrote:

> Archaeologists have found a scroll with the following texts:

First you need to visit the hidden Temple of Offler, where you will find 
the Tears of Offler (they're the big gems set into the statue just above 
the teeth) after making your way through the maze with the big rolling 
ball, spiky pit, deadfall and spinning blade traps.

Then you offer the tears off Offler up on the altar of Cthulhu[1].

Either your brain will melt when Cthulhu appears to you, or you will be 
blessed with the inspiration to write the code to solve your problem.

If the latter, and the code doesn't work, show us the code and we might 
be able to make suggestions for you.

[1] If you thought getting Offler's Tears was hard, wait until you try 
reaching the altar of Cthulhu.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From rob.clewley at gmail.com  Wed May  6 19:31:09 2015
From: rob.clewley at gmail.com (Rob Clewley)
Date: Wed, 6 May 2015 19:31:09 -0400
Subject: [SciPy-User] Is there existing code to log-with-bells-on for
 runtime algorithm diagnostics?
In-Reply-To: <CAF6FJitWApUdez8N9HbQA4zTXYhwWp34e0-cB+B3p5uu_smjCA@mail.gmail.com>
References: <CA+7tCyR=YZstmdZi9smie4=cZc+sXs61-oxzkSLhMW73+0FQVA@mail.gmail.com>
 <CAF6FJitWApUdez8N9HbQA4zTXYhwWp34e0-cB+B3p5uu_smjCA@mail.gmail.com>
Message-ID: <CA+7tCyQ=uPUqreAwBxYW3dAjDiD7LbU0tWg9hzap1mA7Fk+AtA@mail.gmail.com>

Just to follow up on this thread, for interested readers' future reference...

On Tue, Apr 21, 2015 at 4:22 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On Tue, Apr 21, 2015 at 8:02 PM, Rob Clewley <rob.clewley at gmail.com> wrote:
>> In fact, I'm trying to build a general purpose tool for exploring the
>> inner workings of numerical algorithms for teaching and learning
>> purposes, e.g. for graduate student training or for figuring out
>> parameter choices in difficult applications.
>
> The term you want to search for is "structured logging".
>
> http://www.structlog.org/en/stable/
> http://eliot.readthedocs.org/en/stable/
> https://twiggy.readthedocs.org/en/latest/logging.html#structured-logging
> http://netlogger.lbl.gov/
>

I posted a new blog entry about my prototypical diagnosis and
visualization tools for python numerical algorithms, built over
matplotlib:
http://robclewley.github.io/logging-and-diagnostic-tools-for-numeric-python-algorithms/

They utilize structlog, and I hooked up a noSQL DB (tinydb) to the
logging to enable search capabilities of the log post-mortem. Comments
and PRs most welcome.

Thanks to everyone for the pointers.
-Rob


From denismfmcmahon at gmail.com  Wed May  6 19:31:52 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Wed, 6 May 2015 23:31:52 +0000 (UTC)
Subject: Writing list of dictionaries to CSV
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
Message-ID: <mie898$meu$2@dont-email.me>

On Tue, 05 May 2015 22:32:28 -0700, Kashif Rana wrote:

> thanks for the feedback. I think its problem with excel itself, showing
> wrong value. Because when I opened the csv file in text editor, I can
> see correct value but opening in excel showing wrong value. What I can
> do to see correct in excel as well.

You need to format your CSV date into a date format that Excel 
understands when it imports it.

First thing to try would be to export some dates from excel as CSV and 
see what format excel puts them in.

The see if excel recognises them as dates when you re-import the same 
file.

If excel recognises it's own csv exported dates, reformat your dates to 
match the excel ones when you generate the csv.

Otherwise, you might need to convert the dates to a numeric value and 
tell excel to format the field as date after input.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From nimit at nimitkalra.com  Wed May  6 20:08:33 2015
From: nimit at nimitkalra.com (Nimit Kalra)
Date: Wed, 6 May 2015 19:08:33 -0500
Subject: Simple Interactive Python Version Management for Managing Python 2,
 Python 3, and Development Builds
Message-ID: <CAChB-fX9OhMpdxVDFehgScXCQZcNwuJHCR=4+9Xeyye+pb=4tQ@mail.gmail.com>

Hey all,

Wrote a ~600 line Bash script that takes the pain out of managing Python
versions (from 2.x.x to 3.x.x to development builds). Below are some more
details. The entire project can be found on GitHub at
https://github.com/qw3rtman/p. Let me know if you find it useful. :)

p: Python Version Management, Simplified.
=================================

This GIF sums up the essence of p: http://i.imgur.com/rFrAIlc.gif.

p is powerful and feature-packed, yet simple; both in setup and use. There
are no tricky settings, options, or crazy dependencies. p is just a helpful
~600 line Bash script that gets the job done.

p let's you quickly switch between Python versions whenever you need to,
removing the barrier between Python 2.x.x and 3.x.x.

p was heavily inspired by n, a version manager for Node.js.

p is also great for getting started using Python development versions. Use
p latest to get up and running with the latest development version of
Python!


Getting Started
============

After the super painless drag-and-drop installation, you can start using p
right away.


Usage
=====

Usage: p [COMMAND] [args]

Commands:

p                              Output versions installed
p status                       Output current status
p <version>                    Activate to Python <version>
    p latest                     Activate to the latest Python release
    p stable                     Activate to the latest stable Python
release
p use <version> [args ...]     Execute Python <version> with [args ...]
p bin <version>                Output bin path for <version>
p rm <version ...>             Remove the given version(s)
p prev                         Revert to the previously activated version
p ls                           Output the versions of Python available
    p ls latest                  Output the latest Python version available
    p ls stable                  Output the latest stable Python version
available

Options:

-V, --version   Output current version of p
-h, --help      Display help information


Installation
========

After downloading the Bash script, simply copy it over to your $PATH and p
will take care of the rest.

$ wget https://github.com/qw3rtman/p/releases/download/v0.1.0/p
$ chmod +x p
$ mv p /usr/local/bin

If you don't have wget on your system, you can download the p binary from
the releases page and follow the above steps from the second one onward.

Updating
=======

Simply follow the above steps and swap out the old Bash script with the new
one!


TODO
=====

greater abstraction between Python 2.x.x and 3.x.x
also manage pip
per-directory/project Python version
also manage PyPy

FAQs
====

How does p differ from pyenv?

 - p is designed for the average Python user. You can get up and running
with the latest development build of Python with one simple command: p
latest. No configuration is necessary; p manages everything for you.

 - On the other hand, pyenv is for the more advanced user who is
comfortable configuring their Python environment to all their needs. p
provides the basics in one easy to use aesthetically-pleasing command.

 - Additionally, p is easier to use. To switch your Python version, simply
run p and you'll be presented with a list of installed Python versions to
select from that you can scroll through with your arrow keys and select
with the return key.

 - p is great at dealing with any version of Python. If it's not installed,
running p <version>will download the source, configure it for your system,
and compile it, all in one simple command.


How does p work?

 - p stores each Python version installed in /usr/local/p/versions/python.
When a Python version is activated, p creates a symbolic link to the Python
binary located at/usr/local/p/versions/python/python. Since p prefixes the
$PATH with/usr/local/p/versions/python, this version of python is found
first; hence, it is used over the default version of Python installed on
your system.


How do I revert back to my default Python version?

 - Simply run p default and p will remove the symbolic link described
above; therefore reverting back to your default Python version.


Does p download the source each time I activate or install a version?

 - Nope. p stores the source for each of the versions installed, allowing
for quick activations between already-installed versions.


How do I get this working on Windows?

 - Unfortunately, p is not supported on Windows at the time. If you know of
a workaround, send in a pull request!

Let me know what you think.

Thanks,
-- 
Nimit Kalra
nimit at nimitkalra.com | @qw3rtman
http://www.nimitkalra.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150506/70526440/attachment.html>

From davea at davea.name  Wed May  6 20:58:13 2015
From: davea at davea.name (Dave Angel)
Date: Wed, 06 May 2015 20:58:13 -0400
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
Message-ID: <554AB8A5.708@davea.name>

On 05/06/2015 06:11 PM, Stefan Zimmermann wrote:
> Hi.
>
> I don't like that subprocess.Popen(['command']) only works on Windows if there is a command.exe in %PATH%.

  As a Windows user you would normally expect that also command.bat and 
command.cmd can be run that way.
>

and command.com.

If it's really unfortunate that you picked "command" for your sample 
program name.  Since command.com was the shell in MSDOS, I was about to 
point you to COMSPEC to address your problem.

There's nothing Windows-specific about that behaviour.  In Linux, there 
are bash commands that can only be run by using shell=True.  Fortunately 
Popen didn't make the mistake of pretending it's a shell.

There is lots more to running a batch file than launching it.  The whole 
syntax of the rest of the commandline differs when you're doing that.

> There are simple workarounds like Popen(..., shell=True) but that is a heavy overhead for .exe files.

And the reason there's such an overhead is because you're requesting the 
services of the shell.  If you don't need those services, use shell=False.

>
> Currently I use pywin32 and call Popen([win32api.FindExecutable('command')[1]]) as a workaround. This has zero overhead.
>
> It should be default for Popen to call FindExecutable internally.
>
> Was this discussed before?
> Is it worth a PEP?
> Or at least an issue?
>
> Cheers,
> Stefan
>


-- 
DaveA


From python.list at tim.thechases.com  Wed May  6 21:22:05 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Wed, 6 May 2015 20:22:05 -0500
Subject: Writing list of dictionaries to CSV
In-Reply-To: <mie898$meu$2@dont-email.me>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
 <mie898$meu$2@dont-email.me>
Message-ID: <20150506202205.06be72fd@bigbox.christie.dr>

On 2015-05-06 23:31, Denis McMahon wrote:
> On Tue, 05 May 2015 22:32:28 -0700, Kashif Rana wrote:
> > thanks for the feedback. I think its problem with excel itself,
> > showing wrong value. Because when I opened the csv file in text
> > editor, I can see correct value but opening in excel showing
> > wrong value. What I can do to see correct in excel as well.
> 
> You need to format your CSV date into a date format that Excel 
> understands when it imports it.

That's part of the problem.  Excel is taking the OP's literal values
and interpreting them as dates (and formatting them in the way Excel
wants to) rather than accepting them as literal values.

As ChrisA posted earlier, you have to use Excel's Import
functionality (there are several ways to get this wizard, but not all
ways of opening a .csv trigger the wizard), then specify those
particular columns as "Text" rather than "General" (which
auto-detects as a Date).  This prevents Excel from overthinking the
problem, allowing Excel to import the text as-is like the OP wants.

> First thing to try would be to export some dates from excel as CSV
> and see what format excel puts them in.
> 
> The see if excel recognises them as dates when you re-import the
> same file.

If you create the formatting on such cells the way that the OP wants
them, then export, then re-import, Excel still mangles them since
the CSV spec doesn't preserve formatting.

-tkc






From python.list at tim.thechases.com  Wed May  6 21:52:27 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Wed, 6 May 2015 20:52:27 -0500
Subject: Writing list of dictionaries to CSV [correction]
In-Reply-To: <20150506202205.06be72fd@bigbox.christie.dr>
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
 <mie898$meu$2@dont-email.me>
 <20150506202205.06be72fd@bigbox.christie.dr>
Message-ID: <20150506205227.20b1115a@bigbox.christie.dr>

On 2015-05-06 20:22, Tim Chase wrote:
> As ChrisA posted earlier, you have to use Excel's Import
> functionality (there are several ways to get this wizard, but not
> all ways of opening a .csv trigger the wizard), then specify those
> particular columns as "Text" rather than "General"

Sorry, it was Dennis Lee Bieber responding to ChrisA with that
suggestion.

-tkc





From jon+usenet at unequivocal.co.uk  Wed May  6 21:55:27 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Thu, 7 May 2015 01:55:27 +0000 (UTC)
Subject: Writing list of dictionaries to CSV
References: <5bd0afb3-d2d4-469f-be39-5869f01adc92@googlegroups.com>
 <072badd6-0542-468a-99bd-cc3a4ffc7156@googlegroups.com>
 <mie898$meu$2@dont-email.me>
Message-ID: <slrnmklhif.5bn.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-06, Denis McMahon <denismfmcmahon at gmail.com> wrote:
> You need to format your CSV date into a date format that Excel 
> understands when it imports it.
>
> First thing to try would be to export some dates from excel as CSV and 
> see what format excel puts them in.

Beware of assuming that Excel can import its own exported data.
You may be disappointed.


From rosuav at gmail.com  Wed May  6 22:19:46 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 7 May 2015 12:19:46 +1000
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <554AB8A5.708@davea.name>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
Message-ID: <CAPTjJmpC22Vo64X70NkUce7npRgu5Oip0YQD=Eh4-m2O4_jCTQ@mail.gmail.com>

On Thu, May 7, 2015 at 10:58 AM, Dave Angel <davea at davea.name> wrote:
> There's nothing Windows-specific about that behaviour.  In Linux, there are
> bash commands that can only be run by using shell=True.  Fortunately Popen
> didn't make the mistake of pretending it's a shell.

But bash commands aren't the same as shell scripts. For instance, if
you want to enumerate bash aliases, you can't exec() to the 'alias'
command, because there isn't one. But shell scripts *can* be exec'd:

$ grep $ exec_demo.*
exec_demo.c:#include <stdio.h>
exec_demo.c:#include <unistd.h>
exec_demo.c:int main()
exec_demo.c:{
exec_demo.c: printf("This part is coming from C code.\n");
exec_demo.c: int err=execl("./exec_demo.sh", 0);
exec_demo.c: printf("exec() failed! %d\n",err);
exec_demo.c:}
exec_demo.sh:#!/bin/sh
exec_demo.sh:echo This part ran from the shell.
exec_demo.sh:echo Hello, world!
$ ./a.out
This part is coming from C code.
This part ran from the shell.
Hello, world!
$ pike -e 'Process.exec("./exec_demo.sh");'
This part ran from the shell.
Hello, world!
$ python -c 'import subprocess; subprocess.call(["./exec_demo.sh"])'
This part ran from the shell.
Hello, world!
(Python doesn't seem to have any way to 'exec', but a subprocess comes
to the same thing.)

I don't know about Windows, but it seems reasonable to be able to be
able to run many types of program equally, including batch files. But
maybe Windows is just weak that way.

ChrisA


From dan at tombstonezero.net  Wed May  6 23:16:53 2015
From: dan at tombstonezero.net (Dan Sommers)
Date: Thu, 7 May 2015 03:16:53 +0000 (UTC)
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
 <mailman.175.1430919653.12865.python-list@python.org>
Message-ID: <mielf4$omr$1@dont-email.me>

On Wed, 06 May 2015 09:12:05 -0400, Dave Angel wrote:

> Remember the days when you knew how many cycles each assembly
> instruction took, and could simply add them up to compare algorithms?

I do!  I do!  :-)

And then the MC68020 came out, and instruction execution overlapped in
weird (but predictable) ways, and interrupts would have different
(unpredictable) latency depending on how much internal state of the CPU
had to be saved and restored.

Man, am I *old*.

Dan


From steve+comp.lang.python at pearwood.info  Wed May  6 23:33:11 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 07 May 2015 13:33:11 +1000
Subject: PEP idea: On Windows,
 subprocess should implicitly support .bat and .cmd scripts by using
 FindExecutable from win32 API
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
Message-ID: <554adcf8$0$11103$c3e8da3@news.astraweb.com>

On Thursday 07 May 2015 12:19, Chris Angelico wrote:

> On Thu, May 7, 2015 at 10:58 AM, Dave Angel <davea at davea.name> wrote:
>> There's nothing Windows-specific about that behaviour.  In Linux, there
>> are
>> bash commands that can only be run by using shell=True.  Fortunately
>> Popen didn't make the mistake of pretending it's a shell.
> 
> But bash commands aren't the same as shell scripts. For instance, if
> you want to enumerate bash aliases, you can't exec() to the 'alias'
> command, because there isn't one. But shell scripts *can* be exec'd:

Um, are we still talking about Python here? exec("alias") fails with 
NameError on all the versions of Python I know. *semi-wink*

I'm guessing you're taking about some other exec, it might be a good idea 
that on a Python mailing list you don't assume that we're all going to 
understand the context :-)


> $ grep $ exec_demo.*
> exec_demo.c:#include <stdio.h>
> exec_demo.c:#include <unistd.h>
> exec_demo.c:int main()
> exec_demo.c:{
> exec_demo.c: printf("This part is coming from C code.\n");
> exec_demo.c: int err=execl("./exec_demo.sh", 0);
> exec_demo.c: printf("exec() failed! %d\n",err);
> exec_demo.c:}
> exec_demo.sh:#!/bin/sh
> exec_demo.sh:echo This part ran from the shell.
> exec_demo.sh:echo Hello, world!
> $ ./a.out
> This part is coming from C code.
> This part ran from the shell.
> Hello, world!

> $ pike -e 'Process.exec("./exec_demo.sh");'
> This part ran from the shell.
> Hello, world!

Okay, so C code can call the shell. So can Pike.


> $ python -c 'import subprocess; subprocess.call(["./exec_demo.sh"])'
> This part ran from the shell.
> Hello, world!

And so can Python. I'm not entirely sure what point you are trying to make 
here, or how it relates to the OP's problem that when he calls 

subprocess.Popen(['foo'])

he expects it to run any of foo.exe, foo.cmd, foo.bat (and possibly any 
other number of executable files). Are you agreeing with him or disagreeing?

Apart from any other number of problems, surely having "foo" alone run 
foo.exe, foo.bat etc. is at best confusing and at worst a security risk? 
What if you have *both* foo.exe and foo.bat in the same directory?


> (Python doesn't seem to have any way to 'exec', but a subprocess comes
> to the same thing.)

According to `man exec` on my Linux system, I don't think that is correct. 
The exec* family of functions "replaces the current process image with a new 
process image", they don't run in a subprocess.

I think the Python equivalent of Unix exec* commands are the various 
os.exec* functions.


> I don't know about Windows, but it seems reasonable to be able to be
> able to run many types of program equally, including batch files. But
> maybe Windows is just weak that way.


Hmmm. I'm not sure if this is relevant, or if I'm going off on a tangent, 
but if I write a short bash script and set the execute permission:

steve at runes:~$ chmod u+x test.sh
steve at runes:~$ cat test.sh 
echo "Running shell script"


subprocess.call fails unless I set shell=True:


py> p = subprocess.Popen('./test.sh', shell=True)
py> Running shell script

py> p = subprocess.Popen('./test.sh', shell=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/local/lib/python2.7/subprocess.py", line 1308, in 
_execute_child
    raise child_exception
OSError: [Errno 8] Exec format error


How is this any different from needing to specify shell=True for .bat and 
.cmd files under Windows? This is not a rhetorical question, I actually want 
to know.



-- 
Steve



From rosuav at gmail.com  Wed May  6 23:57:07 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 7 May 2015 13:57:07 +1000
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <554adcf8$0$11103$c3e8da3@news.astraweb.com>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
Message-ID: <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>

On Thu, May 7, 2015 at 1:33 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Thursday 07 May 2015 12:19, Chris Angelico wrote:
>
>> On Thu, May 7, 2015 at 10:58 AM, Dave Angel <davea at davea.name> wrote:
>>> There's nothing Windows-specific about that behaviour.  In Linux, there
>>> are
>>> bash commands that can only be run by using shell=True.  Fortunately
>>> Popen didn't make the mistake of pretending it's a shell.
>>
>> But bash commands aren't the same as shell scripts. For instance, if
>> you want to enumerate bash aliases, you can't exec() to the 'alias'
>> command, because there isn't one. But shell scripts *can* be exec'd:
>
> Um, are we still talking about Python here? exec("alias") fails with
> NameError on all the versions of Python I know. *semi-wink*
>
> I'm guessing you're taking about some other exec, it might be a good idea
> that on a Python mailing list you don't assume that we're all going to
> understand the context :-)

Fair point. I'm talking about the underlying exec* family of
functions, which are used on Unix-like systems to do normal process
execution. More on that later.

>> $ grep $ exec_demo.*
>> exec_demo.c:#include <stdio.h>
>> exec_demo.c:#include <unistd.h>
>> exec_demo.c:int main()
>> exec_demo.c:{
>> exec_demo.c: printf("This part is coming from C code.\n");
>> exec_demo.c: int err=execl("./exec_demo.sh", 0);
>> exec_demo.c: printf("exec() failed! %d\n",err);
>> exec_demo.c:}
>> exec_demo.sh:#!/bin/sh
>> exec_demo.sh:echo This part ran from the shell.
>> exec_demo.sh:echo Hello, world!
>> $ ./a.out
>> This part is coming from C code.
>> This part ran from the shell.
>> Hello, world!
>
>> $ pike -e 'Process.exec("./exec_demo.sh");'
>> This part ran from the shell.
>> Hello, world!
>
> Okay, so C code can call the shell. So can Pike.
>
>
>> $ python -c 'import subprocess; subprocess.call(["./exec_demo.sh"])'
>> This part ran from the shell.
>> Hello, world!
>
> And so can Python. I'm not entirely sure what point you are trying to make
> here, or how it relates to the OP's problem that when he calls
>
> subprocess.Popen(['foo'])
>
> he expects it to run any of foo.exe, foo.cmd, foo.bat (and possibly any
> other number of executable files). Are you agreeing with him or disagreeing?

I'm stating that this works on Unix with shell=False. Consequently,
the OP's request that it work thusly on Windows ought to be possible,
and ought to be reasonable.

> Apart from any other number of problems, surely having "foo" alone run
> foo.exe, foo.bat etc. is at best confusing and at worst a security risk?
> What if you have *both* foo.exe and foo.bat in the same directory?

There's a specific search order. Back in the days of DOS, it was
simply "com, then exe, then bat", but on modern Windowses, I think
it's governed by an environment variable (similarly to PATH for the
directories to search in). It's identical security risk to the
possibility of putting something earlier in $PATH; if you insist on
running a specific executable, you put the entire path, and then
there's no problem. (Actually, I'd consider this to be a feature, not
a bug - it's the equivalent of shadowing built-ins in a Python module.
When you want it, it's really useful.)

>> (Python doesn't seem to have any way to 'exec', but a subprocess comes
>> to the same thing.)
>
> According to `man exec` on my Linux system, I don't think that is correct.
> The exec* family of functions "replaces the current process image with a new
> process image", they don't run in a subprocess.
>
> I think the Python equivalent of Unix exec* commands are the various
> os.exec* functions.

Oh, I forgot about os.exec. That's a better equivalent:

$ python3 -c 'import os; os.execl("./exec_demo.sh","exec_demo.sh")'
This part ran from the shell.
Hello, world!

When you call subprocess.* to run something, what happens is generally
that Python forks and exec's to the new process. The parent of the
fork keeps running (or maybe waits on the child immediately), the
child uses one of the exec family of functions to replace itself with
the new process. This is different from Windows, where there's a
standard API function for "start this program in a new process".
Advantages of the fork/exec model include that you can do stuff in
between the two steps - for instance, you fork, then you change your
user credentials, root directory, current directory, nice value, usage
limits, etc, etc, etc, etc, prior to exec'ing. Unix systems don't need
a way to say "run this process with this root directory", because it
can all be built up from primitives.

So yes, in terms of the ability to locate other executables, it makes
no difference whether you fork first or not - the exec call is the
same.

> Hmmm. I'm not sure if this is relevant, or if I'm going off on a tangent,
> but if I write a short bash script and set the execute permission:
>
> steve at runes:~$ chmod u+x test.sh
> steve at runes:~$ cat test.sh
> echo "Running shell script"
>
> subprocess.call fails unless I set shell=True:
>
> py> p = subprocess.Popen('./test.sh', shell=True)
> py> Running shell script
>
> py> p = subprocess.Popen('./test.sh', shell=False)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/local/lib/python2.7/subprocess.py", line 711, in __init__
>     errread, errwrite)
>   File "/usr/local/lib/python2.7/subprocess.py", line 1308, in
> _execute_child
>     raise child_exception
> OSError: [Errno 8] Exec format error
>
>
> How is this any different from needing to specify shell=True for .bat and
> .cmd files under Windows? This is not a rhetorical question, I actually want
> to know.

Hmm... hm... Ha! Found the difference. I had an explicit shebang on my
script; yours just starts out with shell commands. That means that
your shell script wasn't truly executable, and thus requires a shell
to execute it. Try adding "#!/bin/sh" to the top and rerun that - at
that point, it becomes kernel-executable instead of just
shell-executable.

ChrisA


From gvanem at yahoo.no  Thu May  7 03:15:36 2015
From: gvanem at yahoo.no (Gisle Vanem)
Date: Thu, 07 May 2015 09:15:36 +0200
Subject: PEP idea: On Windows, subprocess should implicitly
 support .bat and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
Message-ID: <554B1118.6010805@yahoo.no>

Chris Angelico wrote:

> There's a specific search order. Back in the days of DOS, it was
> simply "com, then exe, then bat", but on modern Windowses, I think
> it's governed by an environment variable.

You probably mean '%PATHEXT'. Mine is:
  .COM;.EXE;.BAT;.BTM;.CMD;.JS;.JSE;.WSF;.WSH;.MSC;.tcl;.py;.pyw;.pl;.htm;.html

In my favourite shell 4NT, I simply can have:
   set .py=python

Instead of the Explorer associations that the Python-installer puts
in my registry. Revealed from my shell:
   c:\> assoc .py
    .py=py_auto_file
   c:\> ftype py_auto_file
    py_auto_file="F:\ProgramFiler\Python27\python.exe" "%1"

In ShellExecuteEx(), what program gets launched for "py_auto_file" in this
case, seems to be determined by the 'SHELLEXECUTEINFO:lpClass' member.
I fail to see that Python uses this structure anywhere.

-- 
--gv


From news at blinne.net  Thu May  7 04:00:51 2015
From: news at blinne.net (Alexander Blinne)
Date: Thu, 07 May 2015 10:00:51 +0200
Subject: Converting 5.223701009526849e-05 to 5e-05
References: <87vbgakrlr.fsf@Equus.decebal.nl> <85mw1mdpfd.fsf@benfinney.id.au>
 <mailman.54.1430642933.12865.python-list@python.org>
Message-ID: <mif63i$k9a$1@speranza.aioe.org>

Am 03.05.2015 um 10:48 schrieb Ben Finney:
> That's not as clear as it could be. Better is to be explicit about
> choosing ?exponential? format::
> 
>     >>> foo = 5.223701009526849e-05
>     >>> "{foo:5.0e}".format(foo=foo)
>     '5e-05'
> 

Or even better the "general" format, which also works for 0.9999:

    >>> "{foo:.1g}".format(foo=5.223701009526849e-5)
    '5e-05'
    >>> "{foo:.1g}".format(foo=0.9999)
    '1'

I guess all roads lead to Rome...



From alain at universite-de-strasbourg.fr.invalid  Thu May  7 05:14:30 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Thu, 07 May 2015 11:14:30 +0200
Subject: Throw the cat among the pigeons
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
 <8761853fkd.fsf@jester.gateway.sonic.net>
 <87pp6doh09.fsf@universite-de-strasbourg.fr.invalid>
 <mailman.189.1430943800.12865.python-list@python.org>
Message-ID: <87lhh0oikp.fsf@universite-de-strasbourg.fr.invalid>

Dave Angel <davea at davea.name> writes:

> On 05/06/2015 11:36 AM, Alain Ketterlin wrote:
>> Yes, plus the time for memory allocation. Since the code uses "r *=
>> ...", space is reallocated when the result doesn't fit. The new size is
>> probably proportional to the current (insufficient) size. This means
>> that overall, you'll need fewer reallocations, because allocations are
>> made in bigger chunks.
>
> That sounds plausible, but  a=5; a*=4  does not update in place. It
> calculates and creates a new object.  Updating lists can work as you
> say, but an int is immutable.

Ah, okay. Even for big ints? If that is the case, my suggestion doesn't
explain anything. Anyway, with so many allocations for so little
arithmetic, the difference is probably due to the behavior of the
allocator (which maybe always finds blocks big enough, since one was
released after the previous multiplication, or something like that). The
only way to know would be to profile the VM.

> It's an optimization that might be applied if the code generator were
> a lot smarter, (and if the ref count is exactly 1), but it would then
> be confusing to anyone who used id().

"Abandon all hope, ye [optimizer] who enter here."

Thanks for the clarification.

-- Alain.


From rosuav at gmail.com  Thu May  7 05:38:13 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 7 May 2015 19:38:13 +1000
Subject: Throw the cat among the pigeons
In-Reply-To: <87lhh0oikp.fsf@universite-de-strasbourg.fr.invalid>
References: <87h9rvm576.fsf@Equus.decebal.nl>
 <55499304$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <5549b41f$0$12927$c3e8da3$5496439d@news.astraweb.com>
 <8761853fkd.fsf@jester.gateway.sonic.net>
 <87pp6doh09.fsf@universite-de-strasbourg.fr.invalid>
 <mailman.189.1430943800.12865.python-list@python.org>
 <87lhh0oikp.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <CAPTjJmoNQXCPqPb7QmD7fnWm_WWDuxvehcXv39umpA=9azssTg@mail.gmail.com>

On Thu, May 7, 2015 at 7:14 PM, Alain Ketterlin
<alain at universite-de-strasbourg.fr.invalid> wrote:
> Dave Angel <davea at davea.name> writes:
>
>> On 05/06/2015 11:36 AM, Alain Ketterlin wrote:
>>> Yes, plus the time for memory allocation. Since the code uses "r *=
>>> ...", space is reallocated when the result doesn't fit. The new size is
>>> probably proportional to the current (insufficient) size. This means
>>> that overall, you'll need fewer reallocations, because allocations are
>>> made in bigger chunks.
>>
>> That sounds plausible, but  a=5; a*=4  does not update in place. It
>> calculates and creates a new object.  Updating lists can work as you
>> say, but an int is immutable.
>
> Ah, okay. Even for big ints? If that is the case, my suggestion doesn't
> explain anything. Anyway, with so many allocations for so little
> arithmetic, the difference is probably due to the behavior of the
> allocator (which maybe always finds blocks big enough, since one was
> released after the previous multiplication, or something like that). The
> only way to know would be to profile the VM.

Yes, all integers are immutable. This is true regardless of the size
of the integer, because:

x = some_big_long_calculation()
y = x
y += 1

should never change the value of x.

ChrisA


From zimmermann.code at gmail.com  Thu May  7 05:38:16 2015
From: zimmermann.code at gmail.com (Stefan Zimmermann)
Date: Thu, 7 May 2015 02:38:16 -0700 (PDT)
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <mailman.200.1430983119.12865.python-list@python.org>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
Message-ID: <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>

Nice to see that my topic gains that interest :)
And I see that I should have gone more into detail about what I'm actually trying to point out.

Chris Angelico wrote:
> Hmm... hm... Ha! Found the difference. I had an explicit shebang on my
> script; yours just starts out with shell commands. That means that
> your shell script wasn't truly executable, and thus requires a shell
> to execute it. Try adding "#!/bin/sh" to the top and rerun that - at
> that point, it becomes kernel-executable instead of just
> shell-executable.

That's the big advantage of Unix. You can write an kernel-executable script without any file extension, just by putting a shebang to the beginning of that file. And for the caller it makes no difference if 'command' is a binary or a script and Popen('command') works in both cases, without the shell=True overhead.

Steven D'Aprano wrote:
> Apart from any other number of problems, surely having "foo" alone run 
> foo.exe, foo.bat etc. is at best confusing and at worst a security risk? 
> What if you have *both* foo.exe and foo.bat in the same directory?

On Unix you can shadow any binary with a wrapper script of the same name located in a path appearing earlier in $PATH. Any caller will automatically run your script instead of the original binary. An that's usually seen as a big advantage on Unix.
On Windows executability depends on the file extension and if you want to wrap some command.exe you usually write a command.bat in a path with higher precedence. And in Windows it's standard that .exe, .com, .bat and .cmd files should be callable without writing the file extension.
And as already mentioned, there is a defined precedence order if they are in the same directory.
That's not more or less security risky as shadowing binaries with scripts on Unix.

My point is that compared to Unix it's just a big disadvantage on Windows that the subprocess.Popen(['command']) can only call command.exe implicitly, which makes it impossible to work with custom wrapper .bat or .cmd scripts without the shell=True overhead.
And this is acutally confusing for a Windows user.
You write a wrapper .bat for some .exe and are wondering why your Python script doesn't use it.

And the FindExecutable() function from the win32 API would just be the perfect solution for implementing this.


From zimmermann.code at gmail.com  Thu May  7 06:03:09 2015
From: zimmermann.code at gmail.com (Stefan Zimmermann)
Date: Thu, 7 May 2015 03:03:09 -0700 (PDT)
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
Message-ID: <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>

And last but not least, Popen behavior on Windows makes it difficult to write OS-independent Python code which calls external commands that are not binary by default:

2 examples:

1. I wrote a coffeetools package which wraps the CoffeeScript compiler in a Python API. The 'coffee' command is a Node.js script. And under Windows it is installed with a 'coffee.cmd' wrapper to make it callable. So to make Popen work you have to switch and call 'coffee' on Unix and 'coffee.cmd' on Windows. But from the Windows shell you can just call 'coffee'. Maybe in the future the .cmd changes to .bat ...

2. I the embedded portable git from SourceTree instead of the standard Windows git installation. It has a git.bat wrapper which calls the internal git.exe (which must be in the same dir with a lot of other included ported Unix tools and therefore not recommended to add that dir to PATH). That made the dulwich package unworkable for me because it just tries to Popen(['git', ...]). And I am currently trying to make the dulwich developers accept my pull request with a workaround...


From marko at pacujo.net  Thu May  7 06:10:36 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 07 May 2015 13:10:36 +0300
Subject: PEP idea: On Windows,
 subprocess should implicitly support .bat and .cmd scripts by using
 FindExecutable from win32 API
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
Message-ID: <87ioc4k89v.fsf@elektro.pacujo.net>

Stefan Zimmermann <zimmermann.code at gmail.com>:

> And last but not least, Popen behavior on Windows makes it difficult
> to write OS-independent Python code which calls external commands that
> are not binary by default:

Then, write OS-dependent Python code.

I don't think it's Python's job to pave over OS differences. Java does
that by not offering precious system facilities -- very painful. Python
is taking steps in that direction, but I hope it won't go too far.


Marko


From rosuav at gmail.com  Thu May  7 06:24:21 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 7 May 2015 20:24:21 +1000
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <87ioc4k89v.fsf@elektro.pacujo.net>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>

On Thu, May 7, 2015 at 8:10 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Stefan Zimmermann <zimmermann.code at gmail.com>:
>
>> And last but not least, Popen behavior on Windows makes it difficult
>> to write OS-independent Python code which calls external commands that
>> are not binary by default:
>
> Then, write OS-dependent Python code.
>
> I don't think it's Python's job to pave over OS differences. Java does
> that by not offering precious system facilities -- very painful. Python
> is taking steps in that direction, but I hope it won't go too far.

On the contrary, I think it *is* a high level language's job to pave
over those differences. Portable C code generally has to have a
whopping 'configure' script that digs into your hardware, OS, library,
etc availabilities, and lets you figure out which way to do things.
Python code shouldn't need to worry about that. You don't need to care
whether you're on a 32-bit or 64-bit computer; you don't need to care
whether it's an Intel chip or a RISCy one; you shouldn't have to
concern yourself with the difference between BSD networking and
WinSock. There'll be a handful of times when you do care, and for
those, it's nice to have some facilities exposed; but the bulk of code
shouldn't need to know about the platform it's running on.

Java went for a philosophy of "write once, run anywhere" in its early
days, and while that hasn't exactly been stuck to completely, it's
still the reasoning behind the omission of certain system facilities.
Python accepts and understands that there will be differences, so you
can't call os.getuid() on Windows, and there are a few restrictions on
the subprocess module if you want maximum portability, but the bulk of
your code won't be any different on Linux, Windows, Mac OS, OS/2,
Amiga, OS/400, Solaris, or a MicroPython board.

ChrisA


From davea at davea.name  Thu May  7 07:28:09 2015
From: davea at davea.name (Dave Angel)
Date: Thu, 07 May 2015 07:28:09 -0400
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
Message-ID: <554B4C49.7010500@davea.name>

On 05/07/2015 06:24 AM, Chris Angelico wrote:
> On Thu, May 7, 2015 at 8:10 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
>> Stefan Zimmermann <zimmermann.code at gmail.com>:
>>
>>> And last but not least, Popen behavior on Windows makes it difficult
>>> to write OS-independent Python code which calls external commands that
>>> are not binary by default:
>>
>> Then, write OS-dependent Python code.
>>
>> I don't think it's Python's job to pave over OS differences. Java does
>> that by not offering precious system facilities -- very painful. Python
>> is taking steps in that direction, but I hope it won't go too far.
>
> On the contrary, I think it *is* a high level language's job to pave
> over those differences. Portable C code generally has to have a
> whopping 'configure' script that digs into your hardware, OS, library,
> etc availabilities, and lets you figure out which way to do things.
> Python code shouldn't need to worry about that. You don't need to care
> whether you're on a 32-bit or 64-bit computer; you don't need to care
> whether it's an Intel chip or a RISCy one; you shouldn't have to
> concern yourself with the difference between BSD networking and
> WinSock. There'll be a handful of times when you do care, and for
> those, it's nice to have some facilities exposed; but the bulk of code
> shouldn't need to know about the platform it's running on.
>
> Java went for a philosophy of "write once, run anywhere" in its early
> days, and while that hasn't exactly been stuck to completely, it's
> still the reasoning behind the omission of certain system facilities.
> Python accepts and understands that there will be differences, so you
> can't call os.getuid() on Windows, and there are a few restrictions on
> the subprocess module if you want maximum portability, but the bulk of
> your code won't be any different on Linux, Windows, Mac OS, OS/2,
> Amiga, OS/400, Solaris, or a MicroPython board.
>
> ChrisA
>

It's a nice goal.  But these aren't OS features in Windows, they're 
shell features.  And there are several shells.  If the user has 
installed a different shell, is it Python's job to ignore it and 
simulate what cmd.exe does?

Seems to me that's what shell=True is for.  it signals Python that we're 
willing to trust the shell to do whatever magic it chooses, from adding 
extensions, to calling interpreters, to changing search order, to 
parsing the line in strange ways, to setting up temporary environment 
contexts, etc.

If there were just one shell, it might make sense to emulate its 
features.  Or it might make sense to contort its features to look like a 
Unix shell.  But with multiple possibilities, seems that's more like 
space for a 3rd party library.

-- 
DaveA


From rosuav at gmail.com  Thu May  7 07:43:34 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 7 May 2015 21:43:34 +1000
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <554B4C49.7010500@davea.name>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
Message-ID: <CAPTjJmqkFibCweiFOZLaOFtCt_z4jhcrxAH1tM8i5obXXcMKWA@mail.gmail.com>

On Thu, May 7, 2015 at 9:28 PM, Dave Angel <davea at davea.name> wrote:
> It's a nice goal.  But these aren't OS features in Windows, they're shell
> features.  And there are several shells.  If the user has installed a
> different shell, is it Python's job to ignore it and simulate what cmd.exe
> does?

It might be an unattainable goal (in fact, it almost certainly is),
but I was specifically disagreeing with the notion that it's right and
normal to write a bunch of platform-specific code in Python. That
should be the rarity.

ChrisA


From marko at pacujo.net  Thu May  7 08:41:30 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 07 May 2015 15:41:30 +0300
Subject: PEP idea: On Windows,
 subprocess should implicitly support .bat and .cmd scripts by using
 FindExecutable from win32 API
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
Message-ID: <87a8xgk1ad.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> I was specifically disagreeing with the notion that it's right and
> normal to write a bunch of platform-specific code in Python. That
> should be the rarity.

Why is that?

Code is written for a specific need and environment. Often trying to
write generic solutions leads to cumbersome and clunky results on *all*
platforms.

A software system is defined through its interfaces. Natural system
interfaces are very different under different operating systems. The
chosen programming language for whatever component is often an
afterthought. I'm glad I can still write native Linux code using Python.
I couldn't do that with Java, which doesn't have things like os.fork(),
file descriptors or process ids.


Marko


From rosuav at gmail.com  Thu May  7 08:53:16 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 7 May 2015 22:53:16 +1000
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <87a8xgk1ad.fsf@elektro.pacujo.net>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmpQ14qA0x401i9gYPkha-gGVGe=Gr60tjVVXdFSLU4AuQ@mail.gmail.com>

On Thu, May 7, 2015 at 10:41 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Chris Angelico <rosuav at gmail.com>:
>
>> I was specifically disagreeing with the notion that it's right and
>> normal to write a bunch of platform-specific code in Python. That
>> should be the rarity.
>
> Why is that?
>
> Code is written for a specific need and environment. Often trying to
> write generic solutions leads to cumbersome and clunky results on *all*
> platforms.
>
> A software system is defined through its interfaces.

And the most important interface is with a human. Humans are the same
whether you're running under Windows, Linux, or anything else. If you
want to write single-platform code, go for it; but if you want to
write cross-platform code, the best way is to let someone else take
care of the differences, abstracting them away into a nice tidy thing
that we call a high-level language.

I don't need forking, file descriptors, or process IDs, to describe
how a person uses my code. Those are *implementation details*. Now, it
might be that I have to concern myself with some of them. Maybe I want
to get optimal performance out of something, and that means using
multiple processes and managing them properly. Maybe I need to
interface with systemd, respond to dozens of different process-level
signals, use directory notifications, and do a bunch of other
Linux-only things, so maybe it's just completely impractical to
consider supporting even BSD-based Unixes, much less Windows. So be
it. But to the greatest extent possible, Python should let me write
code that doesn't care about any of that.

ChrisA


From marko at pacujo.net  Thu May  7 09:44:39 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 07 May 2015 16:44:39 +0300
Subject: PEP idea: On Windows,
 subprocess should implicitly support .bat and .cmd scripts by using
 FindExecutable from win32 API
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
 <mailman.207.1431003199.12865.python-list@python.org>
Message-ID: <876184jyd4.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

>> A software system is defined through its interfaces.
>
> And the most important interface is with a human.

I barely ever program anything for the human interface.

> If you want to write single-platform code, go for it; but if you want
> to write cross-platform code, the best way is to let someone else take
> care of the differences, abstracting them away into a nice tidy thing
> that we call a high-level language.

You suggested most software should be platform-agnostic. Now you are
qualifying the statement.

But still, I challenge the notion that you could write a web site, game
or application that feels natural on the XBox, iPhone, Windows PC and
LXDE at the same time without significant amounts of
platform-conditioned parts.

> I don't need forking, file descriptors, or process IDs, to describe
> how a person uses my code. Those are *implementation details*.

Even if I programmed for the human and the UI experience were
more-or-less identical between platforms, the system interfaces can be
conceptually quite different. Heroic attempts have been made to overcome
those differences with generic APIs. However, Python should stay out of
that crusade.

Whole programming cultures, idioms and "right ways" differ between
platforms. What's the right way to write a service (daemon)? That's
probably completely different between Windows and Linux. Linux itself is
undergoing a biggish transformation there: an exemplary daemon of last
year will likely be deprecated within a few years.


Marko


From rosuav at gmail.com  Thu May  7 10:03:59 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 00:03:59 +1000
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <876184jyd4.fsf@elektro.pacujo.net>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
 <mailman.207.1431003199.12865.python-list@python.org>
 <876184jyd4.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmrTND2Ou+Tib3nvdkODvuB6dMrL4FTMECKq5QHLQhx1fA@mail.gmail.com>

On Thu, May 7, 2015 at 11:44 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Chris Angelico <rosuav at gmail.com>:
>
>>> A software system is defined through its interfaces.
>>
>> And the most important interface is with a human.
>
> I barely ever program anything for the human interface.
>
>> If you want to write single-platform code, go for it; but if you want
>> to write cross-platform code, the best way is to let someone else take
>> care of the differences, abstracting them away into a nice tidy thing
>> that we call a high-level language.
>
> You suggested most software should be platform-agnostic. Now you are
> qualifying the statement.

I'm qualifying it because it's impossible to write 100%
platform-agnostic code without restricting yourself far too much; but
that doesn't mean that it isn't a worthwhile aim.

> But still, I challenge the notion that you could write a web site, game
> or application that feels natural on the XBox, iPhone, Windows PC and
> LXDE at the same time without significant amounts of
> platform-conditioned parts.

Hmm, you're picking up some very different things there. When a human
picks up an iPhone, s/he expects to use it with a touch-based
interface; I don't know what the normal UI for an Xbox is, but Xbox
users would; and the most normal interface for LXDE would be a
mouse+keyboard. The ideal UI for each of them will differ. This is the
same as coding your application differently if you expect a blind
person to use it, or if you want to make it possible to use your
program in a theatre without disturbing the audience, or any other UI
constraint you wish to concoct. That's nothing to do with platform. If
you write a program for Ubuntu, it might go onto a tablet or a
desktop, and the ideal UI for those is (in my opinion, though not
apparently in Unity's) different.

But if you design your program to be used with the same fundamental
human interface - say, a mouse and a keyboard - then you should be
able to do that the same way on many platforms. I've seen libraries
that let you build an ncurses-like interface or a full GUI window,
using exactly the same application code. It's not difficult.

>> I don't need forking, file descriptors, or process IDs, to describe
>> how a person uses my code. Those are *implementation details*.
>
> Even if I programmed for the human and the UI experience were
> more-or-less identical between platforms, the system interfaces can be
> conceptually quite different. Heroic attempts have been made to overcome
> those differences with generic APIs. However, Python should stay out of
> that crusade.
>
> Whole programming cultures, idioms and "right ways" differ between
> platforms. What's the right way to write a service (daemon)? That's
> probably completely different between Windows and Linux. Linux itself is
> undergoing a biggish transformation there: an exemplary daemon of last
> year will likely be deprecated within a few years.

And that's where a library function can be really awesome. What's the
right way to daemonize? "import daemonize; daemonize.daemonize()"
seems good to me. Maybe there's platform-specific code in the
*implementation* of that, but in your application, no. That's the job
of a layer underneath you.

Incidentally, the way I'm seeing things shift these days is mainly
toward *not* daemonizing your services at all. That makes life a lot
easier; instead of writing special code to put yourself in the
background, you just write your code to the standard basic "glass
teletype" model, and then add a little config file that makes it run
in the background. But a Python module could provide a generic
"install as service" function, which will create a systemd config
file, or a Windows service whatever-it-is, or the equivalent on a Mac,
or an Upstart job file, or whatever it detects. Same difference. A
library takes care of all of that.

In Python, we have the 'subprocess' module. Due to Windows
limitations, you have to restrict yourself to having an importable
main file if you want perfect cross-platform compatibility, but that
doesn't affect how your code runs on Linux or Mac OS. What's the best
way to farm work off to a bunch of processes and have them communicate
their results back? You use the subprocess module, and then it doesn't
matter whether they use Unix sockets, named pipes, physical files on
the disk, or miniature nuclear explosions, they'll communicate back
just fine. And when someone develops a new platform that uses nuclear
fusion instead of fission for interprocess communication, Python's
standard library gets enhanced, and your code instantly works - you
don't have to specifically handle the new case.

That's Python's job. Abstracting away all those differences so you
don't have to look at them.

ChrisA


From ian.g.kelly at gmail.com  Thu May  7 11:14:34 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 7 May 2015 09:14:34 -0600
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <CAPTjJmrTND2Ou+Tib3nvdkODvuB6dMrL4FTMECKq5QHLQhx1fA@mail.gmail.com>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
 <mailman.207.1431003199.12865.python-list@python.org>
 <876184jyd4.fsf@elektro.pacujo.net>
 <CAPTjJmrTND2Ou+Tib3nvdkODvuB6dMrL4FTMECKq5QHLQhx1fA@mail.gmail.com>
Message-ID: <CALwzidnG72x8a185_3BfaWeeuzFTb5P6WiPqNYPyz7ozePmSiQ@mail.gmail.com>

On Thu, May 7, 2015 at 8:03 AM, Chris Angelico <rosuav at gmail.com> wrote:
> On Thu, May 7, 2015 at 11:44 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
>> Whole programming cultures, idioms and "right ways" differ between
>> platforms. What's the right way to write a service (daemon)? That's
>> probably completely different between Windows and Linux. Linux itself is
>> undergoing a biggish transformation there: an exemplary daemon of last
>> year will likely be deprecated within a few years.
>
> And that's where a library function can be really awesome. What's the
> right way to daemonize? "import daemonize; daemonize.daemonize()"
> seems good to me. Maybe there's platform-specific code in the
> *implementation* of that, but in your application, no. That's the job
> of a layer underneath you.

https://www.python.org/dev/peps/pep-3143/


From marko at pacujo.net  Thu May  7 11:24:27 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 07 May 2015 18:24:27 +0300
Subject: PEP idea: On Windows,
 subprocess should implicitly support .bat and .cmd scripts by using
 FindExecutable from win32 API
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
 <mailman.207.1431003199.12865.python-list@python.org>
 <876184jyd4.fsf@elektro.pacujo.net>
 <mailman.208.1431007449.12865.python-list@python.org>
Message-ID: <8761845s2c.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> What's the best way to farm work off to a bunch of processes and have
> them communicate their results back? You use the subprocess module,
> and then it doesn't matter whether they use Unix sockets, named pipes,
> physical files on the disk, or miniature nuclear explosions, they'll
> communicate back just fine. And when someone develops a new platform
> that uses nuclear fusion instead of fission for interprocess
> communication, Python's standard library gets enhanced, and your code
> instantly works - you don't have to specifically handle the new case.
>
> That's Python's job. Abstracting away all those differences so you
> don't have to look at them.

That's the difference between our opinions: you want Python to work the
same on different OS's. I want Python's system programming facilities to
closely mirror those of C.

Take your example of subprocess.Popen. It may be essential to know that
the communication channel is a pipe with standard pipe semantics. The
child program might not be written in Python, after all. In fact, at
system design level you shouldn't care what language you use as long as
the communication interfaces are specified.

Java is lousy at system programming (while excellent in many other
respects). It has traditionally tried to avoid the associated issues by
effectively mandating that all parts of a system be written in Java.
Gladly, Python hasn't (yet) made the same mistake.


Marko


From zimmermann.code at gmail.com  Thu May  7 11:45:19 2015
From: zimmermann.code at gmail.com (Stefan Zimmermann)
Date: Thu, 7 May 2015 08:45:19 -0700 (PDT)
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <8761845s2c.fsf@elektro.pacujo.net>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
 <mailman.207.1431003199.12865.python-list@python.org>
 <876184jyd4.fsf@elektro.pacujo.net>
 <mailman.208.1431007449.12865.python-list@python.org>
 <8761845s2c.fsf@elektro.pacujo.net>
Message-ID: <41a6326f-69c2-4a44-990b-7b714cc96b9e@googlegroups.com>

This discussion is getting really interesting and far beyond the actual topic :)
I want to add some additional thoughts on Popen:

Marko Rauhamaa wrote:
> Stefan Zimmermann:
> 
> > And last but not least, Popen behavior on Windows makes it difficult
> > to write OS-independent Python code which calls external commands that
> > are not binary by default:
> 
> Then, write OS-dependent Python code.

Then you even have to write tool-distribution-dependent code.
Especially Unix tools are often distributed in many different variants for Windows. Some installers expose .exe, some .bat, some .cmd files to the user. So you always have to explicitly support any variant. Or run everything through 'cmd /c ...', which is as mentioned a real overhead for .exe files. Or you always have to manually use the win32 API, just to call a simple external tool. 

Calling an external command should be one of the simplest tasks in a high level scripting language like Python. And that should not involve any OS-specific differences, unless you want to use some advanced process handling features which are only supported by some specific OS.

Meanwhile I checked Ruby and Perl regarding this feature. Both support it. In both langs every standard function that calls external commands (like Perl's exec() or system() or Ruby's exec() or system() or IO.popen()), whether they invoke a shell or call it directly, support running 'tool.bat' or 'tool.cmd' by just writing 'tool'. Python almost seems to be the only major scripting language which does not support this implicitly.

Dave Angel wrote:
> It's a nice goal.  But these aren't OS features in Windows, they're 
> shell features.  And there are several shells.  If the user has 
> installed a different shell, is it Python's job to ignore it and 
> simulate what cmd.exe does?

In fact, it's something between OS and shell. Yes, .bat and .cmd files are always run thgrough cmd.exe. But on the OS level they are also condsidered executable files. And that doesn't depend on %PATHEXT% or any registered applications for file extensions on the explorer or shell level. On the OS level .exe, .com, .bat and .cmd is the exclusive set of extensions which are considered as executable. Not more and not less. When you search for the path of an executable with the win32 API, you can call FindExecutable with only 'tool' and you will get '...\tool.exe', '.com', '.bat' or '.cmd'. Whatever is found first according to PATH and precedence. Not more and not less. You can try that via pywin32 using win32api.FindExecutable().

And interestingly, Popen can call .bat and .cmd scripts directly if you explicitly specify the extension, also with shell=False. But it can't call any other file types even if that works in the shell because some application is registered for them on the explorer level, unless shell=True.

On Windows .bat and .cmd scripts have a special status beginning on the lowest OS level and a Windows user normally expects that any scripting language should be able to run them without explicit extension. Other major languages do it. Why not Python, too?


From marko at pacujo.net  Thu May  7 14:13:27 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 07 May 2015 21:13:27 +0300
Subject: PEP idea: On Windows,
 subprocess should implicitly support .bat and .cmd scripts by using
 FindExecutable from win32 API
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
 <mailman.207.1431003199.12865.python-list@python.org>
 <876184jyd4.fsf@elektro.pacujo.net>
 <mailman.208.1431007449.12865.python-list@python.org>
 <8761845s2c.fsf@elektro.pacujo.net>
 <41a6326f-69c2-4a44-990b-7b714cc96b9e@googlegroups.com>
Message-ID: <87pp6c45o8.fsf@elektro.pacujo.net>

Stefan Zimmermann <zimmermann.code at gmail.com>:

> Calling an external command should be one of the simplest tasks in a
> high level scripting language like Python.

Actually, that's quite a tricky operation in any OS. For example, bash's
simplicity is a trap that claims a lot of victims.

Anyway, Python has os.system() that does the quick and dirty thing you
might be looking for.

> And that should not involve any OS-specific differences, unless you
> want to use some advanced process handling features which are only
> supported by some specific OS.

I can't speak for Windows, but under Linux, it gets advanced pretty
quickly (what gets inherited, how about zombies, how about signals, etc
etc).

> Meanwhile I checked Ruby and Perl regarding this feature. Both support
> it. In both langs every standard function that calls external commands
> (like Perl's exec() or system() or Ruby's exec() or system() or
> IO.popen()), whether they invoke a shell or call it directly, support
> running 'tool.bat' or 'tool.cmd' by just writing 'tool'. Python almost
> seems to be the only major scripting language which does not support
> this implicitly.

I'm not against subprocess.Popen() doing its work under Windows the way
Windows system programmers would expect. I'm against trying to force
Windows and Linux into the same mold where there are genuine
differences.

If I were a Windows developer, I'd expect Python to support something
analogous to what I'd have in C++ or C#. If pipes are natural IPC
channels under Windows, then subprocess.Popen() is probably pretty close
to its mark. However, most of the IPC facilities listed here:

   <URL: https://msdn.microsoft.com/en-us/library/windows/desktop/aa36
   5574%28v=vs.85%29.aspx>

seem to be absent in Python (clipboard, COM, data copy, DDE, file
mapping, mailslots, rpc).


Marko


From mark.r.bannister at googlemail.com  Thu May  7 18:20:27 2015
From: mark.r.bannister at googlemail.com (mark.r.bannister at googlemail.com)
Date: Thu, 7 May 2015 15:20:27 -0700 (PDT)
Subject: Multi-threaded TCP server class for general use
Message-ID: <6e3e810e-b50f-48ef-a8eb-ff85bfa29420@googlegroups.com>

Hi,

I needed to develop a highly scalable multi-threaded TCP server in Python and when I started writing it in 2013 I could not find a suitable library that would scale the way I needed but also easy to use.

So I invented one - it's called Pyloom.  If you want to take a look, it's part of my DBIS project at the moment: https://sourceforge.net/p/dbis/code/ci/default/tree/src/pyloom

Question: does anyone see the value of this library?  Would you like me to fork it as a separate project?  Is SF.net good enough or do people prefer Python libraries to be hosted somewhere else?

In a nutshell, Pyloom is a multi-threaded TCP server class which you overload in your own program.  The library provides:

* Connection management.

* 1 or more marshal threads that listen and pick up new connections, passing to a pool of dedicated worker threads, each of which can manage multiple sessions.

* Methods that you override for handling various states: new connection, data received, send data, close connection.

* Can track custom sockets and wake up a session when there is I/O on the socket.

* Has a notification service so that one session can wait for data to be processed/collected by a different session, and can be woken up when the data is ready.

Let me know if anyone is interested in re-using this library.  Tested ok on Linux and Solaris, not tried on Windows yet.

Best regards,
Mark.


From zimmermann.code at gmail.com  Thu May  7 19:27:24 2015
From: zimmermann.code at gmail.com (Stefan Zimmermann)
Date: Thu, 7 May 2015 16:27:24 -0700 (PDT)
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <87pp6c45o8.fsf@elektro.pacujo.net>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
 <mailman.207.1431003199.12865.python-list@python.org>
 <876184jyd4.fsf@elektro.pacujo.net>
 <mailman.208.1431007449.12865.python-list@python.org>
 <8761845s2c.fsf@elektro.pacujo.net>
 <41a6326f-69c2-4a44-990b-7b714cc96b9e@googlegroups.com>
 <87pp6c45o8.fsf@elektro.pacujo.net>
Message-ID: <c14eedb6-872c-4dcb-bcc0-1434cc196e85@googlegroups.com>

Marko Rauhamaa wrote:
> Anyway, Python has os.system() that does the quick and dirty thing you
> might be looking for.

Always invokes shell ==> overhead for .exe files


From rosuav at gmail.com  Thu May  7 21:46:22 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 11:46:22 +1000
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <CALwzidnG72x8a185_3BfaWeeuzFTb5P6WiPqNYPyz7ozePmSiQ@mail.gmail.com>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
 <mailman.207.1431003199.12865.python-list@python.org>
 <876184jyd4.fsf@elektro.pacujo.net>
 <CAPTjJmrTND2Ou+Tib3nvdkODvuB6dMrL4FTMECKq5QHLQhx1fA@mail.gmail.com>
 <CALwzidnG72x8a185_3BfaWeeuzFTb5P6WiPqNYPyz7ozePmSiQ@mail.gmail.com>
Message-ID: <CAPTjJmpcfCotGjf_KxMGdsucmKRezz-V6kLQwzM-qN=bvjPmsA@mail.gmail.com>

On Fri, May 8, 2015 at 1:14 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Thu, May 7, 2015 at 8:03 AM, Chris Angelico <rosuav at gmail.com> wrote:
>> On Thu, May 7, 2015 at 11:44 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
>>> Whole programming cultures, idioms and "right ways" differ between
>>> platforms. What's the right way to write a service (daemon)? That's
>>> probably completely different between Windows and Linux. Linux itself is
>>> undergoing a biggish transformation there: an exemplary daemon of last
>>> year will likely be deprecated within a few years.
>>
>> And that's where a library function can be really awesome. What's the
>> right way to daemonize? "import daemonize; daemonize.daemonize()"
>> seems good to me. Maybe there's platform-specific code in the
>> *implementation* of that, but in your application, no. That's the job
>> of a layer underneath you.
>
> https://www.python.org/dev/peps/pep-3143/

Precisely. It's definitely within the language's purview; that that
PEP is deferred is not due to it being a bad idea for the
language/stdlib to deal with these differences.

ChrisA


From rosuav at gmail.com  Thu May  7 21:50:00 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 11:50:00 +1000
Subject: PEP idea: On Windows, subprocess should implicitly support .bat
 and .cmd scripts by using FindExecutable from win32 API
In-Reply-To: <8761845s2c.fsf@elektro.pacujo.net>
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
 <mailman.207.1431003199.12865.python-list@python.org>
 <876184jyd4.fsf@elektro.pacujo.net>
 <mailman.208.1431007449.12865.python-list@python.org>
 <8761845s2c.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmpuMMPSqUhkntwUHuf5zBCbduSEC7LbOiyk7Y2owEKDng@mail.gmail.com>

On Fri, May 8, 2015 at 1:24 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
>> That's Python's job. Abstracting away all those differences so you
>> don't have to look at them.
>
> That's the difference between our opinions: you want Python to work the
> same on different OS's. I want Python's system programming facilities to
> closely mirror those of C.

In that case, what you should do is devise an alternative language
that compiles as a thin layer over C. Have a simple translation script
that turns your code into actual C, then passes it along for
compilation. You could add whatever Python-like features you want
(memory management, maybe), but still be executing as C code. But you
don't want a high level language, if your greatest goal is "closely
mirror C".

> Take your example of subprocess.Popen. It may be essential to know that
> the communication channel is a pipe with standard pipe semantics. The
> child program might not be written in Python, after all. In fact, at
> system design level you shouldn't care what language you use as long as
> the communication interfaces are specified.

Ah, but that's a completely different thing. If you're working with a
child that isn't written in Python, then you're not working at the
level of the multiprocessing library - you're working with "I need to
give this something on its stdin and get the result back from its
stdout". For that, yes, you need something a bit more concrete; but
now it's become part of the API for that child process, whereas the
example I was giving was about the multiprocessing library, where it's
part of the implementation.

ChrisA


From rosuav at gmail.com  Thu May  7 21:53:05 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 11:53:05 +1000
Subject: Multi-threaded TCP server class for general use
In-Reply-To: <6e3e810e-b50f-48ef-a8eb-ff85bfa29420@googlegroups.com>
References: <6e3e810e-b50f-48ef-a8eb-ff85bfa29420@googlegroups.com>
Message-ID: <CAPTjJmp03SjBDeqt_-yyNm3KNdH+DATbakByYEL-bj64Qzz8eA@mail.gmail.com>

On Fri, May 8, 2015 at 8:20 AM,  <mark.r.bannister at googlemail.com> wrote:
> I needed to develop a highly scalable multi-threaded TCP server in Python and when I started writing it in 2013 I could not find a suitable library that would scale the way I needed but also easy to use.
>
> So I invented one - it's called Pyloom.  If you want to take a look, it's part of my DBIS project at the moment: https://sourceforge.net/p/dbis/code/ci/default/tree/src/pyloom
>
> Question: does anyone see the value of this library?

I haven't looked at your actual code, but one thing I would suggest
considering is the new asyncio facilities that are coming in Python
3.5 - or the existing asyncio that came in provisionally with 3.4. For
ultimate scalability, you may find yourself wanting less threads and
more asynchronicity, and this is something that's looking pretty cool
and awesome.

(It's also looking like over a thousand, maybe over two thousand,
posts on python-ideas. I have not, I regret to say, been reading every
single post.)

ChrisA


From ben+python at benfinney.id.au  Thu May  7 22:26:05 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 08 May 2015 12:26:05 +1000
Subject: PEP idea: On Windows,
 subprocess should implicitly support .bat and .cmd scripts by using
 FindExecutable from win32 API
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
 <mailman.207.1431003199.12865.python-list@python.org>
 <876184jyd4.fsf@elektro.pacujo.net>
 <mailman.208.1431007449.12865.python-list@python.org>
 <8761845s2c.fsf@elektro.pacujo.net>
 <CAPTjJmpuMMPSqUhkntwUHuf5zBCbduSEC7LbOiyk7Y2owEKDng@mail.gmail.com>
Message-ID: <85fv77dcua.fsf@benfinney.id.au>

Chris Angelico <rosuav at gmail.com> writes:

> On Fri, May 8, 2015 at 1:24 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> >> That's Python's job. Abstracting away all those differences so you
> >> don't have to look at them.
> >
> > That's the difference between our opinions: you want Python to work
> > the same on different OS's. I want Python's system programming
> > facilities to closely mirror those of C.
>
> In that case, what you should do is devise an alternative language
> that compiles as a thin layer over C. [?] But you don't want a high
> level language, if your greatest goal is "closely mirror C".

+1.

Marko, you have many times criticised Python on the basis, essentially,
that it is not some other platform. It's quite unproductive, and leads
only to discussions that are at best frustrating for all involved.

If you want a platform that is fundamentally different from Python,
there are plenty available for you. Arguing that Python should be
fundamentally different will not avail us anything good.

-- 
 \      ?Anyone who believes exponential growth can go on forever in a |
  `\        finite world is either a madman or an economist.? ?Kenneth |
_o__)                                                         Boulding |
Ben Finney



From rustompmody at gmail.com  Fri May  8 00:06:00 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Thu, 7 May 2015 21:06:00 -0700 (PDT)
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <5549ab43$0$11108$c3e8da3@news.astraweb.com>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
 <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>
 <5549ab43$0$11108$c3e8da3@news.astraweb.com>
Message-ID: <b0022fbc-74e2-4990-9773-9e5e07d51f14@googlegroups.com>

On Wednesday, May 6, 2015 at 11:19:07 AM UTC+5:30, Steven D'Aprano wrote:
> On Wednesday 06 May 2015 14:47, Rustom Mody wrote:
> 
> > It strikes me that the FP crowd has stretched the notion of function
> > beyond recognition And the imperative/OO folks have distorted it beyond
> > redemption.
> 
> In what way?
> 
> 
> > And the middle road shown by Pascal has been overgrown with weeds for some
> > 40 years...
> 
> As much as I like Pascal, and am pleased to see someone defending it, I'm 
> afraid I have no idea what you mean by this.

There are many hats...

As a programmer what you say is fine.
As a language-designer maybe even required -- one would expect a language designer to invoke Occm's razor and throw away needless distinctions (read syntax categories)

But there are other hats.
Even if you disregard the teacher-hat as irrelevant, the learner-hat is universal
-- whatever you are master of now is because at some past point you walked its
learning curve.

> 
> 
> > If the classic Pascal (or Fortran or Basic) sibling balanced abstractions
> > of function-for-value procedure-for-effect were more in the collective
> > consciousness rather than C's travesty of function, things might not have
> > been so messy.
> 
> I'm not really sure that having distinct procedures, as opposed to functions 
> that you just ignore their return result, makes *such* a big difference. Can 
> you explain what is the difference between these?
> 
> sort(stuff)  # A procedure.
> sort(stuff)  # ignore the function return result
> 
> And why the first is so much better than the second?

Here are 3 None-returning functions/methods in python.
ie semantically the returns are identical. Are they conceptually identical?

>>> x=print(1)
1
>>> x
>>> ["hello",None].__getitem__(1)
>>> {"a":1, "b":2}.get("c")
>>> 



> 
> 
> 
> > Well... Dont feel right bashing C without some history...
> > 
> > C didn't start the mess of mixing procedure and function -- Lisp/Apl did.
> > Nor the confusion of = for assignment; Fortran did that.
> 
> Pardon, but = has been used for "assignment" for centuries, long before 
> George Boole and Ada Lovelace even started working on computer theory. Just 
> ask mathematicians:
> 
> "let y = 23"
> 
> Is that not a form of assignment?

Truth?? Lets see...

Here is a set of assignments (as you call) that could occur in a school text
teaching business-math, viz how to calculate 'simple-interest'

amt = prin + si
si = prin * n * rate/100.0 
# for instance
prin = 1000.0 
n = 4.0
rate = 12.0

Put it into python and you get Name errors.

Put it into Haskell or some such; (after changing the comment char from '#' to '--')
and you'll get amt and si

Now you can view this operationally (which in some cases even the Haskell docs do)
and say a bunch of ='s in python is a sequence whereas in haskell its
'simultaneous' ie a set.

But this would miss the point viz that in Python
amt = prin + si 
denotes an *action*
whereas in Haskell it denotes a *fact* and a bunch of facts that modified by being
permuted would be a strange bunch!

Note I am not saying Haskell is right; particularly in its semantics of '='
there are serious issues:
http://blog.languager.org/2012/08/functional-programming-philosophical.html

Just that
x = y
in a functional/declarative/math framework means something quite different 
than in an imperative one


From rustompmody at gmail.com  Fri May  8 00:20:43 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Thu, 7 May 2015 21:20:43 -0700 (PDT)
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <mailman.171.1430917883.12865.python-list@python.org>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
 <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>
 <mailman.171.1430917883.12865.python-list@python.org>
Message-ID: <a6332bc0-ec50-497a-a6bb-0ffe48610bf8@googlegroups.com>

On Wednesday, May 6, 2015 at 6:41:38 PM UTC+5:30, Dennis Lee Bieber wrote:
> On Tue, 5 May 2015 21:47:17 -0700 (PDT), Rustom Mody declaimed the following:
> 
> >If the classic Pascal (or Fortran or Basic) sibling balanced abstractions of function-for-value
> >procedure-for-effect were more in the collective consciousness rather than C's
> >travesty of function, things might not have been so messy.
> >
> 	I suspect just the term "subprogram" (as in "function subprogram" and
> "subroutine subprogram") would confuse a lot of cubs these days...
> 
> >C didn't start the mess of mixing procedure and function -- Lisp/Apl did.
> >Nor the confusion of = for assignment; Fortran did that.
> 
> 	I don't think you can blame FORTRAN for that, given that it was one of
> the first of the higher level languages, and had no confusion internally...

BLAME?? Ha You are being funny Dennis!

There are fat regulation-books that pilots need to follow. Breaking them can
make one culpable all the way to homicide.
The Wright-brothers probably broke them all. Should we call them homicidal maniacs?

Shakespeare sometimes has funny spellings. I guess he's illiterate for not turning on the spell-checker in Word?

Or [my favorite] Abraham Lincoln used the word 'negro'. So he's a racist?

Speaking more conceptually, there are pioneers and us ordinary folks.?
The world as we know it is largely a creation of these pioneers.
And if you take them and stuff them into the statistically ordinary mold then fit badly.

That puts people especially teachers into a bind.
If I expect my students to be 1/100 as pioneering as Backus, Thomson, Ritchie 
etc, I would be foolish
And if I dont spell out all their mistakes in minute detail and pull them up
for repeating them, I'd not be doing due diligence

--------------------
I guess this is nowadays called the 'romantic' view.
Ask the family-members of any of these greats for the 'other' view  :-)


From rosuav at gmail.com  Fri May  8 00:33:06 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 14:33:06 +1000
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <b0022fbc-74e2-4990-9773-9e5e07d51f14@googlegroups.com>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
 <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>
 <5549ab43$0$11108$c3e8da3@news.astraweb.com>
 <b0022fbc-74e2-4990-9773-9e5e07d51f14@googlegroups.com>
Message-ID: <CAPTjJmpvt_94L4rH0swYZC5f7=Qtem=+A9tO5Gv_iNmX1UkieA@mail.gmail.com>

On Fri, May 8, 2015 at 2:06 PM, Rustom Mody <rustompmody at gmail.com> wrote:
>> > If the classic Pascal (or Fortran or Basic) sibling balanced abstractions
>> > of function-for-value procedure-for-effect were more in the collective
>> > consciousness rather than C's travesty of function, things might not have
>> > been so messy.
>>
>> I'm not really sure that having distinct procedures, as opposed to functions
>> that you just ignore their return result, makes *such* a big difference. Can
>> you explain what is the difference between these?
>>
>> sort(stuff)  # A procedure.
>> sort(stuff)  # ignore the function return result
>>
>> And why the first is so much better than the second?
>
> Here are 3 None-returning functions/methods in python.
> ie semantically the returns are identical. Are they conceptually identical?
>
>>>> x=print(1)
> 1
>>>> x
>>>> ["hello",None].__getitem__(1)
>>>> {"a":1, "b":2}.get("c")
>>>>

Your second example is a poor one, as it involves calling a dunder
method. But all you've proven with that is that the empty return value
of Python is a real value, and can thus be stored and retrieved.

With print(), you have a conceptual procedure - it invariably returns
None, so it'll normally be called in contexts that don't care about
the return value. What about sys.stdout.write(), though? That's most
often going to be called procedurally - you just write your piece and
move on - but it has a meaningful return value. In normal usage, both
are procedures. The ability to upgrade a function from "always returns
None" to "returns some occasionally-useful information" is one of the
strengths of a unified function/procedure system.

With .get(), it's actually nothing to do with procedures vs functions,
but more to do with Python's use of None where C would most likely use
NULL. By default, .get() uses None as its default return value, so
something that isn't there will be treated as None. In the same way as
your second example, that's just a result of None being a real value
that you can pass around - nothing more special than that.

The other thing you're seeing here is that the *interactive
interpreter* treats None specially. In a program, an expression
statement simply discards its result, whether it's None or 42 or
[1,2,3] or anything else. You could write an interactive interpreter
that has some magic that recognizes that certain functions always
return None (maybe by checking their annotations), and omits printing
their return values, while still printing the return values of other
functions. I'm not sure what it'd gain you, but it wouldn't change the
concepts or semantics surrounding None returns.

ChrisA


From rustompmody at gmail.com  Fri May  8 00:53:56 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Thu, 7 May 2015 21:53:56 -0700 (PDT)
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <mailman.216.1431059590.12865.python-list@python.org>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
 <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>
 <5549ab43$0$11108$c3e8da3@news.astraweb.com>
 <b0022fbc-74e2-4990-9773-9e5e07d51f14@googlegroups.com>
 <mailman.216.1431059590.12865.python-list@python.org>
Message-ID: <aca29698-6b7f-4ac7-ba1f-0e60fd28d639@googlegroups.com>

On Friday, May 8, 2015 at 10:04:02 AM UTC+5:30, Chris Angelico wrote:
> On Fri, May 8, 2015 at 2:06 PM, Rustom Mody wrote:
> >> > If the classic Pascal (or Fortran or Basic) sibling balanced abstractions
> >> > of function-for-value procedure-for-effect were more in the collective
> >> > consciousness rather than C's travesty of function, things might not have
> >> > been so messy.
> >>
> >> I'm not really sure that having distinct procedures, as opposed to functions
> >> that you just ignore their return result, makes *such* a big difference. Can
> >> you explain what is the difference between these?
> >>
> >> sort(stuff)  # A procedure.
> >> sort(stuff)  # ignore the function return result
> >>
> >> And why the first is so much better than the second?
> >
> > Here are 3 None-returning functions/methods in python.
> > ie semantically the returns are identical. Are they conceptually identical?
> >
> >>>> x=print(1)
> > 1
> >>>> x
> >>>> ["hello",None].__getitem__(1)
> >>>> {"a":1, "b":2}.get("c")
> >>>>
> 
> Your second example is a poor one, as it involves calling a dunder
> method. But all you've proven with that is that the empty return value
> of Python is a real value, and can thus be stored and retrieved.
> 
> With print(), you have a conceptual procedure - it invariably returns
> None, so it'll normally be called in contexts that don't care about
> the return value. What about sys.stdout.write(), though? That's most
> often going to be called procedurally - you just write your piece and
> move on - but it has a meaningful return value. In normal usage, both
> are procedures. The ability to upgrade a function from "always returns
> None" to "returns some occasionally-useful information" is one of the
> strengths of a unified function/procedure system.
> 
> With .get(), it's actually nothing to do with procedures vs functions,

That's backwards.

get is very much a function and the None return is semantically significant.
print is just round peg -- what you call conceptual function -- stuffed into
square hole -- function the only available syntax-category

> but more to do with Python's use of None where C would most likely use
> NULL. By default, .get() uses None as its default return value, so
> something that isn't there will be treated as None. In the same way as
> your second example, that's just a result of None being a real value
> that you can pass around - nothing more special than that.
> 
> The other thing you're seeing here is that the *interactive
> interpreter* treats None specially. 


Yeah I know
And if python did not try to be so clever, I'd save some time with
student-surprises

> In a program, an expression
> statement simply discards its result, whether it's None or 42 or
> [1,2,3] or anything else. You could write an interactive interpreter
> that has some magic that recognizes that certain functions always
> return None (maybe by checking their annotations), and omits printing
> their return values, while still printing the return values of other
> functions.


Hoo Boy!  You seem to be in the 'the-more-the-better' (of magic) camp


> I'm not sure what it'd gain you, but it wouldn't change the
> concepts or semantics surrounding None returns.

It would sure help teachers who get paid by the hour and would rather spend
time on technical irrelevantia than grapple with significant concepts


From rustompmody at gmail.com  Fri May  8 00:55:58 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Thu, 7 May 2015 21:55:58 -0700 (PDT)
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <aca29698-6b7f-4ac7-ba1f-0e60fd28d639@googlegroups.com>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
 <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>
 <5549ab43$0$11108$c3e8da3@news.astraweb.com>
 <b0022fbc-74e2-4990-9773-9e5e07d51f14@googlegroups.com>
 <mailman.216.1431059590.12865.python-list@python.org>
 <aca29698-6b7f-4ac7-ba1f-0e60fd28d639@googlegroups.com>
Message-ID: <8e5a3021-ecbc-4eca-aa5b-68d85efd060d@googlegroups.com>

On Friday, May 8, 2015 at 10:24:06 AM UTC+5:30, Rustom Mody wrote:

> get is very much a function and the None return is semantically significant.
> print is just round peg -- what you call conceptual function -- stuffed into
> square hole -- function the only available syntax-category

Sorry "Conceptual procedure" of course


From rosuav at gmail.com  Fri May  8 01:09:21 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 15:09:21 +1000
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <aca29698-6b7f-4ac7-ba1f-0e60fd28d639@googlegroups.com>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
 <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>
 <5549ab43$0$11108$c3e8da3@news.astraweb.com>
 <b0022fbc-74e2-4990-9773-9e5e07d51f14@googlegroups.com>
 <mailman.216.1431059590.12865.python-list@python.org>
 <aca29698-6b7f-4ac7-ba1f-0e60fd28d639@googlegroups.com>
Message-ID: <CAPTjJmqcK1qpqC8UqF+yYjOAVwkewsLjM=wSndKnw9TrVD6qMA@mail.gmail.com>

On Fri, May 8, 2015 at 2:53 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> Yeah I know
> And if python did not try to be so clever, I'd save some time with
> student-surprises
>
>> In a program, an expression
>> statement simply discards its result, whether it's None or 42 or
>> [1,2,3] or anything else. You could write an interactive interpreter
>> that has some magic that recognizes that certain functions always
>> return None (maybe by checking their annotations), and omits printing
>> their return values, while still printing the return values of other
>> functions.
>
>
> Hoo Boy!  You seem to be in the 'the-more-the-better' (of magic) camp

No way! I wouldn't want the interactive interpreter to go too magical.
I'm just saying that it wouldn't break Python to have it do things
differently.

>> I'm not sure what it'd gain you, but it wouldn't change the
>> concepts or semantics surrounding None returns.
>
> It would sure help teachers who get paid by the hour and would rather spend
> time on technical irrelevantia than grapple with significant concepts

Why have the concept of a procedure? Python's rule is simple: Every
function either returns a value or raises an exception. (Even
generators. When you call a generator function, you get back a return
value which is the generator state object.) The procedure/function
distinction is irrelevant.

ChrisA


From ernest.moloko at gmail.com  Fri May  8 02:06:45 2015
From: ernest.moloko at gmail.com (Lesego Moloko)
Date: Fri, 8 May 2015 08:06:45 +0200
Subject: Matlab to Python problem
Message-ID: <CAPphnGMJjAeKOMdRKOwxmK0wyv-GAUwJzqeiVZ9-n+v7JE4xMA@mail.gmail.com>

Dear All

I am a Python novice and have recently converted the attached Matlab file
to Python (attached). The problem is that I cannot reproduce the Matlab
plots (attached) with the converted Python program. I am also trying to go
line by line through my converted Python file to see where the problem
could be.

I attached:

   1. Matlab plots (5_p3.png and 4_p3.png)
   2. The Matlab file (test.m)
   3. Converted Python file (test.py)
   4. data files (a2_all_out_P3.dat and t4_all_out.dat)

You advise and assistance would be highly appreciated.
Regards
Lesego MOLOKO
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/f4754d0b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 5_p3.png
Type: image/png
Size: 2811 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/f4754d0b/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 4_p3.png
Type: image/png
Size: 14041 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/f4754d0b/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: a2_all_out_P3.dat
Type: application/x-ns-proxy-autoconfig
Size: 28281 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/f4754d0b/attachment.dat>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: t4_all_out.dat
Type: application/x-ns-proxy-autoconfig
Size: 91104 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/f4754d0b/attachment-0001.dat>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.m
Type: text/x-objcsrc
Size: 23921 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/f4754d0b/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.py
Type: text/x-python
Size: 36846 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/f4754d0b/attachment.py>

From marko at pacujo.net  Fri May  8 02:14:44 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 08 May 2015 09:14:44 +0300
Subject: PEP idea: On Windows,
 subprocess should implicitly support .bat and .cmd scripts by using
 FindExecutable from win32 API
References: <a0a85a0a-555f-4fff-bc1f-49873dd297c5@googlegroups.com>
 <554AB8A5.708@davea.name>
 <mailman.198.1430965194.12865.python-list@python.org>
 <554adcf8$0$11103$c3e8da3@news.astraweb.com>
 <CAPTjJmq_hpk4tNYWcip88MYo8MtyV=+Y=qR-CALFUfGTEupB9A@mail.gmail.com>
 <mailman.200.1430983119.12865.python-list@python.org>
 <1c51085e-7795-4afc-9a4c-ad8b3f3a73a6@googlegroups.com>
 <cff48c78-07f5-43cc-82b0-5b92a866f7a8@googlegroups.com>
 <87ioc4k89v.fsf@elektro.pacujo.net>
 <CAPTjJmoeVX8zsmCfps8FxrkVFTJv61e2Pxi9evKVm1JdE3ZPDw@mail.gmail.com>
 <554B4C49.7010500@davea.name>
 <mailman.206.1430999023.12865.python-list@python.org>
 <87a8xgk1ad.fsf@elektro.pacujo.net>
 <mailman.207.1431003199.12865.python-list@python.org>
 <876184jyd4.fsf@elektro.pacujo.net>
 <mailman.208.1431007449.12865.python-list@python.org>
 <8761845s2c.fsf@elektro.pacujo.net>
 <CAPTjJmpuMMPSqUhkntwUHuf5zBCbduSEC7LbOiyk7Y2owEKDng@mail.gmail.com>
 <mailman.214.1431051977.12865.python-list@python.org>
Message-ID: <87bnhv4muj.fsf@elektro.pacujo.net>

Ben Finney <ben+python at benfinney.id.au>:

> Chris Angelico <rosuav at gmail.com> writes:
>
>> On Fri, May 8, 2015 at 1:24 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
>> >> That's Python's job. Abstracting away all those differences so you
>> >> don't have to look at them.
>> >
>> > That's the difference between our opinions: you want Python to work
>> > the same on different OS's. I want Python's system programming
>> > facilities to closely mirror those of C.
>>
>> In that case, what you should do is devise an alternative language
>> that compiles as a thin layer over C. [?] But you don't want a high
>> level language, if your greatest goal is "closely mirror C".
>
> +1.
>
> Marko, you have many times criticised Python on the basis,
> essentially, that it is not some other platform. It's quite
> unproductive, and leads only to discussions that are at best
> frustrating for all involved.
>
> If you want a platform that is fundamentally different from Python,
> there are plenty available for you. Arguing that Python should be
> fundamentally different will not avail us anything good.

I don't. Python *does* provide OS-dependent facilities. Somebody
complained about that. I said Python was the way it should be, even
though there are signs Python "wants" to become more Java-esque.

For example, Python provides the errno module. Its use and meanings can
be looked up with man pages.

So Python has the great advantage that it *can* be used as a Linux
system programming language. And I am.


Marko


From rustompmody at gmail.com  Fri May  8 02:36:21 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Thu, 7 May 2015 23:36:21 -0700 (PDT)
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <mailman.217.1431061769.12865.python-list@python.org>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
 <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>
 <5549ab43$0$11108$c3e8da3@news.astraweb.com>
 <b0022fbc-74e2-4990-9773-9e5e07d51f14@googlegroups.com>
 <mailman.216.1431059590.12865.python-list@python.org>
 <aca29698-6b7f-4ac7-ba1f-0e60fd28d639@googlegroups.com>
 <mailman.217.1431061769.12865.python-list@python.org>
Message-ID: <e60dc222-a837-4a65-9ce2-eff80ee2527f@googlegroups.com>

On Friday, May 8, 2015 at 10:39:38 AM UTC+5:30, Chris Angelico wrote:
> Why have the concept of a procedure? 

On Friday, Chris Angelico ALSO wrote:
> With print(), you have a conceptual procedure...

So which do you want to stand by?


Just to be clear I am not saying python should be any different on this front.

G?del's (2nd) theorem guarantees that no formalism (aka programming language in our case)
can ever be complete and so informal borrowing is inevitable.
Its just that Pascal, Fortran, Basic, by ingesting this informal requirement into
the formal language make THIS aspect easier to learn/teach...
... at the cost of eleventeen others

[Regular expressions in Fortran anyone?]


From rosuav at gmail.com  Fri May  8 02:42:24 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 16:42:24 +1000
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <e60dc222-a837-4a65-9ce2-eff80ee2527f@googlegroups.com>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
 <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>
 <5549ab43$0$11108$c3e8da3@news.astraweb.com>
 <b0022fbc-74e2-4990-9773-9e5e07d51f14@googlegroups.com>
 <mailman.216.1431059590.12865.python-list@python.org>
 <aca29698-6b7f-4ac7-ba1f-0e60fd28d639@googlegroups.com>
 <mailman.217.1431061769.12865.python-list@python.org>
 <e60dc222-a837-4a65-9ce2-eff80ee2527f@googlegroups.com>
Message-ID: <CAPTjJmrGFHm3i-OEHdYUgMM4zrN3BdYS7-dVgJoA5k3GO_-=DQ@mail.gmail.com>

On Fri, May 8, 2015 at 4:36 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> On Friday, May 8, 2015 at 10:39:38 AM UTC+5:30, Chris Angelico wrote:
>> Why have the concept of a procedure?
>
> On Friday, Chris Angelico ALSO wrote:
>> With print(), you have a conceptual procedure...
>
> So which do you want to stand by?

A procedure, in Python, is simply a function which returns None.
That's all. It's not any sort of special concept. It doesn't need to
be taught. If your students are getting confused by it, stop teaching
it!

ChrisA


From frank at chagford.com  Fri May  8 04:01:52 2015
From: frank at chagford.com (Frank Millman)
Date: Fri, 8 May 2015 10:01:52 +0200
Subject: Is this unpythonic?
Message-ID: <mihqhb$r1k$1@ger.gmane.org>

Hi all

I have often read about the gotcha regarding 'mutable default arguments' 
that frequently trips people up.

I use them from time to time, but I have never had a problem. I have just 
delved a bit deeper to see if I am skating on thin ice.

AFAICT my usage is safe. If I use a list as an argument, I only use it to 
pass values *into* the function, but I never modify the list. So if I omit 
the argument, the original default empty list is used, otherwise the list 
that I pass in is used.

However, every time I look at my own code, and I see   "def x(y, z=[]): 
....."   it looks wrong because I have been conditioned to think of it as a 
gotcha.

Would it be more pythonic to change them all to use the alternative 
"z=None", or is it ok to leave it as it is? Or to phrase it differently, how 
would an experienced pythonista react on seeing this when reviewing my code?

Thanks

Frank Millman





From jarausch at skynet.be  Fri May  8 04:17:02 2015
From: jarausch at skynet.be (jarausch at skynet.be)
Date: Fri, 8 May 2015 01:17:02 -0700 (PDT)
Subject: www.python.org - Backend is unhealthy
Message-ID: <6fccd558-c78d-4225-919d-f129c8aafae4@googlegroups.com>

I cannot access www.python.org.
I always get 

Error 503 Backend is unhealthy

Backend is unhealthy

Guru Mediation:

Details: cache-ams4149-AMS 1431072956 2041303800

Varnish cache server


Is it only me?

Thanks for a hint,
Helmut


From rosuav at gmail.com  Fri May  8 04:25:14 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 18:25:14 +1000
Subject: Is this unpythonic?
In-Reply-To: <mihqhb$r1k$1@ger.gmane.org>
References: <mihqhb$r1k$1@ger.gmane.org>
Message-ID: <CAPTjJmpmvY07003E_opgzkbzh_rZ5GBuusqrm_HS4MkRCr+ryA@mail.gmail.com>

On Fri, May 8, 2015 at 6:01 PM, Frank Millman <frank at chagford.com> wrote:
> I have often read about the gotcha regarding 'mutable default arguments'
> that frequently trips people up.
>
> I use them from time to time, but I have never had a problem. I have just
> delved a bit deeper to see if I am skating on thin ice.
>
> AFAICT my usage is safe. If I use a list as an argument, I only use it to
> pass values *into* the function, but I never modify the list. So if I omit
> the argument, the original default empty list is used, otherwise the list
> that I pass in is used.

In that case, you may want to consider using a default of () rather
than [], unless there's some reason that it has to be a list. With a
tuple, you have a guarantee that it's not going to be changed, so it's
as safe as a default argument of True or 42 or "Hello".

That may or may not help, but it's worth considering, especially if
all you really need is an iterable (or a sequence).

ChrisA


From rosuav at gmail.com  Fri May  8 04:28:33 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 18:28:33 +1000
Subject: www.python.org - Backend is unhealthy
In-Reply-To: <6fccd558-c78d-4225-919d-f129c8aafae4@googlegroups.com>
References: <6fccd558-c78d-4225-919d-f129c8aafae4@googlegroups.com>
Message-ID: <CAPTjJmpKmtTiBKoFLhc2sMSyE7sZH_dAziOPOM2XNrf1QqFm+w@mail.gmail.com>

On Fri, May 8, 2015 at 6:17 PM,  <jarausch at skynet.be> wrote:
> I cannot access www.python.org.
> I always get
>
> Error 503 Backend is unhealthy
>
> Backend is unhealthy
>
> Guru Mediation:
>
> Details: cache-ams4149-AMS 1431072956 2041303800
>
> Varnish cache server
>
>
> Is it only me?
>

No, it's not only you. I get the same but with different details:

Details: cache-syd1627-SYD 1431073575 864283876

It looks to me as if my result is coming from a cache node in Sydney;
yours is coming from some other cache node, so it's not just one node
that's down.

Cc'ing in the www list in case someone there knows, and I'll create a
github issue to ping the people there.

ChrisA


From noah at coderanger.net  Fri May  8 04:32:03 2015
From: noah at coderanger.net (Noah Kantrowitz)
Date: Fri, 8 May 2015 10:32:03 +0200
Subject: [pydotorg-www] www.python.org - Backend is unhealthy
In-Reply-To: <CAPTjJmpKmtTiBKoFLhc2sMSyE7sZH_dAziOPOM2XNrf1QqFm+w@mail.gmail.com>
References: <6fccd558-c78d-4225-919d-f129c8aafae4@googlegroups.com>
 <CAPTjJmpKmtTiBKoFLhc2sMSyE7sZH_dAziOPOM2XNrf1QqFm+w@mail.gmail.com>
Message-ID: <C0ADC633-FBE7-4B24-A5AD-F26ED35F36D9@coderanger.net>


On May 8, 2015, at 10:28 AM, Chris Angelico <rosuav at gmail.com> wrote:

> On Fri, May 8, 2015 at 6:17 PM,  <jarausch at skynet.be> wrote:
>> I cannot access www.python.org.
>> I always get
>> 
>> Error 503 Backend is unhealthy
>> 
>> Backend is unhealthy
>> 
>> Guru Mediation:
>> 
>> Details: cache-ams4149-AMS 1431072956 2041303800
>> 
>> Varnish cache server
>> 
>> 
>> Is it only me?
>> 
> 
> No, it's not only you. I get the same but with different details:
> 
> Details: cache-syd1627-SYD 1431073575 864283876
> 
> It looks to me as if my result is coming from a cache node in Sydney;
> yours is coming from some other cache node, so it's not just one node
> that's down.
> 
> Cc'ing in the www list in case someone there knows, and I'll create a
> github issue to ping the people there.

Should be recovering now.

--Noah

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 163 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/62c2f956/attachment.sig>

From rosuav at gmail.com  Fri May  8 04:34:31 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 18:34:31 +1000
Subject: [pydotorg-www] www.python.org - Backend is unhealthy
In-Reply-To: <C0ADC633-FBE7-4B24-A5AD-F26ED35F36D9@coderanger.net>
References: <6fccd558-c78d-4225-919d-f129c8aafae4@googlegroups.com>
 <CAPTjJmpKmtTiBKoFLhc2sMSyE7sZH_dAziOPOM2XNrf1QqFm+w@mail.gmail.com>
 <C0ADC633-FBE7-4B24-A5AD-F26ED35F36D9@coderanger.net>
Message-ID: <CAPTjJmoBos1kcx3e9mHOmYSse2JSkOiKhQc+3TvaWzNqM7MiEQ@mail.gmail.com>

On Fri, May 8, 2015 at 6:32 PM, Noah Kantrowitz <noah at coderanger.net> wrote:
>> Cc'ing in the www list in case someone there knows, and I'll create a
>> github issue to ping the people there.
>
> Should be recovering now.

Thanks, yep, working for me now!

ChrisA


From albert at spenarnc.xs4all.nl  Fri May  8 05:14:50 2015
From: albert at spenarnc.xs4all.nl (Albert van der Horst)
Date: 08 May 2015 09:14:50 GMT
Subject: Boolean Operator Confusion
References: <fa297093-4de1-49d2-be0e-45aa2017c75a@googlegroups.com>
 <553a5ded$0$12978$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <554c7e8a$0$3069$e4fe514c@dreader35.news.xs4all.nl>

In article <553a5ded$0$12978$c3e8da3$5496439d at news.astraweb.com>,
Steven D'Aprano  <steve+comp.lang.python at pearwood.info> wrote:
>On Sat, 25 Apr 2015 12:50 am, subhabrata.banerji at gmail.com wrote:
>
>> Dear Group,
>>
>> I am trying to understand the use of Boolean operator in Python. I am
>> trying to write small piece of script, as follows,
>>
>> def input_test():
>>     str1=raw_input("PRINT QUERY:")
>>     if "AND" or "OR" or "NOT" in str1:
>>         print "It is a Boolean Query"
>>     elif "AND" or "OR" or "NOT" not in str1:
>>         print "It is not a Boolean Query"
>>     else:
>>         print "None"
>
>First problem: why do you sometimes return "None"? You have two possible
>answers: either something is a boolean query, or it is not. There is no
>third choice. ("It's a boolean query, but only on Wednesdays.")

In the context of testcode where the OP is not knowing what is going on
it is absolutely useful, and legit. His problem is that it is Wednesday and
weird things happen.


>Steven
>
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst



From jonathan.slenders at gmail.com  Fri May  8 05:14:52 2015
From: jonathan.slenders at gmail.com (jonathan.slenders at gmail.com)
Date: Fri, 8 May 2015 02:14:52 -0700 (PDT)
Subject: Why is array.array('u') deprecated?
Message-ID: <80167ebf-c5ee-4699-89ca-ef544fe3decc@googlegroups.com>

Why is array.array('u') deprecated?

Will we get an alternative for a character array or mutable unicode string?

Thanks!
Jonathan


From jarausch at skynet.be  Fri May  8 05:28:59 2015
From: jarausch at skynet.be (jarausch at skynet.be)
Date: Fri, 8 May 2015 02:28:59 -0700 (PDT)
Subject: [pydotorg-www] www.python.org - Backend is unhealthy
In-Reply-To: <mailman.227.1431073931.12865.python-list@python.org>
References: <6fccd558-c78d-4225-919d-f129c8aafae4@googlegroups.com>
 <CAPTjJmpKmtTiBKoFLhc2sMSyE7sZH_dAziOPOM2XNrf1QqFm+w@mail.gmail.com>
 <mailman.227.1431073931.12865.python-list@python.org>
Message-ID: <c4ce8897-54f9-4e2c-b494-f7681a70d0f2@googlegroups.com>

On Friday, 8 May 2015 10:32:21 UTC+2, Coderanger  wrote:
 Should be recovering now.

Thanks, it's fine here, as well.
Helmut



From Cecil at decebal.nl  Fri May  8 05:58:45 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 08 May 2015 11:58:45 +0200
Subject: To pickle or not to pickle
Message-ID: <87h9rnz8yy.fsf@Equus.decebal.nl>

I first used marshal in my filebasedMessages module. Then I read that
you should not use it, because it changes per Python version and it
was better to use pickle. So I did that and now I find:
    https://wiki.python.org/moin/Pickle

Is it really that bad and should I change again?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From steve+comp.lang.python at pearwood.info  Fri May  8 06:08:09 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 08 May 2015 20:08:09 +1000
Subject: Is this unpythonic?
References: <mailman.224.1431072121.12865.python-list@python.org>
Message-ID: <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>

On Fri, 8 May 2015 06:01 pm, Frank Millman wrote:

> Hi all
> 
> I have often read about the gotcha regarding 'mutable default arguments'
> that frequently trips people up.
> 
> I use them from time to time, but I have never had a problem. I have just
> delved a bit deeper to see if I am skating on thin ice.
> 
> AFAICT my usage is safe. If I use a list as an argument, I only use it to
> pass values *into* the function, but I never modify the list. So if I omit
> the argument, the original default empty list is used, otherwise the list
> that I pass in is used.
> 
> However, every time I look at my own code, and I see   "def x(y, z=[]):
> ....."   it looks wrong because I have been conditioned to think of it as
> a gotcha.

It is a gotcha, and a code smell.

http://www.joelonsoftware.com/articles/Wrong.html

You can use it, but with care: code smells aren't necessarily wrong, they
just need to be looked at a little more carefully.


> Would it be more pythonic to change them all to use the alternative
> "z=None", or is it ok to leave it as it is? Or to phrase it differently,
> how would an experienced pythonista react on seeing this when reviewing my
> code?

I would change it to z=None *unless* you actually required the list to be
mutated, e.g. if you were using it as a cache.

Does z have to be a list? Could you use an empty tuple instead?

def x(y, z=()): ...



-- 
Steven



From cfkaran2 at gmail.com  Fri May  8 06:27:30 2015
From: cfkaran2 at gmail.com (Cem Karan)
Date: Fri, 8 May 2015 06:27:30 -0400
Subject: To pickle or not to pickle
In-Reply-To: <87h9rnz8yy.fsf@Equus.decebal.nl>
References: <87h9rnz8yy.fsf@Equus.decebal.nl>
Message-ID: <3ADD8745-D2A4-4277-B67C-FFBBF2A519D8@gmail.com>

What are you using pickle for?  If this is just for yourself, go for it.  If you're planning on interchanging with different languages/platforms/etc., JSON or XML might be better.  If you're after something that is smaller and faster, maybe MessagePack or Google Protocol Buffers.  If you're after something that can hold a planet's worth of data, maybe HDF5.  It really depends on your use-case.

MessagePack - http://en.wikipedia.org/wiki/MessagePack
Google Protocol Buffers - http://en.wikipedia.org/wiki/Protocol_Buffers
HDF5 - http://en.wikipedia.org/wiki/Hierarchical_Data_Format

Thanks,
Cem Karan

On May 8, 2015, at 5:58 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:

> I first used marshal in my filebasedMessages module. Then I read that
> you should not use it, because it changes per Python version and it
> was better to use pickle. So I did that and now I find:
>    https://wiki.python.org/moin/Pickle
> 
> Is it really that bad and should I change again?
> 
> -- 
> Cecil Westerhof
> Senior Software Engineer
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
> -- 
> https://mail.python.org/mailman/listinfo/python-list



From steve+comp.lang.python at pearwood.info  Fri May  8 06:29:04 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 08 May 2015 20:29:04 +1000
Subject: Why is array.array('u') deprecated?
References: <80167ebf-c5ee-4699-89ca-ef544fe3decc@googlegroups.com>
Message-ID: <554c8ff2$0$12990$c3e8da3$5496439d@news.astraweb.com>

On Fri, 8 May 2015 07:14 pm, jonathan.slenders at gmail.com wrote:

> Why is array.array('u') deprecated?
> 
> Will we get an alternative for a character array or mutable unicode
> string?


Good question.

Of the three main encodings for Unicode, two are variable-width: 

* UTF-8 uses 1-4 bytes per character 
* UTF-16 uses 2 or 4 bytes per character

while UTF-32 is fixed-width (4 bytes per character). So you could try faking
it with a 32-bit array and filling it with string.encode('utf-32').



-- 
Steven



From __peter__ at web.de  Fri May  8 06:32:50 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 08 May 2015 12:32:50 +0200
Subject: To pickle or not to pickle
References: <87h9rnz8yy.fsf@Equus.decebal.nl>
Message-ID: <mii3cj$djb$1@ger.gmane.org>

Cecil Westerhof wrote:

> I first used marshal in my filebasedMessages module. Then I read that
> you should not use it, because it changes per Python version and it
> was better to use pickle. So I did that and now I find:
>     https://wiki.python.org/moin/Pickle
> 
> Is it really that bad and should I change again?

Let's say it the other way around: pickle is fine for short term storage 
when the generation of the file is under your control and you only need to 
access it from Python. 

Does that description fit your requirements?




From __peter__ at web.de  Fri May  8 06:39:04 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 08 May 2015 12:39:04 +0200
Subject: Matlab to Python problem
References: <CAPphnGMJjAeKOMdRKOwxmK0wyv-GAUwJzqeiVZ9-n+v7JE4xMA@mail.gmail.com>
Message-ID: <mii3o9$j9f$1@ger.gmane.org>

Lesego Moloko wrote:

> I am a Python novice and have recently converted the attached Matlab file
> to Python (attached). The problem is that I cannot reproduce the Matlab
> plots (attached) with the converted Python program. I am also trying to go
> line by line through my converted Python file to see where the problem
> could be.
> 
> I attached:
> 
>    1. Matlab plots (5_p3.png and 4_p3.png)
>    2. The Matlab file (test.m)
>    3. Converted Python file (test.py)
>    4. data files (a2_all_out_P3.dat and t4_all_out.dat)
> 
> You advise and assistance would be highly appreciated.
> Regards

Your chance of getting help is higher with a smaller piece of code inlined 
in the text of your post and a description of the problem that is more 
concrete.

> I am also trying to go
> line by line through my converted Python file to see where the problem
> could be.

Once you have located the problem you might follow up here or on the 
matplotlib mailing list.



From albert at spenarnc.xs4all.nl  Fri May  8 06:47:56 2015
From: albert at spenarnc.xs4all.nl (Albert van der Horst)
Date: 08 May 2015 10:47:56 GMT
Subject: Bitten by my C/Java experience
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <5547ad48$0$2864$e4fe514c@news.xs4all.nl>
 <mailman.96.1430761253.12865.python-list@python.org>
Message-ID: <554c945c$0$2952$e4fe514c@dreader35.news.xs4all.nl>

In article <mailman.96.1430761253.12865.python-list at python.org>,
Chris Angelico  <rosuav at gmail.com> wrote:
>On Tue, May 5, 2015 at 3:32 AM, Irmen de Jong <irmen.NOSPAM at xs4all.nl> wrote:
>> That is a broad question, but one thing that comes to mind is the
>current (python 3)
>> behavior of integer division. It gives the exact result and doesn't
>truncate to integers:
>>
>>
>>>>> 5/4
>> 1.25
>
>Using the word "exact" around non-integer values can be a little
>ambiguous, since floats are often inexact. But yes, int/int -> float,
>and yes, it WILL bite C programmers.

This should not be presented as a somewhat arbitrary decision.

Formerly we had

3e0/4e0
0.75

and

3/4
0

So the / operator worked on reals giving reals and integers
giving integers. Great if you're used to it, but sometimes a pitfall.
Also in practice it sometimes leads to rounding in unexpected places.
The greatest disadvantage is when you have i and j and want their
ratio. You have to do something like (real)i/j which feels unnatural.
So we need two different division operators on the integers.

Solution:
introduce // for integer by integer given integer, giving the
quotient (with a possible remainder).
Now / is free to be used for the more rational  "i/j gives a ratio",
i.e. a real number.

Bottom line
3e0/4e0 and 3/4 gives the same result. It is nice to no longer
have to be very careful in floating point calculation to avoid
integer constants.

On the other hand integer division is still available to solve
the familiar egg-farm problems:

I have 103 eggs. 12 eggs go in a box. How many boxes can I fill?

(Similar problems crop up in graphics all the time.)

>
>ChrisA

Groetjes Albert
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst



From denismfmcmahon at gmail.com  Fri May  8 06:51:49 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Fri, 8 May 2015 10:51:49 +0000 (UTC)
Subject: Encrypt python files
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
Message-ID: <mii4g5$olm$6@dont-email.me>

On Tue, 05 May 2015 23:37:02 -0700, Palpandi wrote:

> What are the ways to encrypt python files?

Depends why you want to encrypt them, and what you want to do with the 
encrypted files.

Do you mean executable python code files, or do you mean data files 
generated by python.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From frank at chagford.com  Fri May  8 06:53:19 2015
From: frank at chagford.com (Frank Millman)
Date: Fri, 8 May 2015 12:53:19 +0200
Subject: Is this unpythonic?
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mii4il$vv$1@ger.gmane.org>


"Steven D'Aprano" <steve+comp.lang.python at pearwood.info> wrote in message 
news:554c8b0a$0$12992$c3e8da3$5496439d at news.astraweb.com...
> On Fri, 8 May 2015 06:01 pm, Frank Millman wrote:
>
>> Hi all
>>
[...]
>>
>> However, every time I look at my own code, and I see   "def x(y, z=[]):
>> ....."   it looks wrong because I have been conditioned to think of it as
>> a gotcha.
>
> It is a gotcha, and a code smell.
>
> http://www.joelonsoftware.com/articles/Wrong.html
>

Interesting read - thanks.

> You can use it, but with care: code smells aren't necessarily wrong, they
> just need to be looked at a little more carefully.
>
>
>> Would it be more pythonic to change them all to use the alternative
>> "z=None", or is it ok to leave it as it is? Or to phrase it differently,
>> how would an experienced pythonista react on seeing this when reviewing 
>> my
>> code?
>
> I would change it to z=None *unless* you actually required the list to be
> mutated, e.g. if you were using it as a cache.
>

Ok, you and Joel have convinced me. I will change it to z=None.

> Does z have to be a list? Could you use an empty tuple instead?
>
> def x(y, z=()): ...
>

That was Chris' suggestion as well (thanks Chris).

The idea appealed to me, but then I found a situation where I pass in a 
dictionary instead of a list, so that would not work.

Replacing them all with None is cleaner and, I now agree, more pythonic.

Thanks for the good advice.

Frank





From tahiri.noria1 at gmail.com  Fri May  8 06:53:28 2015
From: tahiri.noria1 at gmail.com (tahiri noria)
Date: Fri, 8 May 2015 10:53:28 +0000
Subject: about orthogonal series
Message-ID: <CAOEuXWuDJO8Tf=6=+tWjifTGtg6Orv72BkTEVVNVV2zyRK5jiw@mail.gmail.com>

hello,

I read your article on the blog: http://jpktd.blogspot.com/ about
orthogonal series estimation , I would like to ask you if you allow .
First, I based on Article Sam Efromovich about "Orthogonal series density
estimates" .

he worked with the Fourier series with a compact support assumed [0,1],
whose aim is to estimate the probability density function. and it is said
in his Article that:


"The choice depends Primarily we of the carrier density. In general, When
the real line (-?, ?) or the half line [0, ?) are the support, Hermite and
Laguerre Then series are recommended; 3,17,23,24 see Refs. If f: has a
compact carrier, for instance [0, 1], Then trigonometric (or Fourier)
series are recommended. "

my question is that if I wanted to draw my probability density function on
a support [-2, -2] (because my data focus on -1 or 1) .What should I do?
is that these results are necessarily  just to support [0,1] for fourier
series ?

Cordially

TAHIRI Noria
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/ace9ddb8/attachment.html>

From steve+comp.lang.python at pearwood.info  Fri May  8 06:54:22 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 08 May 2015 20:54:22 +1000
Subject: To pickle or not to pickle
References: <87h9rnz8yy.fsf@Equus.decebal.nl>
Message-ID: <554c95df$0$12988$c3e8da3$5496439d@news.astraweb.com>

On Fri, 8 May 2015 07:58 pm, Cecil Westerhof wrote:

> I first used marshal in my filebasedMessages module. Then I read that
> you should not use it, because it changes per Python version and it
> was better to use pickle. So I did that and now I find:
>     https://wiki.python.org/moin/Pickle
> 
> Is it really that bad and should I change again?

marshal is really only for Python's internal use. I think that if Python was
created today, marshal would probably be an undocumented and internal-only
module.

pickle is quite safe provided you trust the environment you are running in
and the source of the pickle files. If you don't trust them, then you
should avoid pickle and use a format which doesn't execute code.

You could use JSON, plists, ini-files, or XML, all of which are text-based
and handled by the standard library. There is also YAML, but you have to
use a third-party library for that.

You might also look at the "serpent" serialisation format used by Pyro:

https://pypi.python.org/pypi/serpent

If your code is only going to be used by yourself, I'd just use pickle. If
you are creating an application for others to use, I would spend the extra
effort to build in support for at least pickle, JSON and plists, and let
the user decide what they prefer.



-- 
Steven



From tahiri.noria1 at gmail.com  Fri May  8 06:55:52 2015
From: tahiri.noria1 at gmail.com (tahiri noria)
Date: Fri, 8 May 2015 10:55:52 +0000
Subject: about Orthogonal series
Message-ID: <CAOEuXWuFp_37EtaheDYgyf5JK+wL1ABCNbQTXojC3t04DiAxjA@mail.gmail.com>

hello,

I read your article on the blog: http://jpktd.blogspot.com/ about
orthogonal series estimation , I would like to ask you if you allow .
First, I based on Article Sam Efromovich about "Orthogonal series density
estimates" .

he worked with the Fourier series with a compact support assumed [0,1],
whose aim is to estimate the probability density function. and it is said
in his Article that:


"The choice depends Primarily we of the carrier density. In general, When
the real line (-?, ?) or the half line [0, ?) are the support, Hermite and
Laguerre Then series are recommended; 3,17,23,24 see Refs. If f: has a
compact carrier, for instance [0, 1], Then trigonometric (or Fourier)
series are recommended. "

my question is that if I wanted to draw my probability density function on
a support [-2, -2] (because my data focus on -1 or 1) .What should I do?
is that these results are necessarily  just to support [0,1] for fourier
series ?

Cordially
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/a28b7544/attachment.html>

From denismfmcmahon at gmail.com  Fri May  8 06:59:10 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Fri, 8 May 2015 10:59:10 +0000 (UTC)
Subject: Encrypt python files
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
 <72bd42d6-6926-44ce-bd6f-956ce40afe08@googlegroups.com>
Message-ID: <mii4tu$olm$7@dont-email.me>

On Wed, 06 May 2015 00:23:39 -0700, Palpandi wrote:

> On Wednesday, May 6, 2015 at 12:07:13 PM UTC+5:30, Palpandi wrote:
>> Hi,
>> 
>> What are the ways to encrypt python files?
> 
> No, I just want to hide the scripts from others.

You can do that by deleting the scripts. Make sure you use a secure 
deletion tool.

I'm not aware of any mechanism for encrypted executable python scripts. 
You can obfuscate the code, but you can't encrypt it because the python 
interpreter needs the script file to execute it.

The same holds true for any mechanism designed to encrypt executable code 
regardless of whether it's script or compiled. At the lowest level the 
processor only understands the instruction set, and encrypted code has to 
be decrypted to execute.

As the decryption method is always available to anyone who has legitimate 
access to execute the code, it's impossible to hide the code at that 
point.

Example - if I give you an encrypted binary to run on your system, it 
either has to be unencryptable using tools you already have, or using a 
built in unencrypter, both of which you have access to and can use to 
unencrypt the encrypted executable code.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From jonathan.slenders at gmail.com  Fri May  8 07:05:16 2015
From: jonathan.slenders at gmail.com (jonathan.slenders at gmail.com)
Date: Fri, 8 May 2015 04:05:16 -0700 (PDT)
Subject: Why is array.array('u') deprecated?
In-Reply-To: <554c8ff2$0$12990$c3e8da3$5496439d@news.astraweb.com>
References: <80167ebf-c5ee-4699-89ca-ef544fe3decc@googlegroups.com>
 <554c8ff2$0$12990$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <93aaa9cd-5d30-44db-9324-4fffe292e9e4@googlegroups.com>

Le vendredi 8 mai 2015 12:29:15 UTC+2, Steven D'Aprano a ?crit?:
> On Fri, 8 May 2015 07:14 pm, jonathan.slenders wrote:
> 
> > Why is array.array('u') deprecated?
> > 
> > Will we get an alternative for a character array or mutable unicode
> > string?
> 
> 
> Good question.
> 
> Of the three main encodings for Unicode, two are variable-width: 
> 
> * UTF-8 uses 1-4 bytes per character 
> * UTF-16 uses 2 or 4 bytes per character
> 
> while UTF-32 is fixed-width (4 bytes per character). So you could try faking
> it with a 32-bit array and filling it with string.encode('utf-32').


I guess that doesn't work. I need to have something that I can pass to the re module for searching through it. Creating new strings all the time is no option. (Think about gigabyte strings.)




From denismfmcmahon at gmail.com  Fri May  8 07:22:03 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Fri, 8 May 2015 11:22:03 +0000 (UTC)
Subject: Json Comaprision
References: <mailman.164.1430902487.12865.python-list@python.org>
Message-ID: <mii68r$olm$8@dont-email.me>

On Tue, 05 May 2015 12:55:20 -0700, pra devOPS wrote:

> I wanted to compare two json files ignoring few of the keys in the json
> files.
> 
> Can anybody suggest me few things?

json files usually get imported to either a list or a dictionary (unless 
they're a simple string or number).

If the files are json arrays they'll get imported as lists. otherwise if 
they're key:value pairs they'll get imported as a dictionary.

Basically you need a routine that can take two arbitrary objects, check 
if they're the same data type, and then check their values. If they're of 
a collection type, checking their values means recursively checking that 
every element in the collection matches in type and value.

You need to be able to handle lists, dictionaries, strings, ordinals and 
floats, and for floats you might want to consider that if two floats are 
generated by two different systems, perhaps a cray xmp running compiled 
fortran and an acorn risc os box running compiled c++ for example, they 
might disagree at the least significant digit but still be considered 
equal for your purposes.

It's much easier to prove things are different than to prove they are the 
same, and you're much more likely to falsely detect difference than you 
are to correctly detect equality as the complexity of the objects you are 
checking increases unless you get the code right.

For a list, you need to consider if the order of elements is important. 
If not, then for every element in list a you need to detect if an 
identical element is in list b.

Supposing you have two lists of lists. This might mean that for every sub 
list in a, you need to check every sub list in b to see if it matches. 
You probably also want to check for lists and dictionaries if the sizes 
are equivalent too. Here's an example why:

a = [ [a,b,c] ]

b = [ [a,b,c], [a,c,b], [b,a,c], [b,c,a], [c,a,b], [c,b,a] ]

If you compare sublists purely on the basis that they contain the same 
elements, then a[0] == each of b[0..5]

Does that make a and b equal?

-- 
Denis McMahon, denismfmcmahon at gmail.com


From denismfmcmahon at gmail.com  Fri May  8 07:23:33 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Fri, 8 May 2015 11:23:33 +0000 (UTC)
Subject: Beacon controller
References: <b7943bd5-e2e6-4fcd-8239-750d33b6cbec@googlegroups.com>
Message-ID: <mii6bl$olm$9@dont-email.me>

On Tue, 05 May 2015 16:37:06 -0700, bmanogna.17 wrote:

> I'm new to pyhton, can anyone suggest me how can I implement a DOS or
> DDOS attack in Beacon Controller.

Contact your nearest LEO cybercrime unit and ask them.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From ben+python at benfinney.id.au  Fri May  8 07:31:44 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 08 May 2015 21:31:44 +1000
Subject: Is this unpythonic?
References: <mihqhb$r1k$1@ger.gmane.org>
Message-ID: <85y4kzb90f.fsf@benfinney.id.au>

"Frank Millman" <frank at chagford.com> writes:

> If I use a list as an argument, I only use it to pass values *into*
> the function, but I never modify the list.

You've had good reasons why a mutable default argument is a good idea.

I'll address another aspect of the question: passing a structured
collection of values via a single parameter.

If those values *always* go together, it sounds like you have a class
which should be defined to make it explicit that they go together.

E.g., a contrived example::

    def deliver_flowers(species, location=[0, 0, 0]):
        """ Deliver the species of flower to the specified location. """
        (x, y, z) = location
        vehicle.load_with(species)
        vehicle.drive_to(x, y, z)
        vehicle.unload()

If the only reason you're using a collection is because those values
sensibly go together as a single named object, then define a class for
that::

    class Point:
        """ A three-dimensional point in space. """

        def __init__(self, x, y, z):
            self.x = x
            self.y = y
            self.z = z

    centre = Point(0, 0, 0)

    def deliver_flowers(species, location=centre):
        """ Deliver the species of flower to the specified location. """
        vehicle.load_with(species)
        vehicle.drive_to(location.x, location.y, location.z)
        vehicle.unload()

Too much overhead? If you want to collect a meaningful collection of
values together in the same way each time, but don't need any special
behaviour, then a class might be overkill.

Python has the ?namedtuple? type::

    import collections

    Point = collections.namedtuple('Point', ['x', 'y', 'z'])

    centre = Point(0, 0, 0)

    def deliver_flowers(species, location=centre):
        """ Deliver the species of flower to the specified location. """
        vehicle.load_with(species)
        vehicle.drive_to(location.x, location.y, location.z)
        vehicle.unload()

<URL:https://docs.python.org/3/library/collections.html#collections.namedtuple>
documents the surprisingly useful ?namedtuple? factory.

-- 
 \          ?It's a terrible paradox that most charities are driven by |
  `\     religious belief.? if you think altruism without Jesus is not |
_o__)          altruism, then you're a dick.? ?Tim Minchin, 2010-11-28 |
Ben Finney



From rosuav at gmail.com  Fri May  8 07:32:01 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 21:32:01 +1000
Subject: Bitten by my C/Java experience
In-Reply-To: <554c945c$0$2952$e4fe514c@dreader35.news.xs4all.nl>
References: <87r3qwid3u.fsf@Equus.decebal.nl>
 <5547ad48$0$2864$e4fe514c@news.xs4all.nl>
 <mailman.96.1430761253.12865.python-list@python.org>
 <554c945c$0$2952$e4fe514c@dreader35.news.xs4all.nl>
Message-ID: <CAPTjJmpmAoqxFj-HkcACw5FQFW0T7iJxOcULaJLpZ5=731a9eA@mail.gmail.com>

On Fri, May 8, 2015 at 8:47 PM, Albert van der Horst
<albert at spenarnc.xs4all.nl> wrote:
> In article <mailman.96.1430761253.12865.python-list at python.org>,
> Chris Angelico  <rosuav at gmail.com> wrote:
>>On Tue, May 5, 2015 at 3:32 AM, Irmen de Jong <irmen.NOSPAM at xs4all.nl> wrote:
>>> That is a broad question, but one thing that comes to mind is the
>>current (python 3)
>>> behavior of integer division. It gives the exact result and doesn't
>>truncate to integers:
>>>
>>>
>>>>>> 5/4
>>> 1.25
>>
>>Using the word "exact" around non-integer values can be a little
>>ambiguous, since floats are often inexact. But yes, int/int -> float,
>>and yes, it WILL bite C programmers.
>
> This should not be presented as a somewhat arbitrary decision.

I didn't, but my main point was that "exact" is a poor choice of word.

> Formerly we had
>
> 3e0/4e0
> 0.75
>
> and
>
> 3/4
> 0
>
> So the / operator worked on reals giving reals and integers
> giving integers. Great if you're used to it, but sometimes a pitfall.
> Also in practice it sometimes leads to rounding in unexpected places.
> The greatest disadvantage is when you have i and j and want their
> ratio. You have to do something like (real)i/j which feels unnatural.
> So we need two different division operators on the integers.
>
> Solution:
> introduce // for integer by integer given integer, giving the
> quotient (with a possible remainder).
> Now / is free to be used for the more rational  "i/j gives a ratio",
> i.e. a real number.

Where's that "i.e." come from? Why shouldn't i/j yield a Fraction, for
instance? That would be every bit as logical, and would mean we
effectively get "fraction literals" in their most obvious and natural
form. I can accept that int/int should yield something capable of
storing non-integral values, but I don't see that it absolutely has to
be IEEE floating point. However, that point has been much argued, and
it's definitely not changing now.

> Bottom line
> 3e0/4e0 and 3/4 gives the same result. It is nice to no longer
> have to be very careful in floating point calculation to avoid
> integer constants.
>
> On the other hand integer division is still available to solve
> the familiar egg-farm problems:
>
> I have 103 eggs. 12 eggs go in a box. How many boxes can I fill?

Of course they do... but you still actually have a difference. Compare:

>>> (1<<1000)//(1<<10)
10463951242053391806136963369726580181263718864311851635192874886429209483641954321222640418122029864527291727710479949464718215680589004332016189037791576956967351342601788071700268169006221818240189631008834448226154239518944108944497601509840881752510934060240763835605888507473266002770708660224

>>> (1<<1000)//(2.0**10)
1.0463951242053392e+298

>>> (1<<2000)//(2.0**10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: int too large to convert to float

So you do still need to worry about whether, with integer division,
you're working with integers or floats. Lots of programmers wish they
could just ignore all these problems and they'll go away, but floats
have inexactitude and range limits, where ints have neither (and
Fraction and Decimal values have their own notables), and ultimately,
you're going to have to know what data type your numbers are. That's
why I think it'd be better to have int/int -> Fraction; you might have
shocking performance, but then you can simply tell people to use
float() to speed up calculations at the expense of precision. At least
operations between int and Fraction are guaranteed to maintain
precision.

ChrisA


From __peter__ at web.de  Fri May  8 07:35:30 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 08 May 2015 13:35:30 +0200
Subject: Why is array.array('u') deprecated?
References: <80167ebf-c5ee-4699-89ca-ef544fe3decc@googlegroups.com>
 <554c8ff2$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <93aaa9cd-5d30-44db-9324-4fffe292e9e4@googlegroups.com>
Message-ID: <mii722$7db$1@ger.gmane.org>

jonathan.slenders at gmail.com wrote:

> Le vendredi 8 mai 2015 12:29:15 UTC+2, Steven D'Aprano a ?crit :
>> On Fri, 8 May 2015 07:14 pm, jonathan.slenders wrote:
>> 
>> > Why is array.array('u') deprecated?
>> > 
>> > Will we get an alternative for a character array or mutable unicode
>> > string?
>> 
>> 
>> Good question.
>> 
>> Of the three main encodings for Unicode, two are variable-width:
>> 
>> * UTF-8 uses 1-4 bytes per character
>> * UTF-16 uses 2 or 4 bytes per character
>> 
>> while UTF-32 is fixed-width (4 bytes per character). So you could try
>> faking it with a 32-bit array and filling it with
>> string.encode('utf-32').
> 
> 
> I guess that doesn't work. I need to have something that I can pass to the
> re module for searching through it. Creating new strings all the time is
> no option. (Think about gigabyte strings.)

Can you expand a bit on how array("u") helps here? Are the matches in the 
gigabyte range?



From Cecil at decebal.nl  Fri May  8 07:51:58 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 08 May 2015 13:51:58 +0200
Subject: To pickle or not to pickle
References: <87h9rnz8yy.fsf@Equus.decebal.nl>
 <mailman.229.1431081179.12865.python-list@python.org>
Message-ID: <87d22bz3q9.fsf@Equus.decebal.nl>

Op Friday 8 May 2015 12:32 CEST schreef Peter Otten:

> Cecil Westerhof wrote:
>
>> I first used marshal in my filebasedMessages module. Then I read
>> that you should not use it, because it changes per Python version
>> and it was better to use pickle. So I did that and now I find:
>> https://wiki.python.org/moin/Pickle
>>
>> Is it really that bad and should I change again?
>
> Let's say it the other way around: pickle is fine for short term
> storage when the generation of the file is under your control and
> you only need to access it from Python.
>
> Does that description fit your requirements?

Certainly. I use it to store which messages are ?recently? used, so I
will not use them for the next. I will keep it like this for the
moment being then.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From davea at davea.name  Fri May  8 07:53:21 2015
From: davea at davea.name (Dave Angel)
Date: Fri, 08 May 2015 07:53:21 -0400
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <CAPTjJmrGFHm3i-OEHdYUgMM4zrN3BdYS7-dVgJoA5k3GO_-=DQ@mail.gmail.com>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
 <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>
 <5549ab43$0$11108$c3e8da3@news.astraweb.com>
 <b0022fbc-74e2-4990-9773-9e5e07d51f14@googlegroups.com>
 <mailman.216.1431059590.12865.python-list@python.org>
 <aca29698-6b7f-4ac7-ba1f-0e60fd28d639@googlegroups.com>
 <mailman.217.1431061769.12865.python-list@python.org>
 <e60dc222-a837-4a65-9ce2-eff80ee2527f@googlegroups.com>
 <CAPTjJmrGFHm3i-OEHdYUgMM4zrN3BdYS7-dVgJoA5k3GO_-=DQ@mail.gmail.com>
Message-ID: <554CA3B1.1080309@davea.name>

On 05/08/2015 02:42 AM, Chris Angelico wrote:
> On Fri, May 8, 2015 at 4:36 PM, Rustom Mody <rustompmody at gmail.com> wrote:
>> On Friday, May 8, 2015 at 10:39:38 AM UTC+5:30, Chris Angelico wrote:
>>> Why have the concept of a procedure?
>>
>> On Friday, Chris Angelico ALSO wrote:
>>> With print(), you have a conceptual procedure...
>>
>> So which do you want to stand by?
>
> A procedure, in Python, is simply a function which returns None.
> That's all. It's not any sort of special concept. It doesn't need to
> be taught. If your students are getting confused by it, stop teaching
> it!

One thing newbies get tripped up by is having some path through their 
code that doesn't explicitly return.  And in Python that path therefore 
returns None.  It's most commonly confusing when there are nested ifs, 
and one of the "inner ifs" doesn't have an else clause.

Anyway, it's marginally more useful to that newbie if the compiler would 
produce an error instead of later seeing a runtime error due to an 
unexpected None result.

I don't think Python would be improved by detecting such a condition and 
reporting on it.  That's a job for a linter, or a style guide program.

No different than the compile time checks for variable type that most 
languages impose.  They don't belong in Python.

-- 
DaveA


From Cecil at decebal.nl  Fri May  8 07:55:15 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 08 May 2015 13:55:15 +0200
Subject: To pickle or not to pickle
References: <87h9rnz8yy.fsf@Equus.decebal.nl>
 <554c95df$0$12988$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <878uczz3ks.fsf@Equus.decebal.nl>

Op Friday 8 May 2015 12:54 CEST schreef Steven D'Aprano:

> If your code is only going to be used by yourself, I'd just use
> pickle. If you are creating an application for others to use, I
> would spend the extra effort to build in support for at least
> pickle, JSON and plists, and let the user decide what they prefer.

Well, I put it on GitHub, so I hope it is going to be used by others
also. ;-) There are other things that are more urgent at the moment,
but in the future I will implement JSON and plists then.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From davea at davea.name  Fri May  8 08:04:34 2015
From: davea at davea.name (Dave Angel)
Date: Fri, 08 May 2015 08:04:34 -0400
Subject: Is this unpythonic?
In-Reply-To: <mii4il$vv$1@ger.gmane.org>
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mii4il$vv$1@ger.gmane.org>
Message-ID: <554CA652.1000607@davea.name>

On 05/08/2015 06:53 AM, Frank Millman wrote:
>
> "Steven D'Aprano" <steve+comp.lang.python at pearwood.info> wrote in message
> news:554c8b0a$0$12992$c3e8da3$5496439d at news.astraweb.com...
>> On Fri, 8 May 2015 06:01 pm, Frank Millman wrote:
>>
>>> Hi all
>>>
> [...]
>>>
>>> However, every time I look at my own code, and I see   "def x(y, z=[]):
>>> ....."   it looks wrong because I have been conditioned to think of it as
>>> a gotcha.
>>

It might be appropriate to define the list at top-level, as

EMPTY_LIST=[]

and in your default argument as
     def x(y, z=EMPTY_LIST):

and with the all-caps, you're thereby promising that nobody will modify 
that list.

(I'd tend to do the None trick, but I think this alternative would be 
acceptable)

-- 
DaveA


From rustompmody at gmail.com  Fri May  8 08:09:42 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Fri, 8 May 2015 05:09:42 -0700 (PDT)
Subject: functions, optional parameters
In-Reply-To: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
Message-ID: <cad2930a-a176-4329-810d-d195ed8e4ea1@googlegroups.com>

On Friday, May 8, 2015 at 5:30:15 PM UTC+5:30, Michael Welle wrote:
> Hello,
> 
> assume the following function definition:
> 
> def bar(foo = []):
>     print("foo: %s" % foo)
>     foo.append("foo")
> 
> It doesn't work like one would expect (or as I would expect ;-)). As I
> understand it the assignment of the empty list to the optional parameter
> foo take place when the function object is created, not when it is
> called. I think from the perspective of a user this is very strange.
> Anyways, what would be a good idiom to get the desired behaviour?

A standard gotcha of python.

Short version: Dont default optionals to mutable data structures

Longer: See http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments

[Or just run a search for "python mutable default" ]



From jonathan.slenders at gmail.com  Fri May  8 08:12:00 2015
From: jonathan.slenders at gmail.com (jonathan.slenders at gmail.com)
Date: Fri, 8 May 2015 05:12:00 -0700 (PDT)
Subject: Why is array.array('u') deprecated?
In-Reply-To: <mailman.237.1431084939.12865.python-list@python.org>
References: <80167ebf-c5ee-4699-89ca-ef544fe3decc@googlegroups.com>
 <554c8ff2$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <93aaa9cd-5d30-44db-9324-4fffe292e9e4@googlegroups.com>
 <mailman.237.1431084939.12865.python-list@python.org>
Message-ID: <7c33ef30-5923-49eb-933c-bfb568cb4753@googlegroups.com>

> Can you expand a bit on how array("u") helps here? Are the matches in the 
> gigabyte range?

I have a string of unicode characters, e.g.:

data = array.array('u', u'x' * 1000000000)

Then I need to change some data in the middle of this string, for instance:

data[500000] = 'y'

Then I want to use re to search in this text:

re.search('y', data)

This has to be fast. I really don't want to split and concatenate strings. Re should be able to process it and the expressions can be much more complex than this. (I think it should be anything that implements the buffer protocol).

So, this works perfectly fine and fast. But it scares me that it's deprecated and Python 4 will not support it anymore.


From davea at davea.name  Fri May  8 08:13:57 2015
From: davea at davea.name (Dave Angel)
Date: Fri, 08 May 2015 08:13:57 -0400
Subject: Encrypt python files
In-Reply-To: <mii4tu$olm$7@dont-email.me>
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
 <72bd42d6-6926-44ce-bd6f-956ce40afe08@googlegroups.com>
 <mii4tu$olm$7@dont-email.me>
Message-ID: <554CA885.60100@davea.name>

On 05/08/2015 06:59 AM, Denis McMahon wrote:
> On Wed, 06 May 2015 00:23:39 -0700, Palpandi wrote:
>
>> On Wednesday, May 6, 2015 at 12:07:13 PM UTC+5:30, Palpandi wrote:
>>> Hi,
>>>
>>> What are the ways to encrypt python files?
>>
>> No, I just want to hide the scripts from others.
>
> You can do that by deleting the scripts. Make sure you use a secure
> deletion tool.
>
> I'm not aware of any mechanism for encrypted executable python scripts.
> You can obfuscate the code, but you can't encrypt it because the python
> interpreter needs the script file to execute it.
>
> The same holds true for any mechanism designed to encrypt executable code
> regardless of whether it's script or compiled. At the lowest level the
> processor only understands the instruction set, and encrypted code has to
> be decrypted to execute.
>
> As the decryption method is always available to anyone who has legitimate
> access to execute the code, it's impossible to hide the code at that
> point.
>
> Example - if I give you an encrypted binary to run on your system, it
> either has to be unencryptable

It'd be clearer if you used  decryptable, since unencryptable has a very 
different meaning.

http://en.wiktionary.org/wiki/unencryptable


> using tools you already have, or using a
> built in unencrypter, both of which you have access to and can use to
> unencrypt the encrypted executable code.
>

likewise decrypter and decrypt.

-- 
DaveA


From skip.montanaro at gmail.com  Fri May  8 08:24:23 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Fri, 8 May 2015 07:24:23 -0500
Subject: A __call__ "method" which is itself another callable class instance?
Message-ID: <CANc-5UycxL5GUxBnk1CsjZOekpc0WG1575WX8HdKAHFwBEimdA@mail.gmail.com>

This is just a pedantic post. It has no real useful application, just
something to think about. I apologize that the setup is rather long. The
stuff to think about is at the end. (I also apologize for using rich
text/html. In this HTML world in which we live nowadays, I'm never certain
that mail systems preserve leading whitespace, even in plain text mail.)

I have a script which accepts a CSV file as input and the definition of a
transform function. It applies the transform function to each row of the
CSV file and produces a new CSV file on standard out.

One of the things I discovered I needed in certain circumstances was to
inject user-defined variables into the system. Since it's a single pass
sort of thing (not long-lived), I decided to inject those variables into
the function's globals. (I could have added them to builtins, I suppose,
but it didn't occur to me when I first wrote the script.)

The challenge was to find the appropriate globlas dictionary into which to
inject these variable definitions. My code wound up looking like this:

    if inspect.ismethod(func):
func_globals = func.im_func.func_globals
    elif inspect.isfunction(func):
func_globals = func.func_globals
    elif inspect.isbuiltin(func):
func_globals = sys.modules[func.__module__].__dict__
    elif inspect.ismethoddescriptor(func):
raise TypeError("Can't get globals of a method descriptor")

What if func is actually an instance of a class with a __call__ method? So
I added another branch:

    elif hasattr(func, "__call__"):
func = func.__call__

But now what? This required me to to start the process over again. I also
needed an else clause. Let's add that first:

    else:
raise TypeError("Don't know how to find globals from %s objects" %
type(func))

I need to repeat the tests in the case where I found a non-function-like
object with a __call__ attribute. I decided to wrap this logic in a for
loop which could only run twice:

    for _i in (0, 1):
        if inspect.ismethod(func):
            func_globals = func.im_func.func_globals
        elif inspect.isfunction(func):
            func_globals = func.func_globals
        elif inspect.isbuiltin(func):
            func_globals = sys.modules[func.__module__].__dict__
        elif inspect.ismethoddescriptor(func):
            raise TypeError("Can't get globals of a method descriptor")
        elif hasattr(func, "__call__"):
            func = func.__call__
            # Try again...
            continue
        else:
            raise TypeError("Don't know how to find globals from %s
objects" % type(func))
        break
    else:
        raise TypeError("Don't know how to find globals from %s objects" %
type(func))

I thought about using while True, but thought, "nah, that's never going to
happen." Then I wondered if it *could* happen. I constructed a short chain
of __call__ attributes which might require the loop to repeat more than
twice, but I can't think of a situation where it would actually be useful:

    class C1(object):
       def __call__(self):
           print "called"

    class C2(object):
       __call__ = C1()

If you instantiate C2 and call the instance, it works as you would expect:

    >>> o = C2()
    >>> o()
    called

>From that simple example, it's straightforward to create a chain of classes
of any length, thus requiring my for loop to potentially be a while True
loop. The question for the esteemed gathering here: have you ever
encountered the need for this class-instance-as-a-dunder-call-method
before? Perhaps in some sort of code generation situation? Or cleaner
functions-with-attributes?

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/1c6113dd/attachment.html>

From rosuav at gmail.com  Fri May  8 08:24:34 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 22:24:34 +1000
Subject: Beacon controller
In-Reply-To: <mii6bl$olm$9@dont-email.me>
References: <b7943bd5-e2e6-4fcd-8239-750d33b6cbec@googlegroups.com>
 <mii6bl$olm$9@dont-email.me>
Message-ID: <CAPTjJmoV+sneGmuHh01XmdB=9RYU7JbDjSbW4fLDyKM0-QU76w@mail.gmail.com>

On Fri, May 8, 2015 at 9:23 PM, Denis McMahon <denismfmcmahon at gmail.com> wrote:
> On Tue, 05 May 2015 16:37:06 -0700, bmanogna.17 wrote:
>
>> I'm new to pyhton, can anyone suggest me how can I implement a DOS or
>> DDOS attack in Beacon Controller.
>
> Contact your nearest LEO cybercrime unit and ask them.

Now I'm thinking about the possibilities of the International Space
Station getting involved in internet crime. Because to me... LEO
doesn't mean Law Enforcement Officer...

https://xkcd.com/713/

ChrisA


From storchaka at gmail.com  Fri May  8 08:28:54 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Fri, 08 May 2015 15:28:54 +0300
Subject: Stripping unencodable characters from a string
In-Reply-To: <24ef6c6d-a47a-4d8c-8651-c581e25161cb@googlegroups.com>
References: <24ef6c6d-a47a-4d8c-8651-c581e25161cb@googlegroups.com>
Message-ID: <miia68$tlf$1@ger.gmane.org>

On 05.05.15 21:19, Paul Moore wrote:
> I want to write a string to an already-open file (sys.stdout, typically). However, I *don't* want encoding errors, and the string could be arbitrary Unicode (in theory). The best way I've found is
>
>      data = data.encode(file.encoding, errors='replace').decode(file.encoding)
>      file.write(data)
>
> (I'd probably use backslashreplace rather than replace, but that's a minor point).
>
> Is that the best way? The multiple re-encoding dance seems a bit clumsy, but it was the best I could think of.

There are flaws in this approach.

1) file.encoding can be None (StringIO) or absent (general file-like 
object, that implements only write()).

2) When the encoding is UTF-16, UTF-32, UTF-8-SIG, the output will 
contain superfluous byte order marks.

This is not easy problem and there is no simple solution. In particular 
cases you can create TextIOWrapper(file.buffer, 'w', 
encoding=file.encoding, errors='replace', newline=file.newlines, 
write_through=True) and write to it, but be aware of limitations.




From rosuav at gmail.com  Fri May  8 08:39:17 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 22:39:17 +1000
Subject: functions, optional parameters
In-Reply-To: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
Message-ID: <CAPTjJmqgoaH3ZzdqHGvWp_xZhT4xUxvV_pTmqsyKsyYM8p2YLA@mail.gmail.com>

On Fri, May 8, 2015 at 9:59 PM, Michael Welle <mwe012008 at gmx.net> wrote:
> Hello,
>
> assume the following function definition:
>
> def bar(foo = []):
>     print("foo: %s" % foo)
>     foo.append("foo")
>
> It doesn't work like one would expect (or as I would expect ;-)). As I
> understand it the assignment of the empty list to the optional parameter
> foo take place when the function object is created, not when it is
> called. I think from the perspective of a user this is very strange.
> Anyways, what would be a good idiom to get the desired behaviour?

I'm not sure what the desired behaviour _is_, given that you're not
doing anything with the list after appending to it. But argument
defaults are evaluated when the function's defined. It's basically
like this:

DEFAULT_FOO = []
def bar(foo | nothing):
    if no argument passed: foo = DEFAULT_FOO
    print("foo: %s" % foo)
    foo.append("foo")

There are times when this is incredibly useful, but if you don't want
this behaviour, the most common idiom is this:

def bar(foo=None):
    if foo is None: foo = []
    print("foo: %s" % foo)
    foo.append("foo")

As an example of this, a project I'm involved in had a transition
function which could work in one of two modes:

1) Transition one file into another
2) Transition one file into another, then that into a third, then a fourth, etc

In order to maintain state cleanly, a dictionary was passed in, which
provided a "previous position" marker, which could then be updated.
For the first form, though, all I needed to do was give it an empty
dictionary, which would then be discarded. So the function went
something like this:

def transition(from, to, state=None):
    if not state: state={"cursor": 0}
    # ...
    state["cursor"] = last_position

(I can't easily show you the code; subsequent edits meant that the
first case actually wanted to retrieve the cursor position, so it now
always has a state dict, and has no default argument. But this is
still a valid concept.)

Both idioms are very common, and you need to decide whether you want a
single mutable default, or a None default that gets replaced by a
brand new dict/list/etc every time. Just remember that the
construction of a new list is quite different from the assignment of a
pre-existing list to a new name.

ChrisA


From rosuav at gmail.com  Fri May  8 08:50:22 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 22:50:22 +1000
Subject: A __call__ "method" which is itself another callable class
 instance?
In-Reply-To: <CANc-5UycxL5GUxBnk1CsjZOekpc0WG1575WX8HdKAHFwBEimdA@mail.gmail.com>
References: <CANc-5UycxL5GUxBnk1CsjZOekpc0WG1575WX8HdKAHFwBEimdA@mail.gmail.com>
Message-ID: <CAPTjJmoge61-bhbjdZQir9qWJ0jMAquNQ=s0O-EB+d=W4tULvQ@mail.gmail.com>

On Fri, May 8, 2015 at 10:24 PM, Skip Montanaro
<skip.montanaro at gmail.com> wrote:
> The question for the esteemed gathering here: have you ever encountered the
> need for this class-instance-as-a-dunder-call-method before? Perhaps in some
> sort of code generation situation? Or cleaner functions-with-attributes?

I've never actually done it, but there've been times when I've
considered setting __call__ to be a class, rather than an actual
function. (Though the time I wanted it, I was wanting to set __call__
on a module, and I ended up not bothering with the hassle that that
entails.) Python classes are automatically factories of themselves
[1], so there should be no difference between calling a class and
calling a function that passes through to the class:

foo = Bar
@functools.wraps(Bar)
def foo(*args, **kw): return Bar(*args, **kw)

In fact, I would fully expect things like functools.wraps to work just
fine on classes, types, and instances of classes with __call__
methods, just as they would with functions - in the same way that
you'd expect a def function to work just as well as a lambda function.

If your code _does_ get into an infinite loop, so would calling the
function. I wouldn't see a problem with that being rewritten as "while
True", just to ensure the equivalencies.

ChrisA

[1] https://xkcd.com/387/


From rosuav at gmail.com  Fri May  8 08:53:33 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 22:53:33 +1000
Subject: To pickle or not to pickle
In-Reply-To: <878uczz3ks.fsf@Equus.decebal.nl>
References: <87h9rnz8yy.fsf@Equus.decebal.nl>
 <554c95df$0$12988$c3e8da3$5496439d@news.astraweb.com>
 <878uczz3ks.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmrXRK-=ND0fhXYpGSu_8Fz8sXihydObdatkDV37oUrMYg@mail.gmail.com>

On Fri, May 8, 2015 at 9:55 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> Op Friday 8 May 2015 12:54 CEST schreef Steven D'Aprano:
>
>> If your code is only going to be used by yourself, I'd just use
>> pickle. If you are creating an application for others to use, I
>> would spend the extra effort to build in support for at least
>> pickle, JSON and plists, and let the user decide what they prefer.
>
> Well, I put it on GitHub, so I hope it is going to be used by others
> also. ;-) There are other things that are more urgent at the moment,
> but in the future I will implement JSON and plists then.

But will the pickle files be shared? If not, they're still nice and
private, and fairly safe. The problem comes when, for instance, you
have a client Python program that pickles data and sends it over a
network to a server Python program to be unpickled, because then
someone could craft a malicious pickle and send it to you to eat. If
they're only ever saved locally and re-read, there shouldn't be any
security risk (anyone who could reach in and edit the pickle file
could probably reach in and change the code anyway).

That said, if your needs are sufficiently simple, it may be worth
using something plain text just for the debuggability.

ChrisA


From rosuav at gmail.com  Fri May  8 09:02:13 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 23:02:13 +1000
Subject: asyncio: What is the difference between tasks, futures,
 and coroutines?
In-Reply-To: <554CA3B1.1080309@davea.name>
References: <344fd8f6-75c1-4b7d-888d-c5c9d4498ec3@googlegroups.com>
 <878ud27waw.fsf@elektro.pacujo.net>
 <4ea2d5ac-8c19-4a53-9a09-fe6dbe4a52bd@googlegroups.com>
 <5549ab43$0$11108$c3e8da3@news.astraweb.com>
 <b0022fbc-74e2-4990-9773-9e5e07d51f14@googlegroups.com>
 <mailman.216.1431059590.12865.python-list@python.org>
 <aca29698-6b7f-4ac7-ba1f-0e60fd28d639@googlegroups.com>
 <mailman.217.1431061769.12865.python-list@python.org>
 <e60dc222-a837-4a65-9ce2-eff80ee2527f@googlegroups.com>
 <CAPTjJmrGFHm3i-OEHdYUgMM4zrN3BdYS7-dVgJoA5k3GO_-=DQ@mail.gmail.com>
 <554CA3B1.1080309@davea.name>
Message-ID: <CAPTjJmqE06pdY9s8R-eCwmWvD1jbYiTRyYikckne93dgfzztQQ@mail.gmail.com>

On Fri, May 8, 2015 at 9:53 PM, Dave Angel <davea at davea.name> wrote:
> One thing newbies get tripped up by is having some path through their code
> that doesn't explicitly return.  And in Python that path therefore returns
> None.  It's most commonly confusing when there are nested ifs, and one of
> the "inner ifs" doesn't have an else clause.
>
> Anyway, it's marginally more useful to that newbie if the compiler would
> produce an error instead of later seeing a runtime error due to an
> unexpected None result.
>
> I don't think Python would be improved by detecting such a condition and
> reporting on it.  That's a job for a linter, or a style guide program.

They're not hard for linters to notice. After all, there's a clear
difference of intent between "return" (or just falling off the end of
the function) and "return None", even though they compile to the same
byte code. Though if you start enforcing that in your checkin policy,
you might have to deal with this kind of thing:

def raise_with_code(code):
    raise SpamError("Error %d in spamination: %s" % (code,
errortext.get(code, "<unknown>")))

def spaminate_text(f):
    rc = low_level_spamination_function(f)
    if not rc: return low_level_get_spamination_result()
    raise_with_code(rc)

Clearly one branch returns a value... but figuring out that the other
one always raises isn't easy. (Though I suppose in this case it could
be dealt with by having a function that constructs the error, and then
you "raise make_spam_error(rc)" instead.)

You're definitely right that Python shouldn't check for it.

ChrisA


From frank at chagford.com  Fri May  8 09:10:52 2015
From: frank at chagford.com (Frank Millman)
Date: Fri, 8 May 2015 15:10:52 +0200
Subject: Is this unpythonic?
References: <mihqhb$r1k$1@ger.gmane.org> <85y4kzb90f.fsf@benfinney.id.au>
Message-ID: <miickg$7md$1@ger.gmane.org>


"Ben Finney" <ben+python at benfinney.id.au> wrote in message 
news:85y4kzb90f.fsf at benfinney.id.au...
> "Frank Millman" <frank at chagford.com> writes:
>
>> If I use a list as an argument, I only use it to pass values *into*
>> the function, but I never modify the list.
>
> You've had good reasons why a mutable default argument is a good idea.
>
> I'll address another aspect of the question: passing a structured
> collection of values via a single parameter.
>
> If those values *always* go together, it sounds like you have a class
> which should be defined to make it explicit that they go together.
>
[...]
>
> Too much overhead? If you want to collect a meaningful collection of
> values together in the same way each time, but don't need any special
> behaviour, then a class might be overkill.
>
> Python has the 'namedtuple' type::
>
[...]
>
> <URL:https://docs.python.org/3/library/collections.html#collections.namedtuple>
> documents the surprisingly useful 'namedtuple' factory.
>

Useful thoughts - thanks, Ben.

Here is a typical use case.

I have a function which constructs a sql SELECT statement from various input 
parameters. One of the parameters is used to build up an ORDER BY clause.

If ordering is required, the caller passes a list of tuples, each tuple 
consisting of a column name and a boolean indicating ascending/descending.

If no ordering is required, it is convenient for the caller to omit the 
argument altogether, but then the callee needs a default argument. 
Previously I had 'order=[]' as a keyword argument, but that gave rise to my 
query. I have now changed it to 'order=None'.

Frank





From __peter__ at web.de  Fri May  8 09:11:35 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 08 May 2015 15:11:35 +0200
Subject: Why is array.array('u') deprecated?
References: <80167ebf-c5ee-4699-89ca-ef544fe3decc@googlegroups.com>
 <554c8ff2$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <93aaa9cd-5d30-44db-9324-4fffe292e9e4@googlegroups.com>
 <mailman.237.1431084939.12865.python-list@python.org>
 <7c33ef30-5923-49eb-933c-bfb568cb4753@googlegroups.com>
Message-ID: <miicm8$8ju$1@ger.gmane.org>

jonathan.slenders at gmail.com wrote:

>> Can you expand a bit on how array("u") helps here? Are the matches in the
>> gigabyte range?
> 
> I have a string of unicode characters, e.g.:
> 
> data = array.array('u', u'x' * 1000000000)
> 
> Then I need to change some data in the middle of this string, for
> instance:
> 
> data[500000] = 'y'
> 
> Then I want to use re to search in this text:
> 
> re.search('y', data)
> 
> This has to be fast. I really don't want to split and concatenate strings.
> Re should be able to process it and the expressions can be much more
> complex than this. (I think it should be anything that implements the
> buffer protocol).
> 
> So, this works perfectly fine and fast. But it scares me that it's
> deprecated and Python 4 will not support it anymore.

Hm, this doesn't even work with Python 3:

>>> data = array.array("u", u"x"*1000)
>>> data[100] = "y"
>>> re.search("y", data)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/re.py", line 166, in search
    return _compile(pattern, flags).search(string)
TypeError: can't use a string pattern on a bytes-like object

You can search for bytes

>>> re.search(b"y", data)
<_sre.SRE_Match object; span=(400, 401), match=b'y'>
>>> data[101] = "z"
>>> re.search(b"y", data)
<_sre.SRE_Match object; span=(400, 401), match=b'y'>
>>> re.search(b"yz", data)
>>> re.search(b"y\0\0\0z", data)
<_sre.SRE_Match object; span=(400, 405), match=b'y\x00\x00\x00z'>

but if that is good enough you can use a bytearray in the first place.



From frank at chagford.com  Fri May  8 09:12:11 2015
From: frank at chagford.com (Frank Millman)
Date: Fri, 8 May 2015 15:12:11 +0200
Subject: Is this unpythonic?
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mii4il$vv$1@ger.gmane.org> <554CA652.1000607@davea.name>
Message-ID: <miicmv$939$1@ger.gmane.org>


"Dave Angel" <davea at davea.name> wrote in message 
news:554CA652.1000607 at davea.name...
> On 05/08/2015 06:53 AM, Frank Millman wrote:
>>
>
> It might be appropriate to define the list at top-level, as
>
> EMPTY_LIST=[]
>
> and in your default argument as
>     def x(y, z=EMPTY_LIST):
>
> and with the all-caps, you're thereby promising that nobody will modify 
> that list.
>
> (I'd tend to do the None trick, but I think this alternative would be 
> acceptable)
>

Thanks, Dave, I like that idea.

However, as you can see from my other replies, I have decided to go with the 
flow, and use the None trick.

Frank





From rosuav at gmail.com  Fri May  8 09:34:47 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 8 May 2015 23:34:47 +1000
Subject: multiprocessing, queue
In-Reply-To: <gheu1cx2pd.ln2@news.c0t0d0s0.de>
References: <gheu1cx2pd.ln2@news.c0t0d0s0.de>
Message-ID: <CAPTjJmqANfCS8E16nnjUAYC=Korjrz2jcdDMA71BD1fT2qCKTg@mail.gmail.com>

On Fri, May 8, 2015 at 8:08 PM, Michael Welle <mwe012008 at gmx.net> wrote:
> Hello,
>
> what's wrong with [0]? As num_tasks gets higher proc.join() seems to
> block forever. First I thought the magical frontier is around 32k tasks,
> but then it seemed to work with 40k tasks. Now I'm stuck around 7k
> tasks. I think I do something fundamentally wrong, but I can't find it.
>
> Regards
> hmw
>
> [0] http://pastebin.com/adfBYgY9

Your code's small enough to include inline, so I'm doing that:

#!/usr/bin/python3
# -*- coding: utf-8 -*-

from multiprocessing import Process, Queue

class Foo(Process):

    def __init__(self, task_queue, result_queue):
        Process.__init__(self)
        self.result_queue = result_queue
        self.task_queue = task_queue

    def run(self):
        while True:
            n = self.task_queue.get()
            if n is None:
                break

            self.result_queue.put(1)

        return


def main():
    results = Queue()
    tasks = Queue()
    procs = []
    num_procs = 8
    num_tasks = 8000

    for i in range(num_procs):
        proc = Foo(tasks, results)
        procs.append(proc)

    for proc in procs:
        proc.start()

    for i in range(num_tasks):
        tasks.put(i)

    for i in range(num_procs):
        tasks.put(None)

    for proc in procs:
        print("join")
        proc.join()

    while not results.empty():
        result = results.get()
        print('Result: {}'.format(result))


if __name__ == '__main__':
    main()

# -- end of code --


First thing I'd look at is the default queue size. If your result
queue fills up, all processes will block until something starts
retrieving results. If you really want to have all your results stay
in the queue like that, you may need to specify a huge queue size,
which may cost you a lot of memory; much better would be to have each
job post something on the result queue when it's done, and then you
wait till they're all done:

from multiprocessing import Process, Queue

def foo(task_queue, result_queue):
    while True:
        n = task_queue.get()
        if n is None: break
        result_queue.put(1)
    # Make sure None is not a possible actual result
    # Otherwise, create an object() to use as a flag.
    result_queue.put(None)

def feed_tasks(num_tasks, num_procs, tasks):
    for i in range(num_tasks):
        tasks.put(i)

    for i in range(num_procs):
        tasks.put(None)

def main():
    results = Queue()
    tasks = Queue()
    num_procs = 8
    num_tasks = 8000
    procs = [Process(target=foo, args=(tasks, results)) for i in
range(num_procs)]

    for proc in procs: proc.start()

    Process(target=feed_tasks, args=(num_tasks, num_procs, tasks)).start()

    while num_procs:
        result = results.get()
        if result is None: num_procs -= 1
        else: print('Result: {}'.format(result))

    for proc in procs:
        print("join")
        proc.join()

if __name__ == '__main__':
    main()


I've also made a few other changes (for instance, no need to subclass
Process just to pass args), but the most important parts are a
result_queue.put() just before the process ends, and switching the
order of the result-queue-pump and process-join loops.

That still might block, though, at the point where the tasks are being
put onto the queue; so I've spun that off into its own process. (It
might not be necessary, depending on how your tasks work.) But I
tested this on 200,000 tasks (with the printing of results replaced
with a simple counter), and it worked fine, churning through the work
in about ten seconds.

As a general rule, queues need to have both ends operating
simultaneously, otherwise you're likely to have them blocking. In
theory, your code should all work with ridiculously low queue sizes;
the only cost will be concurrency (since you'd forever be waiting for
the queue, so your tasks will all be taking turns). I tested this by
changing the Queue() calls to Queue(1), and the code took about twice
as long to complete. :)

Hope that helps!

ChrisA


From sabri.pllana at gmail.com  Fri May  8 09:40:09 2015
From: sabri.pllana at gmail.com (SP)
Date: Fri, 8 May 2015 06:40:09 -0700 (PDT)
Subject: CFP: Workshop on Exascale Multi/many Core Computing Systems
 (E-MuCoCoS)
Message-ID: <b1a6b1dd-5c9a-47f9-a27d-89edc784b284@googlegroups.com>

***********************************************************************
 Paper submission deadline: June 22, 2015
***********************************************************************
 Workshop on Exascale Multi/many Core Computing Systems (E-MuCoCoS) 

 In conjunction with
 the 18th IEEE Conference on Computational Science and Engineering
 Porto, Portugal, October 21 - 23, 2015 
 
 http://homepage.lnu.se/staff/saplaa/e-mucocos15/
***********************************************************************
 Workshop proceedings are published by the IEEE 
***********************************************************************

* CONTEXT

Exascale computing will revolutionize computational science and 
engineering by providing 1000x the capabilities of currently available 
computing systems, while having a similar power footprint. The HPC 
community is working towards the development of the first Exaflop 
computer (expected around 2020) after reaching the Petaflop milestone 
in 2008. There are concerns that computer designs based on existing 
multi-core and many-core solutions will not scale to Exascale 
considering technical challenges (such as, productivity, energy 
consumption or reliability) and reasonable economic constraints. 
Therefore, novel multi-core and many-core solutions are required to 
reach Exascale.
 
E-MuCoCoS will be organized in conjunction with the 18th IEEE 
Conference on Computational Science and Engineering (CSE 2015), Porto, 
Portugal, October 21 - 23, 2015.

Previous editions of MuCoCoS workshop include: MuCoCoS 2014 (Porto, 
PT), MuCoCoS 2013 (Edinburgh, UK), MuCoCoS 2012 (SLC, US), MuCoCoS 
2011 (Seoul, KR), MuCoCoS 2010 (Krakow, PL), MuCoCoS 2009 (Fukuoka, 
JP), MuCoCoS 2008 (Barcelona, ES).
 
* TOPICS OF INTEREST

E-MuCoCoS focuses on multi/many core languages, system software and 
architectural solutions towards Exascale computing systems. The topics 
of the workshop include but are not limited to:

:: Methods and tools for preparing applications for Exascale
:: Programming models, languages, libraries and compilation techniques
:: Run-time systems and hardware support
:: Patterns, algorithms and data structures
:: Performance analysis, modeling, optimization and tuning

* SUBMISSION GUIDELINES

- The papers should be prepared using the IEEE format, and no longer 
than 6 pages. Submitted papers will be carefully evaluated based on 
originality, significance to workshop topics, technical soundness, and 
presentation quality.
- Submission of the paper implies that should the paper be accepted, 
at least one of the authors will register and present the paper at the 
workshop.
- Please submit your paper (as PDF) electronically using the online 
submission system.
 
* IMPORTANT DATES

- Submission: June 22, 2015	  
- Notification: July 28, 2015
- Camera ready: September 4, 2015
- Registration: September 4, 2015
- Workshop: October 21, 2015
 
* WORKSHOP CHAIR

- Sabri Pllana, Linnaeus University, Sweden 
  http://homepage.lnu.se/staff/saplaa/




From Cecil at decebal.nl  Fri May  8 10:34:58 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 08 May 2015 16:34:58 +0200
Subject: To pickle or not to pickle
References: <87h9rnz8yy.fsf@Equus.decebal.nl>
 <554c95df$0$12988$c3e8da3$5496439d@news.astraweb.com>
 <878uczz3ks.fsf@Equus.decebal.nl>
 <mailman.246.1431089615.12865.python-list@python.org>
Message-ID: <87zj5fxhm5.fsf@Equus.decebal.nl>

Op Friday 8 May 2015 14:53 CEST schreef Chris Angelico:

> On Fri, May 8, 2015 at 9:55 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> Op Friday 8 May 2015 12:54 CEST schreef Steven D'Aprano:
>>
>>> If your code is only going to be used by yourself, I'd just use
>>> pickle. If you are creating an application for others to use, I
>>> would spend the extra effort to build in support for at least
>>> pickle, JSON and plists, and let the user decide what they prefer.
>>
>> Well, I put it on GitHub, so I hope it is going to be used by
>> others also. ;-) There are other things that are more urgent at the
>> moment, but in the future I will implement JSON and plists then.
>
> But will the pickle files be shared? If not, they're still nice and
> private, and fairly safe. The problem comes when, for instance, you
> have a client Python program that pickles data and sends it over a
> network to a server Python program to be unpickled, because then
> someone could craft a malicious pickle and send it to you to eat. If
> they're only ever saved locally and re-read, there shouldn't be any
> security risk (anyone who could reach in and edit the pickle file
> could probably reach in and change the code anyway).

I would expect not. But I never know what someone else is going to do.
;-)

But in my case there is a Twitter directory with:
    quotes.txt
    quotes.pickle
    tips.txt
    tips.pickle

All four files are normally only accessed by the Python program. When
I want to extend the messages I use a text editor to append them.

The .txt files contain messages that can be used. And the .pickle
files contain the ?recently? used messages.

When I unpickle quotes.pickle I get:
    [25, 112, 4, 18, 41, 2, 81, 75, 28, 60, 105, 47, 84, 65, 103, 42,
    13, 66, 55, 124, 6, 82, 76, 12, 61, 113, 119, 96, 3, 68, 11, 89,
    98, 107, 118, 29, 57, 33, 88, 121, 110, 49, 90, 72, 87, 114, 43,
    59, 8, 92]

Very simple indeed.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From jonathan.slenders at gmail.com  Fri May  8 10:40:50 2015
From: jonathan.slenders at gmail.com (jonathan.slenders at gmail.com)
Date: Fri, 8 May 2015 07:40:50 -0700 (PDT)
Subject: Why is array.array('u') deprecated?
In-Reply-To: <mailman.249.1431090704.12865.python-list@python.org>
References: <80167ebf-c5ee-4699-89ca-ef544fe3decc@googlegroups.com>
 <554c8ff2$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <93aaa9cd-5d30-44db-9324-4fffe292e9e4@googlegroups.com>
 <mailman.237.1431084939.12865.python-list@python.org>
 <7c33ef30-5923-49eb-933c-bfb568cb4753@googlegroups.com>
 <mailman.249.1431090704.12865.python-list@python.org>
Message-ID: <c603bf41-a9e6-4b28-ab71-1d3ac127c3ec@googlegroups.com>

Le vendredi 8 mai 2015 15:11:56 UTC+2, Peter Otten a ?crit?:
> > So, this works perfectly fine and fast. But it scares me that it's
> > deprecated and Python 4 will not support it anymore.
> 
> Hm, this doesn't even work with Python 3:

My mistake. I should have tested better.

> >>> data = array.array("u", u"x"*1000)
> >>> data[100] = "y"
> >>> re.search("y", data)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib/python3.4/re.py", line 166, in search
>     return _compile(pattern, flags).search(string)
> TypeError: can't use a string pattern on a bytes-like object
> 
> You can search for bytes
> 
> >>> re.search(b"y", data)
> <_sre.SRE_Match object; span=(400, 401), match=b'y'>
> >>> data[101] = "z"
> >>> re.search(b"y", data)
> <_sre.SRE_Match object; span=(400, 401), match=b'y'>
> >>> re.search(b"yz", data)
> >>> re.search(b"y\0\0\0z", data)
> <_sre.SRE_Match object; span=(400, 405), match=b'y\x00\x00\x00z'>
> 
> but if that is good enough you can use a bytearray in the first place.

Maybe I'll try that. Thanks for the suggestions!

Jonathan


From rosuav at gmail.com  Fri May  8 11:05:24 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 01:05:24 +1000
Subject: multiprocessing, queue
In-Reply-To: <autu1cxa2m.ln2@news.c0t0d0s0.de>
References: <gheu1cx2pd.ln2@news.c0t0d0s0.de>
 <mailman.251.1431092090.12865.python-list@python.org>
 <autu1cxa2m.ln2@news.c0t0d0s0.de>
Message-ID: <CAPTjJmrJoZshhYrX9-1Lc9JpCk_4fQ-g=vREmFaByuzAq0M0yQ@mail.gmail.com>

On Sat, May 9, 2015 at 12:31 AM, Michael Welle <mwe012008 at gmx.net> wrote:
>> As a general rule, queues need to have both ends operating
>> simultaneously, otherwise you're likely to have them blocking. In
>> theory, your code should all work with ridiculously low queue sizes;
>> the only cost will be concurrency (since you'd forever be waiting for
>> the queue, so your tasks will all be taking turns). I tested this by
>> changing the Queue() calls to Queue(1), and the code took about twice
>> as long to complete. :)
> ;) I know, as you might guess it's not a real world example. It's just
> to explore the multiprocessing module.

Sure, but even in a real-world example, it shouldn't ever be necessary
to create huge queues. Larger queues allow for inconsistent
performance of producer and/or consumer (eg if your consumer takes 1s
to do each of 500 jobs, then 500s to do one job, it's capable of
coping with a producer that puts one job on the queue every 2s, but
only if the queue can handle ~250 jobs), but otherwise, the only
effect of shrinking the queues is to force the processes into
lock-step. Bigger queues mean that an over-performing producer will
run you out of memory rather than get blocked. At very least, it
should be a safe way to debug your logic - cut the queues down to just
2-3 elements each, and keep on printing out what's in the queue.

ChrisA


From steve+comp.lang.python at pearwood.info  Fri May  8 11:07:05 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sat, 09 May 2015 01:07:05 +1000
Subject: Is this unpythonic?
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mailman.231.1431082398.12865.python-list@python.org>
Message-ID: <554cd119$0$12977$c3e8da3$5496439d@news.astraweb.com>

On Fri, 8 May 2015 08:53 pm, Frank Millman wrote:

>> Does z have to be a list? Could you use an empty tuple instead?
>>
>> def x(y, z=()): ...
>>
> 
> That was Chris' suggestion as well (thanks Chris).
> 
> The idea appealed to me, but then I found a situation where I pass in a
> dictionary instead of a list, so that would not work.


Why wouldn't it work? If it worked with an empty list, it will probably work
with an empty tuple instead.



-- 
Steven



From rosuav at gmail.com  Fri May  8 11:11:50 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 01:11:50 +1000
Subject: To pickle or not to pickle
In-Reply-To: <87zj5fxhm5.fsf@Equus.decebal.nl>
References: <87h9rnz8yy.fsf@Equus.decebal.nl>
 <554c95df$0$12988$c3e8da3$5496439d@news.astraweb.com>
 <878uczz3ks.fsf@Equus.decebal.nl>
 <mailman.246.1431089615.12865.python-list@python.org>
 <87zj5fxhm5.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmr7aQ0BCnPT+PL3KGHz7bnCSC-uiECAbJiug2bqoSAyJA@mail.gmail.com>

On Sat, May 9, 2015 at 12:34 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> When I unpickle quotes.pickle I get:
>     [25, 112, 4, 18, 41, 2, 81, 75, 28, 60, 105, 47, 84, 65, 103, 42,
>     13, 66, 55, 124, 6, 82, 76, 12, 61, 113, 119, 96, 3, 68, 11, 89,
>     98, 107, 118, 29, 57, 33, 88, 121, 110, 49, 90, 72, 87, 114, 43,
>     59, 8, 92]
>
> Very simple indeed.

In that case, I'd probably write it out as JSON, or as a simple
whitespace-separated list of numbers. That way, if anything goes
wrong, you can open up the file and look at it easily.

ChrisA


From steve+comp.lang.python at pearwood.info  Fri May  8 11:24:00 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sat, 09 May 2015 01:24:00 +1000
Subject: functions, optional parameters
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
Message-ID: <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>

On Fri, 8 May 2015 09:59 pm, Michael Welle wrote:

> Hello,
> 
> assume the following function definition:
> 
> def bar(foo = []):
>     print("foo: %s" % foo)
>     foo.append("foo")
> 
> It doesn't work like one would expect (or as I would expect ;-)). As I
> understand it the assignment of the empty list to the optional parameter
> foo take place when the function object is created, not when it is
> called. I think from the perspective of a user this is very strange.

I think it is perfectly expected.

Do you think that Python will re-compile the body of the function every time
you call it? Setting the default is part of the process of compiling the
function.


If we have this function definition:

    def spam(eggs=long_complex_calculation()):
        pass


do you expect the long complex calculation to be repeated every time you
call the function?

How about this definition:

    default = 23
    def spam(eggs=default):
        pass

    del default

    print spam()


Do you expect the function call to fail because `default` doesn't exist?


My answers to those questions are all No. To me, it is not only expected,
but desirable that function defaults are set once, not every time the
function is called. This behaviour is called "early binding" of defaults.

The opposite behaviour is called "late binding".

If your language uses late binding, it is very inconvenient to get early
binding when you want it. But if your language uses early binding, it is
very simple to get late binding when you want it: just put the code you
want to run inside the body of the function:

    # simulate late binding
    def spam(eggs=None):
        if eggs is None:
            # perform the calculation every time you call the function
            eggs = long_complex_calculation()


    default = 23
    def spam(eggs=None):
        if eggs is None:
            # look up the global variable every time you call the function
            eggs = default


On the rare times that you want to allow None as an ordinary value, you can
create your own private sentinel value:


    _SENTINEL = object()

    def spam(eggs=_SENTINEL):
        if eggs is _SENTINEL:
            eggs = something_else



-- 
Steven



From subhabrata.banerji at gmail.com  Fri May  8 11:26:48 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Fri, 8 May 2015 08:26:48 -0700 (PDT)
Subject: IDLE Restoration
Message-ID: <08bb6733-1bc3-4f21-bba1-b97d3f73e010@googlegroups.com>

Dear Group,

In many applications there is a facility to restore its previous sessions, especially if they close accidentally. 

Does IDLE have any such facility? 
If it is there, how may I use it?

If someone may kindly suggest it.

Regards,
Subhabrata Banerjee. 


From rosuav at gmail.com  Fri May  8 11:35:38 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 01:35:38 +1000
Subject: IDLE Restoration
In-Reply-To: <08bb6733-1bc3-4f21-bba1-b97d3f73e010@googlegroups.com>
References: <08bb6733-1bc3-4f21-bba1-b97d3f73e010@googlegroups.com>
Message-ID: <CAPTjJmrNAjovQqtyP4vHMDjFjkfRg+ip9JfM=uCPQvyExjD0GA@mail.gmail.com>

On Sat, May 9, 2015 at 1:26 AM,  <subhabrata.banerji at gmail.com> wrote:
> In many applications there is a facility to restore its previous sessions, especially if they close accidentally.
>
> Does IDLE have any such facility?
> If it is there, how may I use it?
>
> If someone may kindly suggest it.

Generally the notion of session restore is a fairly limited one; for
instance, a text editor might keep track of which files were opened
(and maybe cursor positions), but not the actual contents. Web
browsers tend to do a bit more than that (a good thing, since they
crash more often than text editors do), but still generally depend on
being able to retrieve most of the content from the origin server(s)
on restart.

Are you hoping to have the IDLE editor restore the fact that you had
files X, Y, and Z open, or are you hoping to have the interactive
interpreter retain your entire history of commands and their results?
The former is reasonably plausible (and may well be possible already -
Terry?), but the latter is quite impractical.

ChrisA


From tommyc168168 at gmail.com  Fri May  8 11:40:34 2015
From: tommyc168168 at gmail.com (Tommy C)
Date: Fri, 8 May 2015 08:40:34 -0700 (PDT)
Subject: How to properly apply OOP in the bouncing ball code
Message-ID: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>

I'm trying to apply OOP in this bouncing ball code in order to have multiple balls bouncing around the screen. The objective of this code is to create a method called settings, which controls all the settings for the screen and the bouncing behaviour of multiple balls, under the class Ball. However, I keep on getting errors related to attributes (e.g., speed). I'm new to OOP in Python so your help will be much appreciated. Thanks in advance.


import pygame
import random
import sys

pygame.init()

class Ball:
    def __init__(self, X, Y):
        self.velocity = [1,1]
        self.ball_image = pygame.image.load ("ball.jpg")
        self.ball_boundary = self.ball_image.get_rect ()
        self.black = [0,0,0]
        self.width = 800
        self.height = 600
        self.num = 8
        self.X = random.randint(0, self.width)
        self.Y = random.randint(0, self.height)

    def settings(self):
        #X = random.randint(0, self.width)
        #Y = random.randint(0, self.height)
        clock = pygame.time.Clock()
        size = self.width, self.height
        screen = pygame.display.set_mode(size)
        ball_boundary = self.ball_image.get_rect()
        speed = self.velocity
        pic = self.ball_image
        pygame.display.set_caption("Balls")
        num_balls = self.num
        ball_list = []

        for i in range(num_balls):
           ball_list.append( Ball(random.randint(10, self.width-10),random.randint(10, self.height-10)) )

        while 1:
            for event in pygame.event.get():
                if event.type == pygame.QUIT:
                    sys.exit(0)

            screen.fill(self.black)
            for balls in ball_list:
                if balls.ball_boundary.left < 0 or balls.ball_boundary.right > self.width:
                    balls.speed[0] = -balls.speed[0]
                if balls.ball_boundary.top < 0 or balls.ball_boundary.bottom > self.height:
                    balls.speed[1] = -balls.speed[1]
                balls.ball_boundary = balls.ball_boundary.move (self.velocity)
                screen.blit (balls.ball_image, balls.ball_boundary)
            pygame.display.flip()

play = Ball(random.randint(0, 800), random.randint(0, 600))

play.settings()




Message	File Name	Line	Position	
Traceback				
    <module>	C:\....\Multiple_balls_TC.py	63		
    settings	C:\....\Multiple_balls_TC.py	56		
AttributeError: Ball instance has no attribute 'speed'				


From ian.g.kelly at gmail.com  Fri May  8 11:48:32 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 8 May 2015 09:48:32 -0600
Subject: functions, optional parameters
In-Reply-To: <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CALwzidnkFF3DZws_5q=O6g9DFyFD5yK=BvHY2DtRoCVnZYLdyw@mail.gmail.com>

On May 8, 2015 9:26 AM, "Steven D'Aprano" <
steve+comp.lang.python at pearwood.info> wrote:
>
> Do you think that Python will re-compile the body of the function every
time
> you call it? Setting the default is part of the process of compiling the
> function.

To be a bit pedantic, that's not accurate. The default is evaluated when
the function object is created, i.e. when the def statement is executed at
runtime, not when the underlying code object is compiled.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/8ade46fb/attachment.html>

From rosuav at gmail.com  Fri May  8 12:02:42 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 02:02:42 +1000
Subject: functions, optional parameters
In-Reply-To: <CALwzidnkFF3DZws_5q=O6g9DFyFD5yK=BvHY2DtRoCVnZYLdyw@mail.gmail.com>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <CALwzidnkFF3DZws_5q=O6g9DFyFD5yK=BvHY2DtRoCVnZYLdyw@mail.gmail.com>
Message-ID: <CAPTjJmrR+fP2kHRf05qu92jU=8pjDNuodzQa0=W1GMbh559HfQ@mail.gmail.com>

On Sat, May 9, 2015 at 1:48 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On May 8, 2015 9:26 AM, "Steven D'Aprano"
> <steve+comp.lang.python at pearwood.info> wrote:
>>
>> Do you think that Python will re-compile the body of the function every
>> time
>> you call it? Setting the default is part of the process of compiling the
>> function.
>
> To be a bit pedantic, that's not accurate. The default is evaluated when the
> function object is created, i.e. when the def statement is executed at
> runtime, not when the underlying code object is compiled.

Aside from constructing two closures in the same context and proving
that their __code__ attributes point to the same object, is there any
way to distinguish between "code object compilation time" and "def
execution time"? I just played around with it, and as far as I can
tell, code objects are completely read-only.

ChrisA


From PointedEars at web.de  Fri May  8 12:06:49 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Fri, 08 May 2015 18:06:49 +0200
Subject: IDLE Restoration
References: <08bb6733-1bc3-4f21-bba1-b97d3f73e010@googlegroups.com>
 <mailman.254.1431099341.12865.python-list@python.org>
Message-ID: <4709717.b925lGYb8P@PointedEars.de>

Chris Angelico wrote:

> Are you hoping to have the IDLE editor restore the fact that you had
> files X, Y, and Z open, or are you hoping to have the interactive
> interpreter retain your entire history of commands and their results?
> The former is reasonably plausible (and may well be possible already -
> Terry?), but the latter is quite impractical.

Various shells do the latter.  I do not see why that would be impractical, 
on the contrary.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From rosuav at gmail.com  Fri May  8 12:16:43 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 02:16:43 +1000
Subject: IDLE Restoration
In-Reply-To: <4709717.b925lGYb8P@PointedEars.de>
References: <08bb6733-1bc3-4f21-bba1-b97d3f73e010@googlegroups.com>
 <mailman.254.1431099341.12865.python-list@python.org>
 <4709717.b925lGYb8P@PointedEars.de>
Message-ID: <CAPTjJmr8tUHF7=WDdhMG2GOYakeRCtRnU8kB7EhUV7Pr-FPTjA@mail.gmail.com>

On Sat, May 9, 2015 at 2:06 AM, Thomas 'PointedEars' Lahn
<PointedEars at web.de> wrote:
> Chris Angelico wrote:
>
>> Are you hoping to have the IDLE editor restore the fact that you had
>> files X, Y, and Z open, or are you hoping to have the interactive
>> interpreter retain your entire history of commands and their results?
>> The former is reasonably plausible (and may well be possible already -
>> Terry?), but the latter is quite impractical.
>
> Various shells do the latter.  I do not see why that would be impractical,
> on the contrary.

Exactly what do they recall? A textual form of the scrollback? That
wouldn't be too hard. But what about all your working state - assigned
globals, changed state of imported modules (eg random number seed),
etc, etc, etc? An active Python session is a *lot* more than a shell
session scrollback, because "their results" could literally be any
Python objects.

ChrisA


From ian.g.kelly at gmail.com  Fri May  8 12:22:28 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 8 May 2015 10:22:28 -0600
Subject: How to properly apply OOP in the bouncing ball code
In-Reply-To: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>
References: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>
Message-ID: <CALwzidnk9iJhykAkxWONnCuu3fF-Siu-SCrFDP+jdW9rA-Pmgw@mail.gmail.com>

On May 8, 2015 9:46 AM, "Tommy C" <tommyc168168 at gmail.com> wrote:
>
> I'm trying to apply OOP in this bouncing ball code in order to have
multiple balls bouncing around the screen. The objective of this code is to
create a method called settings, which controls all the settings for the
screen and the bouncing behaviour of multiple balls, under the class Ball.
However, I keep on getting errors related to attributes (e.g., speed). I'm
new to OOP in Python so your help will be much appreciated. Thanks in
advance.

As the error says, the attribute does not exist.

> class Ball:
>     def __init__(self, X, Y):
>         self.velocity = [1,1]

Here where you set it, you call it "velocity".

>         speed = self.velocity

Here you create a local variable called "speed", which you never use.

>                 if balls.ball_boundary.left < 0 or
balls.ball_boundary.right > self.width:
>                     balls.speed[0] = -balls.speed[0]

And here you look up an attribute of Ball called "speed", which doesn't
match the name you used in __init__.

This is a muddled design overall. Your Ball class represents the individual
balls bouncing around the screen. It should not also contain details about
window size and the logic for the event loop. Use a separate class for that.

Similarly, if the purpose of your settings method is to manage settings,
why does it also contain all the bouncing logic? These should probably be
separate methods.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150508/a138a726/attachment.html>

From Cecil at decebal.nl  Fri May  8 12:43:44 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 08 May 2015 18:43:44 +0200
Subject: To pickle or not to pickle
References: <87h9rnz8yy.fsf@Equus.decebal.nl>
 <554c95df$0$12988$c3e8da3$5496439d@news.astraweb.com>
 <878uczz3ks.fsf@Equus.decebal.nl>
 <mailman.246.1431089615.12865.python-list@python.org>
 <87zj5fxhm5.fsf@Equus.decebal.nl>
 <mailman.253.1431097914.12865.python-list@python.org>
Message-ID: <87fv77xbnj.fsf@Equus.decebal.nl>

Op Friday 8 May 2015 17:11 CEST schreef Chris Angelico:

> On Sat, May 9, 2015 at 12:34 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> When I unpickle quotes.pickle I get:
>> [25, 112, 4, 18, 41, 2, 81, 75, 28, 60, 105, 47, 84, 65, 103, 42,
>> 13, 66, 55, 124, 6, 82, 76, 12, 61, 113, 119, 96, 3, 68, 11, 89,
>> 98, 107, 118, 29, 57, 33, 88, 121, 110, 49, 90, 72, 87, 114, 43,
>> 59, 8, 92]
>>
>> Very simple indeed.
>
> In that case, I'd probably write it out as JSON, or as a simple
> whitespace-separated list of numbers. That way, if anything goes
> wrong, you can open up the file and look at it easily.

Done. And the files are even smaller. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From irmen.NOSPAM at xs4all.nl  Fri May  8 13:11:03 2015
From: irmen.NOSPAM at xs4all.nl (Irmen de Jong)
Date: Fri, 08 May 2015 19:11:03 +0200
Subject: To pickle or not to pickle
In-Reply-To: <mailman.229.1431081179.12865.python-list@python.org>
References: <87h9rnz8yy.fsf@Equus.decebal.nl>
 <mailman.229.1431081179.12865.python-list@python.org>
Message-ID: <554cee28$0$2965$e4fe514c@news.xs4all.nl>

On 8-5-2015 12:32, Peter Otten wrote:
> Cecil Westerhof wrote:
> 
>> I first used marshal in my filebasedMessages module. Then I read that
>> you should not use it, because it changes per Python version and it
>> was better to use pickle. So I did that and now I find:
>>     https://wiki.python.org/moin/Pickle
>>
>> Is it really that bad and should I change again?
> 
> Let's say it the other way around: pickle is fine for short term storage 
> when the generation of the file is under your control and you only need to 
> access it from Python. 

The latter is not really a restriction, if you want to use it from Java or .NET.
https://github.com/irmen/Pyrolite  provides an (un)pickler for these platforms.


-irmen



From steve+comp.lang.python at pearwood.info  Fri May  8 13:26:52 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sat, 09 May 2015 03:26:52 +1000
Subject: functions, optional parameters
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <mailman.255.1431100115.12865.python-list@python.org>
Message-ID: <554cf1dd$0$12996$c3e8da3$5496439d@news.astraweb.com>

On Sat, 9 May 2015 01:48 am, Ian Kelly wrote:

> On May 8, 2015 9:26 AM, "Steven D'Aprano" <
> steve+comp.lang.python at pearwood.info> wrote:
>>
>> Do you think that Python will re-compile the body of the function every
> time
>> you call it? Setting the default is part of the process of compiling the
>> function.
> 
> To be a bit pedantic, that's not accurate. The default is evaluated when
> the function object is created, i.e. when the def statement is executed at
> runtime, not when the underlying code object is compiled.

Yes, that is the pedantically correct version.

"Technically correct -- the best kind of correct."



-- 
Steven



From steve+comp.lang.python at pearwood.info  Fri May  8 13:36:32 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sat, 09 May 2015 03:36:32 +1000
Subject: functions, optional parameters
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <CALwzidnkFF3DZws_5q=O6g9DFyFD5yK=BvHY2DtRoCVnZYLdyw@mail.gmail.com>
 <mailman.256.1431100965.12865.python-list@python.org>
Message-ID: <554cf421$0$12978$c3e8da3$5496439d@news.astraweb.com>

On Sat, 9 May 2015 02:02 am, Chris Angelico wrote:

> On Sat, May 9, 2015 at 1:48 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> On May 8, 2015 9:26 AM, "Steven D'Aprano"
>> <steve+comp.lang.python at pearwood.info> wrote:
>>>
>>> Do you think that Python will re-compile the body of the function every
>>> time
>>> you call it? Setting the default is part of the process of compiling the
>>> function.
>>
>> To be a bit pedantic, that's not accurate. The default is evaluated when
>> the function object is created, i.e. when the def statement is executed
>> at runtime, not when the underlying code object is compiled.
> 
> Aside from constructing two closures in the same context and proving
> that their __code__ attributes point to the same object, is there any
> way to distinguish between "code object compilation time" and "def
> execution time"? I just played around with it, and as far as I can
> tell, code objects are completely read-only.

Sure there is. Write this Python code:


# test.py
print("Function definition time.")
def func():
    pass


Now from your shell, run this:


echo "Compile time"
python -m compileall test.py
rm test.py
sleep 5
python test.pyc



(The sleep is just to make it clear that the compilation and definition time
can be very far apart. They could literally be years apart.)



Actually, we don't need external tools, we can do it all in Python!

py> source = """\
... print "Function definition time."
... def func():
...     pass
... """
py> print "Compile time."; code = compile(source, '', 'exec')
Compile time.
py> exec(code)
Function definition time.




-- 
Steven



From rosuav at gmail.com  Fri May  8 13:49:36 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 03:49:36 +1000
Subject: functions, optional parameters
In-Reply-To: <554cf421$0$12978$c3e8da3$5496439d@news.astraweb.com>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <CALwzidnkFF3DZws_5q=O6g9DFyFD5yK=BvHY2DtRoCVnZYLdyw@mail.gmail.com>
 <mailman.256.1431100965.12865.python-list@python.org>
 <554cf421$0$12978$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmonHgMbcc+SoRq4O3CnyFb_VNfCA7QJRDTy5pZzu+3nkw@mail.gmail.com>

On Sat, May 9, 2015 at 3:36 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Sat, 9 May 2015 02:02 am, Chris Angelico wrote:
>> Aside from constructing two closures in the same context and proving
>> that their __code__ attributes point to the same object, is there any
>> way to distinguish between "code object compilation time" and "def
>> execution time"? I just played around with it, and as far as I can
>> tell, code objects are completely read-only.
>
> Sure there is. Write this Python code:
>
> py> source = """\
> ... print "Function definition time."
> ... def func():
> ...     pass
> ... """
> py> print "Compile time."; code = compile(source, '', 'exec')
> Compile time.
> py> exec(code)
> Function definition time.

Yes, but can you *distinguish* them in terms of default argument
versus code object creation? How do you know that the function's code
object was created when compile() happened, rather than being created
when the function was defined? Is there anything that lets you in any
way show different behaviour based on that timing difference?

ChrisA


From mwilson at the-wire.com  Fri May  8 14:44:28 2015
From: mwilson at the-wire.com (Mel Wilson)
Date: Fri, 8 May 2015 18:44:28 +0000 (UTC)
Subject: How to properly apply OOP in the bouncing ball code
References: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>
Message-ID: <mij06a$tgc$1@dont-email.me>

On Fri, 08 May 2015 08:40:34 -0700, Tommy C wrote:

> I'm trying to apply OOP in this bouncing ball code in order to have
> multiple balls bouncing around the screen. The objective of this code is
> to create a method called settings, which controls all the settings for
> the screen and the bouncing behaviour of multiple balls, under the class
> Ball. However, I keep on getting errors related to attributes (e.g.,
> speed). I'm new to OOP in Python so your help will be much appreciated.
> Thanks in advance.

I did something similar, with little characters running around the screen 
under joystick control.  What I think:

1. Each instance of the Ball class should control the behaviour of one 
ball.
2. The screen settings and global environment generally should be global, 
not stuffed inside the behaviour of a Ball.
3. Keeping the list of Ball instances should be something the main 
section of your program does, not stuffed inside the behaviour of a Ball.

Each of my little GamePlayer objects is subclassed from 
pygame.sprite.Sprite .  I forget what I got by doing this (it's old code) 
but looking up the docs for that class might give you some insights.  
Each GamePlayer object has an "update" method which accepts info from the 
joystick and updates the player's movements, and a "draw" method which 
the main loop calls on displaying every frame, to show the player's new 
position and attitude.

You might be able to do something along these lines.

	Mel.


From mwilson at the-wire.com  Fri May  8 14:49:46 2015
From: mwilson at the-wire.com (Mel Wilson)
Date: Fri, 8 May 2015 18:49:46 +0000 (UTC)
Subject: functions, optional parameters
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <CALwzidnkFF3DZws_5q=O6g9DFyFD5yK=BvHY2DtRoCVnZYLdyw@mail.gmail.com>
 <mailman.256.1431100965.12865.python-list@python.org>
 <554cf421$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <mailman.259.1431107379.12865.python-list@python.org>
Message-ID: <mij0g8$tgc$2@dont-email.me>

On Sat, 09 May 2015 03:49:36 +1000, Chris Angelico wrote:

> Yes, but can you *distinguish* them in terms of default argument versus
> code object creation? How do you know that the function's code object
> was created when compile() happened, rather than being created when the
> function was defined? Is there anything that lets you in any way show
> different behaviour based on that timing difference?

This might show that default objects are fixed at run time:

Python 2.7.3 (default, Mar 14 2014, 11:57:14) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> a = []
>>> def b (arr=a):
...   arr.append ('c')
... 
>>> print repr(a)
[]
>>> b()
>>> print repr(a)
['c']
>>> b()
>>> print repr(a)
['c', 'c']
>>> 



From zljubisicmob at gmail.com  Fri May  8 15:00:55 2015
From: zljubisicmob at gmail.com (zljubisicmob at gmail.com)
Date: Fri, 8 May 2015 12:00:55 -0700 (PDT)
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes
 in position 2-3: truncated \UXXXXXXXX escape
Message-ID: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>

The script is very simple (abc.txt exists in ROOTDIR directory):

import os
import shutil

ROOTDIR = 'C:\Users\zoran'

file1 = os.path.join(ROOTDIR, 'abc.txt')
file2 = os.path.join(ROOTDIR, 'def.txt')

shutil.move(file1, file2)


But it returns the following error:


C:\Python34\python.exe C:/Users/bckslash_test.py
  File "C:/Users/bckslash_test.py", line 4
    ROOTDIR = 'C:\Users'
             ^
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

Process finished with exit code 1

As I saw, I could solve the problem by changing line 4 to (small letter "r" before string:
ROOTDIR = r'C:\Users\zoran'

but that is not an option for me because I am using configparser in order to read the ROOTDIR from underlying cfg file.

I need a mechanism to read the path string with single backslashes into a variable, but afterwards to escape every backslash in it. 

How to do that?


From PointedEars at web.de  Fri May  8 15:12:34 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Fri, 08 May 2015 21:12:34 +0200
Subject: IDLE Restoration
References: <08bb6733-1bc3-4f21-bba1-b97d3f73e010@googlegroups.com>
 <mailman.254.1431099341.12865.python-list@python.org>
 <4709717.b925lGYb8P@PointedEars.de>
 <mailman.257.1431101805.12865.python-list@python.org>
Message-ID: <9479712.mIojuHKHzG@PointedEars.de>

Chris Angelico wrote:

> [?] Thomas 'PointedEars' Lahn [?] wrote:
>> Chris Angelico wrote:
>>> Are you hoping to have the IDLE editor restore the fact that you had
>>> files X, Y, and Z open, or are you hoping to have the interactive
>>> interpreter retain your entire history of commands and their results?
>>> The former is reasonably plausible (and may well be possible already -
>>> Terry?), but the latter is quite impractical.
>>
>> Various shells do the latter.  I do not see why that would be
>> impractical, on the contrary.
> 
> Exactly what do they recall?

?your entire history of commands? and

> A textual form of the scrollback?

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From random832 at fastmail.us  Fri May  8 15:29:48 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 08 May 2015 15:29:48 -0400
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
In-Reply-To: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
Message-ID: <1431113388.623600.264600285.05FCF32E@webmail.messagingengine.com>

On Fri, May 8, 2015, at 15:00, zljubisicmob at gmail.com wrote:
> As I saw, I could solve the problem by changing line 4 to (small letter
> "r" before string:
> ROOTDIR = r'C:\Users\zoran'
> 
> but that is not an option for me because I am using configparser in order
> to read the ROOTDIR from underlying cfg file.

configparser won't have that problem, since "escaping" is only an issue
for python source code. No escaping for backslashes is necessary in
files read by configparser.

>>> import sys
>>> import configparser
>>> config = configparser.ConfigParser()
>>> config['DEFAULT'] = {'ROOTDIR': r'C:\Users\zoran'}
>>> config.write(sys.stdout)
[DEFAULT]
rootdir = C:\Users\zoran


From python at mrabarnett.plus.com  Fri May  8 15:33:25 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 08 May 2015 20:33:25 +0100
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
In-Reply-To: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
Message-ID: <554D0F85.5060503@mrabarnett.plus.com>

On 2015-05-08 20:00, zljubisicmob at gmail.com wrote:
> The script is very simple (abc.txt exists in ROOTDIR directory):
>
> import os
> import shutil
>
> ROOTDIR = 'C:\Users\zoran'
>
> file1 = os.path.join(ROOTDIR, 'abc.txt')
> file2 = os.path.join(ROOTDIR, 'def.txt')
>
> shutil.move(file1, file2)
>
>
> But it returns the following error:
>
>
> C:\Python34\python.exe C:/Users/bckslash_test.py
>    File "C:/Users/bckslash_test.py", line 4
>      ROOTDIR = 'C:\Users'
>               ^
> SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
>
> Process finished with exit code 1
>
> As I saw, I could solve the problem by changing line 4 to (small letter "r" before string:
> ROOTDIR = r'C:\Users\zoran'
>
> but that is not an option for me because I am using configparser in order to read the ROOTDIR from underlying cfg file.
>
> I need a mechanism to read the path string with single backslashes into a variable, but afterwards to escape every backslash in it.
>
> How to do that?
>
If you're reading the path from a file, it's not a problem. Try it!



From nagarajuusstaffing at gmail.com  Fri May  8 16:02:30 2015
From: nagarajuusstaffing at gmail.com (nagaraju thoudoju)
Date: Fri, 8 May 2015 13:02:30 -0700 (PDT)
Subject: Direct client Requirement - Immediate Hire: ETL Developer
Message-ID: <4bb895e6-e385-452c-a81c-1cb9964f0e23@googlegroups.com>

Hi,
 
Hope your are doing great,
 
Please find the requirement below and let me know you interest on this position on  nagaraju at intsystech.com or feel free to call me on my
D:908-333-3534. 
Requirement: 
Role: ETL Developer
Location: Boise Idaho 


Must be 10+ Years Experience 

. Should have strong knowledge in SQL Server database including SQL Queries, joins, functions, trigger, store procedure. 
. Should have strong knowledge of ETL concept. 
. Should have knowledge of SSIS Including Data  flow component, control flow component, SSIS Logging, SSIS Configuration, SSIS Security. SSIS Error handling. 
. Should have knowledge of SSIS package deployment. 
. Experience in Product Life Cycle Management development projects. 
. Experience in agile, sprint development projects. 
. Should have knowledge in data warehouse concept. 
. Should have basic knowledge in scripting language like C# or VB. 
 
Thanks & Best Regards....?
Raju
International Systems Technologies Inc.
10 Corporate place south.| Suite 203  |
Piscataway, NJ 08854  |
D:908-333-3534 |Fax:732-348-9533|
E-mail: nagaraju at intsystech.com   |
G-talk: nagarajuusstaffing at gmail.com
URL: www.intsystech.com


From zljubisicmob at gmail.com  Fri May  8 16:39:54 2015
From: zljubisicmob at gmail.com (zljubisicmob at gmail.com)
Date: Fri, 8 May 2015 13:39:54 -0700 (PDT)
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
In-Reply-To: <mailman.260.1431113391.12865.python-list@python.org>
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
 <mailman.260.1431113391.12865.python-list@python.org>
Message-ID: <8360473a-45ac-4270-9bf3-81932da5f223@googlegroups.com>

Thanks for clarifying.
Looks like the error message was wrong.
On windows ntfs I had a file name more than 259 characters which is widows limit.
After cutting file name to 259 characters everything works as it should.
If I cut file name to 260 characters I get the error from subject which is wrong.

Anyway case closed, thank you very much because I was suspecting that something is wrong with configparser.

Best regards.


From rosuav at gmail.com  Fri May  8 18:52:30 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 08:52:30 +1000
Subject: IDLE Restoration
In-Reply-To: <9479712.mIojuHKHzG@PointedEars.de>
References: <08bb6733-1bc3-4f21-bba1-b97d3f73e010@googlegroups.com>
 <mailman.254.1431099341.12865.python-list@python.org>
 <4709717.b925lGYb8P@PointedEars.de>
 <mailman.257.1431101805.12865.python-list@python.org>
 <9479712.mIojuHKHzG@PointedEars.de>
Message-ID: <CAPTjJmpgwE+JYzOdCKVrS=rNf82ftF87Xhg9QtQ97y6DoPMMWw@mail.gmail.com>

On Sat, May 9, 2015 at 5:12 AM, Thomas 'PointedEars' Lahn
<PointedEars at web.de> wrote:
> Chris Angelico wrote:
>
>> [?] Thomas 'PointedEars' Lahn [?] wrote:
>>> Chris Angelico wrote:
>>>> Are you hoping to have the IDLE editor restore the fact that you had
>>>> files X, Y, and Z open, or are you hoping to have the interactive
>>>> interpreter retain your entire history of commands and their results?
>>>> The former is reasonably plausible (and may well be possible already -
>>>> Terry?), but the latter is quite impractical.
>>>
>>> Various shells do the latter.  I do not see why that would be
>>> impractical, on the contrary.
>>
>> Exactly what do they recall?
>
> ?your entire history of commands? and
>
>> A textual form of the scrollback?

Okay. I'm not sure how useful a text-only scrollback would be in IDLE,
if you start with a clean slate. But I'll let the OP decide on whether
that's a worthwhile feature request.

ChrisA


From rosuav at gmail.com  Fri May  8 18:54:49 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 08:54:49 +1000
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
In-Reply-To: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
Message-ID: <CAPTjJmoJhLGAvs76_4tZeoCTd3k1CvpePfSWHAwbHTWWoty9Qw@mail.gmail.com>

On Sat, May 9, 2015 at 5:00 AM,  <zljubisicmob at gmail.com> wrote:
> But it returns the following error:
>
>
> C:\Python34\python.exe C:/Users/bckslash_test.py
>   File "C:/Users/bckslash_test.py", line 4
>     ROOTDIR = 'C:\Users'
>              ^
> SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

Strong suggestion: Use forward slashes for everything other than what
you show to a human - and maybe even then (some programs have always
printed stuff out that way - zip/unzip, for instance). The backslash
has special meaning in many contexts, and you'll just save yourself so
much trouble...

ROOTDIR = 'C:/Users/zoran'

Problem solved!

ChrisA


From rosuav at gmail.com  Fri May  8 18:58:35 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 08:58:35 +1000
Subject: Direct client Requirement - Immediate Hire: ETL Developer
In-Reply-To: <4bb895e6-e385-452c-a81c-1cb9964f0e23@googlegroups.com>
References: <4bb895e6-e385-452c-a81c-1cb9964f0e23@googlegroups.com>
Message-ID: <CAPTjJmofMXQ3-1OuXP9xZMs3GonvT7OutXqHejjKXH6sofyHvg@mail.gmail.com>

On Sat, May 9, 2015 at 6:02 AM, nagaraju thoudoju
<nagarajuusstaffing at gmail.com> wrote:
> Please find the requirement below and let me know you interest on this position on
> nagaraju at intsystech.com or feel free to call me on my
> D:908-333-3534.
> Requirement:
> Role: ETL Developer
> Location: Boise Idaho

This is spam sent on behalf of intsystech.com to a large mailing
list/newsgroup. The posted job has no mention of Python whatsoever,
and will serve only to make hundreds, perhaps thousands, of people
displeased with intsystech.com. Please deal with this as you would
deal with any other spammer abusing your domain, as otherwise we will
be dealing with your domain as we deal with any other domain abusing
our patience.

ChrisA


From rosuav at gmail.com  Fri May  8 19:02:10 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 09:02:10 +1000
Subject: Delivery Status Notification (Failure)
In-Reply-To: <047d7bd6bd96d64621051599f764@google.com>
References: <047d7bd6bd96d64621051599f764@google.com>
Message-ID: <CAPTjJmqTX4ZzW2XMdp6x8Lm4gcd2f+m2X3=rfZJKhk405qCT6w@mail.gmail.com>

On Sat, May 9, 2015 at 8:58 AM, Mail Delivery Subsystem
<mailer-daemon at googlemail.com> wrote:
> Delivery to the following recipient failed permanently:
>
>      abuse at intsystech.com
>
> Technical details of permanent failure:
> Google tried to deliver your message, but it was rejected by the server for the recipient domain intsystech.com by mx01.1and1.com. [74.208.5.21].
>
> The error that the other server returned was:
> 550 Requested action not taken: mailbox unavailable

Thanks, that saves me the trouble of wondering if they'll deal with
it. Flag as spam, folks, flag as spam. They breach protocol by not
accepting abuse at .

ChrisA


From vjp2.at at at.BioStrategist.dot.dot.com  Fri May  8 19:36:12 2015
From: vjp2.at at at.BioStrategist.dot.dot.com (vjp2.at at at.BioStrategist.dot.dot.com)
Date: Fri, 8 May 2015 23:36:12 +0000 (UTC)
Subject: Jython from bathc file?
Message-ID: <mijh9c$nv6$2@reader1.panix.com>

How do I do this in a .bat file?
Do I include the Jython or pipe it?

% CLASSPATH=$CLASSPATH:$RDBASE/Code/JavaWrappers/gmwrapper/org.RDKit.jar; jython
 -Djava.library.path=$RDBASE/Code/JavaWrappers/gmwrapper
Jython 2.2.1 on java1.6.0_20
Type "copyright", "credits" or "license" for more information.
>>> from org.RDKit import *
>>> from java import lang
>>> lang.System.loadLibrary('GraphMolWrap')
>>> m = RWMol.MolFromSmiles('c1ccccc1')
>>> m.getNumAtoms()




From tjreedy at udel.edu  Fri May  8 20:24:11 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 08 May 2015 20:24:11 -0400
Subject: IDLE Restoration
In-Reply-To: <08bb6733-1bc3-4f21-bba1-b97d3f73e010@googlegroups.com>
References: <08bb6733-1bc3-4f21-bba1-b97d3f73e010@googlegroups.com>
Message-ID: <mijk3g$r32$1@ger.gmane.org>

On 5/8/2015 11:26 AM, subhabrata.banerji at gmail.com wrote:
> Dear Group,
>
> In many applications there is a facility to restore its previous sessions, especially if they close accidentally.
>
> Does IDLE have any such facility?

No.  You are prompted to close unsaved files. Filenames are saved in 
Recent files, making it easy to reopen.

I have used Notepad++, which automatically reopens files left open when 
last closed, but I find that to be a nuisance as much as a help.  But it 
is less a nuisance in Notepad++ because it has one tabbed window instead 
of multiple windows (something I would like for Idle).

-- 
Terry Jan Reedy



From greg.ewing at canterbury.ac.nz  Fri May  8 21:03:10 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Sat, 09 May 2015 13:03:10 +1200
Subject: (OT) Re: Encrypt python files
In-Reply-To: <mailman.240.1431087248.12865.python-list@python.org>
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
 <72bd42d6-6926-44ce-bd6f-956ce40afe08@googlegroups.com>
 <mii4tu$olm$7@dont-email.me>
 <mailman.240.1431087248.12865.python-list@python.org>
Message-ID: <cr54mgFb88lU1@mid.individual.net>

Dave Angel wrote:
> It'd be clearer if you used  decryptable, since unencryptable has a very 
> different meaning.
> 
> http://en.wiktionary.org/wiki/unencryptable

The meaning is clear, but does that word have a use?
What could a piece of unencryptable data possibly
be like?

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri May  8 21:41:26 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Sat, 09 May 2015 13:41:26 +1200
Subject: functions, optional parameters
In-Reply-To: <mailman.259.1431107379.12865.python-list@python.org>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <CALwzidnkFF3DZws_5q=O6g9DFyFD5yK=BvHY2DtRoCVnZYLdyw@mail.gmail.com>
 <mailman.256.1431100965.12865.python-list@python.org>
 <554cf421$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <mailman.259.1431107379.12865.python-list@python.org>
Message-ID: <cr56u8FbnksU1@mid.individual.net>

Chris Angelico wrote:
> How do you know that the function's code
> object was created when compile() happened, rather than being created
> when the function was defined?

Python 3.4.2 (default, Feb  4 2015, 20:08:25)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> source = "def f(x = 42): pass"
 >>> code = compile(source, "", "exec")
 >>> c1 = code.co_consts[1]
 >>> c1
<code object f at 0x53d430, file "", line 1>
 >>> e = {}
 >>> exec(code, e)
 >>> c2 = e['f'].__code__
 >>> c2
<code object f at 0x53d430, file "", line 1>
 >>> c1 is c2
True

Is that proof enough for you?

-- 
Greg


From rosuav at gmail.com  Fri May  8 22:02:15 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 12:02:15 +1000
Subject: (OT) Re: Encrypt python files
In-Reply-To: <cr54mgFb88lU1@mid.individual.net>
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
 <72bd42d6-6926-44ce-bd6f-956ce40afe08@googlegroups.com>
 <mii4tu$olm$7@dont-email.me>
 <mailman.240.1431087248.12865.python-list@python.org>
 <cr54mgFb88lU1@mid.individual.net>
Message-ID: <CAPTjJmrFjz928HQzvDFog+_LbD7Vagz26MHWk-Ez_ZvCeprbHw@mail.gmail.com>

On Sat, May 9, 2015 at 11:03 AM, Gregory Ewing
<greg.ewing at canterbury.ac.nz> wrote:
> Dave Angel wrote:
>>
>> It'd be clearer if you used  decryptable, since unencryptable has a very
>> different meaning.
>>
>> http://en.wiktionary.org/wiki/unencryptable
>
>
> The meaning is clear, but does that word have a use?
> What could a piece of unencryptable data possibly
> be like?

When you make an HTTPS request to a remote server, both your IP and
the server's are unencryptable; anyone in the middle can see who is
talking to whom.

ChrisA


From rosuav at gmail.com  Fri May  8 22:05:53 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 12:05:53 +1000
Subject: functions, optional parameters
In-Reply-To: <cr56u8FbnksU1@mid.individual.net>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <CALwzidnkFF3DZws_5q=O6g9DFyFD5yK=BvHY2DtRoCVnZYLdyw@mail.gmail.com>
 <mailman.256.1431100965.12865.python-list@python.org>
 <554cf421$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <mailman.259.1431107379.12865.python-list@python.org>
 <cr56u8FbnksU1@mid.individual.net>
Message-ID: <CAPTjJmrkQ-rAHu7hpC+KzHgf-LUXHJaRmusjpfvv35XrSZDmow@mail.gmail.com>

On Sat, May 9, 2015 at 11:41 AM, Gregory Ewing
<greg.ewing at canterbury.ac.nz> wrote:
> Chris Angelico wrote:
>>
>> How do you know that the function's code
>> object was created when compile() happened, rather than being created
>> when the function was defined?
>
>
> Python 3.4.2 (default, Feb  4 2015, 20:08:25)
> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> source = "def f(x = 42): pass"
>>>> code = compile(source, "", "exec")
>>>> c1 = code.co_consts[1]
>>>> c1
> <code object f at 0x53d430, file "", line 1>
>>>> e = {}
>>>> exec(code, e)
>>>> c2 = e['f'].__code__
>>>> c2
> <code object f at 0x53d430, file "", line 1>
>>>> c1 is c2
> True
>
> Is that proof enough for you?

That's what I reached for as my first try, but it's no different from this:

>>> def f(x=42): return x + 1
...
>>> n1 = f()
>>> n2 = f(n1-1)
>>> n1 is n2
True

Clearly in this case, the "x + 1" is getting evaluated at run-time,
and yet the interpreter is welcome to intern the constants. So no, it
isn't proof - it's equally well explained by the code object being
constant.

ChrisA


From steve+comp.lang.python at pearwood.info  Fri May  8 22:37:03 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sat, 09 May 2015 12:37:03 +1000
Subject: Delivery Status Notification (Failure)
References: <047d7bd6bd96d64621051599f764@google.com>
 <mailman.265.1431126133.12865.python-list@python.org>
Message-ID: <554d72cf$0$12987$c3e8da3$5496439d@news.astraweb.com>

On Sat, 9 May 2015 09:02 am, Chris Angelico wrote:

> Thanks, that saves me the trouble of wondering if they'll deal with
> it. Flag as spam, folks, flag as spam. They breach protocol by not
> accepting abuse at .

OMG, you mean spammers ignore long standing Internet conventions for dealing
with misuse of email???

I see the original email didn't separate the body of the message from the
user's signature with a '-- ' either.

WILL THEIR CRIMES NEVER END????/??



*wink*

-- 
Steven



From steve+comp.lang.python at pearwood.info  Fri May  8 22:52:42 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sat, 09 May 2015 12:52:42 +1000
Subject: functions, optional parameters
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <CALwzidnkFF3DZws_5q=O6g9DFyFD5yK=BvHY2DtRoCVnZYLdyw@mail.gmail.com>
 <mailman.256.1431100965.12865.python-list@python.org>
 <554cf421$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <mailman.259.1431107379.12865.python-list@python.org>
Message-ID: <554d767b$0$13000$c3e8da3$5496439d@news.astraweb.com>

On Sat, 9 May 2015 03:49 am, Chris Angelico wrote:

> On Sat, May 9, 2015 at 3:36 AM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>> On Sat, 9 May 2015 02:02 am, Chris Angelico wrote:
>>> Aside from constructing two closures in the same context and proving
>>> that their __code__ attributes point to the same object, is there any
>>> way to distinguish between "code object compilation time" and "def
>>> execution time"? I just played around with it, and as far as I can
>>> tell, code objects are completely read-only.
>>
>> Sure there is. Write this Python code:
>>
>> py> source = """\
>> ... print "Function definition time."
>> ... def func():
>> ...     pass
>> ... """
>> py> print "Compile time."; code = compile(source, '', 'exec')
>> Compile time.
>> py> exec(code)
>> Function definition time.
> 
> Yes, but can you *distinguish* them in terms of default argument
> versus code object creation? How do you know that the function's code
> object was created when compile() happened, rather than being created
> when the function was defined? Is there anything that lets you in any
> way show different behaviour based on that timing difference?

I think the answer is, "yes, but it's only by peering into the
implementation".

You can read the source code of the Python compiler.

You can compile the code, and then disassemble the byte-code to see that the
code object exists but the function is assembled when the byte-code runs:


py> from dis import dis
py> code = compile("def spam(x): return x + name", "", "exec")
py> dis(code)
  1           0 LOAD_CONST               0 (<code object spam at 0xb7b88160,
file "", line 1>)
              3 LOAD_CONST               1 ('spam')
              6 MAKE_FUNCTION            0
              9 STORE_NAME               0 (spam)
             12 LOAD_CONST               2 (None)
             15 RETURN_VALUE



Contrast that to what happens with a default argument:


py> code = compile("def spam(x=name+1): return x + name", "", "exec")
py> dis(code)
  1           0 LOAD_NAME                0 (name)
              3 LOAD_CONST               0 (1)
              6 BINARY_ADD
              7 LOAD_CONST               1 (<code object spam at 0xb7bce890,
file "", line 1>)
             10 LOAD_CONST               2 ('spam')
             13 MAKE_FUNCTION            1
             16 STORE_NAME               1 (spam)
             19 LOAD_CONST               3 (None)
             22 RETURN_VALUE




-- 
Steven



From steve+comp.lang.python at pearwood.info  Fri May  8 22:53:01 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sat, 09 May 2015 12:53:01 +1000
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
 <mailman.260.1431113391.12865.python-list@python.org>
 <8360473a-45ac-4270-9bf3-81932da5f223@googlegroups.com>
Message-ID: <554d768d$0$13000$c3e8da3$5496439d@news.astraweb.com>

On Sat, 9 May 2015 06:39 am, zljubisicmob at gmail.com wrote:

> Thanks for clarifying.
> Looks like the error message was wrong.

No, the error message was right.

Your problem was that you used backslashes in *Python program code*, rather
than reading it from a text file.

In Python, a string-literal containing \U is an escape sequence which
expects exactly 8 hexadecimal digits to follow:


py> path = '~~~~\U000000a7~~~~'
py> print(path)
~~~~?~~~~

If you don't follow the \U with eight hex digits, you get an error:

py> path = '~~~~\Users~~~~'
  File "<stdin>", line 1
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in
position 4-6: truncated \UXXXXXXXX escape


This applies only to string literals in code. For data read from files,
backslash \ is just an ordinary character which has no special meaning.


> On windows ntfs I had a file name more than 259 characters which is widows
> limit. After cutting file name to 259 characters everything works as it
> should. If I cut file name to 260 characters I get the error from subject
> which is wrong.

What you describe is impossible. You cannot possibly get a SyntaxError at
compile time because the path is too long. You must have made other changes
at the same time, such as using a raw string r'C: ... \Users\ ...'.



-- 
Steven



From mynameisdesmond at gmail.com  Fri May  8 23:14:25 2015
From: mynameisdesmond at gmail.com (Desmond Lim)
Date: Fri, 08 May 2015 23:14:25 -0400
Subject: Question on Pygame
Message-ID: <D172F3C1.1381D%mynameisdesmond@gmail.com>

I am trying to install pygame on my computer to get the webcame working.
After going through the process on
http://juliaelman.com/blog/2013/04/02/installing-pygame-on-osx-mountain-lio
n/, I was able to download the required software and folders, but now I am
met with this problem below with the SDL.

I was thinking if you could provide advice on how to resolve it?

Thanks!
Desmond

--

objc[20586]: Class SDLTranslatorResponder is implemented in both
/opt/local/lib/libSDL-1.2.0.dylib and /usr/local/lib/libSDL-1.2.0.dylib.
One of the two will be used. Which one is undefined.
objc[20586]: Class SDL_QuartzWindow is implemented in both
/opt/local/lib/libSDL-1.2.0.dylib and /usr/local/lib/libSDL-1.2.0.dylib.
One of the two will be used. Which one is undefined.
objc[20586]: Class SDL_QuartzWindowDelegate is implemented in both
/opt/local/lib/libSDL-1.2.0.dylib and /usr/local/lib/libSDL-1.2.0.dylib.
One of the two will be used. Which one is undefined.
objc[20586]: Class SDL_QuartzView is implemented in both
/opt/local/lib/libSDL-1.2.0.dylib and /usr/local/lib/libSDL-1.2.0.dylib.
One of the two will be used. Which one is undefined.
Traceback (most recent call last):
  File "cam.py", line 4, in <module>
    pygame.camera.init()
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packa
ges/pygame/camera.py", line 42, in init
    from pygame import _camera
ImportError: cannot import name _camera



On 5/8/15, 11:11 PM, "python-list-request at python.org"
<python-list-request at python.org> wrote:

>Welcome to the Python-list at python.org mailing list! The purpose of
>this mailing list is to support general discussions about the Python
>programming language.  Please remember that this list is mirrored to
>the Usenet newsgroup comp.lang.python.
>
>For more information on the Python programming language see
><http://www.python.org/>
>
>To post to this list, send your message to:
>
>  python-list at python.org
>
>General information about the mailing list is at:
>
>  https://mail.python.org/mailman/listinfo/python-list
>
>If you ever want to unsubscribe or change your options (eg, switch to
>or from digest mode, change your password, etc.), visit your
>subscription page at:
>
>  
>https://mail.python.org/mailman/options/python-list/mynameisdesmond%40gmai
>l.com
>
>
>You can also make such adjustments via email by sending a message to:
>
>  Python-list-request at python.org
>
>with the word `help' in the subject or body (don't include the
>quotes), and you will get back a message with instructions.
>
>You must know your password to change your options (including changing
>the password, itself) or to unsubscribe without confirmation.  It is:
>
>  bball007
>
>Normally, Mailman will remind you of your python.org mailing list
>passwords once every month, although you can disable this if you
>prefer.  This reminder will also include instructions on how to
>unsubscribe or change your account options.  There is also a button on
>your options page that will email your current password to you.




From rosuav at gmail.com  Fri May  8 23:56:01 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 13:56:01 +1000
Subject: Delivery Status Notification (Failure)
In-Reply-To: <554d72cf$0$12987$c3e8da3$5496439d@news.astraweb.com>
References: <047d7bd6bd96d64621051599f764@google.com>
 <mailman.265.1431126133.12865.python-list@python.org>
 <554d72cf$0$12987$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmqkFNxHytnP4xPJwSVBR5uvBd1cTzZ8itvxf=Bb=tgArg@mail.gmail.com>

On Sat, May 9, 2015 at 12:37 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Sat, 9 May 2015 09:02 am, Chris Angelico wrote:
>
>> Thanks, that saves me the trouble of wondering if they'll deal with
>> it. Flag as spam, folks, flag as spam. They breach protocol by not
>> accepting abuse at .
>
> OMG, you mean spammers ignore long standing Internet conventions for dealing
> with misuse of email???
>
> I see the original email didn't separate the body of the message from the
> user's signature with a '-- ' either.
>
> WILL THEIR CRIMES NEVER END????/??

Yeah, I know, shocking. But I wanted to at least *try* doing the
normal and official thing, in the hopes that they were a legit company
that perhaps didn't realize what this looked like.

ChrisA


From shdwkeeper at gmail.com  Sat May  9 01:46:31 2015
From: shdwkeeper at gmail.com (shdwkeeper at gmail.com)
Date: Fri, 8 May 2015 22:46:31 -0700 (PDT)
Subject: SSL Socket Error
Message-ID: <b8712610-2956-4954-83dd-cdc83498f5ab@googlegroups.com>

Ive setup a Debian server with Python 2.7.9.  Have everything running and SSL as well, but when I try and login to using HTTPS:// I get this error:

Incoming web connection from ('192.168.15.177', 53202)
error: uncaptured python exception, closing channel <main.AlarmServer listening :8111 at 0x75ea7b48> (:certfile must be specified for server-side operations [/usr/lib/python2.7/asyncore.py|read|83] [/usr/lib/python2.7/asyncore.py|handle_read_event|443] [./alarmserver.py|handle_accept|456] [/usr/lib/python2.7/ssl.py|wrap_socket|891] [/usr/lib/python2.7/ssl.py|init|498])

Any ideas how to resolve?


From frank at chagford.com  Sat May  9 01:56:54 2015
From: frank at chagford.com (Frank Millman)
Date: Sat, 9 May 2015 07:56:54 +0200
Subject: Is this unpythonic?
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mailman.231.1431082398.12865.python-list@python.org>
 <554cd119$0$12977$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mik7j6$59n$1@ger.gmane.org>


"Steven D'Aprano" <steve+comp.lang.python at pearwood.info> wrote in message 
news:554cd119$0$12977$c3e8da3$5496439d at news.astraweb.com...
> On Fri, 8 May 2015 08:53 pm, Frank Millman wrote:
>
>>> Does z have to be a list? Could you use an empty tuple instead?
>>>
>>> def x(y, z=()): ...
>>>
>>
>> That was Chris' suggestion as well (thanks Chris).
>>
>> The idea appealed to me, but then I found a situation where I pass in a
>> dictionary instead of a list, so that would not work.
>
>
> Why wouldn't it work? If it worked with an empty list, it will probably 
> work
> with an empty tuple instead.
>

Sorry, I should have been more explicit. In the case of a dictionary, I used 
'def x(y, z={}'

I have not checked, but I assume that as dictionaries are mutable, this 
suffers from the same drawback as a default list.

Unlike a list, it cannot be replaced by an empty tuple without changing the 
body of the function.

Dave's suggestion would have worked here -

    EMPTY_LIST = []
    EMPTY_DICT = {}

But as I have decided to use the None trick, I use it for a default 
dictionary as well.

Frank







From djhon9813 at gmail.com  Sat May  9 02:41:52 2015
From: djhon9813 at gmail.com (david jhon)
Date: Sat, 9 May 2015 11:41:52 +0500
Subject: calling base class method fetches no results
Message-ID: <CANPUzQ1QJrg7GKRTpfvKLivG+vbNAY+gYeBSsB_zrJmX=4tRFQ@mail.gmail.com>

Hello everyone,

I am new to python and trying to run an example code from mininet tests.
Basically, I am trying to call a method in Hcontroller.py from base class
Routing defined in DCRouting.py which runs and fetches all the required
results in install_reactive_path() method, but it returns None when it is
called from _GlobalFirstFit. I hope someone here could help me fix this
bug..

I am attaching all the three files(DCRouting.py, HController.py, util.py)
to have a look into. Thanks in advance for your time, help or suggestion.
Thanks a lot!

kind regards,
David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150509/df507503/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: DCRouting.py
Type: text/x-python
Size: 3821 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150509/df507503/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: DCTopo.py
Type: text/x-python
Size: 5835 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150509/df507503/attachment-0001.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: DemandEstimation.py
Type: text/x-python
Size: 3288 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150509/df507503/attachment-0002.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: HController.py
Type: text/x-python
Size: 14245 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150509/df507503/attachment-0003.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: util.py
Type: text/x-python
Size: 387 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20150509/df507503/attachment-0004.py>

From frank at chagford.com  Sat May  9 02:51:40 2015
From: frank at chagford.com (Frank Millman)
Date: Sat, 9 May 2015 08:51:40 +0200
Subject: Is this unpythonic?
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mailman.231.1431082398.12865.python-list@python.org>
 <554cd119$0$12977$c3e8da3$5496439d@news.astraweb.com>
 <mik7j6$59n$1@ger.gmane.org>
Message-ID: <mikapr$gv3$1@ger.gmane.org>


"Frank Millman" <frank at chagford.com> wrote in message 
news:mik7j6$59n$1 at ger.gmane.org...
>
> "Steven D'Aprano" <steve+comp.lang.python at pearwood.info> wrote in message 
> news:554cd119$0$12977$c3e8da3$5496439d at news.astraweb.com...
>> On Fri, 8 May 2015 08:53 pm, Frank Millman wrote:
>>
>>>> Does z have to be a list? Could you use an empty tuple instead?
>>>>
>>>> def x(y, z=()): ...
>>>>
>>>
>>> That was Chris' suggestion as well (thanks Chris).
>>>
>>> The idea appealed to me, but then I found a situation where I pass in a
>>> dictionary instead of a list, so that would not work.
>>
>>
>> Why wouldn't it work? If it worked with an empty list, it will probably 
>> work
>> with an empty tuple instead.
>>
>
> Sorry, I should have been more explicit. In the case of a dictionary, I 
> used 'def x(y, z={}'
>
> I have not checked, but I assume that as dictionaries are mutable, this 
> suffers from the same drawback as a default list.
>
> Unlike a list, it cannot be replaced by an empty tuple without changing 
> the body of the function.
>
> Dave's suggestion would have worked here -
>
>    EMPTY_LIST = []
>    EMPTY_DICT = {}
>
> But as I have decided to use the None trick, I use it for a default 
> dictionary as well.
>

Cough, cough, I really should have given that a moment's thought before 
posting.

It just dawned on me that a dictionary *can* be replaced by an empty tuple 
without changing the body of the function.

There are two operations I might perform on the dictionary -

1. iterate over the keys and retrieve the values

2: use 'in' to test if a given string exists as a key

Both of these operations will work on a tuple and give the desired result, 
so it is a very valid workaround.

More testing needed ...

Frank





From kevin.p.dwyer at gmail.com  Sat May  9 03:00:27 2015
From: kevin.p.dwyer at gmail.com (Kev Dwyer)
Date: Sat, 09 May 2015 08:00:27 +0100
Subject: SSL Socket Error
References: <b8712610-2956-4954-83dd-cdc83498f5ab@googlegroups.com>
Message-ID: <mikbac$nme$1@ger.gmane.org>

shdwkeeper at gmail.com wrote:

> Ive setup a Debian server with Python 2.7.9.  Have everything running and
> SSL as well, but when I try and login to using HTTPS:// I get this error:
> 
> Incoming web connection from ('192.168.15.177', 53202)
> error: uncaptured python exception, closing channel <main.AlarmServer
> listening :8111 at 0x75ea7b48> (:certfile must be specified for
> server-side operations [/usr/lib/python2.7/asyncore.py|read|83]
> [/usr/lib/python2.7/asyncore.py|handle_read_event|443]
> [./alarmserver.py|handle_accept|456]
> [/usr/lib/python2.7/ssl.py|wrap_socket|891]
> [/usr/lib/python2.7/ssl.py|init|498])
> 
> Any ideas how to resolve?

It looks like the exception is coming up through alarmserver.py, which seems 
to be based on https://github.com/juggie/AlarmServer (I'm not entirely
certain because the line numbers don't seem to match).  Anyway, looking at 
the code for alarmserver.py it expects to find certificate paths defined in
its config file, but they are undefined.

The sample config file for the project includes this section:

## The server runs with SSL. You need a certificate and key
## server.crt and server.key are included but you should 
## generate your own.
## If left blank the default included cert/key will be used
#certfile=/etc/apache2/ssl/server.crt
#keyfile=/etc/apache2/ssl/server.key
certfile=
keyfile=

So I think you need to start by examining the config file on your server.

Good luck,

Kev



From breamoreboy at yahoo.co.uk  Sat May  9 03:07:13 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 09 May 2015 08:07:13 +0100
Subject: calling base class method fetches no results
In-Reply-To: <CANPUzQ1QJrg7GKRTpfvKLivG+vbNAY+gYeBSsB_zrJmX=4tRFQ@mail.gmail.com>
References: <CANPUzQ1QJrg7GKRTpfvKLivG+vbNAY+gYeBSsB_zrJmX=4tRFQ@mail.gmail.com>
Message-ID: <mikbn3$tc3$1@ger.gmane.org>

On 09/05/2015 07:41, david jhon wrote:
> Hello everyone,
>
> I am new to python and trying to run an example code from mininet tests.
> Basically, I am trying to call a method in Hcontroller.py from base class
> Routing defined in DCRouting.py which runs and fetches all the required
> results in install_reactive_path() method, but it returns None when it is
> called from _GlobalFirstFit. I hope someone here could help me fix this
> bug..
>
> I am attaching all the three files(DCRouting.py, HController.py, util.py)
> to have a look into. Thanks in advance for your time, help or suggestion.
> Thanks a lot!
>
> kind regards,
> David
>

I'm sorry but I'm not wading through nearly 30kb of code in five 
attachments.  Please see http://sscce.org/ for how to put your question 
so you're more likely to get answers.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From greg.ewing at canterbury.ac.nz  Sat May  9 03:27:56 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Sat, 09 May 2015 19:27:56 +1200
Subject: functions, optional parameters
In-Reply-To: <mailman.268.1431137161.12865.python-list@python.org>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <CALwzidnkFF3DZws_5q=O6g9DFyFD5yK=BvHY2DtRoCVnZYLdyw@mail.gmail.com>
 <mailman.256.1431100965.12865.python-list@python.org>
 <554cf421$0$12978$c3e8da3$5496439d@news.astraweb.com>
 <mailman.259.1431107379.12865.python-list@python.org>
 <cr56u8FbnksU1@mid.individual.net>
 <mailman.268.1431137161.12865.python-list@python.org>
Message-ID: <cr5r7uFg6vhU1@mid.individual.net>

Chris Angelico wrote:

> So no, it
> isn't proof - it's equally well explained by the code object being
> constant.

I suppose, strictly speaking, that's true -- but
then the code object *might as well* be created
at compile time, since the semantics are identical.

In any case, it's easy to see from the data structure
that the default values are kept in the function object:

Python 3.4.2 (default, Feb  4 2015, 20:08:25)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> y = "spam"
 >>> def f(x = y): pass
...
 >>> f.__defaults__
('spam',)

I suppose you could argue that f.__defaults__
could be a computed property that's looking inside
f.__code__ somewhere, but that would be stretching
credibility.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Sat May  9 03:47:16 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Sat, 09 May 2015 19:47:16 +1200
Subject: Is this unpythonic?
In-Reply-To: <mailman.273.1431154310.12865.python-list@python.org>
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mailman.231.1431082398.12865.python-list@python.org>
 <554cd119$0$12977$c3e8da3$5496439d@news.astraweb.com>
 <mik7j6$59n$1@ger.gmane.org>
 <mailman.273.1431154310.12865.python-list@python.org>
Message-ID: <cr5sc6FgfmiU1@mid.individual.net>

Frank Millman wrote:
> There are two operations I might perform on the dictionary -
> 
> 1. iterate over the keys and retrieve the values
> 
> 2: use 'in' to test if a given string exists as a key
> 
> Both of these operations will work on a tuple and give the desired result, 
> so it is a very valid workaround.

Although if I were reviewing a function like that,
it would probably make me pause for a moment or two
to consider why a tuple was being used as a value
for a parameter that is ostensibly supposed to be
a dict, and convince myself that it was okay.

The absolutely clearest way to write it would
probably be

    def f(things = None):
       "things is a mapping of stuff to be operated on"
       if things:
          for key in things:
             value = things[key]
             ...

A default value of None is a well-established idiom
for "this parameter is optional", and "if x:" is
idiomatic for "if I've been given an x", so writing it
that way has the best chance of passing the "pretty much
what you expect" test for good code. :-)

-- 
Greg


From djhon9813 at gmail.com  Sat May  9 03:59:20 2015
From: djhon9813 at gmail.com (david jhon)
Date: Sat, 9 May 2015 12:59:20 +0500
Subject: calling base class method fetches no results
In-Reply-To: <mikbn3$tc3$1@ger.gmane.org>
References: <CANPUzQ1QJrg7GKRTpfvKLivG+vbNAY+gYeBSsB_zrJmX=4tRFQ@mail.gmail.com>
 <mikbn3$tc3$1@ger.gmane.org>
Message-ID: <CANPUzQ0vTW6a+Umd6nXUOQ+Uds9dhHO-i44EVuAr0hYgcADv0Q@mail.gmail.com>

Hi, I am sorry for sending in five attachments, I cloned the code from here
<https://bitbucket.org/msharif/hedera/src>: Let me explain it here:

Routing Base class defined in DCRouting.py:

import logging
from copy import copy

class Routing(object):
    '''Base class for data center network routing.

    Routing engines must implement the get_route() method.
    '''

    def __init__(self, topo):
        '''Create Routing object.

        @param topo Topo object from Net parent
        '''
        self.topo = topo

    def get_route(self, src, dst, hash_):
        '''Return flow path.

        @param src source host
        @param dst destination host
        @param hash_ hash value

        @return flow_path list of DPIDs to traverse (including hosts)
        '''
        raise NotImplementedError

    def routes(self, src, dst):
        ''' Return list of paths

        Only works for Fat-Tree topology

        @ param src source host
        @ param dst destination host

        @ return list of DPIDs (including inputs)
        '''

        complete_paths = [] # List of complete dpid routes

        src_paths = { src : [[src]] }
        dst_paths = { dst : [[dst]] }

        dst_layer = self.topo.layer(dst)
        src_layer = self.topo.layer(src)

        lower_layer = src_layer
        if dst_layer > src_layer:
            lower_layer = dst_layer


        for front_layer in range(lower_layer-1, -1, -1):
            if src_layer > front_layer:
            # expand src frontier
                new_src_paths = {}
                for node in sorted(src_paths):
                    path_list = src_paths[node]
                    for path in path_list:
                        last_node = path[-1]
                        for frontier_node in
self.topo.upper_nodes(last_node):
                            new_src_paths[frontier_node] = [path +
[frontier_node]]

                            if frontier_node in dst_paths:
                                dst_path_list = dst_paths[frontier_node]
                                for dst_path in dst_path_list:
                                    dst_path_copy = copy ( dst_path )
                                    dst_path_copy.reverse()
                                    complete_paths.append( path +
dst_path_copy)
                src_paths = new_src_paths

            if dst_layer > front_layer:
            # expand dst frontier
                new_dst_paths = {}
                for node in sorted(dst_paths):
                    path_list = dst_paths[node]
                    for path in path_list:
                        last_node = path[-1]
                        for frontier_node in
self.topo.upper_nodes(last_node):
                            new_dst_paths[frontier_node] = [ path +
[frontier_node]]

                            if frontier_node in src_paths:
                                src_path_list = src_paths[frontier_node]
                                dst_path_copy = copy( path )
                                dst_path_copy.reverse()
                                for src_path in src_path_list:
                                    complete_paths.append( src_path +
dst_path_copy)

                dst_paths = new_dst_paths

            if complete_paths:
                return complete_paths
class HashedRouting(Routing):
    ''' Hashed routing '''

    def __init__(self, topo):
        self.topo = topo

    def get_route(self, src, dst, hash_):
        ''' Return flow path. '''

        if src == dst:
            return [src]

        paths = self.routes(src,dst)
        if paths:
            #print 'hash_:', hash_
            choice = hash_ % len(paths)
            #print 'choice:', choice
            path = sorted(paths)[choice]
            #print 'path:', path
            return path

============>
Instantiated in util.py:

from DCTopo import FatTreeTopo
from mininet.util import makeNumeric
from DCRouting import HashedRouting, Routing

TOPOS = {'ft': FatTreeTopo}
ROUTING = {'ECMP' : HashedRouting}


def buildTopo(topo):
    topo_name, topo_param = topo.split( ',' )
    return TOPOS[topo_name](makeNumeric(topo_param))


def getRouting(routing, topo):
    return ROUTING[routing](topo)

============================> utilized in HController. py:
A Piece of code which works with self.r.routes() method:
Following list of methods are defined in HController.py

    def _ecmp_hash(self, packet):
        ''' Return an ECMP-style 5-tuple hash for TCP/IP packets, otherwise
0.
        RFC2992 '''
        hash_input = [0] * 5
        if isinstance(packet.next, ipv4):
            ip = packet.next
            hash_input[0] = ip.srcip.toUnsigned()
            hash_input[1] = ip.dstip.toUnsigned()
            hash_input[2] = ip.protocol
            if isinstance(ip.next, tcp) or isinstance(ip.next, udp):
                l4 = ip.next
                hash_input[3] = l4.srcport
                hash_input[4] = l4.dstport
                return crc32(pack('LLHHH', *hash_input))
        return 0

    def _install_reactive_path(self, event, out_dpid, final_out_port,
packet):
        ''' Install entries on route between two switches. '''
        in_name = self.t.node_gen(dpid = event.dpid).name_str()
        out_name = self.t.node_gen(dpid = out_dpid).name_str()
        hash_ = self._ecmp_hash(packet)
        paths = self.r.routes(src_name, dst_name)
        if paths == None:
             print "PATH is None :("
             return
        route = self.r.get_route(in_name, out_name, hash_)
        print "Route:",route
        print '-'*80
        if route == None:
            print None, "route between", in_name, "and", out_name
            return

        match = of.ofp_match.from_packet(packet)

        for i, node in enumerate(route):
            node_dpid = self.t.node_gen(name = node).dpid
            if i < len(route) - 1:
                next_node = route[i + 1]
                out_port, next_in_port = self.t.port(node, next_node)
            else:
                out_port = final_out_port
            self.switches[node_dpid].install(out_port, match, idle_timeout
= 10)

        if isinstance(packet.next, of.ipv4) and
isinstance(packet.next.next, of.tcp):
            self.matchDict[(packet.next.srcip, packet.next.dstip,
packet.next.next.srcport, packet.next.next.dstport)] = (route, match)

    def _handle_PacketIn(self, event):
        if not self.all_switches_up:
            #log.info("Saw PacketIn before all switches were up -
ignoring." )
            return

        packet = event.parsed
        dpid = event.dpid
        in_port = event.port

        # Learn MAC address of the sender on every packet-in.
        self.macTable[packet.src] = (dpid, in_port)
        sw_name = self.t.node_gen(dpid = dpid).name_str()
        #print "Sw:", sw_name, packet.src, packet.dst,"port", in_port,
packet.dst.isMulticast(),"macTable", packet.dst in self.macTable
        #print '-'*80

        # Insert flow, deliver packet directly to destination.

        if packet.dst in self.macTable:
            out_dpid, out_port = self.macTable[packet.dst]
            self._install_reactive_path(event, out_dpid, out_port, packet)

            self.switches[out_dpid].send_packet_data(out_port, event.data)

        else:
            self._flood(event)

===================> code snippet which returns 'None' number of paths.
    def _GlobalFirstFit(self,flow):
        '''do the Hedera global first fit here'''
        src_name = self.t.node_gen(dpid = flow['src']).name_str()
        dst_name = self.t.node_gen(dpid = flow['dst']).name_str()
        print 'Global Fisrt Fit for the elephant flow from ',src_name,'to',
dst_name
        paths = self.r.routes(src_name,dst_name)
        print 'all routes found for the big flow:\n',paths
        GFF_route = None
        if paths == None:
           return
        else:
          for path in paths:
            fitCheck = True

            for i in range(1,len(path)):
                fitCheck = False
                if self.bwReservation.has_key(path[i-1]) and
self.bwReservation[path[i-1]].has_key(path[i]):
                    if
self.bwReservation[path[i-1]][path[i]]['reserveDemand'] + flow['demand'] >
1 :
                        break
                    else:

#self.bwReservation[path[i-1]][path[i]]['reserveDemand'] += flow['demand']
                        fitCheck = True
                else:
                    self.bwReservation[path[i-1]]={}

self.bwReservation[path[i-1]][path[i]]={'reserveDemand':0}
                    fitCheck = True
            if fitCheck == True:
                for i in range(1,len(path)):
                    self.bwReservation[path[i-1]][path[i]]['reserveDemand']
+= flow['demand']
                GFF_route = path
                print "GFF route found:", path
                break
        if GFF_route != None:
            """install new GFF_path between source and destintaion"""
            self. _install_GFF_path(GFF_route,flow['match'])

def launch(topo = None, routing = None, bw = None ):
    #print topo
    if not topo:
        raise Exception ("Please specify the topology")
    else:
        t = buildTopo(topo)

    r = getRouting(routing, t)
    if bw == None:
        bw = 10.0 #Mb/s
        bw = float(bw/1000) #Gb/s
    else:
        bw = float(bw)/1000
    core.registerNew(HController, t, r, bw)
    log.info("** HController is running

I am really sorry for any inconvenience caused. I, ve tried to make it a
bit clear here. I am not even able to debug the code by setting a python
debugging point pdb. I need help from you guys. Thanks a lot again for your
time and help.

Best Regards,
David


On Sat, May 9, 2015 at 12:07 PM, Mark Lawrence <breamoreboy at yahoo.co.uk>
wrote:

> On 09/05/2015 07:41, david jhon wrote:
>
>> Hello everyone,
>>
>> I am new to python and trying to run an example code from mininet tests.
>> Basically, I am trying to call a method in Hcontroller.py from base class
>> Routing defined in DCRouting.py which runs and fetches all the required
>> results in install_reactive_path() method, but it returns None when it is
>> called from _GlobalFirstFit. I hope someone here could help me fix this
>> bug..
>>
>> I am attaching all the three files(DCRouting.py, HController.py, util.py)
>> to have a look into. Thanks in advance for your time, help or suggestion.
>> Thanks a lot!
>>
>> kind regards,
>> David
>>
>>
> I'm sorry but I'm not wading through nearly 30kb of code in five
> attachments.  Please see http://sscce.org/ for how to put your question
> so you're more likely to get answers.
>
> --
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
>
> Mark Lawrence
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150509/8466c240/attachment.html>

From breamoreboy at yahoo.co.uk  Sat May  9 04:11:45 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 09 May 2015 09:11:45 +0100
Subject: calling base class method fetches no results
In-Reply-To: <CANPUzQ0vTW6a+Umd6nXUOQ+Uds9dhHO-i44EVuAr0hYgcADv0Q@mail.gmail.com>
References: <CANPUzQ1QJrg7GKRTpfvKLivG+vbNAY+gYeBSsB_zrJmX=4tRFQ@mail.gmail.com>
 <mikbn3$tc3$1@ger.gmane.org>
 <CANPUzQ0vTW6a+Umd6nXUOQ+Uds9dhHO-i44EVuAr0hYgcADv0Q@mail.gmail.com>
Message-ID: <mikfg3$jap$1@ger.gmane.org>

On 09/05/2015 08:59, david jhon wrote:
> Hi, I am sorry for sending in five attachments, I cloned the code from here
> <https://bitbucket.org/msharif/hedera/src>: Let me explain it here:
>

[nearly 300 lines of code snipped]

> I am really sorry for any inconvenience caused. I, ve tried to make it a
> bit clear here. I am not even able to debug the code by setting a python
> debugging point pdb. I need help from you guys. Thanks a lot again for your
> time and help.
>
> Best Regards,
> David

You clearly didn't bother to read this http://sscce.org/ about how to 
put a question so have another go.  Three strikes and you're out.

And please don't top post here, it's extremely annoying when trying to 
follow long threads.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From Cecil at decebal.nl  Sat May  9 04:28:32 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 09 May 2015 10:28:32 +0200
Subject: Useful module to be written by a newbie
References: <87y4lbasvf.fsf@Equus.decebal.nl>
 <mailman.88.1430324895.3680.python-list@python.org>
 <87pp6mc100.fsf@Equus.decebal.nl>
 <mailman.93.1430334242.3680.python-list@python.org>
 <87h9rybr0w.fsf@Equus.decebal.nl> <00l02cx1i1.ln2@news.c0t0d0s0.de>
Message-ID: <87a8xexihb.fsf@Equus.decebal.nl>

Op Saturday 9 May 2015 08:10 CEST schreef Michael Welle:

> Cecil Westerhof <Cecil at decebal.nl> writes:
>
>> Op Wednesday 29 Apr 2015 21:03 CEST schreef Peter Otten:
>>
>>>>> Realistically a Python coder with a little experience will have
>>>>> a glance at your code and run away.
>>>>
>>>> Oops, that is not nice to hear. :'-( 
>>>
>>> Sorry, I did not mean to discourage you or insult you, I just
>>> wanted to make it clear that your code is not there yet.
>>
>> You did not. Of-course it is not nice to hear, but if it is true,
>> it is very useful. If there is a lot to be desired, then it is good
>> when someone point this out.
>>
>>
>>>> But can you enlighten me? Then I can learn from it.
> learning a new language looks like an easy job, in most cases. All
> the language's keywords and stuff, you can shuffle that into your
> head in a weekend or so. But what it makes it a hard task is all the
> idioms. It takes a long time to learn them. I like your approach of
> hacking random algorithms, like happynumbers and friends, (everyone
> does it to learn a new language I think) and show them for criticism
> (not everyone does that).

Well in my experience the fastest way to learn something is let people
?burn you down?. Of-course you need to be able to take it. Also
important: ?C'est le ton qui fait la musique?. But no problems here.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sat May  9 04:37:47 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 09 May 2015 10:37:47 +0200
Subject: Seralization
Message-ID: <876182xi1w.fsf@Equus.decebal.nl>

To make serialization a bit easier I made a few functions to get, save
and convert between the different types. As I see it pickle and json
are probably used the most. I also have a get and save for marshal.
But no conversion to marshal, because in principle you should not use
it, so a conversion to it is not useful. I did define conversion from.

Are there other seralizations that is handy to take care of?

I understood that because the differences between platforms it is
better to do the file operatins in binary mode. Is that true?


The code:
    def get_json(json_file):
        with open(json_file, 'rb') as in_f:
            return json.load(in_f)

    def get_marshal(marshal_file):
        with open(marshal_file, 'rb') as in_f:
            return marshal.load(in_f)

    def get_pickle(pickle_file):
        with open(pickle_file, 'rb') as in_f:
            return pickle.load(in_f)

    def save_json(data, json_file):
        with open(json_file, 'wb') as out_f:
            json.dump(data, out_f)

    def save_marshal(data, marshal_file):
        with open(marshal_file, 'wb') as out_f:
            marshal.dump(data, out_f)

    def save_pickle(data, pickle_file):
        with open(pickle_file, 'wb') as out_f:
            pickle.dump(data, out_f)

    def marshal_to_pickle(marshal_file, pickle_file):
        data_in = get_marshal(marshal_file)
        save_pickle(data_in, pickle_file)
        data_out = get_pickle(pickle_file)
        if data_in != data_out:
            raise SerializationError('Serialization from {0} to {1} not succesfull'.
                                     format(marshal_file, pickle_file))

    def marshal_to_json(marshal_file, json_file):
        data_in = get_marshal(marshal_file)
        save_json(data_in, json_file)
        data_out = get_json(json_file)
        if data_in != data_out:
            raise SerializationError('Serialization from {0} to {1} not succesfull'.
                                     format(marshal_file, json_file))

    def pickle_to_json(pickle_file, json_file):
        data_in = get_pickle(pickle_file)
        save_json(data_in, json_file)
        data_out = get_json(json_file)
        if data_in != data_out:
            raise SerializationError('Serialization from {0} to {1} not succesfull'.
                                     format(pickle_file, json_file))

    def json_to_pickle(json_file, pickle_file):
        data_in = get_json(json_file)
        save_pickle(data_in, pickle_file)
        data_out = get_pickle(pickle_file)
        if data_in == data_out:
            raise SerializationError('Serialization from {0} to {1} not succesfull'.
                                     format(json_file, pickle_file))

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From frank at chagford.com  Sat May  9 05:10:00 2015
From: frank at chagford.com (Frank Millman)
Date: Sat, 9 May 2015 11:10:00 +0200
Subject: Is this unpythonic?
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mailman.231.1431082398.12865.python-list@python.org>
 <554cd119$0$12977$c3e8da3$5496439d@news.astraweb.com>
 <mik7j6$59n$1@ger.gmane.org>
 <mailman.273.1431154310.12865.python-list@python.org>
 <cr5sc6FgfmiU1@mid.individual.net>
Message-ID: <mikit6$3th$1@ger.gmane.org>


"Gregory Ewing" <greg.ewing at canterbury.ac.nz> wrote in message 
news:cr5sc6FgfmiU1 at mid.individual.net...
> Frank Millman wrote:
>
> The absolutely clearest way to write it would
> probably be
>
>    def f(things = None):
>       "things is a mapping of stuff to be operated on"
>       if things:
>          for key in things:
>             value = things[key]
>             ...
>
> A default value of None is a well-established idiom
> for "this parameter is optional", and "if x:" is
> idiomatic for "if I've been given an x", so writing it
> that way has the best chance of passing the "pretty much
> what you expect" test for good code. :-)
>

Thanks, Greg, that makes a lot of sense.

My original method of using 'z=[]' worked, but would cause a reviewer to 
stop and think about it for a moment.

Changing it to 'z=()' would have pretty much the same effect.

Using 'z=None' would cause the least hesitation, and is therefore I think 
the most pythonic.

It does require an additional test every time the function is called, but 
the effect of that in my application is virtually zero.

If the overhead did become an issue, Dave's suggestion of 'z=EMPTY_LIST' 
would be a good solution.

Frank





From rosuav at gmail.com  Sat May  9 05:16:54 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 9 May 2015 19:16:54 +1000
Subject: Seralization
In-Reply-To: <876182xi1w.fsf@Equus.decebal.nl>
References: <876182xi1w.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmoBgiTeSp27_QnZ9tJ7pULtrM9VdDJ8DfU5GNXhWCgS1Q@mail.gmail.com>

On Sat, May 9, 2015 at 6:37 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> The code:
>     def get_json(json_file):
>         with open(json_file, 'rb') as in_f:
>             return json.load(in_f)
>
>     def get_marshal(marshal_file):
>         with open(marshal_file, 'rb') as in_f:
>             return marshal.load(in_f)
>
>     def get_pickle(pickle_file):
>         with open(pickle_file, 'rb') as in_f:
>             return pickle.load(in_f)

def get_any(format, filename):
    with open(filename, 'rb') as in_f:
        return format.load(in_f)

>     def save_json(data, json_file):
>         with open(json_file, 'wb') as out_f:
>             json.dump(data, out_f)
>
>     def save_marshal(data, marshal_file):
>         with open(marshal_file, 'wb') as out_f:
>             marshal.dump(data, out_f)
>
>     def save_pickle(data, pickle_file):
>         with open(pickle_file, 'wb') as out_f:
>             pickle.dump(data, out_f)

def save_any(format, data, filename):
    with open(filename,'wb') as out_f:
        format.dump(data, out_f)

>     def marshal_to_pickle(marshal_file, pickle_file):
>         data_in = get_marshal(marshal_file)
>         save_pickle(data_in, pickle_file)
>         data_out = get_pickle(pickle_file)
>         if data_in != data_out:
>             raise SerializationError('Serialization from {0} to {1} not succesfull'.
>                                      format(marshal_file, pickle_file))
>
>     def marshal_to_json(marshal_file, json_file):
>         data_in = get_marshal(marshal_file)
>         save_json(data_in, json_file)
>         data_out = get_json(json_file)
>         if data_in != data_out:
>             raise SerializationError('Serialization from {0} to {1} not succesfull'.
>                                      format(marshal_file, json_file))
>
>     def pickle_to_json(pickle_file, json_file):
>         data_in = get_pickle(pickle_file)
>         save_json(data_in, json_file)
>         data_out = get_json(json_file)
>         if data_in != data_out:
>             raise SerializationError('Serialization from {0} to {1} not succesfull'.
>                                      format(pickle_file, json_file))

def any_to_any(fmt1, fmt2, fn1, fn2):
    data_in = get_any(fmt1, fn1)
    save_any(fmt2, data_in, fn2)
    data_out = get_any(fmt2, fn2)
    if data_in != data_out:
        raise SerializationError('Serialization from {0} to {1} not successful'.
                                     format(fn1, fn2))

formats = [json, pickle, marshal]
for fmt1 in formats:
    for fmt2 in formats:
        globals()["%s_to_%s" % (fmt1.__name__, fmt2.__name__)] = \
            functools.partial(any_to_any, fmt1, fmt2)

>     def json_to_pickle(json_file, pickle_file):
>         data_in = get_json(json_file)
>         save_pickle(data_in, pickle_file)
>         data_out = get_pickle(pickle_file)
>         if data_in == data_out:
>             raise SerializationError('Serialization from {0} to {1} not succesfull'.
>                                      format(json_file, pickle_file))

def json_to_pickle(json_file, pickle_file):
    try:
        any_to_any(json, pickle, json_file, pickle_file)
    except SerializationError:
        pass
    else:
        raise SerializationError('Serialization from {0} to {1} not successful'.
            format(json_file, pickle_file))

There. Much simpler. And maybe you can see that the last one actually
shouldn't exist :)

ChrisA


From tlvargo at sbcglobal.netz  Sat May  9 06:17:12 2015
From: tlvargo at sbcglobal.netz (Todd Vargo)
Date: Sat, 09 May 2015 06:17:12 -0400
Subject: Jython from bathc file?
In-Reply-To: <mijh9c$nv6$2@reader1.panix.com>
References: <mijh9c$nv6$2@reader1.panix.com>
Message-ID: <mikmrd$f3q$1@news.albasani.net>

On 5/8/2015 7:36 PM, vjp2.at at at.BioStrategist.dot.dot.com wrote:
> How do I do this in a .bat file?
> Do I include the Jython or pipe it?
>
> % CLASSPATH=$CLASSPATH:$RDBASE/Code/JavaWrappers/gmwrapper/org.RDKit.jar; jython
>   -Djava.library.path=$RDBASE/Code/JavaWrappers/gmwrapper
> Jython 2.2.1 on java1.6.0_20
> Type "copyright", "credits" or "license" for more information.
>>>> from org.RDKit import *
>>>> from java import lang
>>>> lang.System.loadLibrary('GraphMolWrap')
>>>> m = RWMol.MolFromSmiles('c1ccccc1')
>>>> m.getNumAtoms()
>

This does not do that but for those who don't know Jython it can help.

@echo off
set "x=thequickbrownfoxjumpsoverthelazydog"
set "x1=%x:~11,1%%x:~1,1%%x:~29,1%%x:~0,1%"
set "x2= %x:~32,2%%x:~2,1%%x:~20,1%"
set "x3= %x:~5,1%%x:~0,1% %x:~32,2%"
echo %x1%%x2%%x3%?
pause>nul

-- 
Todd Vargo
(Post questions to group only. Remove "z" to email personal messages)


From steve+comp.lang.python at pearwood.info  Sat May  9 06:29:53 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sat, 09 May 2015 20:29:53 +1000
Subject: Jython from bathc file?
References: <mijh9c$nv6$2@reader1.panix.com>
Message-ID: <554de1a2$0$13006$c3e8da3$5496439d@news.astraweb.com>

On Sat, 9 May 2015 09:36 am, vjp2.at at at.BioStrategist.dot.dot.com wrote:

> How do I do this in a .bat file?
> Do I include the Jython or pipe it?
> 
> % CLASSPATH=$CLASSPATH:$RDBASE/Code/JavaWrappers/gmwrapper/org.RDKit.jar;
> jython
>  -Djava.library.path=$RDBASE/Code/JavaWrappers/gmwrapper
> Jython 2.2.1 on java1.6.0_20
> Type "copyright", "credits" or "license" for more information.
>>>> from org.RDKit import *
>>>> from java import lang
>>>> lang.System.loadLibrary('GraphMolWrap')
>>>> m = RWMol.MolFromSmiles('c1ccccc1')
>>>> m.getNumAtoms()


I'm not sure I fully understand your question, but perhaps this will help.

I haven't used Windows for anything like this for decades, but I would
expect something like this:


(1) Create a file containing your Python code and call it "myscript.py" (or
any other meaningful name):


from org.RDKit import *
from java import lang
lang.System.loadLibrary('GraphMolWrap')
m = RWMol.MolFromSmiles('c1ccccc1')
m.getNumAtoms()


(2) Create a .bat file which calls Jython. I don't remember .bat syntax, but
perhaps it is something like this?


CLASSPATH=$CLASSPATH:$RDBASE/Code/JavaWrappers/gmwrapper/org.RDKit.jar;
jython -Djava.library.path=$RDBASE/Code/JavaWrappers/gmwrapper myscript.py


And that's it.

Does that work? Any Windows users like to comment?



-- 
Steven



From zljubisicmob at gmail.com  Sat May  9 06:31:08 2015
From: zljubisicmob at gmail.com (zljubisicmob at gmail.com)
Date: Sat, 9 May 2015 03:31:08 -0700 (PDT)
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
In-Reply-To: <554d768d$0$13000$c3e8da3$5496439d@news.astraweb.com>
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
 <mailman.260.1431113391.12865.python-list@python.org>
 <8360473a-45ac-4270-9bf3-81932da5f223@googlegroups.com>
 <554d768d$0$13000$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <580ee0d6-a703-4da3-af2d-105589a1780f@googlegroups.com>

Steven,

please do look at the code bellow:

# C:\Users\zoran\PycharmProjects\mm_align\hrt3.cfg contents
# [Dir]
# ROOTDIR = C:\Users\zoran\hrt


import os
import shutil
import configparser
import requests
import re

Config = configparser.ConfigParser()
Config.optionxform = str # preserve case in ini file
cfg_file = os.path.join('C:\\Users\\zoran\\PycharmProjects\\mm_align\\hrt3.cfg' )
Config.read(cfg_file)



ROOTDIR = Config.get('Dir', 'ROOTDIR')

print(ROOTDIR)

html = requests.get("http://radio.hrt.hr/prvi-program/arhiva/ujutro-prvi-poligraf-politicki-grafikon/118/").text

art_html = re.search('<article id="aod_0">(.+?)</article>', html, re.DOTALL).group(1)
for p_tag in re.finditer(r'<p>(.*?)</p>', art_html, re.DOTALL):
    if '<strong>' not in p_tag.group(1):
        title = p_tag.group(1)

title = title[:232]
title = title.replace(" ", "_").replace("/", "_").replace("!", "_").replace("?", "_")\
                    .replace('"', "_").replace(':', "_").replace(',', "_").replace('&#34;', '')\
                    .replace('\n', '_').replace('&#39', '')

print(title)

src_file = os.path.join(ROOTDIR, 'src_' + title + '.txt')
dst_file = os.path.join(ROOTDIR, 'des_' + title + '.txt')

print(len(src_file), src_file)
print(len(dst_file), dst_file)

with open(src_file, mode='w', encoding='utf-8') as s_file:
    s_file.write('test')


shutil.move(src_file, dst_file)

It works, but if you change title = title[:232] to title = title[:233], you will get "FileNotFoundError: [Errno 2] No such file or directory".
As you can see ROOTDIR contains \U.

Regards.


From davea at davea.name  Sat May  9 07:36:43 2015
From: davea at davea.name (Dave Angel)
Date: Sat, 09 May 2015 07:36:43 -0400
Subject: calling base class method fetches no results
In-Reply-To: <CANPUzQ0vTW6a+Umd6nXUOQ+Uds9dhHO-i44EVuAr0hYgcADv0Q@mail.gmail.com>
References: <CANPUzQ1QJrg7GKRTpfvKLivG+vbNAY+gYeBSsB_zrJmX=4tRFQ@mail.gmail.com>
 <mikbn3$tc3$1@ger.gmane.org>
 <CANPUzQ0vTW6a+Umd6nXUOQ+Uds9dhHO-i44EVuAr0hYgcADv0Q@mail.gmail.com>
Message-ID: <554DF14B.6020604@davea.name>

On 05/09/2015 03:59 AM, david jhon wrote:
> Hi, I am sorry for sending in five attachments, I cloned the code from here
> <https://bitbucket.org/msharif/hedera/src>: Let me explain it here:
>

Please don't top-post.  Your earlier problem description, which I could 
make no sense of, is now located after your later "clarification".

Thanks for eliminating the attachments.  Many cannot see them.  And for 
extracting only part of the code into the message.  It's still too much 
for me, but others may manage it okay.  To me, it seems likely that most 
of that code will not have any part in the problem you're describing. 
And in some places you have code that's missing its context.

Now, eliminate the pieces of code that are irrelevant to your question, 
and state the problem in terms that make sense.  Somebody is 
instantiating an object.  Exactly which class is being used for that? 
Somebody else is calling a particular method (be specific, rather than 
just saying "some method"), and it's giving the wrong results.  And 
apparently, those wrong results depend on which source file something 
happens in.


> Routing Base class defined in DCRouting.py:
>
> import logging
> from copy import copy
>
> class Routing(object):
>      '''Base class for data center network routing.
>
>      Routing engines must implement the get_route() method.
>      '''
>
>      def __init__(self, topo):
>          '''Create Routing object.
>
>          @param topo Topo object from Net parent
>          '''
>          self.topo = topo
>
>      def get_route(self, src, dst, hash_):
>          '''Return flow path.
>
>          @param src source host
>          @param dst destination host
>          @param hash_ hash value
>
>          @return flow_path list of DPIDs to traverse (including hosts)
>          '''
>          raise NotImplementedError
>
>      def routes(self, src, dst):
>          ''' Return list of paths
>
>          Only works for Fat-Tree topology
>
>          @ param src source host
>          @ param dst destination host
>
>          @ return list of DPIDs (including inputs)
>          '''
>
>          complete_paths = [] # List of complete dpid routes
>
>          src_paths = { src : [[src]] }
>          dst_paths = { dst : [[dst]] }
>
>          dst_layer = self.topo.layer(dst)
>          src_layer = self.topo.layer(src)
>
>          lower_layer = src_layer
>          if dst_layer > src_layer:
>              lower_layer = dst_layer
>
>
>          for front_layer in range(lower_layer-1, -1, -1):
>              if src_layer > front_layer:
>              # expand src frontier
>                  new_src_paths = {}
>                  for node in sorted(src_paths):
>                      path_list = src_paths[node]
>                      for path in path_list:
>                          last_node = path[-1]
>                          for frontier_node in
> self.topo.upper_nodes(last_node):
>                              new_src_paths[frontier_node] = [path +
> [frontier_node]]
>
>                              if frontier_node in dst_paths:
>                                  dst_path_list = dst_paths[frontier_node]
>                                  for dst_path in dst_path_list:
>                                      dst_path_copy = copy ( dst_path )
>                                      dst_path_copy.reverse()
>                                      complete_paths.append( path +
> dst_path_copy)
>                  src_paths = new_src_paths
>
>              if dst_layer > front_layer:
>              # expand dst frontier
>                  new_dst_paths = {}
>                  for node in sorted(dst_paths):
>                      path_list = dst_paths[node]
>                      for path in path_list:
>                          last_node = path[-1]
>                          for frontier_node in
> self.topo.upper_nodes(last_node):
>                              new_dst_paths[frontier_node] = [ path +
> [frontier_node]]
>
>                              if frontier_node in src_paths:
>                                  src_path_list = src_paths[frontier_node]
>                                  dst_path_copy = copy( path )
>                                  dst_path_copy.reverse()
>                                  for src_path in src_path_list:
>                                      complete_paths.append( src_path +
> dst_path_copy)
>
>                  dst_paths = new_dst_paths
>
>              if complete_paths:
>                  return complete_paths
> class HashedRouting(Routing):
>      ''' Hashed routing '''
>
>      def __init__(self, topo):
>          self.topo = topo
>
>      def get_route(self, src, dst, hash_):
>          ''' Return flow path. '''
>
>          if src == dst:
>              return [src]
>
>          paths = self.routes(src,dst)
>          if paths:
>              #print 'hash_:', hash_
>              choice = hash_ % len(paths)
>              #print 'choice:', choice
>              path = sorted(paths)[choice]
>              #print 'path:', path
>              return path
>
> ============>
> Instantiated in util.py:
>
> from DCTopo import FatTreeTopo
> from mininet.util import makeNumeric
> from DCRouting import HashedRouting, Routing
>
> TOPOS = {'ft': FatTreeTopo}
> ROUTING = {'ECMP' : HashedRouting}
>
>
> def buildTopo(topo):
>      topo_name, topo_param = topo.split( ',' )
>      return TOPOS[topo_name](makeNumeric(topo_param))
>
>
> def getRouting(routing, topo):
>      return ROUTING[routing](topo)

A comment on that last line that says it instantiates the child class 
would have been very helpful.

But even though I can now search for calls to that function, I still 
cannot make sense out of that context, either.

>
> ============================> utilized in HController. py:
> A Piece of code which works with self.r.routes() method:
> Following list of methods are defined in HController.py
>
>      def _ecmp_hash(self, packet):
>          ''' Return an ECMP-style 5-tuple hash for TCP/IP packets, otherwise
> 0.
>          RFC2992 '''
>          hash_input = [0] * 5
>          if isinstance(packet.next, ipv4):
>              ip = packet.next
>              hash_input[0] = ip.srcip.toUnsigned()
>              hash_input[1] = ip.dstip.toUnsigned()
>              hash_input[2] = ip.protocol
>              if isinstance(ip.next, tcp) or isinstance(ip.next, udp):
>                  l4 = ip.next
>                  hash_input[3] = l4.srcport
>                  hash_input[4] = l4.dstport
>                  return crc32(pack('LLHHH', *hash_input))
>          return 0
>
>      def _install_reactive_path(self, event, out_dpid, final_out_port,
> packet):
>          ''' Install entries on route between two switches. '''
>          in_name = self.t.node_gen(dpid = event.dpid).name_str()
>          out_name = self.t.node_gen(dpid = out_dpid).name_str()
>          hash_ = self._ecmp_hash(packet)
>          paths = self.r.routes(src_name, dst_name)
>          if paths == None:
>               print "PATH is None :("
>               return
>          route = self.r.get_route(in_name, out_name, hash_)
>          print "Route:",route
>          print '-'*80
>          if route == None:
>              print None, "route between", in_name, "and", out_name
>              return
>
>          match = of.ofp_match.from_packet(packet)
>
>          for i, node in enumerate(route):
>              node_dpid = self.t.node_gen(name = node).dpid
>              if i < len(route) - 1:
>                  next_node = route[i + 1]
>                  out_port, next_in_port = self.t.port(node, next_node)
>              else:
>                  out_port = final_out_port
>              self.switches[node_dpid].install(out_port, match, idle_timeout
> = 10)
>
>          if isinstance(packet.next, of.ipv4) and
> isinstance(packet.next.next, of.tcp):
>              self.matchDict[(packet.next.srcip, packet.next.dstip,
> packet.next.next.srcport, packet.next.next.dstport)] = (route, match)
>
>      def _handle_PacketIn(self, event):
>          if not self.all_switches_up:
>              #log.info("Saw PacketIn before all switches were up -
> ignoring." )
>              return
>
>          packet = event.parsed
>          dpid = event.dpid
>          in_port = event.port
>
>          # Learn MAC address of the sender on every packet-in.
>          self.macTable[packet.src] = (dpid, in_port)
>          sw_name = self.t.node_gen(dpid = dpid).name_str()
>          #print "Sw:", sw_name, packet.src, packet.dst,"port", in_port,
> packet.dst.isMulticast(),"macTable", packet.dst in self.macTable
>          #print '-'*80
>
>          # Insert flow, deliver packet directly to destination.
>
>          if packet.dst in self.macTable:
>              out_dpid, out_port = self.macTable[packet.dst]
>              self._install_reactive_path(event, out_dpid, out_port, packet)
>
>              self.switches[out_dpid].send_packet_data(out_port, event.data)
>
>          else:
>              self._flood(event)
>
> ===================> code snippet which returns 'None' number of paths.
>      def _GlobalFirstFit(self,flow):
>          '''do the Hedera global first fit here'''
>          src_name = self.t.node_gen(dpid = flow['src']).name_str()
>          dst_name = self.t.node_gen(dpid = flow['dst']).name_str()
>          print 'Global Fisrt Fit for the elephant flow from ',src_name,'to',
> dst_name
>          paths = self.r.routes(src_name,dst_name)
>          print 'all routes found for the big flow:\n',paths
>          GFF_route = None
>          if paths == None:
>             return
>          else:
>            for path in paths:
>              fitCheck = True
>
>              for i in range(1,len(path)):
>                  fitCheck = False
>                  if self.bwReservation.has_key(path[i-1]) and
> self.bwReservation[path[i-1]].has_key(path[i]):
>                      if
> self.bwReservation[path[i-1]][path[i]]['reserveDemand'] + flow['demand'] >
> 1 :
>                          break
>                      else:
>
> #self.bwReservation[path[i-1]][path[i]]['reserveDemand'] += flow['demand']
>                          fitCheck = True
>                  else:
>                      self.bwReservation[path[i-1]]={}
>
> self.bwReservation[path[i-1]][path[i]]={'reserveDemand':0}
>                      fitCheck = True
>              if fitCheck == True:
>                  for i in range(1,len(path)):
>                      self.bwReservation[path[i-1]][path[i]]['reserveDemand']
> += flow['demand']
>                  GFF_route = path
>                  print "GFF route found:", path
>                  break
>          if GFF_route != None:
>              """install new GFF_path between source and destintaion"""
>              self. _install_GFF_path(GFF_route,flow['match'])
>
> def launch(topo = None, routing = None, bw = None ):
>      #print topo
>      if not topo:
>          raise Exception ("Please specify the topology")
>      else:
>          t = buildTopo(topo)
>
>      r = getRouting(routing, t)
>      if bw == None:
>          bw = 10.0 #Mb/s
>          bw = float(bw/1000) #Gb/s
>      else:
>          bw = float(bw)/1000
>      core.registerNew(HController, t, r, bw)
>      log.info("** HController is running
>
> I am really sorry for any inconvenience caused. I, ve tried to make it a
> bit clear here. I am not even able to debug the code by setting a python
> debugging point pdb. I need help from you guys. Thanks a lot again for your
> time and help.
>
> Best Regards,
> David
>
>
> On Sat, May 9, 2015 at 12:07 PM, Mark Lawrence <breamoreboy at yahoo.co.uk>
> wrote:
>
>> On 09/05/2015 07:41, david jhon wrote:
>>
>>> Hello everyone,
>>>
>>> I am new to python and trying to run an example code from mininet tests.
>>> Basically, I am trying to call a method in Hcontroller.py from base class
>>> Routing defined in DCRouting.py which runs and fetches all the required
>>> results in install_reactive_path() method, but it returns None when it is
>>> called from _GlobalFirstFit. I hope someone here could help me fix this
>>> bug..
>>>
>>> I am attaching all the three files(DCRouting.py, HController.py, util.py)
>>> to have a look into. Thanks in advance for your time, help or suggestion.
>>> Thanks a lot!
>>>
>>> kind regards,
>>> David
>>>
>>>
>> I'm sorry but I'm not wading through nearly 30kb of code in five
>> attachments.  Please see http://sscce.org/ for how to put your question
>> so you're more likely to get answers.
>>
>> --
>> My fellow Pythonistas, ask not what our language can do for you, ask
>> what you can do for our language.
>>
>> Mark Lawrence
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
>
>


-- 
DaveA


From Cecil at decebal.nl  Sat May  9 07:38:16 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 09 May 2015 13:38:16 +0200
Subject: Seralization
References: <876182xi1w.fsf@Equus.decebal.nl>
 <mailman.279.1431163016.12865.python-list@python.org>
Message-ID: <87y4kyvv4n.fsf@Equus.decebal.nl>

Op Saturday 9 May 2015 11:16 CEST schreef Chris Angelico:

> On Sat, May 9, 2015 at 6:37 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> The code:
>> def get_json(json_file):
>> with open(json_file, 'rb') as in_f:
>> return json.load(in_f)
>>
>> def get_marshal(marshal_file):
>> with open(marshal_file, 'rb') as in_f:
>> return marshal.load(in_f)
>>
>> def get_pickle(pickle_file):
>> with open(pickle_file, 'rb') as in_f:
>> return pickle.load(in_f)
>
> def get_any(format, filename):
> with open(filename, 'rb') as in_f:
> return format.load(in_f)

I was thinking about something like that and then let the other three
call this one. I think that:
    data = get_json(json_file)
is nicer to do as:
    data = get_any(json, json_file)


> def any_to_any(fmt1, fmt2, fn1, fn2): data_in = get_any(fmt1, fn1)
> save_any(fmt2, data_in, fn2) data_out = get_any(fmt2, fn2) if
> data_in != data_out: raise SerializationError('Serialization from
> {0} to {1} not successful'. format(fn1, fn2))
>
> formats = [json, pickle, marshal]
> for fmt1 in formats:
> for fmt2 in formats:
> globals()["%s_to_%s" % (fmt1.__name__, fmt2.__name__)] = \
> functools.partial(any_to_any, fmt1, fmt2)
>
>> def json_to_pickle(json_file, pickle_file): data_in =
>> get_json(json_file) save_pickle(data_in, pickle_file) data_out =
>> get_pickle(pickle_file) if data_in == data_out: raise
>> SerializationError('Serialization from {0} to {1} not succesfull'.
>> format(json_file, pickle_file))
>
> def json_to_pickle(json_file, pickle_file): try: any_to_any(json,
> pickle, json_file, pickle_file) except SerializationError: pass
> else: raise SerializationError('Serialization from {0} to {1} not
> successful'. format(json_file, pickle_file))

Was thinking about that also. Only should there be no conversion to
marshal I think.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From djhon9813 at gmail.com  Sat May  9 08:08:05 2015
From: djhon9813 at gmail.com (david jhon)
Date: Sat, 9 May 2015 17:08:05 +0500
Subject: calling base class method fetches no results
In-Reply-To: <554DF14B.6020604@davea.name>
References: <CANPUzQ1QJrg7GKRTpfvKLivG+vbNAY+gYeBSsB_zrJmX=4tRFQ@mail.gmail.com>
 <mikbn3$tc3$1@ger.gmane.org>
 <CANPUzQ0vTW6a+Umd6nXUOQ+Uds9dhHO-i44EVuAr0hYgcADv0Q@mail.gmail.com>
 <554DF14B.6020604@davea.name>
Message-ID: <CANPUzQ06Njz57w2OttGspEuoTgY6KQAnvZ5VwhXT_zt=n5yJBQ@mail.gmail.com>

I am really sorry for any inconvenience caused, I was trying to fix this
bug from last 2 days so I had to post it here. It now has been resolved.
Thanks a lot for your time. I'll be careful again. Have a great weekend!

On Sat, May 9, 2015 at 4:36 PM, Dave Angel <davea at davea.name> wrote:

> On 05/09/2015 03:59 AM, david jhon wrote:
>
>> Hi, I am sorry for sending in five attachments, I cloned the code from
>> here
>> <https://bitbucket.org/msharif/hedera/src>: Let me explain it here:
>>
>>
> Please don't top-post.  Your earlier problem description, which I could
> make no sense of, is now located after your later "clarification".
>
> Thanks for eliminating the attachments.  Many cannot see them.  And for
> extracting only part of the code into the message.  It's still too much for
> me, but others may manage it okay.  To me, it seems likely that most of
> that code will not have any part in the problem you're describing. And in
> some places you have code that's missing its context.
>
> Now, eliminate the pieces of code that are irrelevant to your question,
> and state the problem in terms that make sense.  Somebody is instantiating
> an object.  Exactly which class is being used for that? Somebody else is
> calling a particular method (be specific, rather than just saying "some
> method"), and it's giving the wrong results.  And apparently, those wrong
> results depend on which source file something happens in.
>
>
>
>  Routing Base class defined in DCRouting.py:
>>
>> import logging
>> from copy import copy
>>
>> class Routing(object):
>>      '''Base class for data center network routing.
>>
>>      Routing engines must implement the get_route() method.
>>      '''
>>
>>      def __init__(self, topo):
>>          '''Create Routing object.
>>
>>          @param topo Topo object from Net parent
>>          '''
>>          self.topo = topo
>>
>>      def get_route(self, src, dst, hash_):
>>          '''Return flow path.
>>
>>          @param src source host
>>          @param dst destination host
>>          @param hash_ hash value
>>
>>          @return flow_path list of DPIDs to traverse (including hosts)
>>          '''
>>          raise NotImplementedError
>>
>>      def routes(self, src, dst):
>>          ''' Return list of paths
>>
>>          Only works for Fat-Tree topology
>>
>>          @ param src source host
>>          @ param dst destination host
>>
>>          @ return list of DPIDs (including inputs)
>>          '''
>>
>>          complete_paths = [] # List of complete dpid routes
>>
>>          src_paths = { src : [[src]] }
>>          dst_paths = { dst : [[dst]] }
>>
>>          dst_layer = self.topo.layer(dst)
>>          src_layer = self.topo.layer(src)
>>
>>          lower_layer = src_layer
>>          if dst_layer > src_layer:
>>              lower_layer = dst_layer
>>
>>
>>          for front_layer in range(lower_layer-1, -1, -1):
>>              if src_layer > front_layer:
>>              # expand src frontier
>>                  new_src_paths = {}
>>                  for node in sorted(src_paths):
>>                      path_list = src_paths[node]
>>                      for path in path_list:
>>                          last_node = path[-1]
>>                          for frontier_node in
>> self.topo.upper_nodes(last_node):
>>                              new_src_paths[frontier_node] = [path +
>> [frontier_node]]
>>
>>                              if frontier_node in dst_paths:
>>                                  dst_path_list = dst_paths[frontier_node]
>>                                  for dst_path in dst_path_list:
>>                                      dst_path_copy = copy ( dst_path )
>>                                      dst_path_copy.reverse()
>>                                      complete_paths.append( path +
>> dst_path_copy)
>>                  src_paths = new_src_paths
>>
>>              if dst_layer > front_layer:
>>              # expand dst frontier
>>                  new_dst_paths = {}
>>                  for node in sorted(dst_paths):
>>                      path_list = dst_paths[node]
>>                      for path in path_list:
>>                          last_node = path[-1]
>>                          for frontier_node in
>> self.topo.upper_nodes(last_node):
>>                              new_dst_paths[frontier_node] = [ path +
>> [frontier_node]]
>>
>>                              if frontier_node in src_paths:
>>                                  src_path_list = src_paths[frontier_node]
>>                                  dst_path_copy = copy( path )
>>                                  dst_path_copy.reverse()
>>                                  for src_path in src_path_list:
>>                                      complete_paths.append( src_path +
>> dst_path_copy)
>>
>>                  dst_paths = new_dst_paths
>>
>>              if complete_paths:
>>                  return complete_paths
>> class HashedRouting(Routing):
>>      ''' Hashed routing '''
>>
>>      def __init__(self, topo):
>>          self.topo = topo
>>
>>      def get_route(self, src, dst, hash_):
>>          ''' Return flow path. '''
>>
>>          if src == dst:
>>              return [src]
>>
>>          paths = self.routes(src,dst)
>>          if paths:
>>              #print 'hash_:', hash_
>>              choice = hash_ % len(paths)
>>              #print 'choice:', choice
>>              path = sorted(paths)[choice]
>>              #print 'path:', path
>>              return path
>>
>> ============>
>> Instantiated in util.py:
>>
>> from DCTopo import FatTreeTopo
>> from mininet.util import makeNumeric
>> from DCRouting import HashedRouting, Routing
>>
>> TOPOS = {'ft': FatTreeTopo}
>> ROUTING = {'ECMP' : HashedRouting}
>>
>>
>> def buildTopo(topo):
>>      topo_name, topo_param = topo.split( ',' )
>>      return TOPOS[topo_name](makeNumeric(topo_param))
>>
>>
>> def getRouting(routing, topo):
>>      return ROUTING[routing](topo)
>>
>
> A comment on that last line that says it instantiates the child class
> would have been very helpful.
>
> But even though I can now search for calls to that function, I still
> cannot make sense out of that context, either.
>
>
>
>> ============================> utilized in HController. py:
>> A Piece of code which works with self.r.routes() method:
>> Following list of methods are defined in HController.py
>>
>>      def _ecmp_hash(self, packet):
>>          ''' Return an ECMP-style 5-tuple hash for TCP/IP packets,
>> otherwise
>> 0.
>>          RFC2992 '''
>>          hash_input = [0] * 5
>>          if isinstance(packet.next, ipv4):
>>              ip = packet.next
>>              hash_input[0] = ip.srcip.toUnsigned()
>>              hash_input[1] = ip.dstip.toUnsigned()
>>              hash_input[2] = ip.protocol
>>              if isinstance(ip.next, tcp) or isinstance(ip.next, udp):
>>                  l4 = ip.next
>>                  hash_input[3] = l4.srcport
>>                  hash_input[4] = l4.dstport
>>                  return crc32(pack('LLHHH', *hash_input))
>>          return 0
>>
>>      def _install_reactive_path(self, event, out_dpid, final_out_port,
>> packet):
>>          ''' Install entries on route between two switches. '''
>>          in_name = self.t.node_gen(dpid = event.dpid).name_str()
>>          out_name = self.t.node_gen(dpid = out_dpid).name_str()
>>          hash_ = self._ecmp_hash(packet)
>>          paths = self.r.routes(src_name, dst_name)
>>          if paths == None:
>>               print "PATH is None :("
>>               return
>>          route = self.r.get_route(in_name, out_name, hash_)
>>          print "Route:",route
>>          print '-'*80
>>          if route == None:
>>              print None, "route between", in_name, "and", out_name
>>              return
>>
>>          match = of.ofp_match.from_packet(packet)
>>
>>          for i, node in enumerate(route):
>>              node_dpid = self.t.node_gen(name = node).dpid
>>              if i < len(route) - 1:
>>                  next_node = route[i + 1]
>>                  out_port, next_in_port = self.t.port(node, next_node)
>>              else:
>>                  out_port = final_out_port
>>              self.switches[node_dpid].install(out_port, match,
>> idle_timeout
>> = 10)
>>
>>          if isinstance(packet.next, of.ipv4) and
>> isinstance(packet.next.next, of.tcp):
>>              self.matchDict[(packet.next.srcip, packet.next.dstip,
>> packet.next.next.srcport, packet.next.next.dstport)] = (route, match)
>>
>>      def _handle_PacketIn(self, event):
>>          if not self.all_switches_up:
>>              #log.info("Saw PacketIn before all switches were up -
>> ignoring." )
>>              return
>>
>>          packet = event.parsed
>>          dpid = event.dpid
>>          in_port = event.port
>>
>>          # Learn MAC address of the sender on every packet-in.
>>          self.macTable[packet.src] = (dpid, in_port)
>>          sw_name = self.t.node_gen(dpid = dpid).name_str()
>>          #print "Sw:", sw_name, packet.src, packet.dst,"port", in_port,
>> packet.dst.isMulticast(),"macTable", packet.dst in self.macTable
>>          #print '-'*80
>>
>>          # Insert flow, deliver packet directly to destination.
>>
>>          if packet.dst in self.macTable:
>>              out_dpid, out_port = self.macTable[packet.dst]
>>              self._install_reactive_path(event, out_dpid, out_port,
>> packet)
>>
>>              self.switches[out_dpid].send_packet_data(out_port,
>> event.data)
>>
>>          else:
>>              self._flood(event)
>>
>> ===================> code snippet which returns 'None' number of paths.
>>      def _GlobalFirstFit(self,flow):
>>          '''do the Hedera global first fit here'''
>>          src_name = self.t.node_gen(dpid = flow['src']).name_str()
>>          dst_name = self.t.node_gen(dpid = flow['dst']).name_str()
>>          print 'Global Fisrt Fit for the elephant flow from
>> ',src_name,'to',
>> dst_name
>>          paths = self.r.routes(src_name,dst_name)
>>          print 'all routes found for the big flow:\n',paths
>>          GFF_route = None
>>          if paths == None:
>>             return
>>          else:
>>            for path in paths:
>>              fitCheck = True
>>
>>              for i in range(1,len(path)):
>>                  fitCheck = False
>>                  if self.bwReservation.has_key(path[i-1]) and
>> self.bwReservation[path[i-1]].has_key(path[i]):
>>                      if
>> self.bwReservation[path[i-1]][path[i]]['reserveDemand'] + flow['demand'] >
>> 1 :
>>                          break
>>                      else:
>>
>> #self.bwReservation[path[i-1]][path[i]]['reserveDemand'] += flow['demand']
>>                          fitCheck = True
>>                  else:
>>                      self.bwReservation[path[i-1]]={}
>>
>> self.bwReservation[path[i-1]][path[i]]={'reserveDemand':0}
>>                      fitCheck = True
>>              if fitCheck == True:
>>                  for i in range(1,len(path)):
>>
>>  self.bwReservation[path[i-1]][path[i]]['reserveDemand']
>> += flow['demand']
>>                  GFF_route = path
>>                  print "GFF route found:", path
>>                  break
>>          if GFF_route != None:
>>              """install new GFF_path between source and destintaion"""
>>              self. _install_GFF_path(GFF_route,flow['match'])
>>
>> def launch(topo = None, routing = None, bw = None ):
>>      #print topo
>>      if not topo:
>>          raise Exception ("Please specify the topology")
>>      else:
>>          t = buildTopo(topo)
>>
>>      r = getRouting(routing, t)
>>      if bw == None:
>>          bw = 10.0 #Mb/s
>>          bw = float(bw/1000) #Gb/s
>>      else:
>>          bw = float(bw)/1000
>>      core.registerNew(HController, t, r, bw)
>>      log.info("** HController is running
>>
>> I am really sorry for any inconvenience caused. I, ve tried to make it a
>> bit clear here. I am not even able to debug the code by setting a python
>> debugging point pdb. I need help from you guys. Thanks a lot again for
>> your
>> time and help.
>>
>> Best Regards,
>> David
>>
>>
>> On Sat, May 9, 2015 at 12:07 PM, Mark Lawrence <breamoreboy at yahoo.co.uk>
>> wrote:
>>
>>  On 09/05/2015 07:41, david jhon wrote:
>>>
>>>  Hello everyone,
>>>>
>>>> I am new to python and trying to run an example code from mininet tests.
>>>> Basically, I am trying to call a method in Hcontroller.py from base
>>>> class
>>>> Routing defined in DCRouting.py which runs and fetches all the required
>>>> results in install_reactive_path() method, but it returns None when it
>>>> is
>>>> called from _GlobalFirstFit. I hope someone here could help me fix this
>>>> bug..
>>>>
>>>> I am attaching all the three files(DCRouting.py, HController.py,
>>>> util.py)
>>>> to have a look into. Thanks in advance for your time, help or
>>>> suggestion.
>>>> Thanks a lot!
>>>>
>>>> kind regards,
>>>> David
>>>>
>>>>
>>>>  I'm sorry but I'm not wading through nearly 30kb of code in five
>>> attachments.  Please see http://sscce.org/ for how to put your question
>>> so you're more likely to get answers.
>>>
>>> --
>>> My fellow Pythonistas, ask not what our language can do for you, ask
>>> what you can do for our language.
>>>
>>> Mark Lawrence
>>>
>>> --
>>> https://mail.python.org/mailman/listinfo/python-list
>>>
>>>
>>
>>
>>
>
> --
> DaveA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150509/ee92463b/attachment.html>

From davea at davea.name  Sat May  9 08:25:20 2015
From: davea at davea.name (Dave Angel)
Date: Sat, 09 May 2015 08:25:20 -0400
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
In-Reply-To: <580ee0d6-a703-4da3-af2d-105589a1780f@googlegroups.com>
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
 <mailman.260.1431113391.12865.python-list@python.org>
 <8360473a-45ac-4270-9bf3-81932da5f223@googlegroups.com>
 <554d768d$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <580ee0d6-a703-4da3-af2d-105589a1780f@googlegroups.com>
Message-ID: <554DFCB0.2020601@davea.name>

On 05/09/2015 06:31 AM, zljubisicmob at gmail.com wrote:
>
> title = title[:232]
> title = title.replace(" ", "_").replace("/", "_").replace("!", "_").replace("?", "_")\
>                      .replace('"', "_").replace(':', "_").replace(',', "_").replace('&#34;', '')\
>                      .replace('\n', '_').replace('&#39', '')
>
> print(title)
>
> src_file = os.path.join(ROOTDIR, 'src_' + title + '.txt')
> dst_file = os.path.join(ROOTDIR, 'des_' + title + '.txt')
>
> print(len(src_file), src_file)
> print(len(dst_file), dst_file)
>
> with open(src_file, mode='w', encoding='utf-8') as s_file:
>      s_file.write('test')
>
>
> shutil.move(src_file, dst_file)
>
> It works, but if you change title = title[:232] to title = title[:233], you will get "FileNotFoundError: [Errno 2] No such file or directory".
> As you can see ROOTDIR contains \U.

No, we can't see what ROOTDIR is, since you read it from the config 
file.  And you don't show us the results of those prints.  You don't 
even show us the full exception, or even the line it fails on.

I doubt that the problem is in the ROODIR value, but of course nothing 
in your program bothers to check that that directory exists.  I expect 
you either have too many characters total, or the 232th character is a 
strange one.  Or perhaps title has a backslash in it (you took care of 
forward slash).

While we're at it, if you do have an OS limitation on size, your code is 
truncating at the wrong point.  You need to truncate the title based on 
the total size of src_file and dst_file, and since the code cannot know 
the size of ROOTDIR, you need to include that in your figuring.




-- 
DaveA


From Cecil at decebal.nl  Sat May  9 09:04:37 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 09 May 2015 15:04:37 +0200
Subject: Seralization
References: <876182xi1w.fsf@Equus.decebal.nl>
Message-ID: <87twvlx5p6.fsf@Equus.decebal.nl>

Op Saturday 9 May 2015 10:37 CEST schreef Cecil Westerhof:

> To make serialization a bit easier I made a few functions to get,
> save and convert between the different types. As I see it pickle and
> json are probably used the most. I also have a get and save for
> marshal. But no conversion to marshal, because in principle you
> should not use it, so a conversion to it is not useful. I did define
> conversion from.

I used the example of Chris to make it a lot cleaner. I like DRY, but
did not know it was that easy in Python.

The code is now:
    def convert_serialization(format_in, format_out, filename_in, filename_out):
        data_in = get_serialization(format_in, filename_in)
        save_serialization(format_out, data_in, filename_out)
        data_out = get_serialization(format_out, filename_out)
        if data_in != data_out:
            raise SerializationError('Serialization from {0} to {1} not successful'.
                                         format(filename_in, filename_out))

    def get_serialization(format, filename):
        with open(filename, 'rb') as in_f:
            return format.load(in_f)

    def save_serialization(format, data, filename):
        with open(filename, 'wb') as out_f:
            format.dump(data, out_f)

    _all_serialization_formats           = [json, marshal, pickle]
    _convert_to_serialization_formats    = [json, pickle]

    for format_in in _all_serialization_formats:
        globals()[
            'save_%s' % (format_in.__name__)
        ] = functools.partial(save_serialization, format_in)
        globals()[
            'get_%s' % (format_in.__name__)
        ] = functools.partial(get_serialization, format_in)
        for format_out in _convert_to_serialization_formats:
            if format_in == format_out:
                continue
            globals()[
                '%s_to_%s' % (format_in.__name__, format_out.__name__)
            ] = functools.partial(convert_serialization, format_in, format_out)

    del format_in, format_out

I keep the all_serialization_formats and
_convert_to_serialization_formats so you can peek which ones are
supported at the moment.


> Are there other seralizations that is handy to take care of?

That is now a cinch to implement: you only have to append the two
lists. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From pauld11718 at gmail.com  Sat May  9 10:51:50 2015
From: pauld11718 at gmail.com (pauld11718)
Date: Sat, 9 May 2015 07:51:50 -0700 (PDT)
Subject: numpy and cython
Message-ID: <9a85a810-8052-40be-a95d-084f0cb4e2d0@googlegroups.com>

Unable to compile :

import numpy as np
cimport numpy as np
import math as m

DTYPE = np.float
ctypedef np.float_t DTYPE_t

def visc1(float t, float dcal):
    cdef float h, tr, trinv, rhor
    cdef float eta0, sumi, i, sumj, im1, jm1, eta
   
    cdef np.ndarray vb = np.array([1.00000, 0.940695, 0.578377, -0.202044], dtype = DTYPE)
   
    cdef np.ndarray[DTYPE_t, ndim=2] va = np.array([[.4864192, .3509007, -.2847572, .07013759,.0164122, -.01163815,.0],
        [-.2448372,1.315436, -1.037026, .4660127, -.02884911,-.008239587,.0],
        [-.8702035, 1.297752, -1.287846, .2292075, .0,  .0, .0],
        [.8716056, 1.353448,  .0,  -.4857462, .1607171,.0,    -.003886659],
        [-1.051126, .0,   .0,   .0,   .0,      .0,   .0],
        [.3458395, .0,  -.02148229, .0, -.009603846, .004559914,.0]], dtype=DTYPE, ndim = 2)
   

    h=55.2651e-06;
    tr = t/643.89;
    trinv=643.89/t;
    rhor=dcal/0.358;

    eta0 = h*(m.pow(tr,0.5))/(vb[0] + vb[1]/tr + vb[2]/(tr*tr) + vb[3]/(tr**3));
    sumi=0.0
    for i in range(6):
        sumj=va[i,0]
        for j in range(2,7):
            jm1=j-1;
            sumj=sumj+va[i,j]*((rhor-1.0)**jm1);
           
        im1 = i-1
        sumi = sumi+sumj*((trinv-1.0)**im1);
       
    eta = eta0*m.exp(rhor*sumi)
    return eta

Error :
Compiling visco.pyx because it changed.
Cythonizing visco.pyx
running build_ext
building 'visco' extension
gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/deepraj/miniconda3/envs/venv1/include/python3.4m -c visco.c -o build/temp.linux-x86_64-3.4/visco.o
In file included from /usr/include/numpy/ndarraytypes.h:1761:0,
                 from /usr/include/numpy/ndarrayobject.h:17,
                 from /usr/include/numpy/arrayobject.h:4,
                 from visco.c:258:
/usr/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it by " \
  ^
gcc -pthread -shared build/temp.linux-x86_64-3.4/visco.o -L/home/abcd/miniconda3/envs/venv1/lib -lpython3.4m -o /media/abcd/Man_UTD/pythoncode/venv1/visco.cpython-34m.so


From rosuav at gmail.com  Sat May  9 11:04:03 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 10 May 2015 01:04:03 +1000
Subject: numpy and cython
In-Reply-To: <9a85a810-8052-40be-a95d-084f0cb4e2d0@googlegroups.com>
References: <9a85a810-8052-40be-a95d-084f0cb4e2d0@googlegroups.com>
Message-ID: <CAPTjJmqBdJg7JD8AiNi6ppm45CmvOZx1-wQGdE4kABEARm8xPA@mail.gmail.com>

On Sun, May 10, 2015 at 12:51 AM, pauld11718 <pauld11718 at gmail.com> wrote:
> Unable to compile :
> /usr/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
>  #warning "Using deprecated NumPy API, disable it by " \
>   ^
> gcc -pthread -shared build/temp.linux-x86_64-3.4/visco.o -L/home/abcd/miniconda3/envs/venv1/lib -lpython3.4m -o /media/abcd/Man_UTD/pythoncode/venv1/visco.cpython-34m.so

Looks like a warning to me. Are you sure the compilation isn't
working? Check to see if you have an output file. If you don't,
there's something else wrong, not just this.

ChrisA


From steve+comp.lang.python at pearwood.info  Sat May  9 11:13:56 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 10 May 2015 01:13:56 +1000
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
 <mailman.260.1431113391.12865.python-list@python.org>
 <8360473a-45ac-4270-9bf3-81932da5f223@googlegroups.com>
 <554d768d$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <580ee0d6-a703-4da3-af2d-105589a1780f@googlegroups.com>
Message-ID: <554e2435$0$12992$c3e8da3$5496439d@news.astraweb.com>

On Sat, 9 May 2015 08:31 pm, zljubisicmob at gmail.com wrote:

> It works, but if you change title = title[:232] to title = title[:233],
> you will get "FileNotFoundError: [Errno 2] No such file or directory".


Which is a *completely different* error from 

SyntaxError: 'unicodeescape' codec can't decode bytes in position 2-3:
truncated \UXXXXXXXX escape


> As you can see ROOTDIR contains \U.

How can I possibly see that? Your code reads ROOTDIR from the config file,
which you don't show us.

I agree with you that Windows has limitations on the length of file names,
and that you get an error if you give a file name that cannot be found. The
point is that before you can get that far, you *first* have to fix the
SyntaxError. That's a completely different problem.

You can't fix the \U syntax error by truncating the total file length. But
you can fix that syntax error by changing your code so it reads the ROOTDIR
from a config file instead of a hard-coded string literal -- exactly like
we told you to do!

An essential skill when programming is to read and understand the error
messages. One of the most painful things to use is a programming language
that just says 

"An error occurred"

with no other explanation. Python gives you lots of detail to explain what
went wrong:

SyntaxError means you made an error in the syntax of the code and the
program cannot even run.

FileNotFoundError means that the program did run, it tried to open a file,
but the file doesn't exist.

They're a little bit different, don't you agree?



-- 
Steven



From rosuav at gmail.com  Sat May  9 11:22:07 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 10 May 2015 01:22:07 +1000
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
In-Reply-To: <554e2435$0$12992$c3e8da3$5496439d@news.astraweb.com>
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
 <mailman.260.1431113391.12865.python-list@python.org>
 <8360473a-45ac-4270-9bf3-81932da5f223@googlegroups.com>
 <554d768d$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <580ee0d6-a703-4da3-af2d-105589a1780f@googlegroups.com>
 <554e2435$0$12992$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmp5KVdBq87CTcOOV6hcmAhED62UMEd7tBJ7H=scNxPvnQ@mail.gmail.com>

On Sun, May 10, 2015 at 1:13 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> FileNotFoundError means that the program did run, it tried to open a file,
> but the file doesn't exist.

Normally it does, at least. Sometimes it means that a *directory*
doesn't exist (for instance, you can get this when you try to create a
new file, which otherwise wouldn't make sense), and occasionally,
Windows will give you rather peculiar errors when weird things go
wrong, which may be what's going on here (maximum path length - though
that can be overridden by switching to a UNC-style path).

Steven's point still stands - very different from SyntaxError - but
unfortunately it's not always as simple as the name suggests. Thank
you oh so much, Windows.

ChrisA


From pauld11718 at gmail.com  Sat May  9 11:56:41 2015
From: pauld11718 at gmail.com (pauld11718)
Date: Sat, 9 May 2015 08:56:41 -0700 (PDT)
Subject: numpy and cython
In-Reply-To: <mailman.283.1431183852.12865.python-list@python.org>
References: <9a85a810-8052-40be-a95d-084f0cb4e2d0@googlegroups.com>
 <mailman.283.1431183852.12865.python-list@python.org>
Message-ID: <a3d2979c-491e-4d99-a5c3-77f62344fe0c@googlegroups.com>

NO the compilation isn't working...

The setup.py : 

from distutils.core import setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize("visco.pyx")
)


From pauld11718 at gmail.com  Sat May  9 12:09:21 2015
From: pauld11718 at gmail.com (pauld11718)
Date: Sat, 9 May 2015 09:09:21 -0700 (PDT)
Subject: numpy and cython
In-Reply-To: <a3d2979c-491e-4d99-a5c3-77f62344fe0c@googlegroups.com>
References: <9a85a810-8052-40be-a95d-084f0cb4e2d0@googlegroups.com>
 <mailman.283.1431183852.12865.python-list@python.org>
 <a3d2979c-491e-4d99-a5c3-77f62344fe0c@googlegroups.com>
Message-ID: <272aea55-c8a6-4b3f-ac54-4492f8ef586d@googlegroups.com>

An interesting observation : 

Any simple program with just :

cimport numpy as np 

doesnot compile.


From breamoreboy at yahoo.co.uk  Sat May  9 12:53:31 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 09 May 2015 17:53:31 +0100
Subject: numpy and cython
In-Reply-To: <a3d2979c-491e-4d99-a5c3-77f62344fe0c@googlegroups.com>
References: <9a85a810-8052-40be-a95d-084f0cb4e2d0@googlegroups.com>
 <mailman.283.1431183852.12865.python-list@python.org>
 <a3d2979c-491e-4d99-a5c3-77f62344fe0c@googlegroups.com>
Message-ID: <mile2d$hi4$1@ger.gmane.org>

On 09/05/2015 16:56, pauld11718 wrote:
> NO the compilation isn't working...
>
> The setup.py :
>
> from distutils.core import setup
> from Cython.Build import cythonize
>
> setup(
>      ext_modules = cythonize("visco.pyx")
> )
>

If you cannot be bothered to supply any context so I've no idea what 
you're talking about, then I cannot be bothered to try and help.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Sat May  9 12:54:55 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 09 May 2015 17:54:55 +0100
Subject: numpy and cython
In-Reply-To: <272aea55-c8a6-4b3f-ac54-4492f8ef586d@googlegroups.com>
References: <9a85a810-8052-40be-a95d-084f0cb4e2d0@googlegroups.com>
 <mailman.283.1431183852.12865.python-list@python.org>
 <a3d2979c-491e-4d99-a5c3-77f62344fe0c@googlegroups.com>
 <272aea55-c8a6-4b3f-ac54-4492f8ef586d@googlegroups.com>
Message-ID: <mile50$hi4$2@ger.gmane.org>

On 09/05/2015 17:09, pauld11718 wrote:
> An interesting observation :
>
> Any simple program with just :
>
> cimport numpy as np
>
> doesnot compile.
>

Fascinating.  What has this got to do with the general election results 
from the UK?  Or is there some other context that you're not prepared to 
let us in on?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From ian.g.kelly at gmail.com  Sat May  9 12:57:01 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 9 May 2015 10:57:01 -0600
Subject: functions, optional parameters
In-Reply-To: <ui2v1cx5co.ln2@news.c0t0d0s0.de>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <ui2v1cx5co.ln2@news.c0t0d0s0.de>
Message-ID: <CALwzidmb_+fpSr939_AVUREbYt2_4ZsHZUMDLD5rWA=cRtzi8g@mail.gmail.com>

On Fri, May 8, 2015 at 9:50 AM, Michael Welle <mwe012008 at gmx.net> wrote:
>
> Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
>>
>> If your language uses late binding, it is very inconvenient to get early
>> binding when you want it. But if your language uses early binding, it is
>> very simple to get late binding when you want it: just put the code you
>> want to run inside the body of the function:
> And you have to do it all the time again and again. I can't provide hard
> numbers, but I think usually I want late binding.

You could perhaps write a decorator to evaluate your defaults at call
time. This one relies on inspect.signature, so it requires Python 3.3
or newer:

import inspect
from functools import wraps

def late_defaults(**defaults):
    def decorator(f):
        sig = inspect.signature(f)
        @wraps(f)
        def wrapped(*args, **kwargs):
            bound_args = sig.bind_partial(*args, **kwargs)
            for name, get_value in defaults.items():
                if name not in bound_args.arguments:
                    bound_args.arguments[name] = get_value()
            return f(*bound_args.args, **bound_args.kwargs)
        return wrapped
    return decorator

@late_defaults(b=lambda: x+1, c=lambda: y*2)
def f(a, b, c=None):
    print(a, b, c)

x = 14
y = 37
f(10)
x = 30
y = 19
f(10)
f(10, 11)
f(10, 11, c=12)

Output:

10 15 74
10 31 38
10 11 38
10 11 12

For documentation purposes I suggest using default values of None in
the function spec to indicate that the arguments are optional, and
elaborating on the actual defaults in the docstring. Alternatively you
could put the lambdas in the the actual function spec and then just
tell the decorator which ones to apply if not supplied, but that would
result in less readable pydoc.


From antranig at pingvinashen.am  Sat May  9 14:30:21 2015
From: antranig at pingvinashen.am (Antranig Vartanian)
Date: Sat, 9 May 2015 20:30:21 +0200
Subject: Moving to Python 3.x
Message-ID: <alpine.LNX.2.00.1505092025440.10854@Zvartnots>

Hay,

I learned the basics of python using the book "Think Python" 
(http://www.greenteapress.com/thinkpython/) which was good (IMHO), and it 
teaches in Python 2.7. Now I'm trying to write my first python+gtk 
program.

anyways, my question will be, is it so necessary to move to python3.x 
ASAP? or Python2.7 will live for a while (2-3 years)?.

and what do you advice a newbie programmer to do after learning the 
basics?

Thanks all!

--
Antranig Vartanian
http://antranig.pingvinashen.am/


From "i n d e l i b l e b l u e p e n " at gmail.co  Sat May  9 15:29:22 2015
From: "i n d e l i b l e b l u e p e n " at gmail.co (Jason C. McDonald)
Date: Sat, 09 May 2015 12:29:22 -0700
Subject: Moving to Python 3.x
In-Reply-To: <alpine.LNX.2.00.1505092025440.10854@Zvartnots>
References: <alpine.LNX.2.00.1505092025440.10854@Zvartnots>
Message-ID: <miln4h$t15$1@dont-email.me>

On 05/09/2015 11:30 AM, Antranig Vartanian wrote:
> Hay,
>
> I learned the basics of python using the book "Think Python"
> (http://www.greenteapress.com/thinkpython/) which was good (IMHO), and
> it teaches in Python 2.7. Now I'm trying to write my first python+gtk
> program.
>
> anyways, my question will be, is it so necessary to move to python3.x
> ASAP? or Python2.7 will live for a while (2-3 years)?.
>
> and what do you advice a newbie programmer to do after learning the basics?
>
> Thanks all!
>
> --
> Antranig Vartanian
> http://antranig.pingvinashen.am/

I would strongly recommend writing your code to run on both Py2 and Py3. 
NINJA-IDE (an open source Python IDE) will lint your code so it'll run 
in both.

-- 
Jason C. McDonald (CodeMouse92)
[CEO, Lead Dev @ MousePaw Games]


From ian.g.kelly at gmail.com  Sat May  9 15:49:58 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 9 May 2015 13:49:58 -0600
Subject: Moving to Python 3.x
In-Reply-To: <alpine.LNX.2.00.1505092025440.10854@Zvartnots>
References: <alpine.LNX.2.00.1505092025440.10854@Zvartnots>
Message-ID: <CALwzidnsRL8Srmk8i91biV1fzxsJYc6pDcPx0+pn0CykXH0RTA@mail.gmail.com>

On Sat, May 9, 2015 at 12:30 PM, Antranig Vartanian
<antranig at pingvinashen.am> wrote:
> Hay,
>
> I learned the basics of python using the book "Think Python"
> (http://www.greenteapress.com/thinkpython/) which was good (IMHO), and it
> teaches in Python 2.7. Now I'm trying to write my first python+gtk program.
>
> anyways, my question will be, is it so necessary to move to python3.x ASAP?
> or Python2.7 will live for a while (2-3 years)?.

Python 2.7 will continue to be maintained through 2020. If you don't
have any specific reason to use Python 2.7 (such as a library
dependency), then you should try to use 3.x for new projects. You'll
avoid the pain of needing to migrate later, and you'll be able to
start taking advantage of newer features right away.

> and what do you advice a newbie programmer to do after learning the basics?

Find an existing open source project that you'd like to contribute to.
It doesn't have to be anything major, but it will help you learn about
the Python ecosystem, and the opportunities to collaborate will help
you build your skills. It also looks good on a resume, if your plans
include being a professional Python programmer.


From tjreedy at udel.edu  Sat May  9 15:58:13 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 09 May 2015 15:58:13 -0400
Subject: Moving to Python 3.x
In-Reply-To: <alpine.LNX.2.00.1505092025440.10854@Zvartnots>
References: <alpine.LNX.2.00.1505092025440.10854@Zvartnots>
Message-ID: <milosq$mj9$1@ger.gmane.org>

On 5/9/2015 2:30 PM, Antranig Vartanian wrote:
> Hay,
>
> I learned the basics of python using the book "Think Python"
> (http://www.greenteapress.com/thinkpython/) which was good (IMHO), and
> it teaches in Python 2.7. Now I'm trying to write my first python+gtk
> program.
>
> anyways, my question will be, is it so necessary to move to python3.x
> ASAP? or Python2.7 will live for a while (2-3 years)?.

1.5 is still in use, so that is not exactly the issue.

I and many here recommend starting with current 3.x unless there is a 
compelling reason otherwise:
1. learning from a 2.7 book;
2. employer requires 2.7
3. using 2.x only module.  If you are determined to use gtk and are 
using the PyGTK bindings https://pypi.python.org/pypi/PyGTK/2.24.0
then you have no choice.

If you write 2.7 code, make it as 3.x-like as sensible.  Others may post 
some links.

If you use unicode, you might be happier with 3.x or even have to use it.

-- 
Terry Jan Reedy



From vjp2.at at at.BioStrategist.dot.dot.com  Sat May  9 17:04:37 2015
From: vjp2.at at at.BioStrategist.dot.dot.com (vjp2.at at at.BioStrategist.dot.dot.com)
Date: Sat, 9 May 2015 21:04:37 +0000 (UTC)
Subject: Jython from bathc file?
References: <mijh9c$nv6$2@reader1.panix.com>
Message-ID: <milsp5$fg8$1@reader1.panix.com>

Thanks.. I suspected it wasn't meant to be taken as in the file

THe one thing I'm not sure if Jython is suppsosedto keep running
after the initisl stuff is loaded in..


To put the question in purely DOS terms if you run a program can you pipe it
some commands and then keep it running to take the remaining commands from
the console?

				    - = -
 Vasos Panagiotopoulos, Columbia'81+, Reagan, Mozart, Pindus, BioStrategist
		    http://www.panix.com/~vjp2/vasos.htm
  ---{Nothing herein constitutes advice.  Everything fully disclaimed.}---
   [Homeland Security means private firearms not lazy obstructive guards]
 [Urb sprawl confounds terror] [Phooey on GUI: Windows for subprime Bimbos]






From timothy.c.delaney at gmail.com  Sat May  9 17:34:21 2015
From: timothy.c.delaney at gmail.com (Tim Delaney)
Date: Sun, 10 May 2015 07:34:21 +1000
Subject: Delivery Status Notification (Failure)
In-Reply-To: <CAPTjJmqkFNxHytnP4xPJwSVBR5uvBd1cTzZ8itvxf=Bb=tgArg@mail.gmail.com>
References: <047d7bd6bd96d64621051599f764@google.com>
 <mailman.265.1431126133.12865.python-list@python.org>
 <554d72cf$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmqkFNxHytnP4xPJwSVBR5uvBd1cTzZ8itvxf=Bb=tgArg@mail.gmail.com>
Message-ID: <CAN8CLgnuC9mtQo3ZATZ588jkX8A+D=imfaLqDGqw0GrubkLAew@mail.gmail.com>

On 9 May 2015 at 13:56, Chris Angelico <rosuav at gmail.com> wrote:

>
> Yeah, I know, shocking. But I wanted to at least *try* doing the
> normal and official thing, in the hopes that they were a legit company
> that perhaps didn't realize what this looked like.
>

By all means report to abuse@ in the future, but please do not CC the list.
My spam filters have learned to filter out most job spam automatically by
now, but it doesn't filter out your reply.

Tim Delaney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150510/67b11218/attachment.html>

From vincent at vincentdavis.net  Sat May  9 18:46:18 2015
From: vincent at vincentdavis.net (Vincent Davis)
Date: Sat, 9 May 2015 16:46:18 -0600
Subject: getting fieldnames from Dictreader before reading lines
Message-ID: <CALyJZZV8XFjbmiNRYxoAFuGHS2iix4knv-smn5adicVRAJ7ZCg@mail.gmail.com>

I am reading a file with Dictreader and writing a new file. I want use the
fieldnames in the Dictwriter from the reader. See below How should I be
doing this?

See how I am using reader.fieldnames in the the Dictwriter. I get an error
(below)

with open(readfile, 'r', encoding='utf-8', errors='ignore', newline='') as
csvread:
    reader = DictReader(csvread)
    with open(writefile, 'w') as csvwrite:
        writer = DictWriter(csvwrite, delimiter=',',
fieldnames=reader.fieldnames)
        for line in reader:
            pass

ValueError                                Traceback (most recent call
last)<ipython-input-13-0dac622bb8a9> in <module>()----> 1
reader.fieldnames()
/Users/vmd/anaconda/envs/py34/lib/python3.4/csv.py in fieldnames(self)
    94         if self._fieldnames is None:     95
try:---> 96                 self._fieldnames = next(self.reader)
97             except StopIteration:     98                 pass
ValueError: I/O operation on closed file.



Thanks
Vincent
? Davis?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150509/477f3745/attachment.html>

From torriem at gmail.com  Sat May  9 18:54:39 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sat, 09 May 2015 16:54:39 -0600
Subject: Jython from bathc file?
In-Reply-To: <milsp5$fg8$1@reader1.panix.com>
References: <mijh9c$nv6$2@reader1.panix.com> <milsp5$fg8$1@reader1.panix.com>
Message-ID: <554E902F.9030207@gmail.com>

On 05/09/2015 03:04 PM, vjp2.at at at.BioStrategist.dot.dot.com wrote:
> Thanks.. I suspected it wasn't meant to be taken as in the file
> 
> THe one thing I'm not sure if Jython is suppsosedto keep running
> after the initisl stuff is loaded in..
> 
> 
> To put the question in purely DOS terms if you run a program can you pipe it
> some commands and then keep it running to take the remaining commands from
> the console?

No.  When the sending program is finished, it will close the pipe. This
is how it works on both Unix and Windows, if I'm not mistaken.  On Unix
you might be able to read from standard in until it's done, then connect
to a pseudo-tty and do interactive things.  This bypasses standard in
though (which was connected to the pipe).


From breamoreboy at yahoo.co.uk  Sat May  9 19:00:39 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 10 May 2015 00:00:39 +0100
Subject: getting fieldnames from Dictreader before reading lines
In-Reply-To: <CALyJZZV8XFjbmiNRYxoAFuGHS2iix4knv-smn5adicVRAJ7ZCg@mail.gmail.com>
References: <CALyJZZV8XFjbmiNRYxoAFuGHS2iix4knv-smn5adicVRAJ7ZCg@mail.gmail.com>
Message-ID: <mim3iq$jfv$1@ger.gmane.org>

On 09/05/2015 23:46, Vincent Davis wrote:
> I am reading a file with Dictreader and writing a new file. I want use the
> fieldnames in the Dictwriter from the reader. See below How should I be
> doing this?
>
> See how I am using reader.fieldnames in the the Dictwriter. I get an error
> (below)
>
> with open(readfile, 'r', encoding='utf-8', errors='ignore', newline='') as
> csvread:
>      reader = DictReader(csvread)
>      with open(writefile, 'w') as csvwrite:
>          writer = DictWriter(csvwrite, delimiter=',',
> fieldnames=reader.fieldnames)
>          for line in reader:
>              pass
>
> ValueError                                Traceback (most recent call
> last)<ipython-input-13-0dac622bb8a9> in <module>()----> 1
> reader.fieldnames()
> /Users/vmd/anaconda/envs/py34/lib/python3.4/csv.py in fieldnames(self)
>      94         if self._fieldnames is None:     95
> try:---> 96                 self._fieldnames = next(self.reader)
> 97             except StopIteration:     98                 pass
> ValueError: I/O operation on closed file.
>
> Thanks
> Vincent
> ? Davis?
>

 From https://docs.python.org/3/library/csv.html#module-csv

"class csv.DictReader(csvfile, fieldnames=None, restkey=None, 
restval=None, dialect='excel', *args, **kwds)
Create an object which operates like a regular reader but maps the 
information read into a dict whose keys are given by the optional 
fieldnames parameter. The fieldnames parameter is a sequence whose 
elements are associated with the fields of the input data in order. 
These elements become the keys of the resulting dictionary. If the 
fieldnames parameter is omitted, the values in the first row of the 
csvfile will be used as the fieldnames. If the row read has more fields 
than the fieldnames sequence, the remaining data is added as a sequence 
keyed by the value of restkey. If the row read has fewer fields than the 
fieldnames sequence, the remaining keys take the value of the optional 
restval parameter. Any other optional or keyword arguments are passed to 
the underlying reader instance."

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From vincent at vincentdavis.net  Sat May  9 19:01:34 2015
From: vincent at vincentdavis.net (Vincent Davis)
Date: Sat, 9 May 2015 17:01:34 -0600
Subject: getting fieldnames from Dictreader before reading lines
In-Reply-To: <CALyJZZV8XFjbmiNRYxoAFuGHS2iix4knv-smn5adicVRAJ7ZCg@mail.gmail.com>
References: <CALyJZZV8XFjbmiNRYxoAFuGHS2iix4knv-smn5adicVRAJ7ZCg@mail.gmail.com>
Message-ID: <CALyJZZVt_X5pYQg+rXmP4CHx9-LizL4vbTQv-7t4ihD5xGqXng@mail.gmail.com>

Not sure what I was doing wrong, it seems to work now.

Vincent Davis
720-301-3003

On Sat, May 9, 2015 at 4:46 PM, Vincent Davis <vincent at vincentdavis.net>
wrote:

> I am reading a file with Dictreader and writing a new file. I want use the
> fieldnames in the Dictwriter from the reader. See below How should I be
> doing this?
>
> See how I am using reader.fieldnames in the the Dictwriter. I get an error
> (below)
>
> with open(readfile, 'r', encoding='utf-8', errors='ignore', newline='') as
> csvread:
>     reader = DictReader(csvread)
>     with open(writefile, 'w') as csvwrite:
>         writer = DictWriter(csvwrite, delimiter=',',
> fieldnames=reader.fieldnames)
>         for line in reader:
>             pass
>
> ValueError                                Traceback (most recent call last)<ipython-input-13-0dac622bb8a9> in <module>()----> 1 reader.fieldnames()
> /Users/vmd/anaconda/envs/py34/lib/python3.4/csv.py in fieldnames(self)     94         if self._fieldnames is None:     95             try:---> 96                 self._fieldnames = next(self.reader)     97             except StopIteration:     98                 pass
> ValueError: I/O operation on closed file.
>
>
>
> Thanks
> Vincent
> ? Davis?
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150509/2b3b45c4/attachment.html>

From davea at davea.name  Sat May  9 19:51:41 2015
From: davea at davea.name (Dave Angel)
Date: Sat, 09 May 2015 19:51:41 -0400
Subject: Jython from bathc file?
In-Reply-To: <milsp5$fg8$1@reader1.panix.com>
References: <mijh9c$nv6$2@reader1.panix.com> <milsp5$fg8$1@reader1.panix.com>
Message-ID: <554E9D8D.80005@davea.name>

On 05/09/2015 05:04 PM, vjp2.at at at.BioStrategist.dot.dot.com wrote:
> Thanks.. I suspected it wasn't meant to be taken as in the file
>
> THe one thing I'm not sure if Jython is suppsosedto keep running
> after the initisl stuff is loaded in..
>
>
> To put the question in purely DOS terms if you run a program can you pipe it
> some commands and then keep it running to take the remaining commands from
> the console?
>

That's not a built-in feature of cmd.exe.  However, it wouldn't be hard 
to write a data source (funny.exe) that took data from a file, and then 
from stdin, sending both in progression to stdout.  Then you'd run the 
two programs as:

funny.exe infile.txt | newprog.exe


-- 
DaveA


From davea at davea.name  Sat May  9 19:55:05 2015
From: davea at davea.name (Dave Angel)
Date: Sat, 09 May 2015 19:55:05 -0400
Subject: getting fieldnames from Dictreader before reading lines
In-Reply-To: <CALyJZZVt_X5pYQg+rXmP4CHx9-LizL4vbTQv-7t4ihD5xGqXng@mail.gmail.com>
References: <CALyJZZV8XFjbmiNRYxoAFuGHS2iix4knv-smn5adicVRAJ7ZCg@mail.gmail.com>
 <CALyJZZVt_X5pYQg+rXmP4CHx9-LizL4vbTQv-7t4ihD5xGqXng@mail.gmail.com>
Message-ID: <554E9E59.2080906@davea.name>

On 05/09/2015 07:01 PM, Vincent Davis wrote:
> Not sure what I was doing wrong, it seems to work now.
>

I still see two significant things wrong:

1) you're top-posting, putting your response  BEFORE the stuff you're 
responding to.

2) both messages are in html, which thoroughly messed up parts of your 
error messages.

>
> On Sat, May 9, 2015 at 4:46 PM, Vincent Davis <vincent at vincentdavis.net>
> wrote:
>
>> I am reading a file with Dictreader and writing a new file. I want use the
>> fieldnames in the Dictwriter from the reader. See below How should I be
>> doing this?


-- 
DaveA


From vincent at vincentdavis.net  Sat May  9 21:51:05 2015
From: vincent at vincentdavis.net (Vincent Davis)
Date: Sat, 9 May 2015 19:51:05 -0600
Subject: getting fieldnames from Dictreader before reading lines
In-Reply-To: <554E9E59.2080906@davea.name>
References: <CALyJZZV8XFjbmiNRYxoAFuGHS2iix4knv-smn5adicVRAJ7ZCg@mail.gmail.com>
 <CALyJZZVt_X5pYQg+rXmP4CHx9-LizL4vbTQv-7t4ihD5xGqXng@mail.gmail.com>
 <554E9E59.2080906@davea.name>
Message-ID: <CALyJZZXPoRU2v6a29u9UKMJLytP2=XOOaRu71fgxRjr3w5sgeA@mail.gmail.com>

On Sat, May 9, 2015 at 5:55 PM, Dave Angel <davea at davea.name> wrote:
>
> 1) you're top-posting, putting your response  BEFORE the stuff you're
responding to.


I responded to my own email, seemed ok to top post on myself saying it was
resolved.

>
> 2) both messages are in html, which thoroughly messed up parts of your
error messages.

I am posting from google mail (not google groups). Kindly let me know if
this email is also html.



Vincent Davis
720-301-3003
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150509/a59c2b5b/attachment.html>

From vjp2.at at at.BioStrategist.dot.dot.com  Sat May  9 22:18:16 2015
From: vjp2.at at at.BioStrategist.dot.dot.com (vjp2.at at at.BioStrategist.dot.dot.com)
Date: Sun, 10 May 2015 02:18:16 +0000 (UTC)
Subject: Jython from bathc file?
References: <mijh9c$nv6$2@reader1.panix.com> <milsp5$fg8$1@reader1.panix.com>
 <mailman.294.1431215521.12865.python-list@python.org>
Message-ID: <mimf58$gov$1@reader1.panix.com>

Tee from gnuutils??



				    - = -
 Vasos Panagiotopoulos, Columbia'81+, Reagan, Mozart, Pindus, BioStrategist
		    http://www.panix.com/~vjp2/vasos.htm
  ---{Nothing herein constitutes advice.  Everything fully disclaimed.}---
   [Homeland Security means private firearms not lazy obstructive guards]
 [Urb sprawl confounds terror] [Phooey on GUI: Windows for subprime Bimbos]






From vjp2.at at at.BioStrategist.dot.dot.com  Sat May  9 22:25:11 2015
From: vjp2.at at at.BioStrategist.dot.dot.com (vjp2.at at at.BioStrategist.dot.dot.com)
Date: Sun, 10 May 2015 02:25:11 +0000 (UTC)
Subject: Jython from bathc file?
References: <mijh9c$nv6$2@reader1.panix.com> <milsp5$fg8$1@reader1.panix.com>
 <mailman.294.1431215521.12865.python-list@python.org>
 <mimf58$gov$1@reader1.panix.com>
Message-ID: <mimfi7$sda$1@reader1.panix.com>

I have to try this and see if there is ome kind of init file in jython/python
sorta like autoexec.bat. Ialso have no idea if the commands they provide are
all it takes to run the app or I have to stay in jython.. sorry, I'm thinking
at loud.. ok, thanks to all..



				    - = -
 Vasos Panagiotopoulos, Columbia'81+, Reagan, Mozart, Pindus, BioStrategist
		    http://www.panix.com/~vjp2/vasos.htm
  ---{Nothing herein constitutes advice.  Everything fully disclaimed.}---
   [Homeland Security means private firearms not lazy obstructive guards]
 [Urb sprawl confounds terror] [Phooey on GUI: Windows for subprime Bimbos]






From rustompmody at gmail.com  Sat May  9 22:36:18 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sat, 9 May 2015 19:36:18 -0700 (PDT)
Subject: Jython from bathc file?
In-Reply-To: <mimfi7$sda$1@reader1.panix.com>
References: <mijh9c$nv6$2@reader1.panix.com> <milsp5$fg8$1@reader1.panix.com>
 <mailman.294.1431215521.12865.python-list@python.org>
 <mimf58$gov$1@reader1.panix.com> <mimfi7$sda$1@reader1.panix.com>
Message-ID: <4b8e872c-3c95-40c2-b32c-bc6d5d5bac0e@googlegroups.com>

On Sunday, May 10, 2015 at 7:55:22 AM UTC+5:30, vjp... at at.biostrategist.dot.dot.com wrote:
> I have to try this and see if there is ome kind of init file in jython/python
> sorta like autoexec.bat. Ialso have no idea if the commands they provide are
> all it takes to run the app or I have to stay in jython.. sorry, I'm thinking
> at loud.. ok, thanks to all..
> 

Maybe this?

http://www.jython.org/docs/tutorial/interpreter.html?highlight=pythonstartup#the-interactive-startup-file

PS People here tend to prefer avoidance of top-posting
http://en.wikipedia.org/wiki/Posting_style#Top-posting


From steve+comp.lang.python at pearwood.info  Sat May  9 22:45:55 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 10 May 2015 12:45:55 +1000
Subject: functions, optional parameters
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <ui2v1cx5co.ln2@news.c0t0d0s0.de>
Message-ID: <554ec664$0$13001$c3e8da3$5496439d@news.astraweb.com>

On Sat, 9 May 2015 01:50 am, Michael Welle wrote:

[...]
>> How about this definition:
>>
>>     default = 23
>>     def spam(eggs=default):
>>         pass
>>
>>     del default
>>
>>     print spam()
>>
>>
>> Do you expect the function call to fail because `default` doesn't exist?
> 
> If I reference an object, that isn't available in the current context, I
> want to see it fail, yes.

Well, that's an interesting response. Of course I agree with you if the
reference to default is in the code being executed:

def spam():
    value = default


that's quite normal rules for Python functions. 

Aside: note that *closures* behave differently, by design: a closure will
keep non-local values alive even if the parent function is deleted.

py> def outer():
...     default = 23
...     def closure():
...             return default
...     return closure
...
py> f = outer()
py> del outer
py> f()
23


But I don't agree with you about default parameters. Suppose we do this:

default = 23
eggs = default
# some time later
del default
print(eggs)

I trust that you agree that eggs shouldn't raise a NameError here just
because default no longer exists!

Why should that be any different just because the assignment is inside a
parameter list?

def spam(eggs=default):
    ...


One of the nice things about Python's current behaviour is that function
defaults don't behave any differently from any other name binding. Python
uses the same semantics for binding names wherever the name is without the
need for users to memorise a bunch of special rules.

Things which look similar should behave similarly.


>> My answers to those questions are all No.
>
> Different answers are possible as it seems ;).

Obviously :-)

And if Python used late binding, as some other languages do (Lisp, I think),
we would have a FAQ 

"Q: Why does my function run slowly/raise an exception when I use a default
value?"

"A: Because the default is re-evaluated every time you call the function,
not just once when you define it."


>> To me, it is not only expected,
>> but desirable that function defaults are set once, not every time the
>> function is called. This behaviour is called "early binding" of defaults.
>>
>> The opposite behaviour is called "late binding".
>>
>> If your language uses late binding, it is very inconvenient to get early
>> binding when you want it. But if your language uses early binding, it is
>> very simple to get late binding when you want it: just put the code you
>> want to run inside the body of the function:
>
> And you have to do it all the time again and again. I can't provide hard
> numbers, but I think usually I want late binding.

I'm pretty sure that you don't. You just think you do because you're
thinking of the subset of cases where you want to use a mutable default
like [], or perhaps delay looking up a global default until runtime, and
not thinking of all the times you use a default.

I predict that the majority of the time, late binding would just be a
pointless waste of time:

def process_string(thestr, start=0, end=None, slice=1, reverse=True):
    pass

Why would you want 0, None, 1 and True to be re-evaluated every time?
Admittedly it will be fast, but not as fast as evaluating them once, then
grabbing a static default value when needed. (See below for timings.)

Whether you use early or late binding, Python still has to store the
default, then retrieve it at call-time. What happens next depends on the
binding model.

With early binding, Python has the value, and can just use it directly. With
late binding, it needs to store a delayed computation object, an executable
expression if you prefer. There are two obvious ways to implement such a
thunk in Python: a code object, or a function.

thunk = compile('0', '', 'eval')  # when the function is defined
value = eval(thunk)  # when the function is called

# or

thunk = lambda: 0
value = thunk()


Both of those are considerably slower than the current behaviour:

py> from timeit import Timer
py> static = Timer("x = 0")
py> thunk = Timer("x = eval(t)", setup="t = compile('0', '', 'eval')")
py> func = Timer("x = f()", setup="f = lambda: 0")
py> min(static.repeat(repeat=7))  # Best of seven trials.
0.04563648998737335
py> min(thunk.repeat(repeat=7))
1.2324241530150175
py> min(func.repeat(repeat=7))
0.20116623677313328


It would be nice to have syntax for late binding, but given that we don't,
and only have one or the other, using early binding is much more sensible.


This is the point where some people try to suggest some sort of complicated,
fragile, DWIM heuristic where the compiler tries to guess whether the user
actually wants the default to use early or late binding, based on what the
expression looks like. "0 is an immutable int, use early binding; [] is a
mutable list, use late binding." sort of thing. Such a thing might work
well for the obvious cases, but it would be a bugger to debug and
work-around for the non-obvious cases when it guesses wrong -- and it will.


-- 
Steven



From rosuav at gmail.com  Sat May  9 23:33:00 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 10 May 2015 13:33:00 +1000
Subject: functions, optional parameters
In-Reply-To: <554ec664$0$13001$c3e8da3$5496439d@news.astraweb.com>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <ui2v1cx5co.ln2@news.c0t0d0s0.de>
 <554ec664$0$13001$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmq+XKq4SER-Hy-3RK=jAv-rHqNEw9ugJym_yH-d2GwFvQ@mail.gmail.com>

On Sun, May 10, 2015 at 12:45 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> This is the point where some people try to suggest some sort of complicated,
> fragile, DWIM heuristic where the compiler tries to guess whether the user
> actually wants the default to use early or late binding, based on what the
> expression looks like. "0 is an immutable int, use early binding; [] is a
> mutable list, use late binding." sort of thing. Such a thing might work
> well for the obvious cases, but it would be a bugger to debug and
> work-around for the non-obvious cases when it guesses wrong -- and it will.

What you could have is "late-binding semantics, optional early binding
as an optimization but only in cases where the result is
indistinguishable". That would allow common cases (int/bool/str/None
literals) to be optimized, since there's absolutely no way for them to
evaluate differently.

I personally don't think it'd be that good an idea, but it's a simple
enough rule that it wouldn't break anything. As far as anyone's code
is concerned, the rule is "late binding, always". In fact, that would
be the language definition; the rest is an optimization. (It's like
how "x.y()" technically first looks up attribute "y" on object x, then
calls the result; but it's perfectly reasonable for a Python
implementation to notice this extremely common case and do an
"optimized method call" that doesn't actually create a function
object.) The simpler the rule, the easier to grok, and therefore the
less chance of introducing bugs.

ChrisA


From rustompmody at gmail.com  Sat May  9 23:35:33 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sat, 9 May 2015 20:35:33 -0700 (PDT)
Subject: functions, optional parameters
In-Reply-To: <554ec664$0$13001$c3e8da3$5496439d@news.astraweb.com>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <ui2v1cx5co.ln2@news.c0t0d0s0.de>
 <554ec664$0$13001$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <861a42ad-c219-4b1b-bcda-ae5ef4b51968@googlegroups.com>

On Sunday, May 10, 2015 at 8:16:07 AM UTC+5:30, Steven D'Aprano wrote:
> I predict that the majority of the time, late binding would just be a
> pointless waste of time:
> 
> def process_string(thestr, start=0, end=None, slice=1, reverse=True):
>     pass
> 
> Why would you want 0, None, 1 and True to be re-evaluated every time?
> Admittedly it will be fast, but not as fast as evaluating them once, then
> grabbing a static default value when needed. (See below for timings.)
> 

And what is the work involved in (re)computing 0, None, 1, True??

If I write (... arg=square_root_of_grahams_number())
I would expect to pay for it.
If I write trivial defaults, then I expect trivial payment.


From steve+comp.lang.python at pearwood.info  Sun May 10 01:20:18 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 10 May 2015 15:20:18 +1000
Subject: functions, optional parameters
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <ui2v1cx5co.ln2@news.c0t0d0s0.de>
 <554ec664$0$13001$c3e8da3$5496439d@news.astraweb.com>
 <mailman.298.1431228783.12865.python-list@python.org>
Message-ID: <554eea94$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Sun, 10 May 2015 01:33 pm, Chris Angelico wrote:

> On Sun, May 10, 2015 at 12:45 PM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>> This is the point where some people try to suggest some sort of
>> complicated, fragile, DWIM heuristic where the compiler tries to guess
>> whether the user actually wants the default to use early or late binding,
>> based on what the expression looks like. "0 is an immutable int, use
>> early binding; [] is a mutable list, use late binding." sort of thing.
>> Such a thing might work well for the obvious cases, but it would be a
>> bugger to debug and work-around for the non-obvious cases when it guesses
>> wrong -- and it will.
> 
> What you could have is "late-binding semantics, optional early binding
> as an optimization but only in cases where the result is
> indistinguishable". That would allow common cases (int/bool/str/None
> literals) to be optimized, since there's absolutely no way for them to
> evaluate differently.
> 
> I personally don't think it'd be that good an idea, but it's a simple
> enough rule that it wouldn't break anything.

It's a change in semantics, and it would break code that expects early
binding.


> As far as anyone's code 
> is concerned, the rule is "late binding, always".

Sure, other languages have made that choice. I think it is the wrong choice,
but if we went back to 1991 Guido could have made that same choice.


> In fact, that would 
> be the language definition; the rest is an optimization. (It's like
> how "x.y()" technically first looks up attribute "y" on object x, then
> calls the result; but it's perfectly reasonable for a Python
> implementation to notice this extremely common case and do an
> "optimized method call" that doesn't actually create a function
> object.)

class X:
   def y(self): pass

y is already a function object.

I think maybe you've got it backwards, and you mean the *method* object
doesn't have to be created. Well, sure, that's possible, and maybe PyPy
does something like that, and maybe it doesn't. Or maybe the function
descriptor __get__ method could cache the result:

# inside FunctionType class
    def __get__(self, instance, type):
        if type is not None:
            if self._method is None:
                self._method = MethodType(self, instance)
            return self._method
        else:
            return self

(I think that's more or less how function __get__ currently works, apart
from the caching. But don't quote me.)

But that's much simpler than the early/late binding example. You talk
about "the obvious cases" like int, bool, str and None. What about floats
and frozensets, are they obvious? How about tuples? How about
MyExpensiveImmutableObject?


> The simpler the rule, the easier to grok, and therefore the 
> less chance of introducing bugs.

You're still going to surprise people who expect early binding:

FLAG = True

def spam(eggs=FLAG):
    ...


What do you mean, the default value gets recalculated every time I call
spam? It's an obvious immutable type! And why does Python crash when I
delete FLAG?

Worse:


def factory():
    funcs = []
    for i in range(1, 5):
        def adder(x, y=i):
            return x + y
        adder.__name__ = "adder%d" % i
        funcs.append(adder)
    return funcs


The current behaviour with early binding:


py> funcs = factory()
py> [f(100) for f in funcs]
[101, 102, 103, 104]


What would it do with late binding? That's a tricky one. I can see two
likely results:

[f(100) for f in funcs]
=> returns [104, 104, 104, 104]

or

NameError: name 'i' is not defined


both of which are significantly less useful.

As I've said, it is trivial to get late binding semantics if you start with
early binding: just move setting the default value into the body of the
function. 99% of the time you can use None as a sentinel, so the common
case is easy:

def func(x=None):
    if x is None: 
        x = some_complex_calculation(i, want, to, repeat, each, time)


and the rest of the time, you just need *one* persistent variable to hold a
sentinel value to use instead of None:

_sentinel = object
def func(x=_sentinel, y=_sentinel, z=_sentinel):
    if x is _sentinel: ...


But if you start with late binding, it's hard to *cleanly* get early binding
semantics. You need a separate global for each parameter of every function
in the module:

_default_x = some_complex_calculation(do, it, once)
_default_y = another_complex_calculation(do, it, once)
_default_z = a_third_complex_calculation(do, it, once)
_default_x_for_some_other_function = something_else()


def func(x=_default_x, y=_default_x, z=_default_z):  # oops, see the bug
    ...


which is just hideous. And even then, you don't really have early binding,
you have a lousy simulacra of it. If you modify or delete any of the
default globals, you're screwed.

No, early binding by default is the only sensible solution, and Guido got it
right. Having syntax for late binding would be a bonus, but it isn't really
needed. We already have a foolproof and simple way to evaluate an
expression at function call-time: put it in the body of the function.


-- 
Steven



From steve+comp.lang.python at pearwood.info  Sun May 10 01:25:32 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 10 May 2015 15:25:32 +1000
Subject: functions, optional parameters
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <ui2v1cx5co.ln2@news.c0t0d0s0.de>
 <554ec664$0$13001$c3e8da3$5496439d@news.astraweb.com>
 <861a42ad-c219-4b1b-bcda-ae5ef4b51968@googlegroups.com>
Message-ID: <554eebcc$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Sun, 10 May 2015 01:35 pm, Rustom Mody wrote:

> On Sunday, May 10, 2015 at 8:16:07 AM UTC+5:30, Steven D'Aprano wrote:
>> I predict that the majority of the time, late binding would just be a
>> pointless waste of time:
>> 
>> def process_string(thestr, start=0, end=None, slice=1, reverse=True):
>>     pass
>> 
>> Why would you want 0, None, 1 and True to be re-evaluated every time?
>> Admittedly it will be fast, but not as fast as evaluating them once, then
>> grabbing a static default value when needed. (See below for timings.)
>> 
> 
> And what is the work involved in (re)computing 0, None, 1, True??

Re-computing a constant is about 5 times more expensive than re-using it,
according to my earlier timing tests. So if you have four of them, there
will be about 20 times more overhead due to the defaults, each and every
time you call the function.

Setting the defaults isn't the only source of overhead, but my guestimate is
that switching to late binding would probably double the overall overhead
of calling a function with one or two defaults. If your function is
expensive, that's trivial, but for small fast functions, that will be
painful. Python's slow enough without making it slower for dubious gains.


> If I write (... arg=square_root_of_grahams_number())
> I would expect to pay for it.

Sure, but only once. If you think that Graham's Number is likely to change
*wink* then you can put it into the body of the function, like any other
code you want run every time you call the function.



-- 
Steven



From dfnsonfsduifb at gmx.de  Sun May 10 04:04:48 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sun, 10 May 2015 10:04:48 +0200
Subject: Is this unpythonic?
In-Reply-To: <mailman.239.1431086688.12865.python-list@python.org>
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mii4il$vv$1@ger.gmane.org>
 <mailman.239.1431086688.12865.python-list@python.org>
Message-ID: <min3f0$2gh$1@news.albasani.net>

On 08.05.2015 14:04, Dave Angel wrote:

> It might be appropriate to define the list at top-level, as
> 
> EMPTY_LIST=[]
> 
> and in your default argument as
>     def x(y, z=EMPTY_LIST):
> 
> and with the all-caps, you're thereby promising that nobody will modify
> that list.
> 
> (I'd tend to do the None trick, but I think this alternative would be
> acceptable)

I think it's a really bad idea to use a module-global mutable
"EMPTY_LIST". It's much too easy this happens:

# Globally
>>> EMPTY_LIST = [ ]

# At somewhere in the code at some point in time
>>> foo = EMPTY_LIST
>>> foo.append(123)
>>> print(foo)
[123]

# Some other place in code
>>> bar = EMPTY_LIST
>>> print(bar)
[123]

Regards,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From frank at chagford.com  Sun May 10 04:58:44 2015
From: frank at chagford.com (Frank Millman)
Date: Sun, 10 May 2015 10:58:44 +0200
Subject: Is this unpythonic?
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mii4il$vv$1@ger.gmane.org>
 <mailman.239.1431086688.12865.python-list@python.org>
 <min3f0$2gh$1@news.albasani.net>
Message-ID: <min6l4$bob$1@ger.gmane.org>


"Johannes Bauer" <dfnsonfsduifb at gmx.de> wrote in message 
news:min3f0$2gh$1 at news.albasani.net...
On 08.05.2015 14:04, Dave Angel wrote:

> > It might be appropriate to define the list at top-level, as
> >
> > EMPTY_LIST=[]
> >
> > and in your default argument as
> >     def x(y, z=EMPTY_LIST):
> >
> > and with the all-caps, you're thereby promising that nobody will modify
> > that list.

> I think it's a really bad idea to use a module-global mutable
> "EMPTY_LIST". It's much too easy this happens:

> # Globally
> >>> EMPTY_LIST = [ ]

> # At somewhere in the code at some point in time
> >>> foo = EMPTY_LIST
> >>> foo.append(123)
> >>> print(foo)
> [123]

> # Some other place in code
> >>> bar = EMPTY_LIST
> >>> print(bar)
> [123]


A fair point. How about this as an alternative?

If one were to use this technique at all, it would be necessary to add a 
comment at the top explaining the reason for this odd declaration.

It is then a simple extra step to say -

EMPTY_L:IST = ()

and if required -

EMPTY_DICT = ()

and expand the explanation to show why a tuple is used instead.

So if there was a situation where the overhead of testing for None became a 
problem, this solution offers the following -

1. it solves the 'overhead' problem
2. it reads reasonably intuitively in the body of the program
3. it is safe
4. it should not be difficult to write a suitable self-explanatory comment

Frank





From rosuav at gmail.com  Sun May 10 04:59:38 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 10 May 2015 18:59:38 +1000
Subject: functions, optional parameters
In-Reply-To: <554eea94$0$13004$c3e8da3$5496439d@news.astraweb.com>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <ui2v1cx5co.ln2@news.c0t0d0s0.de>
 <554ec664$0$13001$c3e8da3$5496439d@news.astraweb.com>
 <mailman.298.1431228783.12865.python-list@python.org>
 <554eea94$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmpsLRacEhPSgw+fgZSurQA5XqNKTVA11tFVDDT9HbL8jw@mail.gmail.com>

(To clarify, I am *not* talking about this as a change to Python, so
all questions of backward compatibility are immaterial. This is "what
happens if we go back in time and have Python use late binding
semantics". This is the "alternate 1985" of Back to the Future.)

On Sun, May 10, 2015 at 3:20 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Sun, 10 May 2015 01:33 pm, Chris Angelico wrote:
>> In fact, that would
>> be the language definition; the rest is an optimization. (It's like
>> how "x.y()" technically first looks up attribute "y" on object x, then
>> calls the result; but it's perfectly reasonable for a Python
>> implementation to notice this extremely common case and do an
>> "optimized method call" that doesn't actually create a function
>> object.)
>
> class X:
>    def y(self): pass
>
> y is already a function object.
>
> I think maybe you've got it backwards, and you mean the *method* object
> doesn't have to be created. Well, sure, that's possible, and maybe PyPy
> does something like that, and maybe it doesn't. Or maybe the function
> descriptor __get__ method could cache the result:

Apologies, that was indeed an error of terminology. I did indeed mean
the method object that doesn't have to be created. There is already a
function object (which can be identified as X.y - Py2 differences
needn't concern us here), and AFAIK, a peephole optimizer can
transform this safely:

x = X()
x.y()
# into
x = X()
X.y(x)

That's an optimization that can't possibly change the result (at
least, I'm not aware of a way that it can; I may be wrong), and so
it's a viable change for something like PyPy to do. But semantically,
a bound method object is still created, which means it's fully legal
to split that into two parts:

x = X()
f = x.y
f()

The only result should be that this defeats the optimization, so you
end up paying a greater cost in object (de)allocations.

> But that's much simpler than the early/late binding example. You talk
> about "the obvious cases" like int, bool, str and None. What about floats
> and frozensets, are they obvious? How about tuples? How about
> MyExpensiveImmutableObject?

Simple: if the optimizer doesn't know about them, they go by the
regular rule. As there's no semantic difference, there cannot be any
true effect beyond performance. Floats can easily be added to the list
I gave; tuples could be, as long as their members are also immutable;
frozenset doesn't have a literal form, nor would
MyExpensiveImmutableObject, so they would miss out on this benefit.

>> The simpler the rule, the easier to grok, and therefore the
>> less chance of introducing bugs.
>
> You're still going to surprise people who expect early binding:
>
> FLAG = True
>
> def spam(eggs=FLAG):
>     ...
>
>
> What do you mean, the default value gets recalculated every time I call
> spam? It's an obvious immutable type! And why does Python crash when I
> delete FLAG?

Still simple: Since late binding is the semantically-mandated
behaviour, this will always reevaluate FLAG - the optimizer has been
bypassed here. It's not an obvious immutable type - the example I
actually gave was "int/bool/str/None *literals*", not *values*. Here's
a non-toy example that would use this kind of flag-lookup semantics
usefully:

default_timeout = 60 # seconds

def url_get(url, timeout=default_timeout):
    """Perform a GET request and return the data"""

def url_post(url, body, timeout=default_timeout):
    """Perform a POST request and return the data"""

def dns_lookup(server, name, type="A", class="IN", timeout=default_timeout):
    """Send a DNS request and await a response"""


By changing modulename.default_timeout, you instantly change all of
the functions' defaults. In current Python, this would have to be done
as:

def url_get(url, timeout=None):
    if timeout is None: timeout = default_timeout

which duplicates that code down all of them, and it means that
introspection of the function can't show you what it's actually doing.
With late binding, an introspection could yield both the expression
used ("default_timeout") and, with evaluation, the effective default.

Now, this is a rarity. This is far FAR less common than the situations
where early binding is better. But there are places where it would
make sense.

> Worse:
>
>
> def factory():
>     funcs = []
>     for i in range(1, 5):
>         def adder(x, y=i):
>             return x + y
>         adder.__name__ = "adder%d" % i
>         funcs.append(adder)
>     return funcs
>
>
> The current behaviour with early binding:
>
>
> py> funcs = factory()
> py> [f(100) for f in funcs]
> [101, 102, 103, 104]
>
>
> What would it do with late binding? That's a tricky one. I can see two
> likely results:
>
> [f(100) for f in funcs]
> => returns [104, 104, 104, 104]
>
> or
>
> NameError: name 'i' is not defined
>
>
> both of which are significantly less useful.

I'd say the former makes more sense - it's what would happen if you
evaluated the expression "i" in the context of that factory function.
But yes, significantly less useful than early binding; I'm not sure
how to cleanly implement that kind of metaprogramming otherwise.

> As I've said, it is trivial to get late binding semantics if you start with
> early binding: just move setting the default value into the body of the
> function. 99% of the time you can use None as a sentinel, so the common
> case is easy:
>
> def func(x=None):
>     if x is None:
>         x = some_complex_calculation(i, want, to, repeat, each, time)
>
>
> and the rest of the time, you just need *one* persistent variable to hold a
> sentinel value to use instead of None:
>
> _sentinel = object
> def func(x=_sentinel, y=_sentinel, z=_sentinel):
>     if x is _sentinel: ...

Presumably that would instantiate an object() rather than using the
object type itself, but yes. Sometimes it'd be nice to be able to get
something with a more useful repr, but that's not a big deal.

> But if you start with late binding, it's hard to *cleanly* get early binding
> semantics. You need a separate global for each parameter of every function
> in the module:
>
> _default_x = some_complex_calculation(do, it, once)
> _default_y = another_complex_calculation(do, it, once)
> _default_z = a_third_complex_calculation(do, it, once)
> _default_x_for_some_other_function = something_else()
>
>
> def func(x=_default_x, y=_default_x, z=_default_z):  # oops, see the bug
>     ...

Yup, I see it... but I quite probably wouldn't if your variable names
were less toyish. Even as it is, the important info is getting lost in
this sea of "=_default_" that keeps having to be repeated.

> No, early binding by default is the only sensible solution, and Guido got it
> right. Having syntax for late binding would be a bonus, but it isn't really
> needed. We already have a foolproof and simple way to evaluate an
> expression at function call-time: put it in the body of the function.

I agree. As I said at the top, this is all just what happens if Biff
is in charge instead of Guido. It's not instantly internally
inconsistent, but it is a lot less useful than the early binding we
currently have.

The advantage of a late-binding syntax is that it could be visible in
the function signature, instead of being buried inside. If we had
something like this:

def print_list(lst, start=0, end==len(lst)):
    """Print out some or all elements of a given list"""

then it'd be obvious that the one-arg behaviour is to print out the
whole list; otherwise, you'd see None up there, and have to presume
that it means "to end of list". But syntax has to justify itself with
a lot more than uber-rare cases like these.

ChrisA


From dfnsonfsduifb at gmx.de  Sun May 10 05:54:43 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sun, 10 May 2015 11:54:43 +0200
Subject: Is this unpythonic?
In-Reply-To: <mailman.302.1431248365.12865.python-list@python.org>
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mii4il$vv$1@ger.gmane.org>
 <mailman.239.1431086688.12865.python-list@python.org>
 <min3f0$2gh$1@news.albasani.net>
 <mailman.302.1431248365.12865.python-list@python.org>
Message-ID: <min9t3$e56$1@news.albasani.net>

On 10.05.2015 10:58, Frank Millman wrote:

> It is then a simple extra step to say -
> 
> EMPTY_L:IST = ()
> 
> and if required -
> 
> EMPTY_DICT = ()
> 
> and expand the explanation to show why a tuple is used instead.
> 
> So if there was a situation where the overhead of testing for None became a 
> problem, this solution offers the following -
> 
> 1. it solves the 'overhead' problem
> 2. it reads reasonably intuitively in the body of the program
> 3. it is safe
> 4. it should not be difficult to write a suitable self-explanatory comment

I do understand what you're trying to do, but it is my gut-feeling that
you're overengineering this and as a side-effect introducing new problems.

With the above declaration as you describe, the code becomes weird:

foo = EMPTY_LIST
foo.append(123)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'tuple' object has no attribute 'append'

and

foo = EMPTY_DICT
foo["bar"] = "moo"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment

So instead, the user of this construct would have to

foo = list(EMPTY_LIST)

or

foo = dict(EMPTY_DICT)

but, coincidentially, this is easier (and more pythonic) by doing

foo = list()
foo = dict()

to which there are the obvious (pythonic) shortcuts

foo = [ ]
foo = { }

All in all, I'd be more confused why someone would introduct
"EMPTY_LIST" in the first place and think there's some strange weird
reason behind it. Explaining the reason in the comments doesn't really
help in my opinion.

Best regards,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From steve+comp.lang.python at pearwood.info  Sun May 10 05:58:13 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 10 May 2015 19:58:13 +1000
Subject: Calling a function is faster than not calling it?
Message-ID: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>

Why is calling a function faster than bypassing the function object and
evaluating the code object itself? And not by a little, but by a lot?

Here I have a file, eval_test.py:

# === cut ===
from timeit import Timer

def func():
    a = 2
    b = 3
    c = 4
    return (a+b)*(a-b)/(a*c + b*c)


code = func.__code__
assert func() == eval(code)

t1 = Timer("eval; func()", setup="from __main__ import func")
t2 = Timer("eval(code)", setup="from __main__ import code")

# Best of 10 trials.
print (min(t1.repeat(repeat=10)))
print (min(t2.repeat(repeat=10)))

# === cut ===


Note that both tests include a name lookup for eval, so that as much as
possible I am comparing the two pieces of code on an equal footing.

Here are the results I get:


[steve at ando ~]$ python2.7 eval_test.py
0.804041147232
1.74012994766
[steve at ando ~]$ python3.3 eval_test.py
0.7233301624655724
1.7154695875942707

Directly eval'ing the code object is easily more than twice as expensive
than calling the function, but calling the function has to eval the code
object. That suggests that the overhead of calling the function is
negative, which is clearly ludicrous.

I knew that calling eval() on a string was slow, as it has to parse and
compile the source code into byte code before it can evaluate it, but this
is pre-compiled and shouldn't have that overhead.

So what's going on?



-- 
Steven



From auriocus at gmx.de  Sun May 10 06:34:55 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Sun, 10 May 2015 12:34:55 +0200
Subject: Calling a function is faster than not calling it?
In-Reply-To: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <minc6e$loc$1@dont-email.me>

Am 10.05.15 um 11:58 schrieb Steven D'Aprano:
> Why is calling a function faster than bypassing the function object and
> evaluating the code object itself? And not by a little, but by a lot?
>
> Here I have a file, eval_test.py:
>
> # === cut ===
> from timeit import Timer
>
> def func():
>      a = 2
>      b = 3
>      c = 4
>      return (a+b)*(a-b)/(a*c + b*c)
>
>
> code = func.__code__
> assert func() == eval(code)
>
> t1 = Timer("eval; func()", setup="from __main__ import func")
> t2 = Timer("eval(code)", setup="from __main__ import code")
>
> # Best of 10 trials.
> print (min(t1.repeat(repeat=10)))
> print (min(t2.repeat(repeat=10)))

what exactly does this mean, are you calling it 10 times? Are you sure 
hat is enaough to reach the granularity of your clock? A benchmark 
should last at least 100ms IMHO.

> # === cut ===
>
>
> Note that both tests include a name lookup for eval, so that as much as
> possible I am comparing the two pieces of code on an equal footing.
>
> Here are the results I get:
>
>
> [steve at ando ~]$ python2.7 eval_test.py
> 0.804041147232
> 1.74012994766
> [steve at ando ~]$ python3.3 eval_test.py
> 0.7233301624655724
> 1.7154695875942707
>
> Directly eval'ing the code object is easily more than twice as expensive
> than calling the function, but calling the function has to eval the code
> object. That suggests that the overhead of calling the function is
> negative, which is clearly ludicrous.
>
> I knew that calling eval() on a string was slow, as it has to parse and
> compile the source code into byte code before it can evaluate it, but this
> is pre-compiled and shouldn't have that overhead.

Does eval(code) lookup a,b and c in the scope of the eval, whereas in 
the func it is bound to locals and optimized out?

I've got no idea of Python's internals, just guessing.


	Christian


From frank at chagford.com  Sun May 10 06:40:47 2015
From: frank at chagford.com (Frank Millman)
Date: Sun, 10 May 2015 12:40:47 +0200
Subject: Is this unpythonic?
References: <mailman.224.1431072121.12865.python-list@python.org>
 <554c8b0a$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mii4il$vv$1@ger.gmane.org>
 <mailman.239.1431086688.12865.python-list@python.org>
 <min3f0$2gh$1@news.albasani.net>
 <mailman.302.1431248365.12865.python-list@python.org>
 <min9t3$e56$1@news.albasani.net>
Message-ID: <minf2j$3g6$1@ger.gmane.org>


"Johannes Bauer" <dfnsonfsduifb at gmx.de> wrote in message 
news:min9t3$e56$1 at news.albasani.net...
On 10.05.2015 10:58, Frank Millman wrote:

> > It is then a simple extra step to say -
> >
> > EMPTY_L:IST = ()
> >
> > and if required -
> >
> > EMPTY_DICT = ()
> >
> > and expand the explanation to show why a tuple is used instead.
> >
> > So if there was a situation where the overhead of testing for None 
> > became a
> > problem, this solution offers the following -
> >
> > 1. it solves the 'overhead' problem
> > 2. it reads reasonably intuitively in the body of the program
> > 3. it is safe
> > 4. it should not be difficult to write a suitable self-explanatory 
> > comment

> I do understand what you're trying to do, but it is my gut-feeling that
> you're overengineering this and as a side-effect introducing new problems.

This has actually gone beyond a practical suggestion, and become more of an 
academic exercise, so I don't think 'overengineering' comes into it. No-one 
is recommending that this should be used in real-world code.

> With the above declaration as you describe, the code becomes weird:
>
> foo = EMPTY_LIST
> foo.append(123)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> AttributeError: 'tuple' object has no attribute 'append'
>
> and
>
> foo = EMPTY_DICT
> foo["bar"] = "moo"
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: 'tuple' object does not support item assignment

The whole point of this admittedly odd declaration is that no-one should do 
anything with it at all. It is simply a place-holder to be used in the 
absence of a real list or dict provided by the caller of the function. The 
starting premise was that the function would only read from the list/dict, 
not try to modify it.

Frank





From __peter__ at web.de  Sun May 10 06:43:42 2015
From: __peter__ at web.de (Peter Otten)
Date: Sun, 10 May 2015 12:43:42 +0200
Subject: Calling a function is faster than not calling it?
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mincov$2ul$1@ger.gmane.org>

Steven D'Aprano wrote:

> Why is calling a function faster than bypassing the function object and
> evaluating the code object itself? And not by a little, but by a lot?

> Directly eval'ing the code object is easily more than twice as expensive
> than calling the function, but calling the function has to eval the code
> object. That suggests that the overhead of calling the function is
> negative, which is clearly ludicrous.
> 
> I knew that calling eval() on a string was slow, as it has to parse and
> compile the source code into byte code before it can evaluate it, but this
> is pre-compiled and shouldn't have that overhead.
> 
> So what's going on?

A significant part of the extra time is apparently spent on stack 
inspection:

$ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}' 'f()'
10000000 loops, best of 3: 0.179 usec per loop

$ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}' 
'eval(code)'
1000000 loops, best of 3: 0.852 usec per loop

$ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}' 
'eval(code, ns)'
1000000 loops, best of 3: 0.433 usec per loop

$ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}' 'eval; 
ns; f()'
1000000 loops, best of 3: 0.263 usec per loop




From davea at davea.name  Sun May 10 07:13:49 2015
From: davea at davea.name (Dave Angel)
Date: Sun, 10 May 2015 07:13:49 -0400
Subject: getting fieldnames from Dictreader before reading lines
In-Reply-To: <CALyJZZXPoRU2v6a29u9UKMJLytP2=XOOaRu71fgxRjr3w5sgeA@mail.gmail.com>
References: <CALyJZZV8XFjbmiNRYxoAFuGHS2iix4knv-smn5adicVRAJ7ZCg@mail.gmail.com>
 <CALyJZZVt_X5pYQg+rXmP4CHx9-LizL4vbTQv-7t4ihD5xGqXng@mail.gmail.com>
 <554E9E59.2080906@davea.name>
 <CALyJZZXPoRU2v6a29u9UKMJLytP2=XOOaRu71fgxRjr3w5sgeA@mail.gmail.com>
Message-ID: <554F3D6D.9060202@davea.name>

On 05/09/2015 09:51 PM, Vincent Davis wrote:
> On Sat, May 9, 2015 at 5:55 PM, Dave Angel <davea at davea.name> wrote:
>>
>> 1) you're top-posting, putting your response  BEFORE the stuff you're
> responding to.
>
>
> I responded to my own email, seemed ok to top post on myself saying it was
> resolved.

Yeah, I overreacted.  There has been a lot of top-posting lately, but if 
a message is expected to be the end of the thread, I shouldn't care what 
it looks like.  Sorry.

>
>>
>> 2) both messages are in html, which thoroughly messed up parts of your
> error messages.
>
> I am posting from google mail (not google groups). Kindly let me know if
> this email is also html.

Still html.  It didn't matter in this case, but html can cause painful 
formatting.  And worse, it'll be different for different recipients.  So 
some people will see exactly what was sent, while others will see things 
messed up a little, or a lot.  The formatting can get messed up by the 
sender's mailer, or by the receiver's mail program, or both.

Dennis showed you what your current message looked like, so I won't 
repeat the whole thing.  But your original message had a 
"multipart/alternative" which gets interpreted as plain text by many 
mail programs (including mine == Thunderbird), but it was already 
misformatted, with newlines in the wrong places.  And it had an "html", 
which was correctly formatted, or at least could be interpreted 
correctly by mine.

So the output of google mail was in this case just plain wrong.

Any reason you're using google mail when you have your own email address 
+ domain?


-- 
DaveA


From davea at davea.name  Sun May 10 07:25:58 2015
From: davea at davea.name (Dave Angel)
Date: Sun, 10 May 2015 07:25:58 -0400
Subject: functions, optional parameters
In-Reply-To: <CAPTjJmq+XKq4SER-Hy-3RK=jAv-rHqNEw9ugJym_yH-d2GwFvQ@mail.gmail.com>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <ui2v1cx5co.ln2@news.c0t0d0s0.de>
 <554ec664$0$13001$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmq+XKq4SER-Hy-3RK=jAv-rHqNEw9ugJym_yH-d2GwFvQ@mail.gmail.com>
Message-ID: <554F4046.6000101@davea.name>

On 05/09/2015 11:33 PM, Chris Angelico wrote:
> On Sun, May 10, 2015 at 12:45 PM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>> This is the point where some people try to suggest some sort of complicated,
>> fragile, DWIM heuristic where the compiler tries to guess whether the user
>> actually wants the default to use early or late binding, based on what the
>> expression looks like. "0 is an immutable int, use early binding; [] is a
>> mutable list, use late binding." sort of thing. Such a thing might work
>> well for the obvious cases, but it would be a bugger to debug and
>> work-around for the non-obvious cases when it guesses wrong -- and it will.
>
> What you could have is "late-binding semantics, optional early binding
> as an optimization but only in cases where the result is
> indistinguishable". That would allow common cases (int/bool/str/None
> literals) to be optimized, since there's absolutely no way for them to
> evaluate differently.
>

Except for literals, True, False and None, I can't see any way to 
optimize such a thing.  Just because the name on the right side 
references an immutable object at compile time, it doesn't follow that 
it'll still be the same object later.

Unless late binding means something very different than I understood.

-- 
DaveA


From techtonik at gmail.com  Sun May 10 09:57:06 2015
From: techtonik at gmail.com (anatoly techtonik)
Date: Sun, 10 May 2015 16:57:06 +0300
Subject: [pydotorg-www] www.python.org - Backend is unhealthy
In-Reply-To: <C0ADC633-FBE7-4B24-A5AD-F26ED35F36D9@coderanger.net>
References: <6fccd558-c78d-4225-919d-f129c8aafae4@googlegroups.com>
 <CAPTjJmpKmtTiBKoFLhc2sMSyE7sZH_dAziOPOM2XNrf1QqFm+w@mail.gmail.com>
 <C0ADC633-FBE7-4B24-A5AD-F26ED35F36D9@coderanger.net>
Message-ID: <CAPkN8xKH3E5ViifyobQ_wKdRyeMGcM2LqtZmR+iRQz92_K75+w@mail.gmail.com>

On Fri, May 8, 2015 at 11:32 AM, Noah Kantrowitz <noah at coderanger.net> wrote:
>
> On May 8, 2015, at 10:28 AM, Chris Angelico <rosuav at gmail.com> wrote:
>
>> On Fri, May 8, 2015 at 6:17 PM,  <jarausch at skynet.be> wrote:
>>> I cannot access www.python.org.
>>> I always get
>>>
>>> Error 503 Backend is unhealthy
>>>
>>> Backend is unhealthy
>>>
>>> Guru Mediation:
>>>
>>> Details: cache-ams4149-AMS 1431072956 2041303800
>>>
>>> Varnish cache server
>>>
>>>
>>> Is it only me?
>>>
>>
>> No, it's not only you. I get the same but with different details:
>>
>> Details: cache-syd1627-SYD 1431073575 864283876
>>
>> It looks to me as if my result is coming from a cache node in Sydney;
>> yours is coming from some other cache node, so it's not just one node
>> that's down.
>>
>> Cc'ing in the www list in case someone there knows, and I'll create a
>> github issue to ping the people there.
>
> Should be recovering now.

I see those once in a while. Not sure if it is only for PyPI, wiki or both.
What is the reason?

-- 
anatoly t.


From rosuav at gmail.com  Sun May 10 10:39:57 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 11 May 2015 00:39:57 +1000
Subject: functions, optional parameters
In-Reply-To: <554F4046.6000101@davea.name>
References: <72lu1cxvmg.ln2@news.c0t0d0s0.de>
 <554cd511$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <ui2v1cx5co.ln2@news.c0t0d0s0.de>
 <554ec664$0$13001$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmq+XKq4SER-Hy-3RK=jAv-rHqNEw9ugJym_yH-d2GwFvQ@mail.gmail.com>
 <554F4046.6000101@davea.name>
Message-ID: <CAPTjJmp3QKMvRJ_bX7CP3DsV_bZkJGPaOcx_zLH1mPSsEumHZA@mail.gmail.com>

On Sun, May 10, 2015 at 9:25 PM, Dave Angel <davea at davea.name> wrote:
> On 05/09/2015 11:33 PM, Chris Angelico wrote:
>> What you could have is "late-binding semantics, optional early binding
>> as an optimization but only in cases where the result is
>> indistinguishable". That would allow common cases (int/bool/str/None
>> literals) to be optimized, since there's absolutely no way for them to
>> evaluate differently.
>>
>
> Except for literals, True, False and None, I can't see any way to optimize
> such a thing.

I did specifically say "literals" :)

ChrisA


From steve+comp.lang.python at pearwood.info  Sun May 10 10:49:50 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 11 May 2015 00:49:50 +1000
Subject: Calling a function is faster than not calling it?
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <mailman.304.1431254634.12865.python-list@python.org>
Message-ID: <554f700f$0$13011$c3e8da3$5496439d@news.astraweb.com>

On Sun, 10 May 2015 08:43 pm, Peter Otten wrote:

> A significant part of the extra time is apparently spent on stack
> inspection:

I don't know what you mean by "stack inspection", or how you come to that
conclusion.


> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
> 'f()' 10000000 loops, best of 3: 0.179 usec per loop
> 
> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
> 'eval(code)'
> 1000000 loops, best of 3: 0.852 usec per loop
> 
> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
> 'eval(code, ns)'
> 1000000 loops, best of 3: 0.433 usec per loop

Curious.

If I'm reading that correctly, supplying an explicit namespace cuts the time
by a factor of two. That surprises me, as your function doesn't do any name
lookups in the body of the function, so I would have thought that would be
irrelevant.


> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
> 'eval; ns; f()'
> 1000000 loops, best of 3: 0.263 usec per loop

So, roughly speaking, calling eval(code, ns) takes about 1.6 times as long
as calling the function; calling eval(code) without providing a namespace
takes about 3.2 times as long. That's roughly consistent with the results
I'm getting.  



-- 
Steven



From steve+comp.lang.python at pearwood.info  Sun May 10 11:04:12 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 11 May 2015 01:04:12 +1000
Subject: Calling a function is faster than not calling it?
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <minc6e$loc$1@dont-email.me>
Message-ID: <554f736d$0$12981$c3e8da3$5496439d@news.astraweb.com>

On Sun, 10 May 2015 08:34 pm, Christian Gollwitzer wrote:

> Am 10.05.15 um 11:58 schrieb Steven D'Aprano:
>> Why is calling a function faster than bypassing the function object and
>> evaluating the code object itself? And not by a little, but by a lot?
>>
>> Here I have a file, eval_test.py:
>>
>> # === cut ===
>> from timeit import Timer
>>
>> def func():
>>      a = 2
>>      b = 3
>>      c = 4
>>      return (a+b)*(a-b)/(a*c + b*c)
>>
>>
>> code = func.__code__
>> assert func() == eval(code)
>>
>> t1 = Timer("eval; func()", setup="from __main__ import func")
>> t2 = Timer("eval(code)", setup="from __main__ import code")
>>
>> # Best of 10 trials.
>> print (min(t1.repeat(repeat=10)))
>> print (min(t2.repeat(repeat=10)))
> 
> what exactly does this mean, are you calling it 10 times? Are you sure
> hat is enaough to reach the granularity of your clock? A benchmark
> should last at least 100ms IMHO.


No, timeit by default calls the code snippet one million times. So in the
example above, each trial calls "eval(code)" one million times, and I have
ten trials. I pick the fastest trial, and report that.

The time for the fastest trial is 1.7 seconds for one million calls to eval.


>> # === cut ===
>>
>>
>> Note that both tests include a name lookup for eval, so that as much as
>> possible I am comparing the two pieces of code on an equal footing.
>>
>> Here are the results I get:
>>
>>
>> [steve at ando ~]$ python2.7 eval_test.py
>> 0.804041147232
>> 1.74012994766
>> [steve at ando ~]$ python3.3 eval_test.py
>> 0.7233301624655724
>> 1.7154695875942707
>>
>> Directly eval'ing the code object is easily more than twice as expensive
>> than calling the function, but calling the function has to eval the code
>> object. That suggests that the overhead of calling the function is
>> negative, which is clearly ludicrous.
>>
>> I knew that calling eval() on a string was slow, as it has to parse and
>> compile the source code into byte code before it can evaluate it, but
>> this is pre-compiled and shouldn't have that overhead.
> 
> Does eval(code) lookup a,b and c in the scope of the eval, whereas in
> the func it is bound to locals and optimized out?

Obviously I don't *fully* understand the details either, otherwise I
wouldn't be asking, but as far as I understanding, in my example above:

def func():
    a = 2
    b = 3
    c = 4
    return (a+b)*(a-b)/(a*c + b*c)

the function's code object includes a reference to the three constants (2,
3, 4, also None) and the local variable look-ups just grab them from the
internal cache:

func.__code__.co_consts
=> returns (None, 2, 3, 4)

So I can set globals with the same name, then eval the code object, and
still get the right answer:

py> a, b, c = 100, 50, 25
py> func()
-0.25
py> eval(func.__code__)
-0.25



-- 
Steven



From somelauw at gmail.com  Sun May 10 11:53:08 2015
From: somelauw at gmail.com (Somelauw .)
Date: Sun, 10 May 2015 17:53:08 +0200
Subject: Why does unicode-escape decode escape symbols that are already
 escaped?
Message-ID: <CA+gt_a82WGXHUZhcdbTUWG+TRV1Ys1ZSrkGjOxgavZGjAh9FiQ@mail.gmail.com>

In Python 3, decoding "?" with unicode-escape returns '?\x82?' which in my
opinion doesn't make sense.
The ? already is decoded; if it were encoded it would look like this:
'\u20ac'.
So why is it doing this?

In Python 2 the behaviour is similar, but slightly different.

$ python3 -S
Python 3.3.3 (default, Nov 27 2013, 17:12:35)
[GCC 4.8.2] on linux
>>> import codecs
>>> codecs.decode('?', 'unicode-escape')
'?\x82?'
>>> codecs.encode('?', 'unicode-escape')
b'\\u20ac'
>>>

$ python2 -S
Python 2.7.5+ (default, Sep 17 2013, 15:31:50)
[GCC 4.8.1] on linux2
>>> import codecs
>>> codecs.decode('?', 'unicode-escape')
u'\xe2\x82\xac'
>>> codecs.encode('?', 'unicode-escape')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0:
ordinal not in range(128)
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150510/a2f15a96/attachment.html>

From rosuav at gmail.com  Sun May 10 12:06:56 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 11 May 2015 02:06:56 +1000
Subject: Why does unicode-escape decode escape symbols that are already
 escaped?
In-Reply-To: <CA+gt_a82WGXHUZhcdbTUWG+TRV1Ys1ZSrkGjOxgavZGjAh9FiQ@mail.gmail.com>
References: <CA+gt_a82WGXHUZhcdbTUWG+TRV1Ys1ZSrkGjOxgavZGjAh9FiQ@mail.gmail.com>
Message-ID: <CAPTjJmpQ_25UVGVpg5MUsO1+sUeBp+20t5Txo+XhXkSkYPMjqw@mail.gmail.com>

On Mon, May 11, 2015 at 1:53 AM, Somelauw . <somelauw at gmail.com> wrote:
> In Python 3, decoding "?" with unicode-escape returns '?\x82?' which in my
> opinion doesn't make sense.
> The ? already is decoded; if it were encoded it would look like this:
> '\u20ac'.
> So why is it doing this?
>
> In Python 2 the behaviour is similar, but slightly different.
>
> $ python3 -S
> Python 3.3.3 (default, Nov 27 2013, 17:12:35)
> [GCC 4.8.2] on linux
>>>> import codecs
>>>> codecs.decode('?', 'unicode-escape')
> '?\x82?'
>>>> codecs.encode('?', 'unicode-escape')
> b'\\u20ac'
>>>>

Whenever you start encoding and decoding, you need to know whether
you're working with bytes->text, text->bytes, or something else. In
the case of unicode-escape, it expects to encode text into bytes, as
you can see with your second example - you give it a Unicode string,
and get back a byte string. When you attempt to *decode* a Unicode
string, that doesn't actually make sense, so it first gets *encoded*
to bytes, before being decoded. What you're actually seeing there is
that the one-character string is being encoded into a three-byte UTF-8
sequence,and then the unicode-escape decode takes those bytes and
interprets them as characters; as it happens, that's equivalent to a
Latin-1 decode:

>>> '?'.encode('utf-8').decode('latin-1')
'?\x82?'

I strongly suggest leaving the codecs module aside, and working
exclusively with the str.encode() and bytes.decode() methods, if you
possibly can. If you can't, at very least keep track in your head of
what is text and what is bytes, and which way things change in every
transformation.

ChrisA


From __peter__ at web.de  Sun May 10 12:14:17 2015
From: __peter__ at web.de (Peter Otten)
Date: Sun, 10 May 2015 18:14:17 +0200
Subject: Calling a function is faster than not calling it?
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <mailman.304.1431254634.12865.python-list@python.org>
 <554f700f$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mio04q$6gk$1@ger.gmane.org>

Steven D'Aprano wrote:

> On Sun, 10 May 2015 08:43 pm, Peter Otten wrote:
> 
>> A significant part of the extra time is apparently spent on stack
>> inspection:
> 
> I don't know what you mean by "stack inspection", or how you come to that
> conclusion.
> 
> 
>> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
>> 'f()' 10000000 loops, best of 3: 0.179 usec per loop
>> 
>> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
>> 'eval(code)'
>> 1000000 loops, best of 3: 0.852 usec per loop
>> 
>> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
>> 'eval(code, ns)'
>> 1000000 loops, best of 3: 0.433 usec per loop
> 
> Curious.
> 
> If I'm reading that correctly, supplying an explicit namespace cuts the
> time by a factor of two. That surprises me, as your function doesn't do
> any name lookups in the body of the function, so I would have thought that
> would be irrelevant.

I had a quick look at the implementation in bltinmodule.c, and the only 
piece of code that I could prevent from being run was:

    if (globals == Py_None) {
        globals = PyEval_GetGlobals();
        if (locals == Py_None) {
            locals = PyEval_GetLocals();
            if (locals == NULL)
                return NULL;
        }
    }

When there was an actual speed-up I also had a look at 
PyEval_GetGlobals/Locals() which in turn call 

PyEval_GetFrame() 

and

PyEvalPyFrame_FastToLocalsWithError()

whatever these do. (The first function reminded me of sys._getframe() hence 
the mention of stack inspection)

>> $ python3 -m timeit -s 'f = (lambda: 42); code = f.__code__; ns = {}'
>> 'eval; ns; f()'
>> 1000000 loops, best of 3: 0.263 usec per loop
> 
> So, roughly speaking, calling eval(code, ns) takes about 1.6 times as long
> as calling the function; calling eval(code) without providing a namespace
> takes about 3.2 times as long. That's roughly consistent with the results
> I'm getting.
 




From ian.g.kelly at gmail.com  Sun May 10 12:25:24 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sun, 10 May 2015 10:25:24 -0600
Subject: Calling a function is faster than not calling it?
In-Reply-To: <mio04q$6gk$1@ger.gmane.org>
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <mailman.304.1431254634.12865.python-list@python.org>
 <554f700f$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <mio04q$6gk$1@ger.gmane.org>
Message-ID: <CALwzid=+4eprDJWsnZpnZnNdjrDHNNVb0tAF9roH=D1nBJOzqw@mail.gmail.com>

On Sun, May 10, 2015 at 10:14 AM, Peter Otten <__peter__ at web.de> wrote:
> When there was an actual speed-up I also had a look at
> PyEval_GetGlobals/Locals() which in turn call
>
> PyEval_GetFrame()
>
> and
>
> PyEvalPyFrame_FastToLocalsWithError()
>
> whatever these do. (The first function reminded me of sys._getframe() hence
> the mention of stack inspection)

Based on the names, I surmise that the first one gets the top stack
frame object, and that the second one extracts the "fast" local
variables from the frame object and builds a dict of them for use by
eval.


From dreamingforward at gmail.com  Sun May 10 12:34:52 2015
From: dreamingforward at gmail.com (Mark Rosenblitt-Janssen)
Date: Sun, 10 May 2015 11:34:52 -0500
Subject: anomaly
Message-ID: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>

Here's something that might be wrong in Python (tried on v2.7):

>>> class int(str): pass

>>> int(3)
'3'

Mark


From tjreedy at udel.edu  Sun May 10 12:37:10 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 10 May 2015 12:37:10 -0400
Subject: Calling a function is faster than not calling it?
In-Reply-To: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mio1fs$qmc$1@ger.gmane.org>

On 5/10/2015 5:58 AM, Steven D'Aprano wrote:
> Why is calling a function faster than bypassing the function object and
> evaluating the code object itself? And not by a little, but by a lot?
>
> Here I have a file, eval_test.py:
>
> # === cut ===
> from timeit import Timer
>
> def func():
>      a = 2
>      b = 3
>      c = 4
>      return (a+b)*(a-b)/(a*c + b*c)
>
>
> code = func.__code__
> assert func() == eval(code)
>
> t1 = Timer("eval; func()", setup="from __main__ import func")
> t2 = Timer("eval(code)", setup="from __main__ import code")

eval has 3 parameters. I believe omitting the last two results in 
globals() and locals() calls, but at least one of them.  If {} is 
passed, access to builtins is added.

> # Best of 10 trials.
> print (min(t1.repeat(repeat=10)))
> print (min(t2.repeat(repeat=10)))

Adding
g = globals()
t3 = Timer("eval(code, g)", setup="from __main__ import code, g")
print (min(t3.repeat(repeat=10)))

 >>>
0.3992733933018515
0.6967548323372563
0.49210603735894587

2/3s of the extra time disappears, but there is still extra time needed 
for processing the two extra arguments. Passing g twice has no effect

> [steve at ando ~]$ python2.7 eval_test.py
> 0.804041147232
> 1.74012994766

> Directly eval'ing the code object is easily more than twice as expensive
> than calling the function, but calling the function has to eval the code

Calling the function executes the code via its .__call__ method.
Perhaps eval wraps the code object in a function object with .__call__. 
  I don't know the comparative internals.


-- 
Terry Jan Reedy



From ian.g.kelly at gmail.com  Sun May 10 12:42:58 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sun, 10 May 2015 10:42:58 -0600
Subject: anomaly
In-Reply-To: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
Message-ID: <CALwzidk39=wMQ8K38ZAbyBskCey_Oh+fghj9PvSW=n-1acSA5w@mail.gmail.com>

On Sun, May 10, 2015 at 10:34 AM, Mark Rosenblitt-Janssen
<dreamingforward at gmail.com> wrote:
> Here's something that might be wrong in Python (tried on v2.7):
>
>>>> class int(str): pass

This defines a new class named "int" that is a subclass of str. It has
no relation to the builtin class int.

>>>> int(3)
> '3'

This creates an instance of the above "int" class, which is basically
equivalent to calling "str(3)".

Were you expecting a different result?


From rustompmody at gmail.com  Sun May 10 12:48:15 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sun, 10 May 2015 09:48:15 -0700 (PDT)
Subject: anomaly
In-Reply-To: <mailman.318.1431276220.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
Message-ID: <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>

On Sunday, May 10, 2015 at 10:14:36 PM UTC+5:30, Ian wrote:
> On Sun, May 10, 2015 at 10:34 AM, Mark Rosenblitt-Janssen wrote:
> > Here's something that might be wrong in Python (tried on v2.7):
> >
> >>>> class int(str): pass
> 
> This defines a new class named "int" that is a subclass of str. It has
> no relation to the builtin class int.
> 
> >>>> int(3)
> > '3'
> 
> This creates an instance of the above "int" class, which is basically
> equivalent to calling "str(3)".
> 
> Were you expecting a different result?

In C (family) languages int is a keyword
>From that pov this is completely bizarre


From tjreedy at udel.edu  Sun May 10 13:00:52 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 10 May 2015 13:00:52 -0400
Subject: Why does unicode-escape decode escape symbols that are already
 escaped?
In-Reply-To: <CA+gt_a82WGXHUZhcdbTUWG+TRV1Ys1ZSrkGjOxgavZGjAh9FiQ@mail.gmail.com>
References: <CA+gt_a82WGXHUZhcdbTUWG+TRV1Ys1ZSrkGjOxgavZGjAh9FiQ@mail.gmail.com>
Message-ID: <mio2s9$hgk$1@ger.gmane.org>

On 5/10/2015 11:53 AM, Somelauw . wrote:
> In Python 3, decoding "?" with unicode-escape returns '?\x82?' which in
> my opinion doesn't make sense.

Agreed. I think this is a bug in that it should raise an exception 
instead. Decoding a string only makes sense for rot-13

> The ? already is decoded; if it were encoded it would look like this:
> '\u20ac'.
> So why is it doing this?

> $ python3 -S
> Python 3.3.3 (default, Nov 27 2013, 17:12:35)
> [GCC 4.8.2] on linux
>  >>> import codecs
>  >>> codecs.decode('?', 'unicode-escape')
> '?\x82?'
>  >>> codecs.encode('?', 'unicode-escape')
> b'\\u20ac'

-- 
Terry Jan Reedy




From tjreedy at udel.edu  Sun May 10 13:09:12 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 10 May 2015 13:09:12 -0400
Subject: anomaly
In-Reply-To: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
Message-ID: <mio3bt$or9$1@ger.gmane.org>

On 5/10/2015 12:34 PM, Mark Rosenblitt-Janssen wrote:
> Here's something that might be wrong in Python (tried on v2.7):

You are being hypnotized by the fact the 'int' is a builtin name. 
Builtin names are not keywords and can intentionally be rebound.  If you 
rebind randomly, the result may seem strange, but is predictable.  This 
is why be recommend not randomly reusing names of builtins.

>>>> class int(str): pass

This in effect rebinds 'int' to the str class.  The work 'int' no longer 
has any connect to the builtin integer class.

>>>> int(3)

Equivalent to str(3)

> '3'

No surprise that str(3) == '3'.

Replace 'int' with anything else, such as 'zyz' or 'Mark' in both 
statements and the result is the same.

-- 
Terry Jan Reedy



From breamoreboy at yahoo.co.uk  Sun May 10 13:21:42 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 10 May 2015 18:21:42 +0100
Subject: anomaly
In-Reply-To: <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
Message-ID: <mio439$10j$1@ger.gmane.org>

On 10/05/2015 17:48, Rustom Mody wrote:
> On Sunday, May 10, 2015 at 10:14:36 PM UTC+5:30, Ian wrote:
>> On Sun, May 10, 2015 at 10:34 AM, Mark Rosenblitt-Janssen wrote:
>>> Here's something that might be wrong in Python (tried on v2.7):
>>>
>>>>>> class int(str): pass
>>
>> This defines a new class named "int" that is a subclass of str. It has
>> no relation to the builtin class int.
>>
>>>>>> int(3)
>>> '3'
>>
>> This creates an instance of the above "int" class, which is basically
>> equivalent to calling "str(3)".
>>
>> Were you expecting a different result?
>
> In C (family) languages int is a keyword
>  From that pov this is completely bizarre
>

This is a news group for a language called Python.  Why do people keep 
coming here expecting it to behave in the same way that (say) CORAL 
66/250 does?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From gherron at digipen.edu  Sun May 10 13:28:05 2015
From: gherron at digipen.edu (Gary Herron)
Date: Sun, 10 May 2015 10:28:05 -0700
Subject: anomaly
In-Reply-To: <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
Message-ID: <554F9525.5040101@digipen.edu>

On 05/10/2015 09:48 AM, Rustom Mody wrote:
> On Sunday, May 10, 2015 at 10:14:36 PM UTC+5:30, Ian wrote:
>> On Sun, May 10, 2015 at 10:34 AM, Mark Rosenblitt-Janssen wrote:
>>> Here's something that might be wrong in Python (tried on v2.7):
>>>
>>>>>> class int(str): pass
>> This defines a new class named "int" that is a subclass of str. It has
>> no relation to the builtin class int.
>>
>>>>>> int(3)
>>> '3'
>> This creates an instance of the above "int" class, which is basically
>> equivalent to calling "str(3)".
>>
>> Were you expecting a different result?
> In C (family) languages int is a keyword
>  From that pov this is completely bizarre

Not really.  Expecting Python to act like C family languages *is* bizarre.

Common Python thought::  "We're all adults here."    If you want to 
override a builtin within your own namespace, who are we to stop you?
Besides, it's still available as __builtins__.int  (unless you've also 
overridden that).

-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418



From fomcl at yahoo.com  Sun May 10 14:39:34 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Sun, 10 May 2015 11:39:34 -0700
Subject: Why does unicode-escape decode escape symbols that are already
 escaped?
Message-ID: <1431283174.10170.BPMail_high_carrier@web163801.mail.gq1.yahoo.com>


-----------------------------
On Sun, May 10, 2015 5:53 PM CEST Somelauw . wrote:

>In Python 3, decoding "?" with unicode-escape returns '?\x82?' which in my
>opinion doesn't make sense.
>The ? already is decoded; if it were encoded it would look like this:
>'\u20ac'.
>So why is it doing this?
>
>In Python 2 the behaviour is similar, but slightly different.
>
>$ python3 -S
>Python 3.3.3 (default, Nov 27 2013, 17:12:35)
>[GCC 4.8.2] on linux
>>> import codecs
>>> codecs.decode('?', 'unicode-escape')
>'?\x82?'
>>> codecs.encode('?', 'unicode-escape')
>b'\\u20ac'
>>>
>
>$ python2 -S
>Python 2.7.5+ (default, Sep 17 2013, 15:31:50)
>[GCC 4.8.1] on linux2
>>> import codecs
>>> codecs.decode('?', 'unicode-escape')
>u'\xe2\x82\xac'
>>> codecs.encode('?', 'unicode-escape')
>Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0:
>ordinal not in range(128)
>>>

Hi,

I only have Python 2 on my phone, but I am suprised that you (and are able to) decode unicode strings. What result do you get when you do the following in Python 3:

Python 2.7.2 (default, Oct 25 2014, 20:52:15)
[GCC 4.9 20140827 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import codecs
>>> codecs.decode(b'?', 'unicode-escape')
u'\xe2\x82\xac'
>>> codecs.encode(u'?', 'unicode-escape')
'\\xe2\\x82\\xac'
>>>



From robertvstepp at gmail.com  Sun May 10 15:12:44 2015
From: robertvstepp at gmail.com (boB Stepp)
Date: Sun, 10 May 2015 14:12:44 -0500
Subject: anomaly
In-Reply-To: <554F9525.5040101@digipen.edu>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
Message-ID: <CANDiX9+m4=iGWVzJjnS00HTfg7AaZd_-UNKvt8jf-__WVrJo-g@mail.gmail.com>

I am in process learning Python and normally hang out on the Tutor
list, but monitor this one hoping to learn what I can. This thread is
of interest to me from the standpoint of trying to understand the
Python way of doing things.

On Sun, May 10, 2015 at 12:28 PM, Gary Herron <gherron at digipen.edu> wrote:
> On 05/10/2015 09:48 AM, Rustom Mody wrote:
>>
>> On Sunday, May 10, 2015 at 10:14:36 PM UTC+5:30, Ian wrote:
>>>
>>> On Sun, May 10, 2015 at 10:34 AM, Mark Rosenblitt-Janssen wrote:
>>>>
>>>> Here's something that might be wrong in Python (tried on v2.7):
>>>>
>>>>>>> class int(str): pass
>>>
>>> This defines a new class named "int" that is a subclass of str. It has
>>> no relation to the builtin class int.
>>>
>>>>>>> int(3)
>>>>
>>>> '3'
>>>
>>> This creates an instance of the above "int" class, which is basically
>>> equivalent to calling "str(3)".
>>>
>>> Were you expecting a different result?
>>
>> In C (family) languages int is a keyword
>>  From that pov this is completely bizarre
>
>
> Not really.  Expecting Python to act like C family languages *is* bizarre.

I have to admit being surprised by this, too. I am just now studying
on how to write my own classes in Python, and have come to realize
that doing this is *possible*, but the *surprise* to me is why the
language design allowed this to actually be done.

> Common Python thought::  "We're all adults here."    If you want to override
> a builtin within your own namespace, who are we to stop you?

I'm surprised that this thought has not been added to the "Zen Of
Python", as I see it as more and more recurrent as I continue my
studies. What I would like to comprehend is what is the essential
mindset of Python? That is, what do I need to understand, so that I am
no longer likely to be surprised by discovering new possibilities in
Python such as what the current thread is discussing?


-- 
boB


From cseberino at gmail.com  Sun May 10 16:43:03 2015
From: cseberino at gmail.com (Chris Seberino)
Date: Sun, 10 May 2015 13:43:03 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming intro
 course...What about an intro course that uses *BOTH*? Good idea?
Message-ID: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>

Instead of learning only Scheme or only Python for a one semester intro
course, what about learning BOTH?  Maybe that could somehow
get the benefits of both?

I'm thinking that for the VERY beginning, Scheme is the fastest language
to get beginners up and running writing code due to the extremely minimal simple syntax.

I'm thinking half way into the semester, instead of moving into intermediate Scheme, perhaps that is a good time to switch to Python?

Would a little strong intro to 2 nice languages in one semester be
same/good/worse/better than just 1?

cs


From mwilson at the-wire.com  Sun May 10 16:59:52 2015
From: mwilson at the-wire.com (Mel Wilson)
Date: Sun, 10 May 2015 20:59:52 +0000 (UTC)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
Message-ID: <miogs6$1ub$1@dont-email.me>

On Sun, 10 May 2015 13:43:03 -0700, Chris Seberino wrote:

> Instead of learning only Scheme or only Python for a one semester intro
> course, what about learning BOTH?  Maybe that could somehow get the
> benefits of both?
> 
> I'm thinking that for the VERY beginning, Scheme is the fastest language
> to get beginners up and running writing code due to the extremely
> minimal simple syntax.
> 
> I'm thinking half way into the semester, instead of moving into
> intermediate Scheme, perhaps that is a good time to switch to Python?
> 
> Would a little strong intro to 2 nice languages in one semester be
> same/good/worse/better than just 1?

The first course I took, we learned Algol-60, then when we couldn't get 
computer time for compiles, we were asked to pick up FORTRAN-IV on the 
side.  So we "published" our solutions to the class problems in Algol and 
re-wrote them to be run in FORTRAN.  It was a fine first-hand look at 
what the "general purpose" in General Purpose Computer really meant.  
There was no confusing the machine and the language after that.  Scheme/
Python would be even more radical, I think.  If you can put them across 
effectively, I say go for it.

	Mel.
> 
> cs



From bc at freeuk.com  Sun May 10 17:08:25 2015
From: bc at freeuk.com (BartC)
Date: Sun, 10 May 2015 22:08:25 +0100
Subject: Calling a function is faster than not calling it?
In-Reply-To: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <fNP3x.641427$3i4.583649@fx29.am4>

On 10/05/2015 10:58, Steven D'Aprano wrote:
> from timeit import Timer
>
> def func():
>      a = 2
>      b = 3
>      c = 4
>      return (a+b)*(a-b)/(a*c + b*c)
>
>
> code = func.__code__
> assert func() == eval(code)
>
> t1 = Timer("eval; func()", setup="from __main__ import func")
> t2 = Timer("eval(code)", setup="from __main__ import code")
>
> # Best of 10 trials.
> print (min(t1.repeat(repeat=10)))
> print (min(t2.repeat(repeat=10)))

Maybe the overheads of using eval() are significant when calling a 
simple function such as your example.

When I made it do a bit more work:

def func():
     a = 2
     b = 3
     c = 4
     for i in range(50):
         x=(a+b)*(a-b)/(a*c + b*c)
     return (a+b)*(a-b)/(a*c + b*c)

Then the eval() call took only 3% longer rather than 100%.

-- 
Bartc




From zljubisicmob at gmail.com  Sun May 10 17:10:32 2015
From: zljubisicmob at gmail.com (zljubisicmob at gmail.com)
Date: Sun, 10 May 2015 14:10:32 -0700 (PDT)
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
In-Reply-To: <mailman.282.1431174340.12865.python-list@python.org>
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
 <mailman.260.1431113391.12865.python-list@python.org>
 <8360473a-45ac-4270-9bf3-81932da5f223@googlegroups.com>
 <554d768d$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <580ee0d6-a703-4da3-af2d-105589a1780f@googlegroups.com>
 <mailman.282.1431174340.12865.python-list@python.org>
Message-ID: <d06f180f-bc6f-4610-b8cb-a6335ee5a2ce@googlegroups.com>

> No, we can't see what ROOTDIR is, since you read it from the config 
> file.  And you don't show us the results of those prints.  You don't 
> even show us the full exception, or even the line it fails on.

Sorry I forgot. This is the output of the script:

C:\Python34\python.exe C:/Users/zoran/PycharmProjects/mm_align/bckslash_test.py
C:\Users\zoran\hrt
Traceback (most recent call last):
  File "C:/Users/zoran/PycharmProjects/mm_align/bckslash_test.py", line 43, in <module>
    with open(src_file, mode='w', encoding='utf-8') as s_file:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\zoran\\hrt\\src_70._godi?njica_pobjede_nad_fa?izmom_Za?to_ve?ina_?elnika_Europske_unije_bojkotira_vojnu_paradu_u_Moskvi__Kako_?e_se_obljetnica_pobjede_nad_nacisti?kom_Njema?kom_i_njenim_satelitima_obilje?iti_u_na?oj_zemlji__Ho?e_li_Josip_Broz_Tito_o.txt'
70._godi?njica_pobjede_nad_fa?izmom_Za?to_ve?ina_?elnika_Europske_unije_bojkotira_vojnu_paradu_u_Moskvi__Kako_?e_se_obljetnica_pobjede_nad_nacisti?kom_Njema?kom_i_njenim_satelitima_obilje?iti_u_na?oj_zemlji__Ho?e_li_Josip_Broz_Tito_o
260 C:\Users\zoran\hrt\src_70._godi?njica_pobjede_nad_fa?izmom_Za?to_ve?ina_?elnika_Europske_unije_bojkotira_vojnu_paradu_u_Moskvi__Kako_?e_se_obljetnica_pobjede_nad_nacisti?kom_Njema?kom_i_njenim_satelitima_obilje?iti_u_na?oj_zemlji__Ho?e_li_Josip_Broz_Tito_o.txt
260 C:\Users\zoran\hrt\des_70._godi?njica_pobjede_nad_fa?izmom_Za?to_ve?ina_?elnika_Europske_unije_bojkotira_vojnu_paradu_u_Moskvi__Kako_?e_se_obljetnica_pobjede_nad_nacisti?kom_Njema?kom_i_njenim_satelitima_obilje?iti_u_na?oj_zemlji__Ho?e_li_Josip_Broz_Tito_o.txt

Process finished with exit code 1

Cfg file has the following contents:

C:\Users\zoran\PycharmProjects\mm_align\hrt3.cfg contents
[Dir]
ROOTDIR = C:\Users\zoran\hrt 

> I doubt that the problem is in the ROODIR value, but of course nothing 
> in your program bothers to check that that directory exists.  I expect 
> you either have too many characters total, or the 232th character is a 
> strange one.  Or perhaps title has a backslash in it (you took care of 
> forward slash).

How to determine that?
 
> While we're at it, if you do have an OS limitation on size, your code is 
> truncating at the wrong point.  You need to truncate the title based on 
> the total size of src_file and dst_file, and since the code cannot know 
> the size of ROOTDIR, you need to include that in your figuring.

Well, in my program I am defining a file name as category-id-description.mp3.
If the file is too long I am cutting description (it wasn't clear from my example).

Regards.


From zljubisicmob at gmail.com  Sun May 10 17:14:48 2015
From: zljubisicmob at gmail.com (zljubisicmob at gmail.com)
Date: Sun, 10 May 2015 14:14:48 -0700 (PDT)
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
In-Reply-To: <554e2435$0$12992$c3e8da3$5496439d@news.astraweb.com>
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
 <mailman.260.1431113391.12865.python-list@python.org>
 <8360473a-45ac-4270-9bf3-81932da5f223@googlegroups.com>
 <554d768d$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <580ee0d6-a703-4da3-af2d-105589a1780f@googlegroups.com>
 <554e2435$0$12992$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <f176cbcf-96e6-4bbe-869c-27994550ab60@googlegroups.com>

> > It works, but if you change title = title[:232] to title = title[:233],
> > you will get "FileNotFoundError: [Errno 2] No such file or directory".
> 
> 
> Which is a *completely different* error from 
> 
> SyntaxError: 'unicodeescape' codec can't decode bytes in position 2-3:
> truncated \UXXXXXXXX escape

I don't know when the original error disappeared and become this one (confused).

Regards.


From marko at pacujo.net  Sun May 10 17:16:28 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 11 May 2015 00:16:28 +0300
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
Message-ID: <87wq0gyvyr.fsf@elektro.pacujo.net>

Chris Seberino <cseberino at gmail.com>:

> Instead of learning only Scheme or only Python for a one semester
> intro course, what about learning BOTH? Maybe that could somehow get
> the benefits of both?
>
> I'm thinking that for the VERY beginning, Scheme is the fastest
> language to get beginners up and running writing code due to the
> extremely minimal simple syntax.

Scheme is my favorite language. I think, however, it is a pretty
advanced language and requires a pretty solid basis in programming and
computer science.

Python, in contrast, is a great introductory programming language. Sure,
you *can* get quite advanced with it, too, but you can get quite a bit
of fun stuff done with just the basics.

Of course, you could introduce Scheme with similar simplifications.
However, such simplifications (say, iterative constructs) are
nonidiomatic in Scheme. The students should not get into bad habits
that they need to be weaned off of later.

> I'm thinking half way into the semester, instead of moving into
> intermediate Scheme, perhaps that is a good time to switch to Python?

What are you teaching? If you are teaching computer science, you should
use languages to illustrate abstract ideas. Thus, Python can be used to
introduce basic control and data structures, I/O, OOP etc. Scheme should
be used to teach functional programming and maybe combinatory logic and
computability. Prolog could be used to demonstrate logic programming and
automated theorem proving. C could be used to understand the
nitty-gritties under the hood and fear of SIGSEGV.


Marko


From dreamingforward at gmail.com  Sun May 10 18:59:44 2015
From: dreamingforward at gmail.com (Mark Rosenblitt-Janssen)
Date: Sun, 10 May 2015 17:59:44 -0500
Subject: anomaly
In-Reply-To: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
Message-ID: <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>

Here's where this exploration came from.  I've (once again) been
contemplating the OO nature.

It's clear to me that there needs to be a distinction between
specialization of an object vs. expansion of an object (a new term I'm
proposing to the OOP lexicon).  The latter *adds* more functionality
(like what everyone does with the Object class), while the former
changes the behavior of some class for more specific behavior that was
not programmed in the original class.

It's a difference between, for example, concrete base types and ABCs.
Python artificially tried to make int inherit from object, just
because it can, but this is wrong.  It`s messed with the Zen-thing.
"Purity has hammered practicality [like the fact that we actually have
to work on concrete types in the CPU] into the ground. " (addition
mine).

Sorry I can't spend more time clarifying.  I hope that there's at
least one person who sees the issue.

Mark



On 5/10/15, Mark Rosenblitt-Janssen <dreamingforward at gmail.com> wrote:
> Here's something that might be wrong in Python (tried on v2.7):
>
>>>> class int(str): pass
>
>>>> int(3)
> '3'
>
> Mark
>


From dreamingforward at gmail.com  Sun May 10 19:26:06 2015
From: dreamingforward at gmail.com (Mark Rosenblitt-Janssen)
Date: Sun, 10 May 2015 18:26:06 -0500
Subject: anomaly
In-Reply-To: <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
Message-ID: <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>

Along those lines, it makes no sense for mix-in classes to inherit
from Object at all -- they're neither expanding on object nor
specializing it.

For example, I'm refactoring my python Graph class so that people can
have different Vertex behaviors by using different composition of
mix-in classes.  But instead of letting them figure out the object
inheritence order (so there's no conflicts), I'm going to make some
classes that pre-specify the right ordering for different
combiniations of behaviors.

class Vertex(object):
    """Basic Vertex base class, giving a dictionary of edges.""""

class WVertex(Vertex, weighted_mixin):  #the weighted_mixin will
override some of the methods in Vertex.
    """Use this when initializing a Graph class for weighted edges."""

class RWVertex(WVertex, reverse_edge_mixin):
    """Use this class for O(1) access to the edges coming into Vertices."""

class Graph(dict):

    def __init__(self, init={}, VertexType=Vertex):
        """Creates a Graph with the specified behaviors within VertexType."""

-----
No one has to see the mix-in classes.

I'm inventing a new term for this kind of inheritence:  expansion.
I've enclosed one class inside of another, but yet it's not quite
"encapsulation" (in the C++ sense).

Cheers,

Mark J
----
http://wiki.hackerspaces.org/Hacking_with_the_Tao


On 5/10/15, Mark Rosenblitt-Janssen <dreamingforward at gmail.com> wrote:
> Here's where this exploration came from.  I've (once again) been
> contemplating the OO nature.
>
> It's clear to me that there needs to be a distinction between
> specialization of an object vs. expansion of an object (a new term I'm
> proposing to the OOP lexicon).  The latter *adds* more functionality
> (like what everyone does with the Object class), while the former
> changes the behavior of some class for more specific behavior that was
> not programmed in the original class.
>
> It's a difference between, for example, concrete base types and ABCs.
> Python artificially tried to make int inherit from object, just
> because it can, but this is wrong.  It`s messed with the Zen-thing.
> "Purity has hammered practicality [like the fact that we actually have
> to work on concrete types in the CPU] into the ground. " (addition
> mine).
>
> Sorry I can't spend more time clarifying.  I hope that there's at
> least one person who sees the issue.
>
> Mark
>
>
>
> On 5/10/15, Mark Rosenblitt-Janssen <dreamingforward at gmail.com> wrote:
>> Here's something that might be wrong in Python (tried on v2.7):
>>
>>>>> class int(str): pass
>>
>>>>> int(3)
>> '3'
>>
>> Mark
>>
>


From breamoreboy at yahoo.co.uk  Sun May 10 19:34:43 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 11 May 2015 00:34:43 +0100
Subject: anomaly
In-Reply-To: <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
Message-ID: <miopum$4gk$1@ger.gmane.org>

On 10/05/2015 23:59, Mark Rosenblitt-Janssen wrote:
> On 5/10/15, Mark Rosenblitt-Janssen <dreamingforward at gmail.com> wrote:
>> Here's something that might be wrong in Python (tried on v2.7):
>>
>>>>> class int(str): pass
>>
>>>>> int(3)
>> '3'
>>
>> Mark
>>
> Here's where this exploration came from.  I've (once again) been
> contemplating the OO nature.
>
> It's clear to me that there needs to be a distinction between
> specialization of an object vs. expansion of an object (a new term I'm
> proposing to the OOP lexicon).  The latter *adds* more functionality
> (like what everyone does with the Object class), while the former
> changes the behavior of some class for more specific behavior that was
> not programmed in the original class.
>
> It's a difference between, for example, concrete base types and ABCs.
> Python artificially tried to make int inherit from object, just
> because it can, but this is wrong.  It`s messed with the Zen-thing.
> "Purity has hammered practicality [like the fact that we actually have
> to work on concrete types in the CPU] into the ground. " (addition
> mine).
>
> Sorry I can't spend more time clarifying.  I hope that there's at
> least one person who sees the issue.
>
> Mark
>

It strikes me that you haven't a clue about Python and worse still 
you're top posting, please don't.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From somelauw at gmail.com  Sun May 10 19:45:34 2015
From: somelauw at gmail.com (Somelauw .)
Date: Mon, 11 May 2015 01:45:34 +0200
Subject: Why does unicode-escape decode escape symbols that are already
 escaped?
In-Reply-To: <1431283174.10170.BPMail_high_carrier@web163801.mail.gq1.yahoo.com>
References: <1431283174.10170.BPMail_high_carrier@web163801.mail.gq1.yahoo.com>
Message-ID: <CA+gt_a8C86bmk+GYfU-H-7u7vZ5RjtpXSYxc9cUF6eLz_Z+F9w@mail.gmail.com>

2015-05-10 20:39 GMT+02:00 Albert-Jan Roskam <fomcl at yahoo.com>:

>
> Hi,
>
> I only have Python 2 on my phone, but I am suprised that you (and are able
> to) decode unicode strings. What result do you get when you do the
> following in Python 3:
>
> Python 2.7.2 (default, Oct 25 2014, 20:52:15)
> [GCC 4.9 20140827 (prerelease)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import codecs
> >>> codecs.decode(b'?', 'unicode-escape')
> u'\xe2\x82\xac'
> >>> codecs.encode(u'?', 'unicode-escape')
> '\\xe2\\x82\\xac'
> >>>
>
>
Python 3.3.3 (default, Nov 27 2013, 17:12:35)
[GCC 4.8.2] on linux
>>> import codecs
>>> codecs.decode(b'?', 'unicode-escape')
  File "<stdin>", line 1
SyntaxError: bytes can only contain ASCII literal characters.
>>> codecs.encode(u'?', 'unicode-escape')
b'\\u20ac'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150511/e970cf06/attachment.html>

From somelauw at gmail.com  Sun May 10 19:56:46 2015
From: somelauw at gmail.com (Somelauw .)
Date: Mon, 11 May 2015 01:56:46 +0200
Subject: Why does unicode-escape decode escape symbols that are already
 escaped?
In-Reply-To: <CAPTjJmpQ_25UVGVpg5MUsO1+sUeBp+20t5Txo+XhXkSkYPMjqw@mail.gmail.com>
References: <CA+gt_a82WGXHUZhcdbTUWG+TRV1Ys1ZSrkGjOxgavZGjAh9FiQ@mail.gmail.com>
 <CAPTjJmpQ_25UVGVpg5MUsO1+sUeBp+20t5Txo+XhXkSkYPMjqw@mail.gmail.com>
Message-ID: <CA+gt_a_BAh0YSo_htvFpw_o4ioJiy8L+Y48nAh1vjqLE+SQ2ew@mail.gmail.com>

2015-05-10 18:06 GMT+02:00 Chris Angelico <rosuav at gmail.com>:
> Whenever you start encoding and decoding, you need to know whether
> you're working with bytes->text, text->bytes, or something else. In
> the case of unicode-escape, it expects to encode text into bytes, as
> you can see with your second example - you give it a Unicode string,
> and get back a byte string. When you attempt to *decode* a Unicode
> string, that doesn't actually make sense, so it first gets *encoded*
> to bytes, before being decoded. What you're actually seeing there is
> that the one-character string is being encoded into a three-byte UTF-8
> sequence,and then the unicode-escape decode takes those bytes and
> interprets them as characters; as it happens, that's equivalent to a
> Latin-1 decode:

Thanks for your response.
I was using unicode-escape for handling escape characters like
converting "\\n" to actual newlines.
My input argument is already in string format and the decoding from
bytes to string has already been done a couple of layers deeper, so I
really needed a string to string conversion.
I guess that it's not possible to do this operation without converting
to bytes first (even if I use the codecs module, it will convert to
bytes implicitly as you just told me).
What I'm probably going to do is writing my own parser to perform this task.


From dreamingforward at gmail.com  Sun May 10 20:14:18 2015
From: dreamingforward at gmail.com (Mark Rosenblitt-Janssen)
Date: Sun, 10 May 2015 19:14:18 -0500
Subject: anomaly
In-Reply-To: <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
Message-ID: <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>

In case the example given at the start of the thread wasn't
interesting enough, it also works in the other direction:

>>> class str(int):  pass

>>> str('2')
2  #<----- an integer!!!

Mark


From breamoreboy at yahoo.co.uk  Sun May 10 20:19:49 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 11 May 2015 01:19:49 +0100
Subject: anomaly
In-Reply-To: <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
Message-ID: <miosj8$61q$1@ger.gmane.org>

On 11/05/2015 01:14, Mark Rosenblitt-Janssen wrote:
> In case the example given at the start of the thread wasn't
> interesting enough, it also works in the other direction:
>
>>>> class str(int):  pass
>
>>>> str('2')
> 2  #<----- an integer!!!
>
> Mark
>

Thanks for this, I've not found anybody new for my dream team in months.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From dreamingforward at gmail.com  Sun May 10 20:48:30 2015
From: dreamingforward at gmail.com (zipher)
Date: Sun, 10 May 2015 17:48:30 -0700 (PDT)
Subject: anomaly
In-Reply-To: <mailman.318.1431276220.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
Message-ID: <7169d38e-b21c-472e-b0e3-544ed37fbfd2@googlegroups.com>

On Sunday, May 10, 2015 at 11:44:36 AM UTC-5, Ian wrote:
> On Sun, May 10, 2015 at 10:34 AM, Mark Rosenblitt-Janssen
> <dreamingforward at gmail.com> wrote:
> > Here's something that might be wrong in Python (tried on v2.7):
> >
> >>>> class int(str): pass
> 
> This defines a new class named "int" that is a subclass of str. It has
> no relation to the builtin class int.
> 
> >>>> int(3)
> > '3'
> 
> This creates an instance of the above "int" class, which is basically
> equivalent to calling "str(3)".
> 
> Were you expecting a different result?

Sorry, yes.  If you're going to define a concept called "keywords", I don't think you should allow them to be shadowed by a class definition.

Mark


From dreamingforward at gmail.com  Sun May 10 20:57:56 2015
From: dreamingforward at gmail.com (zipher)
Date: Sun, 10 May 2015 17:57:56 -0700 (PDT)
Subject: anomaly
In-Reply-To: <mailman.330.1431303600.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
Message-ID: <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>

On Sunday, May 10, 2015 at 7:20:13 PM UTC-5, Mark Lawrence wrote:
> On 11/05/2015 01:14, Mark Rosenblitt-Janssen wrote:
> > In case the example given at the start of the thread wasn't
> > interesting enough, it also works in the other direction:
> >
> >>>> class str(int):  pass
> >
> >>>> str('2')
> > 2  #<----- an integer!!!
> >
> > Mark
> >
> 
> Thanks for this, I've not found anybody new for my dream team in months.

Sorry, I forgot I unsubscribed from the mailing list and didn't see any replies and deduced that no one had noticed anything unexpected about the idea. 

I guess everyone expects this behavior since Python implemented this idea of "everything is an object", but I think this branch of OOP (on the branch of the Tree of Programming Languages) has to be chopped off.  The idea of everything is an object is backwards (unless your in a LISP machine).  Like I say, it's trying to be too pure and not practical.

Whatever,

Mark


From dreamingforward at gmail.com  Sun May 10 21:03:32 2015
From: dreamingforward at gmail.com (zipher)
Date: Sun, 10 May 2015 18:03:32 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
Message-ID: <c95dd623-a2fe-4d43-8c94-7a12abd6f5de@googlegroups.com>

On Sunday, May 10, 2015 at 3:43:25 PM UTC-5, Chris Seberino wrote:
> Instead of learning only Scheme or only Python for a one semester intro
> course, what about learning BOTH?  Maybe that could somehow
> get the benefits of both?
> 
> I'm thinking that for the VERY beginning, Scheme is the fastest language
> to get beginners up and running writing code due to the extremely minimal simple syntax.
> 
> I'm thinking half way into the semester, instead of moving into intermediate Scheme, perhaps that is a good time to switch to Python?
> 
> Would a little strong intro to 2 nice languages in one semester be
> same/good/worse/better than just 1?

No.  LISP-like languages are very different beasts, requiring different mind-sets.  It's like going from geometry to arithmetic.

Or trying to teach OS/2 (which had great abstract ideas) and switching to Linux without covering the architecture and engineering underneath them.

mark


From gherron at digipen.edu  Sun May 10 21:07:36 2015
From: gherron at digipen.edu (Gary Herron)
Date: Sun, 10 May 2015 18:07:36 -0700
Subject: anomaly
In-Reply-To: <7169d38e-b21c-472e-b0e3-544ed37fbfd2@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <7169d38e-b21c-472e-b0e3-544ed37fbfd2@googlegroups.com>
Message-ID: <555000D8.9080307@digipen.edu>

On 05/10/2015 05:48 PM, zipher wrote:
> On Sunday, May 10, 2015 at 11:44:36 AM UTC-5, Ian wrote:
>> On Sun, May 10, 2015 at 10:34 AM, Mark Rosenblitt-Janssen
>> <dreamingforward at gmail.com> wrote:
>>> Here's something that might be wrong in Python (tried on v2.7):
>>>
>>>>>> class int(str): pass
>> This defines a new class named "int" that is a subclass of str. It has
>> no relation to the builtin class int.
>>
>>>>>> int(3)
>>> '3'
>> This creates an instance of the above "int" class, which is basically
>> equivalent to calling "str(3)".
>>
>> Were you expecting a different result?
> Sorry, yes.  If you're going to define a concept called "keywords", I don't think you should allow them to be shadowed by a class definition.
>
> Mark

Huh?   Python has plenty of keywords, and indeed, none of them can be 
redefined or shadowed.    But you would gain nothing (and lose a bit or 
dynamic-language freedom) by making int a keyword.

-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418



From dreamingforward at gmail.com  Sun May 10 21:18:52 2015
From: dreamingforward at gmail.com (zipher)
Date: Sun, 10 May 2015 18:18:52 -0700 (PDT)
Subject: anomaly
In-Reply-To: <mailman.331.1431306462.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <7169d38e-b21c-472e-b0e3-544ed37fbfd2@googlegroups.com>
 <mailman.331.1431306462.12865.python-list@python.org>
Message-ID: <ad8dd792-a236-4514-9bb5-a90cf21535a4@googlegroups.com>

> Huh?   Python has plenty of keywords, and indeed, none of them can be 
> redefined or shadowed.    But you would gain nothing (and lose a bit or 
> dynamic-language freedom) by making int a keyword.

Okay.  I apologize for thinking in C and believing "int" was a keyword.  It isn't in Python as you remind me.  However, this is where I'm arguing the purity has hammered practicality into the ground.

Python is trying to be as elegant as LISP in trying to make everything an Object.  It's not necessary, and it's trying to put lipstick on a pig instead of making BBQ.  Python will be as elegant, but in a different direction of the axis.

That difference is exactly like how Philosophy is different from Math -- both require logic, but go in different directions with it.  Python makes classes, when LISP tries to make classes to try to be like C++ it also looks equally stupid.  So don't do it.  Let them be different specializations.

mark


From dreamingforward at gmail.com  Sun May 10 21:29:24 2015
From: dreamingforward at gmail.com (zipher)
Date: Sun, 10 May 2015 18:29:24 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <c95dd623-a2fe-4d43-8c94-7a12abd6f5de@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <c95dd623-a2fe-4d43-8c94-7a12abd6f5de@googlegroups.com>
Message-ID: <a079d583-5744-4d50-8a42-d91ee83d561c@googlegroups.com>

> > Instead of learning only Scheme or only Python for a one semester intro
> > course, what about learning BOTH?  Maybe that could somehow
> > get the benefits of both?
> 
> No.  LISP-like languages are very different beasts, requiring different mind-sets.  It's like going from geometry to arithmetic.
> 
> Or trying to teach OS/2 (which had great abstract ideas) and switching to Linux without covering the architecture and engineering underneath them.

Another point.  You're allowing Church`s Thesis to misinform you.  While, in theory, every programming language could be made into any other, the architecture in which to do so is completely different, so it misleads the programmer.

To fit LISP into a Turing Machine architecture (what most every procedural programming language and most every computer sold is/utilizes) requires a very obscene translation table in the TM.  I'm not even sure that it's been analyzed, because the TM has no concept of a stack.

Any PhD's know of who's actually make a stack on the TM?  Usually the tape itself holds the stack, but with LISP architecture, this isn't the natural fit.

Mark


From davea at davea.name  Sun May 10 21:33:09 2015
From: davea at davea.name (Dave Angel)
Date: Sun, 10 May 2015 21:33:09 -0400
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
In-Reply-To: <d06f180f-bc6f-4610-b8cb-a6335ee5a2ce@googlegroups.com>
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
 <mailman.260.1431113391.12865.python-list@python.org>
 <8360473a-45ac-4270-9bf3-81932da5f223@googlegroups.com>
 <554d768d$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <580ee0d6-a703-4da3-af2d-105589a1780f@googlegroups.com>
 <mailman.282.1431174340.12865.python-list@python.org>
 <d06f180f-bc6f-4610-b8cb-a6335ee5a2ce@googlegroups.com>
Message-ID: <555006D5.5060509@davea.name>

On 05/10/2015 05:10 PM, zljubisicmob at gmail.com wrote:
>> No, we can't see what ROOTDIR is, since you read it from the config
>> file.  And you don't show us the results of those prints.  You don't
>> even show us the full exception, or even the line it fails on.
>
> Sorry I forgot. This is the output of the script:
>
> C:\Python34\python.exe C:/Users/zoran/PycharmProjects/mm_align/bckslash_test.py
> C:\Users\zoran\hrt
> Traceback (most recent call last):
>    File "C:/Users/zoran/PycharmProjects/mm_align/bckslash_test.py", line 43, in <module>
>      with open(src_file, mode='w', encoding='utf-8') as s_file:
> FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\zoran\\hrt\\src_70._godi?njica_pobjede_nad_fa?izmom_Za?to_ve?ina_?elnika_Europske_unije_bojkotira_vojnu_paradu_u_Moskvi__Kako_?e_se_obljetnica_pobjede_nad_nacisti?kom_Njema?kom_i_njenim_satelitima_obilje?iti_u_na?oj_zemlji__Ho?e_li_Josip_Broz_Tito_o.txt'
> 70._godi?njica_pobjede_nad_fa?izmom_Za?to_ve?ina_?elnika_Europske_unije_bojkotira_vojnu_paradu_u_Moskvi__Kako_?e_se_obljetnica_pobjede_nad_nacisti?kom_Njema?kom_i_njenim_satelitima_obilje?iti_u_na?oj_zemlji__Ho?e_li_Josip_Broz_Tito_o
> 260 C:\Users\zoran\hrt\src_70._godi?njica_pobjede_nad_fa?izmom_Za?to_ve?ina_?elnika_Europske_unije_bojkotira_vojnu_paradu_u_Moskvi__Kako_?e_se_obljetnica_pobjede_nad_nacisti?kom_Njema?kom_i_njenim_satelitima_obilje?iti_u_na?oj_zemlji__Ho?e_li_Josip_Broz_Tito_o.txt
> 260 C:\Users\zoran\hrt\des_70._godi?njica_pobjede_nad_fa?izmom_Za?to_ve?ina_?elnika_Europske_unije_bojkotira_vojnu_paradu_u_Moskvi__Kako_?e_se_obljetnica_pobjede_nad_nacisti?kom_Njema?kom_i_njenim_satelitima_obilje?iti_u_na?oj_zemlji__Ho?e_li_Josip_Broz_Tito_o.txt
>
> Process finished with exit code 1
>
> Cfg file has the following contents:
>
> C:\Users\zoran\PycharmProjects\mm_align\hrt3.cfg contents
> [Dir]
> ROOTDIR = C:\Users\zoran\hrt
>
>> I doubt that the problem is in the ROODIR value, but of course nothing
>> in your program bothers to check that that directory exists.  I expect
>> you either have too many characters total, or the 232th character is a
>> strange one.  Or perhaps title has a backslash in it (you took care of
>> forward slash).
>
> How to determine that?

Probably by calling os.path.isdir()

>
>> While we're at it, if you do have an OS limitation on size, your code is
>> truncating at the wrong point.  You need to truncate the title based on
>> the total size of src_file and dst_file, and since the code cannot know
>> the size of ROOTDIR, you need to include that in your figuring.
>
> Well, in my program I am defining a file name as category-id-description.mp3.
> If the file is too long I am cutting description (it wasn't clear from my example).

Since you've got non-ASCII characters in that name, the utf-8 version of 
the name will be longer.  I don't run Windows, but perhaps it's just a 
length problem after all.



-- 
DaveA


From ian.g.kelly at gmail.com  Sun May 10 21:37:38 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sun, 10 May 2015 19:37:38 -0600
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <87wq0gyvyr.fsf@elektro.pacujo.net>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
Message-ID: <CALwzidmQuo2Stw8ip0DzG9p4b1Rd77KYvddn0gcrUn0BqvQpSw@mail.gmail.com>

On Sun, May 10, 2015 at 3:16 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Scheme is my favorite language. I think, however, it is a pretty
> advanced language and requires a pretty solid basis in programming and
> computer science.
>
> Python, in contrast, is a great introductory programming language. Sure,
> you *can* get quite advanced with it, too, but you can get quite a bit
> of fun stuff done with just the basics.

MIT famously used Scheme in their introductory course for more than
two decades. Although they switched to Python a few years ago, I don't
think they did so because there was anything wrong with Scheme.
Wikipedia informs me that Yale and Grinnell are still using Scheme for
their introductory courses.

> Of course, you could introduce Scheme with similar simplifications.
> However, such simplifications (say, iterative constructs) are
> nonidiomatic in Scheme. The students should not get into bad habits
> that they need to be weaned off of later.

You don't need iterative constructs to teach an introductory course.
The full text of SICP (the "wizard book") is available on the web for
anyone to read at https://mitpress.mit.edu/sicp/. I don't think it
ever even *mentions* "iterative constructs". Where it distinguishes
recursive algorithms from iterative ones, recursive syntax is used in
both cases.

>> I'm thinking half way into the semester, instead of moving into
>> intermediate Scheme, perhaps that is a good time to switch to Python?

No, stick with one language for at least the first course. Needing to
learn the syntax and semantics of *two* programming languages,
especially two such different ones, is just going to distract students
from the fundamental concepts that the introductory class is intended
to teach.


From dreamingforward at gmail.com  Sun May 10 21:39:02 2015
From: dreamingforward at gmail.com (zipher)
Date: Sun, 10 May 2015 18:39:02 -0700 (PDT)
Subject: code blocks
In-Reply-To: <mailman.49.1430633046.12865.python-list@python.org>
References: <mailman.49.1430633046.12865.python-list@python.org>
Message-ID: <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>

 > I'm thinking how interesting it would be to add code blocks to Python, so that arbitrary strings of code can be passed around.?? It would open up some interesting possibilities for self-modifying code and generic programming.
>
> My suggestion would be to use triple double-quoted strings.? 

Hmmm.  Would be interesting to do this:

>>> codestr = decode(SomeClass)

...and have it return a triple-quoted string of the perhaps-reduced source for the class.  It's like serialization of code objects, but the serialization is just the source string w/LFs that would re-make the object.

Similarly, you'd want:

>>> encode(codestr)

to instantiate all objects in the codestr.  You can't do this with eval, because it doesn't allow assignment (eval(n=2) returns "InvalidSyntax").

mark


From dreamingforward at gmail.com  Sun May 10 21:46:49 2015
From: dreamingforward at gmail.com (zipher)
Date: Sun, 10 May 2015 18:46:49 -0700 (PDT)
Subject: Confessions of a Python fanboy
In-Reply-To: <d44977dc-2473-46e6-914c-1b218f32af85@d36g2000prb.googlegroups.com>
References: <e3725c9b-9f03-494a-a6a3-9d089f4f129c@d32g2000yqh.googlegroups.com>
 <d44977dc-2473-46e6-914c-1b218f32af85@d36g2000prb.googlegroups.com>
Message-ID: <56a4cc19-a6c1-4c55-873f-e944d4ad9165@googlegroups.com>

> > 3.) true OOP
> > Now before you go and get all "huffy" over this statement, hear me
> > out. Python is the best language in the world. But it damn sure has
> > some warts! "len(this)" instead of "obj.length" max(that) instead of
> > [1,2,3,4,5].max().
> 
> As the Zen says: '[P]racticality beats purity'. Personally, I'm not
> sure how a handful of convenient built-in functions make a language in
> which _everything is an object_ somehow "false" OO.
> 
> If you're really that concerned with writing "true" OO (for some
> wildly variable value of "true"), there's nothing stopping you from
> doing so now:
> 
>     obj.__len__()

The issue is that Python currently blurs a very powerful conceptual boundary in CS -- between the abstract space where objects and variables are held and the concrete underlying machine. 

By having methods like len() in your built-in namespace when it's really only relevant to objects that are types of containers, you blur one primary component of OOP:  encapsulation.

Mark


From rosuav at gmail.com  Sun May 10 21:53:24 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 11 May 2015 11:53:24 +1000
Subject: anomaly
In-Reply-To: <ad8dd792-a236-4514-9bb5-a90cf21535a4@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <7169d38e-b21c-472e-b0e3-544ed37fbfd2@googlegroups.com>
 <mailman.331.1431306462.12865.python-list@python.org>
 <ad8dd792-a236-4514-9bb5-a90cf21535a4@googlegroups.com>
Message-ID: <CAPTjJmrfDma6WgAFONQxZM+Owkqtjfo2wCkLtLMKPx=w1Z2cLQ@mail.gmail.com>

On Mon, May 11, 2015 at 11:18 AM, zipher <dreamingforward at gmail.com> wrote:
> Okay.  I apologize for thinking in C and believing "int" was a keyword.  It isn't in Python as you remind me.  However, this is where I'm arguing the purity has hammered practicality into the ground.
>
> Python is trying to be as elegant as LISP in trying to make everything an Object.  It's not necessary, and it's trying to put lipstick on a pig instead of making BBQ.  Python will be as elegant, but in a different direction of the axis.
>

How is it a problem for an integer to be an object? In Python, I can
put together a generic handler for a generic list and depend on
everything being an object:

lst = ["asdf", 2, lambda x: x+1, re.compile("q[a-tv-z]"), 3.14159]

In Java, I would have to box up that integer 2 and that float 3.14159
(and maybe the string too?) to make it possible to put them into a
generic collection, because integers and objects are fundamentally
different things. In Python, ints and floats and strings all follow
the same rules as other objects: they can be passed around, stored in
collections, turned into strings with str() or repr(), etc, etc, etc.
They have methods, they react to operators, everything is done
according to a simple set of rules. This does not harm practicality -
it's extremely helpful in a number of situations.

And I still don't see how this has anything to do with your confusion
about shadowing the name 'int'. As I see it, attempting to redefine
int to be a null subclass of str should have one of two results:
either it's a straight-up error, or it does exactly what happens in
Python - calling 'int' now calls a subclass of 'str'.

ChrisA


From rosuav at gmail.com  Sun May 10 22:00:14 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 11 May 2015 12:00:14 +1000
Subject: Confessions of a Python fanboy
In-Reply-To: <56a4cc19-a6c1-4c55-873f-e944d4ad9165@googlegroups.com>
References: <e3725c9b-9f03-494a-a6a3-9d089f4f129c@d32g2000yqh.googlegroups.com>
 <d44977dc-2473-46e6-914c-1b218f32af85@d36g2000prb.googlegroups.com>
 <56a4cc19-a6c1-4c55-873f-e944d4ad9165@googlegroups.com>
Message-ID: <CAPTjJmq3taac-QcNxyXSL_e34Uu95w9qS0bS+Z+jtNe5mgh1QA@mail.gmail.com>

On Mon, May 11, 2015 at 11:46 AM, zipher <dreamingforward at gmail.com> wrote:
>> > 3.) true OOP
>> > Now before you go and get all "huffy" over this statement, hear me
>> > out. Python is the best language in the world. But it damn sure has
>> > some warts! "len(this)" instead of "obj.length" max(that) instead of
>> > [1,2,3,4,5].max().
>>
>> As the Zen says: '[P]racticality beats purity'. Personally, I'm not
>> sure how a handful of convenient built-in functions make a language in
>> which _everything is an object_ somehow "false" OO.
>>
>> If you're really that concerned with writing "true" OO (for some
>> wildly variable value of "true"), there's nothing stopping you from
>> doing so now:
>>
>>     obj.__len__()
>
> The issue is that Python currently blurs a very powerful conceptual boundary in CS -- between the abstract space where objects and variables are held and the concrete underlying machine.
>
> By having methods like len() in your built-in namespace when it's really only relevant to objects that are types of containers, you blur one primary component of OOP:  encapsulation.
>

Please take care of your quoting. You just quoted two other posts, and
I have no idea who said things without going and digging in the
archive.

ChrisA


From dreamingforward at gmail.com  Sun May 10 22:09:00 2015
From: dreamingforward at gmail.com (zipher)
Date: Sun, 10 May 2015 19:09:00 -0700 (PDT)
Subject: anomaly
In-Reply-To: <mailman.334.1431309212.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <7169d38e-b21c-472e-b0e3-544ed37fbfd2@googlegroups.com>
 <mailman.331.1431306462.12865.python-list@python.org>
 <ad8dd792-a236-4514-9bb5-a90cf21535a4@googlegroups.com>
 <mailman.334.1431309212.12865.python-list@python.org>
Message-ID: <16bfec06-9667-4f10-818f-5b68097731c8@googlegroups.com>

> > Okay.  I apologize for thinking in C and believing "int" was a keyword.  It isn't in Python as you remind me.  However, this is where I'm arguing the purity has hammered practicality into the ground.
> >
> > Python is trying to be as elegant as LISP in trying to make everything an Object.  It's not necessary, and it's trying to put lipstick on a pig instead of making BBQ.  Python will be as elegant, but in a different direction of the axis.
> 
> How is it a problem for an integer to be an object?

It's not any more of a problem than trying to put lipstick on a pig, but at some point the pig might complain.  In this case, it has to be faced that an integer is a very concrete type -- because we're not running on a Symbolics Machine.  

Once again, Church's Thesis is misinforming people -- it doesn't matter that you can do everything on everything -- at some point the rubber has to hit the road and in Python, when I use an int, I expect some guarantee of O(log n) behavior, because I assume the dev team hasn't done something stupid, like implement bigints in LISP.

> In Python, I can
> put together a generic handler for a generic list and depend on
> everything being an object:
> 
> lst = ["asdf", 2, lambda x: x+1, re.compile("q[a-tv-z]"), 3.14159]

Okay, Chris, now you've made a pig out of lipstick.  The lipstick is lambda expressions.  We already know the BDFL opinion of them.  The pig is whatever that toy problem you just concocted is supposed to do.

> In Java, I would have to box up that integer 2 and that float 3.14159
> (and maybe the string too?) to make it possible to put them into a
> generic collection, because integers and objects are fundamentally
> different things.

No. Yes!? I mean no!  They are different, but it's a type, even though (in this thought experiment) it's not an object.  And types can be put into collections (hey just like C++!).

> In Python, ints and floats and strings all follow
> the same rules as other objects: they can be passed around, stored in
> collections, turned into strings with str() or repr(), etc, etc, etc.

But Chris, you are forgetting:  Python could do all this before the type/class unification.

> They have methods, they react to operators, everything is done
> according to a simple set of rules. This does not harm practicality -
> it's extremely helpful in a number of situations.

But you are infected with script-mind.  Scriptmind is a disease, where no one worries about the underlying architecture or the effects on machine performance.  This is fine for the web, but not for CPython.  Many lost souls have gone the way of Javascript and been infected.  Perhaps you are one of them.
 
> And I still don't see how this has anything to do with your confusion
> about shadowing the name 'int'. As I see it, attempting to redefine
> int to be a null subclass of str should have one of two results:
> either it's a straight-up error, or it does exactly what happens in
> Python - calling 'int' now calls a subclass of 'str'.

It just blurs conceptual boundaries, something I call "SeparabilityOfDomains".

Mark


From dreamingforward at gmail.com  Sun May 10 22:11:07 2015
From: dreamingforward at gmail.com (zipher)
Date: Sun, 10 May 2015 19:11:07 -0700 (PDT)
Subject: Confessions of a Python fanboy
In-Reply-To: <mailman.335.1431309617.12865.python-list@python.org>
References: <e3725c9b-9f03-494a-a6a3-9d089f4f129c@d32g2000yqh.googlegroups.com>
 <d44977dc-2473-46e6-914c-1b218f32af85@d36g2000prb.googlegroups.com>
 <56a4cc19-a6c1-4c55-873f-e944d4ad9165@googlegroups.com>
 <mailman.335.1431309617.12865.python-list@python.org>
Message-ID: <2b126a09-04e2-4821-9713-4ac0f66c4258@googlegroups.com>

> Please take care of your quoting. You just quoted two other posts, and
> I have no idea who said things without going and digging in the
> archive.

I'm sorry.  I've been sleeping on the beach, away from civilization, a little too long, and didn't see that this was a post from 6 years ago.

Feel free to disregard.

Mark


From rosuav at gmail.com  Sun May 10 22:11:57 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 11 May 2015 12:11:57 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
Message-ID: <CAPTjJmpzf2k+FhYb2E3Vp1rfWE3of0Ka_D+4whzrkqq2hRhJ7w@mail.gmail.com>

On Mon, May 11, 2015 at 6:43 AM, Chris Seberino <cseberino at gmail.com> wrote:
> Instead of learning only Scheme or only Python for a one semester intro
> course, what about learning BOTH?  Maybe that could somehow
> get the benefits of both?
>
> I'm thinking that for the VERY beginning, Scheme is the fastest language
> to get beginners up and running writing code due to the extremely minimal simple syntax.
>
> I'm thinking half way into the semester, instead of moving into intermediate Scheme, perhaps that is a good time to switch to Python?
>
> Would a little strong intro to 2 nice languages in one semester be
> same/good/worse/better than just 1?

I strongly recommend learning multiple languages, but not at the cost
of proper comprehension of one of them. Pick one and get started with
it, and once you have some basic competence, pick up another; you'll
gain a better appreciation for both that way.

As to which one first... I always recommend Python as a first
language, due to the lack of boilerplate and the simple layout. But if
you have a background that makes LISPy languages a better fit for you,
then by all means, take Scheme first. For most people I work with, an
imperative language makes a better first fit; most people understand
the concept of giving someone a series of instructions and expecting
them to be performed in sequence down the page, but functional
languages take more getting used to. But if you're already accustomed
to a functional style - maybe a heavy mathematical background - then
LISP-family languages will be a logical extension from that.

Eric Raymond recommends [1] learning five languages with distinctly
different styles: Python, C/C++, Java, Perl, and LISP. Of the five, I
would recommend leaving C and Perl for later, as neither is
particularly friendly to a new programmer; the other three you could
pick up in any order, and there are commercial courses using all of
them. (I personally don't think Java offers much that other languages
don't, and haven't used it for anything since the days when Java
applets were the only decent way to embed executable code in web
browsers; these days, I'd much rather do everything in Python or Pike.
But that doesn't mean Java isn't worth learning.) The more languages
you learn, the better you'll be as a programmer - but don't skimp on
one to pick up another.

ChrisA

[1] http://www.catb.org/esr/faqs/hacker-howto.html#skills1


From rustompmody at gmail.com  Sun May 10 22:12:28 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sun, 10 May 2015 19:12:28 -0700 (PDT)
Subject: anomaly
In-Reply-To: <mailman.334.1431309212.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <7169d38e-b21c-472e-b0e3-544ed37fbfd2@googlegroups.com>
 <mailman.331.1431306462.12865.python-list@python.org>
 <ad8dd792-a236-4514-9bb5-a90cf21535a4@googlegroups.com>
 <mailman.334.1431309212.12865.python-list@python.org>
Message-ID: <9f72c817-3127-479a-86b5-132eed7b4f96@googlegroups.com>

On Monday, May 11, 2015 at 7:23:44 AM UTC+5:30, Chris Angelico wrote:
> And I still don't see how this has anything to do with your confusion
> about shadowing the name 'int'.

Speaking as a compiler-writer -- everything :-)

In C 'int' is tagged off as different from 'myvar' earlier than say
'myvar' is different from 'printf'. The "define" in "#define" even earlier

Python sure has different rules of such 'tagging-off'
However all programming languages are the same in that you cant 'do semantics'
until you have finished 'doing syntax'
And name resolution is in a fuzzy area between the two --
Ask a theoretician about 'type-checking' and you will  hear: "This is just
context-sensitive syntax"
Ask a compiler-writer and you get: "Part of the semantic analysis module"


From rosuav at gmail.com  Sun May 10 22:18:11 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 11 May 2015 12:18:11 +1000
Subject: Confessions of a Python fanboy
In-Reply-To: <2b126a09-04e2-4821-9713-4ac0f66c4258@googlegroups.com>
References: <e3725c9b-9f03-494a-a6a3-9d089f4f129c@d32g2000yqh.googlegroups.com>
 <d44977dc-2473-46e6-914c-1b218f32af85@d36g2000prb.googlegroups.com>
 <56a4cc19-a6c1-4c55-873f-e944d4ad9165@googlegroups.com>
 <mailman.335.1431309617.12865.python-list@python.org>
 <2b126a09-04e2-4821-9713-4ac0f66c4258@googlegroups.com>
Message-ID: <CAPTjJmq_2zWywsF+chmeh=CW6prTHDozyuRBHvun3qW0EuL9gA@mail.gmail.com>

On Mon, May 11, 2015 at 12:11 PM, zipher <dreamingforward at gmail.com> wrote:
>> Please take care of your quoting. You just quoted two other posts, and
>> I have no idea who said things without going and digging in the
>> archive.
>
> I'm sorry.  I've been sleeping on the beach, away from civilization, a little too long, and didn't see that this was a post from 6 years ago.
>
> Feel free to disregard.
>

That actually has nothing to do with it. You're still quoting without citation.

This is why Google Groups has such a bad reputation. I think it's time
for me to *plonk* you.

ChrisA


From rosuav at gmail.com  Sun May 10 22:20:26 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 11 May 2015 12:20:26 +1000
Subject: anomaly
In-Reply-To: <9f72c817-3127-479a-86b5-132eed7b4f96@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <7169d38e-b21c-472e-b0e3-544ed37fbfd2@googlegroups.com>
 <mailman.331.1431306462.12865.python-list@python.org>
 <ad8dd792-a236-4514-9bb5-a90cf21535a4@googlegroups.com>
 <mailman.334.1431309212.12865.python-list@python.org>
 <9f72c817-3127-479a-86b5-132eed7b4f96@googlegroups.com>
Message-ID: <CAPTjJmonYnPE+m=M1pJR2n97Q1ZucyO0x0L9D0vuyEVPh71nEA@mail.gmail.com>

On Mon, May 11, 2015 at 12:12 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> On Monday, May 11, 2015 at 7:23:44 AM UTC+5:30, Chris Angelico wrote:
>> And I still don't see how this has anything to do with your confusion
>> about shadowing the name 'int'.
>
> Speaking as a compiler-writer -- everything :-)
>
> In C 'int' is tagged off as different from 'myvar' earlier than say
> 'myvar' is different from 'printf'. The "define" in "#define" even earlier
>
> Python sure has different rules of such 'tagging-off'
> However all programming languages are the same in that you cant 'do semantics'
> until you have finished 'doing syntax'
> And name resolution is in a fuzzy area between the two --
> Ask a theoretician about 'type-checking' and you will  hear: "This is just
> context-sensitive syntax"
> Ask a compiler-writer and you get: "Part of the semantic analysis module"

Right, I understand that 'int' may be a keyword... but that would mean
it's impossible to run the OP's code anyway. If "class int(str): pass"
works, it can't really do anything other than redefine the name "int"
to indicate a null subclass of "str" - or is there something else I've
missed?

ChrisA


From dreamingforward at gmail.com  Sun May 10 22:27:39 2015
From: dreamingforward at gmail.com (zipher)
Date: Sun, 10 May 2015 19:27:39 -0700 (PDT)
Subject: OFFTOPIC, WAS Re: Confessions of a Python fanboy
In-Reply-To: <mailman.337.1431310694.12865.python-list@python.org>
References: <e3725c9b-9f03-494a-a6a3-9d089f4f129c@d32g2000yqh.googlegroups.com>
 <d44977dc-2473-46e6-914c-1b218f32af85@d36g2000prb.googlegroups.com>
 <56a4cc19-a6c1-4c55-873f-e944d4ad9165@googlegroups.com>
 <mailman.335.1431309617.12865.python-list@python.org>
 <2b126a09-04e2-4821-9713-4ac0f66c4258@googlegroups.com>
 <mailman.337.1431310694.12865.python-list@python.org>
Message-ID: <edd03076-2704-433d-aaf9-ada887d54c4a@googlegroups.com>

On Sunday, May 10, 2015 at 9:18:55 PM UTC-5, Chris Angelico wrote:
> On Mon, May 11, 2015 at 12:11 PM, zipher <dreamingforward at gmail.com> wrote:
> >> Please take care of your quoting. You just quoted two other posts, and
> >> I have no idea who said things without going and digging in the
> >> archive.
> >
> > I'm sorry.  I've been sleeping on the beach, away from civilization, a little too long, and didn't see that this was a post from 6 years ago.
> >
> > Feel free to disregard.
> 
> That actually has nothing to do with it. You're still quoting without citation.

Well, I replied right at the point of my correspondent (Alex23).  If googlegroups isn't tracking that fact, it shouldn't put a reply button at every single message, but only at the end of the thread.

> This is why Google Groups has such a bad reputation. I think it's time
> for me to *plonk* you.

I see.  Someone should inform google not to put a reply after every post, but only at the end of a thread so users don't assume that it's tracking thread hierarchies.  So you're point is apropos, if that's the case.

In which case, my reply was to Alex23 who was replying to the OP's first post who goes by the name "r".

Mark


From rustompmody at gmail.com  Sun May 10 22:33:01 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sun, 10 May 2015 19:33:01 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <87wq0gyvyr.fsf@elektro.pacujo.net>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
Message-ID: <2a0d3025-94bd-4aa9-96e8-61ef9fc0dcc9@googlegroups.com>

On Monday, May 11, 2015 at 2:46:38 AM UTC+5:30, Marko Rauhamaa wrote:
> Chris Seberino :
> 
> > Instead of learning only Scheme or only Python for a one semester
> > intro course, what about learning BOTH? Maybe that could somehow get
> > the benefits of both?
> >
> > I'm thinking that for the VERY beginning, Scheme is the fastest
> > language to get beginners up and running writing code due to the
> > extremely minimal simple syntax.
> 
> Scheme is my favorite language. I think, however, it is a pretty
> advanced language and requires a pretty solid basis in programming and
> computer science.

Have you seen this?
http://www-inst.eecs.berkeley.edu/~cs61a/sp12/book/

Yeah from a certain pov scheme is so advanced that even Abelson and Sussman dont
quite get it: http://blog.languager.org/2013/08/applying-si-on-sicp.html
So (from that pov) how reasonable is it to expect students to get it?

Also good to see:
http://www.cs.kent.ac.uk/people/staff/dat/miranda/wadler87.pdf


From ben+python at benfinney.id.au  Sun May 10 22:41:05 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 11 May 2015 12:41:05 +1000
Subject: OFFTOPIC, WAS Re: Confessions of a Python fanboy
References: <e3725c9b-9f03-494a-a6a3-9d089f4f129c@d32g2000yqh.googlegroups.com>
 <d44977dc-2473-46e6-914c-1b218f32af85@d36g2000prb.googlegroups.com>
 <56a4cc19-a6c1-4c55-873f-e944d4ad9165@googlegroups.com>
 <mailman.335.1431309617.12865.python-list@python.org>
 <2b126a09-04e2-4821-9713-4ac0f66c4258@googlegroups.com>
 <mailman.337.1431310694.12865.python-list@python.org>
 <edd03076-2704-433d-aaf9-ada887d54c4a@googlegroups.com>
Message-ID: <85sib3ala6.fsf@benfinney.id.au>

zipher <dreamingforward at gmail.com> writes:

> On Sunday, May 10, 2015 at 9:18:55 PM UTC-5, Chris Angelico wrote:
> > That actually has nothing to do with it. You're still quoting
> > without citation.
>
> Well, I replied right at the point of my correspondent (Alex23).

That's not the issue :-)

The omission Chris is referring to is: in earlier messages, you were
stripping the ?attribution line? from directly above the quoted
material. That makes it anonymous, and it becomes difficult to track who
said what.

The attribution line looks like:

    ?On Sunday, May 10, 2015 at 9:18:55 PM UTC-5, Chris Angelico wrote:?

without the quote marks.

You need to preserve the attribution lines ? as you did on the message
just now, that I'm replying to ? in order to show who wrote what quoted
material.

Only remove attribution lines if you're *also* removing all of the
attributed material. Keep attribution lines for the material you're
quoting.

-- 
 \        ?We have to go forth and crush every world view that doesn't |
  `\                believe in tolerance and free speech.? ?David Brin |
_o__)                                                                  |
Ben Finney



From steve+comp.lang.python at pearwood.info  Sun May 10 23:19:30 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 11 May 2015 13:19:30 +1000
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <mailman.322.1431279306.12865.python-list@python.org>
Message-ID: <55501fc3$0$12987$c3e8da3$5496439d@news.astraweb.com>

On Mon, 11 May 2015 03:28 am, Gary Herron wrote:

> On 05/10/2015 09:48 AM, Rustom Mody wrote:
>> On Sunday, May 10, 2015 at 10:14:36 PM UTC+5:30, Ian wrote:
>>> On Sun, May 10, 2015 at 10:34 AM, Mark Rosenblitt-Janssen wrote:
>>>> Here's something that might be wrong in Python (tried on v2.7):
>>>>
>>>>>>> class int(str): pass
>>> This defines a new class named "int" that is a subclass of str. It has
>>> no relation to the builtin class int.
>>>
>>>>>>> int(3)
>>>> '3'
>>> This creates an instance of the above "int" class, which is basically
>>> equivalent to calling "str(3)".
>>>
>>> Were you expecting a different result?
>> In C (family) languages int is a keyword
>>  From that pov this is completely bizarre
> 
> Not really.  Expecting Python to act like C family languages *is* bizarre.

It's not really *bizarre* to expect that standard functions and types are
reserved words. That applies to many languages other than C, and is not
unique to the C family. E.g. it also applies to Fortran, and the Algol
family of languages.

There are advantages and disadvantages to allowing standard names be
redefined, and it should not surprise us that someone coming from another
language may be taken aback by the fact that

str = 23

doesn't give an error.

What is funny is that one might think it is a *bug* rather than a deliberate
feature, or perhaps misfeature. Python is well over 20 years old. If int
was intended to be a keyword, someone would have noticed by now :-)


> Common Python thought::  "We're all adults here."    If you want to
> override a builtin within your own namespace, who are we to stop you?
> Besides, it's still available as __builtins__.int  (unless you've also
> overridden that).

Don't use __builtins__ that is a private implementation detail. The correct
way to access the built-in namespace explicitly is:


import __builtin__ # Python 2
import builtins # Python 3




-- 
Steven



From ian.g.kelly at gmail.com  Sun May 10 23:31:13 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sun, 10 May 2015 21:31:13 -0600
Subject: code blocks
In-Reply-To: <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>
References: <mailman.49.1430633046.12865.python-list@python.org>
 <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>
Message-ID: <CALwzidmXZmK8V=B7jPQaqBvRLOhvGEj1-ZnWcLvngjzh5KuWjQ@mail.gmail.com>

On Sun, May 10, 2015 at 7:39 PM, zipher <dreamingforward at gmail.com> wrote:
> Similarly, you'd want:
>
>>>> encode(codestr)
>
> to instantiate all objects in the codestr.  You can't do this with eval, because it doesn't allow assignment (eval(n=2) returns "InvalidSyntax").

Is exec what you're looking for?

>>> exec('n = 2')
>>> print(n)
2


From ian.g.kelly at gmail.com  Sun May 10 23:36:38 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sun, 10 May 2015 21:36:38 -0600
Subject: code blocks
In-Reply-To: <CALwzidmXZmK8V=B7jPQaqBvRLOhvGEj1-ZnWcLvngjzh5KuWjQ@mail.gmail.com>
References: <mailman.49.1430633046.12865.python-list@python.org>
 <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>
 <CALwzidmXZmK8V=B7jPQaqBvRLOhvGEj1-ZnWcLvngjzh5KuWjQ@mail.gmail.com>
Message-ID: <CALwzidn_chNap5bqv+LTUtCFSUYUe8qOxHqDp20PDVRzjqsSXg@mail.gmail.com>

On Sun, May 10, 2015 at 9:31 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Sun, May 10, 2015 at 7:39 PM, zipher <dreamingforward at gmail.com> wrote:
>> Similarly, you'd want:
>>
>>>>> encode(codestr)
>>
>> to instantiate all objects in the codestr.  You can't do this with eval, because it doesn't allow assignment (eval(n=2) returns "InvalidSyntax").
>
> Is exec what you're looking for?
>
>>>> exec('n = 2')
>>>> print(n)
> 2

I just found that eval can be used to evaluate a code object compiled
in the 'exec' mode:

>>> eval(compile('n = 42', '', 'exec'))
>>> n
42

Interesting. But I suppose that the mode is really just a compilation
option and there's not really much distinction as far as eval is
concerned once they've been compiled.


From rustompmody at gmail.com  Sun May 10 23:40:15 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sun, 10 May 2015 20:40:15 -0700 (PDT)
Subject: code blocks
In-Reply-To: <mailman.340.1431315116.12865.python-list@python.org>
References: <mailman.49.1430633046.12865.python-list@python.org>
 <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>
 <mailman.340.1431315116.12865.python-list@python.org>
Message-ID: <8ab796df-a966-4b46-a499-36fb9163bb00@googlegroups.com>

On Monday, May 11, 2015 at 9:02:07 AM UTC+5:30, Ian wrote:
> Is exec what you're looking for?

Now Now Ian!

In classical times, corrupting the youth would fetch you a cup of hemlock.

Community service it is nowadays.


From steve+comp.lang.python at pearwood.info  Sun May 10 23:58:15 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 11 May 2015 13:58:15 +1000
Subject: Calling a function is faster than not calling it?
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <fNP3x.641427$3i4.583649@fx29.am4>
Message-ID: <555028d8$0$12997$c3e8da3$5496439d@news.astraweb.com>

On Mon, 11 May 2015 07:08 am, BartC wrote:

> On 10/05/2015 10:58, Steven D'Aprano wrote:
>> from timeit import Timer
>>
>> def func():
>>      a = 2
>>      b = 3
>>      c = 4
>>      return (a+b)*(a-b)/(a*c + b*c)
>>
>>
>> code = func.__code__
>> assert func() == eval(code)
>>
>> t1 = Timer("eval; func()", setup="from __main__ import func")
>> t2 = Timer("eval(code)", setup="from __main__ import code")
>>
>> # Best of 10 trials.
>> print (min(t1.repeat(repeat=10)))
>> print (min(t2.repeat(repeat=10)))
> 
> Maybe the overheads of using eval() are significant when calling a
> simple function such as your example.
> 
> When I made it do a bit more work:
[...]
> Then the eval() call took only 3% longer rather than 100%.

Well duh :-)

If I use:

def func():
    time.sleep(365*24*60*60)  # Sleep for a year.


then the overhead of calling it will be immeasurably small, regardless of
how we call it. But I don't care about big expensive functions. I actually
care about the fastest way to execute a tiny piece of code, and for that,
the overhead of eval is significant.

But regardless of whether the overhead is 3% or 30%, there is an anomaly
here. Conceptually at least, calling a function does more work than
eval'ing the function's code object: the function evaluates the code
object, plus it has overhead of its own which eval'ing the code object
lacks.

Here is one possible, minimal, pseudo-code implementation of function
__call__:


def __call__(self, *args):
    ns = {}
    for i, parameter in enumerate(self.parameters):
        ns[parameter] = args[i]
    return eval(self.__code__, globals(), ns)

If I extract the call to eval, and run that alone, without needing to set up
function arguments, I should avoid the overhead of the __call__ method. We
should expect that:

    eval(self.__code__, globals(), ns)

should be faster than:

    do_something()
    eval(self.__code__, globals(), ns)

no matter how small and insignificant do_something() actually is. The
anomaly is that, according to my tests, and confirmed by others, this is
not the case. However function __call__ works, it doesn't use eval. So what
does it do?




-- 
Steven



From steve+comp.lang.python at pearwood.info  Sun May 10 23:59:21 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 11 May 2015 13:59:21 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
Message-ID: <55502919$0$12997$c3e8da3$5496439d@news.astraweb.com>

On Mon, 11 May 2015 06:43 am, Chris Seberino wrote:

> I'm thinking that for the VERY beginning, Scheme is the fastest language
> to get beginners up and running writing code due to the extremely minimal
> simple syntax.

Do you believe that learning syntax is the hardest part for beginners to get
past?

If so, then perhaps you should start by teaching beginners machine code,
since the syntax is very simple: any sequence of hexadecimal digits is
syntactically valid machine code. If you can teach people to use
0123...9ABCDEF then you have just taught them everything they need to know
about the syntax.

*wink*

The point is, syntax can be *too* minimal, as well as too heavyweight.


-- 
Steven



From steve+comp.lang.python at pearwood.info  Mon May 11 00:00:16 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 11 May 2015 14:00:16 +1000
Subject: Why does unicode-escape decode escape symbols that are already
 escaped?
References: <CA+gt_a82WGXHUZhcdbTUWG+TRV1Ys1ZSrkGjOxgavZGjAh9FiQ@mail.gmail.com>
 <mailman.319.1431277269.12865.python-list@python.org>
Message-ID: <55502951$0$12997$c3e8da3$5496439d@news.astraweb.com>

On Mon, 11 May 2015 03:00 am, Terry Reedy wrote:

> Decoding a string only makes sense for rot-13

Or any other string-to-string encoding.

As has been discussed on python-ideas and python-dev many times, the idea of
a codec is much more general than just bytes -> string and string -> bytes.
It can deal with any transformation of data. The codec machinery can, I
believe, operate on any suitable type, and it can certainly operate on
bytes -> bytes and str -> str.


I have gradually come to agree that bytes and str objects should only
support decode() and encode() operations respectively, but str->str and
bytes->bytes codecs are useful to.


-- 
Steven



From rustompmody at gmail.com  Mon May 11 00:42:41 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sun, 10 May 2015 21:42:41 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <55502919$0$12997$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <55502919$0$12997$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <281c0f90-8353-4480-93d3-9c656505cc61@googlegroups.com>

On Monday, May 11, 2015 at 9:29:31 AM UTC+5:30, Steven D'Aprano wrote:
> On Mon, 11 May 2015 06:43 am, Chris Seberino wrote:
> 
> > I'm thinking that for the VERY beginning, Scheme is the fastest language
> > to get beginners up and running writing code due to the extremely minimal
> > simple syntax.
> 
> Do you believe that learning syntax is the hardest part for beginners to get
> past?

You seem to be asking a rhetorical question.

I regard it as a basic koan in CS
http://blog.languager.org/2015/04/poverty-universality-structure-0.html#fundamentalKoan


From steve+comp.lang.python at pearwood.info  Mon May 11 02:11:14 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 11 May 2015 16:11:14 +1000
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
Message-ID: <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Monday 11 May 2015 10:57, zipher wrote:

> I guess everyone expects this behavior since Python implemented this idea
> of "everything is an object", but I think this branch of OOP (on the
> branch of the Tree of Programming Languages) has to be chopped off.  The
> idea of everything is an object is backwards (unless your in a LISP
> machine).  Like I say, it's trying to be too pure and not practical.


Python is in production use in hundreds of thousands of organisations. It 
has been heavily used for over twenty years, in everything from quick and 
dirty one line scripts to hundred-thousand LOC applications. As well as the 
obvious OOP features, it includes features and idioms from imperative, 
procedural, functional and pipe-lining programming languages, with explicit 
influence from Pascal, C, Haskell, Dylan and Lisp, among others.

Yeah, "too pure and not practical". You're a funny guy.


-- 
Steve



From stefan_ml at behnel.de  Mon May 11 02:11:55 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 11 May 2015 08:11:55 +0200
Subject: Calling a function is faster than not calling it?
In-Reply-To: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <miph7c$26l$1@ger.gmane.org>

Steven D'Aprano schrieb am 10.05.2015 um 11:58:
> Why is calling a function faster than bypassing the function object and
> evaluating the code object itself? And not by a little, but by a lot?
> 
> Here I have a file, eval_test.py:
> 
> # === cut ===
> from timeit import Timer
> 
> def func():
>     a = 2
>     b = 3
>     c = 4
>     return (a+b)*(a-b)/(a*c + b*c)
> 
> 
> code = func.__code__
> assert func() == eval(code)
> 
> t1 = Timer("eval; func()", setup="from __main__ import func")
> t2 = Timer("eval(code)", setup="from __main__ import code")
> 
> # Best of 10 trials.
> print (min(t1.repeat(repeat=10)))
> print (min(t2.repeat(repeat=10)))
> 
> # === cut ===
> 
> 
> Note that both tests include a name lookup for eval, so that as much as
> possible I am comparing the two pieces of code on an equal footing.
> 
> Here are the results I get:
> 
> 
> [steve at ando ~]$ python2.7 eval_test.py
> 0.804041147232
> 1.74012994766
> [steve at ando ~]$ python3.3 eval_test.py
> 0.7233301624655724
> 1.7154695875942707
> 
> Directly eval'ing the code object is easily more than twice as expensive
> than calling the function, but calling the function has to eval the code
> object.

Well, yes, but it does so directly in C code. What you are essentially
doing here is replacing a part of the fast C code path for executing a
Python function by some mostly equivalent but more general Python code. So,
you're basically replacing a function call by another function call to
eval(), plus some extra generic setup overhead.

Python functions know exactly what they have to do internally in order to
execute. eval() cannot make the same streamlined assumptions.

Stefan




From steve+comp.lang.python at pearwood.info  Mon May 11 02:15:24 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 11 May 2015 16:15:24 +1000
Subject: Confessions of a Python fanboy
References: <e3725c9b-9f03-494a-a6a3-9d089f4f129c@d32g2000yqh.googlegroups.com>
 <d44977dc-2473-46e6-914c-1b218f32af85@d36g2000prb.googlegroups.com>
 <56a4cc19-a6c1-4c55-873f-e944d4ad9165@googlegroups.com>
Message-ID: <555048fd$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Monday 11 May 2015 11:46, zipher wrote:

> By having methods like len() in your built-in namespace when it's really
> only relevant to objects that are types of containers, you blur one
> primary component of OOP:  encapsulation.


Gosh, one would almost think that Python's concept of OOP wasn't pure enough 
for you.

You should talk to that other guy who says that Python is too pure, his name 
is Mark Rosenblitt-Janssen. I'm sure you two would have a lot to talk about.



-- 
Steve



From steve+comp.lang.python at pearwood.info  Mon May 11 02:21:52 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 11 May 2015 16:21:52 +1000
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <mailman.329.1431303268.12865.python-list@python.org>
Message-ID: <55504a81$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Monday 11 May 2015 10:14, Mark Rosenblitt-Janssen wrote:

> In case the example given at the start of the thread wasn't
> interesting enough, it also works in the other direction:
> 
>>>> class str(int):  pass
> 
>>>> str('2')
> 2  #<----- an integer!!!


Thank the gods that you're around to point this out to us, because we never 
would have extrapolated from the fact that int can be rebound to discover 
that the same applies to str!



-- 
Steve



From breamoreboy at yahoo.co.uk  Mon May 11 02:31:58 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 11 May 2015 07:31:58 +0100
Subject: Confessions of a Python fanboy
In-Reply-To: <555048fd$0$13004$c3e8da3$5496439d@news.astraweb.com>
References: <e3725c9b-9f03-494a-a6a3-9d089f4f129c@d32g2000yqh.googlegroups.com>
 <d44977dc-2473-46e6-914c-1b218f32af85@d36g2000prb.googlegroups.com>
 <56a4cc19-a6c1-4c55-873f-e944d4ad9165@googlegroups.com>
 <555048fd$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mipid1$j3r$1@ger.gmane.org>

On 11/05/2015 07:15, Steven D'Aprano wrote:
> On Monday 11 May 2015 11:46, zipher wrote:
>
>> By having methods like len() in your built-in namespace when it's really
>> only relevant to objects that are types of containers, you blur one
>> primary component of OOP:  encapsulation.
>
> Gosh, one would almost think that Python's concept of OOP wasn't pure enough
> for you.
>
> You should talk to that other guy who says that Python is too pure, his name
> is Mark Rosenblitt-Janssen. I'm sure you two would have a lot to talk about.
>

Ah, the penny has finally dropped.  I recall years ago somebody pointing 
out his website.  It had clearly been put together after the narcotics 
from several bulk carriers had been consumed.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Mon May 11 03:03:43 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 11 May 2015 08:03:43 +0100
Subject: Why is array.array('u') deprecated?
In-Reply-To: <c603bf41-a9e6-4b28-ab71-1d3ac127c3ec@googlegroups.com>
References: <80167ebf-c5ee-4699-89ca-ef544fe3decc@googlegroups.com>
 <554c8ff2$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <93aaa9cd-5d30-44db-9324-4fffe292e9e4@googlegroups.com>
 <mailman.237.1431084939.12865.python-list@python.org>
 <7c33ef30-5923-49eb-933c-bfb568cb4753@googlegroups.com>
 <mailman.249.1431090704.12865.python-list@python.org>
 <c603bf41-a9e6-4b28-ab71-1d3ac127c3ec@googlegroups.com>
Message-ID: <mipk8i$bdh$1@ger.gmane.org>

On 08/05/2015 15:40, jonathan.slenders at gmail.com wrote:
> Le vendredi 8 mai 2015 15:11:56 UTC+2, Peter Otten a ?crit :
>>> So, this works perfectly fine and fast. But it scares me that it's
>>> deprecated and Python 4 will not support it anymore.
>>
>> Hm, this doesn't even work with Python 3:
>
> My mistake. I should have tested better.
>
>>>>> data = array.array("u", u"x"*1000)
>>>>> data[100] = "y"
>>>>> re.search("y", data)
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>>    File "/usr/lib/python3.4/re.py", line 166, in search
>>      return _compile(pattern, flags).search(string)
>> TypeError: can't use a string pattern on a bytes-like object
>>
>> You can search for bytes
>>
>>>>> re.search(b"y", data)
>> <_sre.SRE_Match object; span=(400, 401), match=b'y'>
>>>>> data[101] = "z"
>>>>> re.search(b"y", data)
>> <_sre.SRE_Match object; span=(400, 401), match=b'y'>
>>>>> re.search(b"yz", data)
>>>>> re.search(b"y\0\0\0z", data)
>> <_sre.SRE_Match object; span=(400, 405), match=b'y\x00\x00\x00z'>
>>
>> but if that is good enough you can use a bytearray in the first place.
>
> Maybe I'll try that. Thanks for the suggestions!
>
> Jonathan
>

http://sourceforge.net/projects/pyropes/ of any use to you?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lorenzo.gatti at gmail.com  Mon May 11 03:16:00 2015
From: lorenzo.gatti at gmail.com (lorenzo.gatti at gmail.com)
Date: Mon, 11 May 2015 00:16:00 -0700 (PDT)
Subject: anomaly
In-Reply-To: <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
Message-ID: <d3d661fd-019a-4cfd-97b5-f2ddf1551d93@googlegroups.com>

On Monday, May 11, 2015 at 2:58:09 AM UTC+2, zipher wrote:
> I guess everyone expects this behavior since Python implemented this idea of "everything is an object", but I think this branch of OOP (on the branch of the Tree of Programming Languages) has to be chopped off.  The idea of everything is an object is backwards (unless your in a LISP machine).  Like I say, it's trying to be too pure and not practical.

Expressing this sort of emphatic, insulting and superficial opinions, to the people who would be most irritated by them (the Python mailing list) and without the slightest interest for contrary viewpoints and constructive discussion, is a very unpleasant form of trolling. 

If you don't like Python, you are welcome to prefer other programming languages. If you want to use Python with C-like primitive types, you can use arrays. Both choices are perfectly good, and routinely made without bothering other people with inane conversations.

Lorenzo Gatti


From breamoreboy at yahoo.co.uk  Mon May 11 03:40:56 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 11 May 2015 08:40:56 +0100
Subject: anomaly
In-Reply-To: <CANDiX9+m4=iGWVzJjnS00HTfg7AaZd_-UNKvt8jf-__WVrJo-g@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <CANDiX9+m4=iGWVzJjnS00HTfg7AaZd_-UNKvt8jf-__WVrJo-g@mail.gmail.com>
Message-ID: <mipmed$hnq$1@ger.gmane.org>

On 10/05/2015 20:12, boB Stepp wrote:
>
> On Sun, May 10, 2015 at 12:28 PM, Gary Herron <gherron at digipen.edu> wrote:
>
>> Common Python thought::  "We're all adults here."    If you want to override
>> a builtin within your own namespace, who are we to stop you?
>
> I'm surprised that this thought has not been added to the "Zen Of
> Python", as I see it as more and more recurrent as I continue my
> studies. What I would like to comprehend is what is the essential
> mindset of Python? That is, what do I need to understand, so that I am
> no longer likely to be surprised by discovering new possibilities in
> Python such as what the current thread is discussing?
>
>

You need to understand that Python is so powerful that after 14 years I 
still can't wrap my mind around all of the possibilities that it offers.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Mon May 11 03:44:31 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 11 May 2015 17:44:31 +1000
Subject: anomaly
In-Reply-To: <CANDiX9+m4=iGWVzJjnS00HTfg7AaZd_-UNKvt8jf-__WVrJo-g@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <CANDiX9+m4=iGWVzJjnS00HTfg7AaZd_-UNKvt8jf-__WVrJo-g@mail.gmail.com>
Message-ID: <CAPTjJmowyBS=VU0=KgMEhM4Xzimw96XqgrAgdi2NFqjB_AWUaA@mail.gmail.com>

On Mon, May 11, 2015 at 5:12 AM, boB Stepp <robertvstepp at gmail.com> wrote:
>> Common Python thought::  "We're all adults here."    If you want to override
>> a builtin within your own namespace, who are we to stop you?
>
> I'm surprised that this thought has not been added to the "Zen Of
> Python", as I see it as more and more recurrent as I continue my
> studies. What I would like to comprehend is what is the essential
> mindset of Python? That is, what do I need to understand, so that I am
> no longer likely to be surprised by discovering new possibilities in
> Python such as what the current thread is discussing?

The Zen of Python is a static document, a historical artifact of a
sort. But in terms of understanding the philosophy of Python, "we're
all adults here" is a big part of it. Once you grok the notion that
nothing can be prevented, you're freed from such considerations as:

* Obfuscating, encrypting, or otherwise hiding your source code
* Private members with restricted access
* Strict type checking, to prevent someone passing in a wrong piece of data
* Prevention of monkey-patching

etc, etc, etc. In actual fact, anyone can bypass any restriction, in
any language; and Python is just more open/honest about it than
languages like C++; for instance, instead of having true private
members where the compiler stops you from looking at or changing them,
Python gives you single-underscore-named attributes, where nobody
stops you from doing anything, but there's a general understanding
that they're not governed by the usual compatibility rules, so
upgrading a library might break your code. Happy with that? Go ahead
then, use the internals.

Hakuna matata, what a wonderful phrase.

ChrisA


From p.f.moore at gmail.com  Mon May 11 04:12:24 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 11 May 2015 09:12:24 +0100
Subject: [Python-Dev] anomaly
In-Reply-To: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
Message-ID: <CACac1F_EDsHyDFJR-H0MnBgqpmmfX-441B7c_nS6xk3h976-cw@mail.gmail.com>

On 10 May 2015 at 17:34, Mark Rosenblitt-Janssen
<dreamingforward at gmail.com> wrote:
> Here's something that might be wrong in Python (tried on v2.7):
>
>>>> class int(str): pass
>
>>>> int(3)
> '3'

It's not wrong as such. It is allowed to define your own class that
subclasses a builtin class, and it's allowed to shadow builtin names.
So while this is (obviously) bad practice, it's not wrong.

For a simpler example:

Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600
64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> str
<class 'str'>
>>> str = "Hello"
>>> str
'Hello'

Paul


From bc at freeuk.com  Mon May 11 05:50:59 2015
From: bc at freeuk.com (BartC)
Date: Mon, 11 May 2015 10:50:59 +0100
Subject: Calling a function is faster than not calling it?
In-Reply-To: <555028d8$0$12997$c3e8da3$5496439d@news.astraweb.com>
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <fNP3x.641427$3i4.583649@fx29.am4>
 <555028d8$0$12997$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <8Y_3x.668959$LI3.568865@fx24.am4>

On 11/05/2015 04:58, Steven D'Aprano wrote:
> On Mon, 11 May 2015 07:08 am, BartC wrote:
>
>> On 10/05/2015 10:58, Steven D'Aprano wrote:

>>> def func():
>>>       a = 2
>>>       b = 3
>>>       c = 4
>>>       return (a+b)*(a-b)/(a*c + b*c)

>>> print (min(t1.repeat(repeat=10)))
>>> print (min(t2.repeat(repeat=10)))
>>
>> Maybe the overheads of using eval() are significant when calling a
>> simple function such as your example.
>>
>> When I made it do a bit more work:
> [...]
>> Then the eval() call took only 3% longer rather than 100%.
>
> Well duh :-)
>
> If I use:
>
> def func():
>      time.sleep(365*24*60*60)  # Sleep for a year.
>
>
> then the overhead of calling it will be immeasurably small, regardless of
> how we call it. But I don't care about big expensive functions. I actually
> care about the fastest way to execute a tiny piece of code, and for that,
> the overhead of eval is significant.

Your subject line is misleading. You're not comparing calling a function 
with not calling it. You're comparing different ways of calling it (if 
'calling' means somehow passing control to the body of the function, 
then eventually getting back to where you were).

You just seem surprised that using eval() to do this is slower than a 
direct call. I, knowing nothing about eval (I thought it liked string 
arguments), except that it can be used to execute the body of a 
function, am not that surprised.

  > But regardless of whether the overhead is 3% or 30%, there is an anomaly
> here. Conceptually at least, calling a function does more work than
> eval'ing the function's code object: the function evaluates the code
> object, plus it has overhead of its own which eval'ing the code object
> lacks.
>
> Here is one possible, minimal, pseudo-code implementation of function
> __call__:
>
>
> def __call__(self, *args):
>      ns = {}
>      for i, parameter in enumerate(self.parameters):
>          ns[parameter] = args[i]
>      return eval(self.__code__, globals(), ns)
>
> If I extract the call to eval, and run that alone, without needing to set up
> function arguments, I should avoid the overhead of the __call__ method. We
> should expect that:

Perhaps you're making assumptions about how a normal call is 
implemented. Maybe it's not just implemented on top of eval(). (And it's 
unlikely to use an actual call to __call__, because how would /that/ 
call be implemented; into another call to a nested __call__, etc?)

(I've no idea how Python actually does it, but performing a quick test 
in another (also interpreted) language, calling an empty function func() 
via a function pointer takes twice as long as calling it directly. 
Because it has to do a bit more work.

In that case, because there are two bytecodes to be executed for the 
call, not one, and it needs to check the number of arguments because it 
can't be done when the bytecode is generated.)

> The
> anomaly is that, according to my tests, and confirmed by others, this is
> not the case. However function __call__ works, it doesn't use eval. So what
> does it do?

Is eval() called like a normal function? Then maybe you're forgetting 
the overhead of calling a function. Whatever is saved on calling func(), 
is expended on calling eval(). And eval() has overheads of its own 
because it can't just immediately execute the code like func() can, it 
has to get to that point first. And the 'return' in the code that is 
being executed, can't (I think) return directly to your program, but has 
to first return to eval(), which then does its own return.

-- 
Bartc


From antoon.pardon at rece.vub.ac.be  Mon May 11 06:15:58 2015
From: antoon.pardon at rece.vub.ac.be (Antoon Pardon)
Date: Mon, 11 May 2015 12:15:58 +0200
Subject: anomaly
In-Reply-To: <554F9525.5040101@digipen.edu>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
Message-ID: <5550815E.5080600@rece.vub.ac.be>

Op 10-05-15 om 19:28 schreef Gary Herron:

> Common Python thought::  "We're all adults here."    If you want to 
> override a builtin within your own namespace, who are we to stop you?
> Besides, it's still available as __builtins__.int  (unless you've also 
> overridden that).

This is a common python myth. That is selectively used when convenient and
ignored when that is convenient.

Try overriding None, True or False in python3 and see what happens.

-- 
Antoon Pardon



From breamoreboy at yahoo.co.uk  Mon May 11 06:40:45 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 11 May 2015 11:40:45 +0100
Subject: anomaly
In-Reply-To: <5550815E.5080600@rece.vub.ac.be>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
Message-ID: <miq0vh$tdg$1@ger.gmane.org>

On 11/05/2015 11:15, Antoon Pardon wrote:
> Op 10-05-15 om 19:28 schreef Gary Herron:
>
>> Common Python thought::  "We're all adults here."    If you want to
>> override a builtin within your own namespace, who are we to stop you?
>> Besides, it's still available as __builtins__.int  (unless you've also
>> overridden that).
>
> This is a common python myth. That is selectively used when convenient and
> ignored when that is convenient.
>
> Try overriding None, True or False in python3 and see what happens.
>

According to 
https://docs.python.org/3/reference/lexical_analysis.html#keywords None, 
True and False are all keywords in Python 3, int isn't as I believe has 
already been pointed out.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From antoon.pardon at rece.vub.ac.be  Mon May 11 07:39:44 2015
From: antoon.pardon at rece.vub.ac.be (Antoon Pardon)
Date: Mon, 11 May 2015 13:39:44 +0200
Subject: anomaly
In-Reply-To: <miq0vh$tdg$1@ger.gmane.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
Message-ID: <55509500.8020102@rece.vub.ac.be>

Op 11-05-15 om 12:40 schreef Mark Lawrence:
> On 11/05/2015 11:15, Antoon Pardon wrote:
>> Op 10-05-15 om 19:28 schreef Gary Herron:
>>
>>> Common Python thought::  "We're all adults here."    If you want to
>>> override a builtin within your own namespace, who are we to stop you?
>>> Besides, it's still available as __builtins__.int  (unless you've also
>>> overridden that).
>>
>> This is a common python myth. That is selectively used when
>> convenient and
>> ignored when that is convenient.
>>
>> Try overriding None, True or False in python3 and see what happens.
>>
>
> According to
> https://docs.python.org/3/reference/lexical_analysis.html#keywords
> None, True and False are all keywords in Python 3, int isn't as I
> believe has already been pointed out.
>
Which is exactly the point! They were turned into keywords because the
developers didn't want to allow them being overridden. There is no
a priori reason why we should turn "True" into a keyword and allow
"int" in the builtins.

We are only allowed to be adults, for as far as the developers let us.
They allow us to be adults with regards to "int" but they don't allow
us to be adults with regards to "True".

Defending "int" being overridable by declating "We're all adults" is
being selective.

-- 
Antoon Pardon



From skybuck2000 at hotmail.com  Mon May 11 07:46:04 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Mon, 11 May 2015 13:46:04 +0200
Subject: Feature Request: Reposition Execution
Message-ID: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>

Hello,

Sometimes it can be handy to "interrupt/reset/reposition" a running script.

For example something externally goes badly wrong.

The script is unaware of this.

Current solution would require to have an "Abort" boolean everywhere.

The abort boolean could then be set to True to indicate all code and all 
loops must abort.

This is far from the ideal solution.

I think a much better solution could be to "reposition" the "instruction 
pointer" so to speak.

However for x86 stack clean up would be necessary. I would assume python has 
a stack too... which would need cleaning up.

Perhaps the entire stack can simply be cleaned up.

So that the point for execution continuation is always "clean".... in a 
"clean state".

So this allows for the stack to be cleaned completely... without any 
trouble.

I also hope this feature gets implemented quickly cause I kinda need it.

Therefore I will give some example code to see how it could look like:

def MyFunction():

  while SomeCondition:
     RunMyCode

return

def OtherFunction():
  while BatmanIsAliveLOL:
     BustPenguins
  return

def DetectProblem:

  if ProblemDetected:
    Aborted = True
    Abort( AbortToPoint )
  return

SpawnThread( DetectProblem)

while MyMainLoop:


   AbortToPoint:
     if Aborted:
        Reset stuff

   MainCode, Call Routines.


Bye,
  Skybuck. 



From breamoreboy at yahoo.co.uk  Mon May 11 07:47:24 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 11 May 2015 12:47:24 +0100
Subject: anomaly
In-Reply-To: <55509500.8020102@rece.vub.ac.be>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org> <55509500.8020102@rece.vub.ac.be>
Message-ID: <miq4sc$rme$1@ger.gmane.org>

On 11/05/2015 12:39, Antoon Pardon wrote:
> Op 11-05-15 om 12:40 schreef Mark Lawrence:
>> On 11/05/2015 11:15, Antoon Pardon wrote:
>>> Op 10-05-15 om 19:28 schreef Gary Herron:
>>>
>>>> Common Python thought::  "We're all adults here."    If you want to
>>>> override a builtin within your own namespace, who are we to stop you?
>>>> Besides, it's still available as __builtins__.int  (unless you've also
>>>> overridden that).
>>>
>>> This is a common python myth. That is selectively used when
>>> convenient and
>>> ignored when that is convenient.
>>>
>>> Try overriding None, True or False in python3 and see what happens.
>>>
>>
>> According to
>> https://docs.python.org/3/reference/lexical_analysis.html#keywords
>> None, True and False are all keywords in Python 3, int isn't as I
>> believe has already been pointed out.
>>
> Which is exactly the point! They were turned into keywords because the
> developers didn't want to allow them being overridden. There is no
> a priori reason why we should turn "True" into a keyword and allow
> "int" in the builtins.
>
> We are only allowed to be adults, for as far as the developers let us.
> They allow us to be adults with regards to "int" but they don't allow
> us to be adults with regards to "True".
>
> Defending "int" being overridable by declating "We're all adults" is
> being selective.
>

If you say so but I disagree and can't be bothered to say anything else.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From bc at freeuk.com  Mon May 11 07:55:25 2015
From: bc at freeuk.com (BartC)
Date: Mon, 11 May 2015 12:55:25 +0100
Subject: anomaly
In-Reply-To: <ad8dd792-a236-4514-9bb5-a90cf21535a4@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <7169d38e-b21c-472e-b0e3-544ed37fbfd2@googlegroups.com>
 <mailman.331.1431306462.12865.python-list@python.org>
 <ad8dd792-a236-4514-9bb5-a90cf21535a4@googlegroups.com>
Message-ID: <QM04x.689546$dT.412345@fx35.am4>

On 11/05/2015 02:18, zipher wrote:
>> Huh?   Python has plenty of keywords, and indeed, none of them can be
>> redefined or shadowed.    But you would gain nothing (and lose a bit or
>> dynamic-language freedom) by making int a keyword.
>
> Okay.  I apologize for thinking in C and believing "int" was a keyword.

Even C allows this:

#define int(a) strlen(a)

(although it doesn't like #define int strlen for some reason.)

-- 
Bartc


From davea at davea.name  Mon May 11 07:57:01 2015
From: davea at davea.name (Dave Angel)
Date: Mon, 11 May 2015 07:57:01 -0400
Subject: Feature Request: Reposition Execution
In-Reply-To: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>
References: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>
Message-ID: <5550990D.5080509@davea.name>

On 05/11/2015 07:46 AM, Skybuck Flying wrote:
> Hello,
>
> Sometimes it can be handy to "interrupt/reset/reposition" a running script.
>
> For example something externally goes badly wrong.
>

os.kill()

then in your process, handle the exception, and do whatever you think is 
worthwhile.




-- 
DaveA


From marko at pacujo.net  Mon May 11 07:58:24 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 11 May 2015 14:58:24 +0300
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
Message-ID: <87fv73iavz.fsf@elektro.pacujo.net>

Antoon Pardon <antoon.pardon at rece.vub.ac.be>:

> Which is exactly the point! They were turned into keywords because the
> developers didn't want to allow them being overridden. There is no a
> priori reason why we should turn "True" into a keyword and allow "int"
> in the builtins.
>
> We are only allowed to be adults, for as far as the developers let us.
> They allow us to be adults with regards to "int" but they don't allow
> us to be adults with regards to "True".
>
> Defending "int" being overridable by declating "We're all adults" is
> being selective.

I'm still failing to see the point. What problem are you having a
difficulty solving?


Marko


From info at egenix.com  Mon May 11 07:59:06 2015
From: info at egenix.com (eGenix Team: M.-A. Lemburg)
Date: Mon, 11 May 2015 13:59:06 +0200
Subject: ANN: eGenix PyRun - One file Python Runtime 2.1.0
Message-ID: <5550998A.7050204@egenix.com>

________________________________________________________________________

ANNOUNCING

                 eGenix PyRun - One file Python Runtime

                            Version 2.1.0


         An easy-to-use single file relocatable Python run-time -
           available for Linux, Mac OS X and Unix platforms,
                 with support for Python 2.6, 2.7 and
                       **also for Python 3.4**.


This announcement is also available on our web-site for online reading:
http://www.egenix.com/company/news/eGenix-PyRun-2.1.0-GA.html

________________________________________________________________________

INTRODUCTION

eGenix PyRun is our open source, one file, no installation version of
Python, making the distribution of a Python interpreter to run based
scripts and applications to Unix based systems as simple as copying a
single file.

eGenix PyRun's executable only needs 11MB for Python 2 and 13MB for
Python 3, but still supports most Python application and scripts - and
it can be compressed to just 3-4MB using upx, if needed.

Compared to a regular Python installation of typically 100MB on disk,
eGenix PyRun is ideal for applications and scripts that need to be
distributed to several target machines, client installations or
customers.

It makes "installing" Python on a Unix based system as simple as
copying a single file.

eGenix has been using eGenix PyRun internally in the mxODBC Connect
Server product since 2008 with great success and decided to make it
available as a stand-alone open-source product.

We provide both the source archive to build your own eGenix PyRun, as
well as pre-compiled binaries for Linux, FreeBSD and Mac OS X, as 32-
and 64-bit versions. The binaries can be downloaded manually, or you
can let our automatic install script install-pyrun take care of the
installation: ./install-pyrun dir and you're done.

Please see the product page for more details:

    http://www.egenix.com/products/python/PyRun/

________________________________________________________________________

NEWS

This major new release of eGenix PyRun 2.1 comes with the following new features and changes:

New Features
------------

 * Upgraded eGenix PyRun to work with and use Python 2.7.9 per
   default.

 * Upgraded eGenix PyRun to work with and use Python 3.4.3 for Python
   3 support.

 * Added support for setting rpath on the PyRun binary to make it easy
   to ship external libraries together with PyRun, without having to
   set the LD_LIBRARY_PATH variable.

 * Added special support for egenix-pyopenssl to load its OpenSSL libs
   during startup.

 * PyRun allows to disable the new default HTTPS certificate
   verification (PEP 476) by setting the env var PYRUN_HTTPSVERIFY to
   0. Default is to apply certificate verification.

 * Add support for more Python command line options: -s and -3 are
   ignored, -R raises an error explaining to use the PYTHONHASHSEED
   env var instead, -B prevents writing of byte code files.This should
   increase compatibility of PyRun with the standard Python command
   line interface.

Enhancements / Changes
----------------------

 * Added patch for Python 2.7.9 and 3.4.3 to make ctypes work again
   after changes to those versions of Python. See
   https://bugs.python.org/issue23042 for details.

 * The PyRun -v command line switch will now be passed to Python and
   supports setting the level using e.g. -vv for a verbose 2 level.

 * Disabled user site configurations in PyRun, since these are not
   needed or wanted for typical PyRun uses and cost startup time.

 * Optimized sys.path setup a bit. PyRun won't check for a Python
   build run and will not check lib/site-python on startup anymore.

 * PyRun for Python 3.4 is no longer compiled to always run in
   optimized mode. This allows using asserts in tests again.

Fixes
-----

 * Entering "license" in the interactive shell now returns the correct
   URL for all supported Python versions.

 * Tilde expansion now works for most arguments of install-pyrun. This
   wasn't working before due to a bug.

install-pyrun Quick Install Enhancements
---------------------------------------------

eGenix PyRun includes a shell script called install-pyrun, which
greatly simplifies installation of PyRun. It works much like the
virtualenv shell script used for creating new virtual environments
(except that there's nothing virtual about PyRun environments).

    https://downloads.egenix.com/python/install-pyrun

With the script, an eGenix PyRun installation is as simple as running:

    ./install-pyrun targetdir

This will automatically detect the platform, download and install the
right pyrun version into targetdir.

We have updated this script since the last release:

 * Updated install-pyrun to default to eGenix PyRun 2.1.0 and its
   feature set.

 * Added -r/--requirements option which allows automatically
   installing a set of required packages using a requirements file, so
   you can easily set up a complete Python environment using a single

   ./install-pyrun -r requirements.txt targetdir

 * Updated install-pyrun to install pyrun 2.1.0, setuptools 15.2 and
   pip 1.5.6 per default.

For a complete list of changes, please see the eGenix PyRun Changelog:

    http://www.egenix.com/products/python/PyRun/changelog.html

________________________________________________________________________

LICENSE

eGenix PyRun is distributed under the eGenix.com Public License 1.1.0
which is an Open Source license similar to the Python license. You can
use eGenix PyRun in both commercial and non-commercial settings
without fee or charge.

Please see our license page for more details:

    http://www.egenix.com/products/python/PyRun/license.html

The package comes with full source code.

________________________________________________________________________

DOWNLOADS

The download archives and instructions for installing eGenix PyRun can
be found at:

    http://www.egenix.com/products/python/PyRun/

As always, we are providing pre-built binaries for all common
platforms: Windows 32/64-bit, Linux 32/64-bit, FreeBSD 32/64-bit, Mac
OS X 32/64-bit. Source code archives are available for installation on
other platforms, such as Solaris, AIX, HP-UX, etc.

_______________________________________________________________________

SUPPORT

Commercial support for this product is available from eGenix.com.
Please see

    http://www.egenix.com/services/support/

for details about our support offerings.

________________________________________________________________________

MORE INFORMATION

For more information about eGenix PyRun, licensing and download
instructions, please visit our web-site:

    http://www.egenix.com/products/python/PyRun/

About Python (http://www.python.org/):

    Python is an object-oriented Open Source programming language
    which runs on all modern platforms. By integrating ease-of-use,
    clarity in coding, enterprise application connectivity and rapid
    application design, Python establishes an ideal programming
    platform for today's IT challenges.

About eGenix (http://www.egenix.com/):

    eGenix is a software project, consulting and product company
    specializing in expert project services and professional quality
    products for companies, Python users and developers.

Enjoy,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 11 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From marko at pacujo.net  Mon May 11 08:02:46 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 11 May 2015 15:02:46 +0300
Subject: Feature Request: Reposition Execution
References: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>
 <mailman.354.1431345441.12865.python-list@python.org>
Message-ID: <87bnhriaop.fsf@elektro.pacujo.net>

Dave Angel <davea at davea.name>:

> On 05/11/2015 07:46 AM, Skybuck Flying wrote:
>> Hello,
>>
>> Sometimes it can be handy to "interrupt/reset/reposition" a running script.
>> For example something externally goes badly wrong.
>
> os.kill()
>
> then in your process, handle the exception, and do whatever you think
> is worthwhile.

One thing that gives me trouble quite often is that Ctrl-C doesn't kill
a multithreaded Python program. Instead, you need to do:

   Ctrl-Z
   [1]+  Stopped
   kill %1


Marko


From robertvstepp at gmail.com  Mon May 11 08:26:58 2015
From: robertvstepp at gmail.com (boB Stepp)
Date: Mon, 11 May 2015 07:26:58 -0500
Subject: anomaly
In-Reply-To: <mipmed$hnq$1@ger.gmane.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <CANDiX9+m4=iGWVzJjnS00HTfg7AaZd_-UNKvt8jf-__WVrJo-g@mail.gmail.com>
 <mipmed$hnq$1@ger.gmane.org>
Message-ID: <CANDiX9JomJNTJGpJG9QvZd2_qDjyBisfKEAfEoXqvsFrq1bOPA@mail.gmail.com>

On Mon, May 11, 2015 at 2:40 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> On 10/05/2015 20:12, boB Stepp wrote:

>> I'm surprised that this thought has not been added to the "Zen Of
>> Python", as I see it as more and more recurrent as I continue my
>> studies. What I would like to comprehend is what is the essential
>> mindset of Python? That is, what do I need to understand, so that I am
>> no longer likely to be surprised by discovering new possibilities in
>> Python such as what the current thread is discussing?
>>
>>
>
> You need to understand that Python is so powerful that after 14 years I
> still can't wrap my mind around all of the possibilities that it offers.

That's quite a statement! I see I have a looonnnnggg journey ahead!!



-- 
boB


From steve+comp.lang.python at pearwood.info  Mon May 11 08:34:26 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 11 May 2015 22:34:26 +1000
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
Message-ID: <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>

On Mon, 11 May 2015 09:39 pm, Antoon Pardon wrote:

> There is no
> a priori reason why we should turn "True" into a keyword and allow
> "int" in the builtins.

Why should there be an *a priori* reason?

There's no a priori reason why I speak English, instead of communicating
through the medium of dance. Nevertheless there are many good, compelling
reasons for speaking English (as well as some reasons that are best
described as historical accidents). Might I be better off if I spoke Latin,
Japanese or Klingon? Perhaps, perhaps not. Those are valid choices too, but
they're not choices I have made.

With programming languages, the designer can take the same route as Pascal
or Java, and define standard functions as keywords that cannot be shadowed
or redefined. Or one can design the language to be like Forth, where there
are no keywords and literally everything can be redefined.

Or one can take a middle-road, where certain syntactic elements, and a very
small number of constant values, are made keywords, and everything else is
free to be redefined.

There's no a priori reason for any of those choices. But there are reasons.


-- 
Steven



From steve+comp.lang.python at pearwood.info  Mon May 11 08:35:46 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 11 May 2015 22:35:46 +1000
Subject: Feature Request: Reposition Execution
References: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>
 <mailman.354.1431345441.12865.python-list@python.org>
Message-ID: <5550a223$0$13013$c3e8da3$5496439d@news.astraweb.com>

On Mon, 11 May 2015 09:57 pm, Dave Angel wrote:

> On 05/11/2015 07:46 AM, Skybuck Flying wrote:
>> Hello,
>>
>> Sometimes it can be handy to "interrupt/reset/reposition" a running
>> script.
>>
>> For example something externally goes badly wrong.
>>
> 
> os.kill()
> 
> then in your process, handle the exception, and do whatever you think is
> worthwhile.


Are you suggesting that the app sends itself a signal?

Is that even allowed?



-- 
Steven



From robertvstepp at gmail.com  Mon May 11 08:43:33 2015
From: robertvstepp at gmail.com (boB Stepp)
Date: Mon, 11 May 2015 07:43:33 -0500
Subject: anomaly
In-Reply-To: <CAPTjJmowyBS=VU0=KgMEhM4Xzimw96XqgrAgdi2NFqjB_AWUaA@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <CANDiX9+m4=iGWVzJjnS00HTfg7AaZd_-UNKvt8jf-__WVrJo-g@mail.gmail.com>
 <CAPTjJmowyBS=VU0=KgMEhM4Xzimw96XqgrAgdi2NFqjB_AWUaA@mail.gmail.com>
Message-ID: <CANDiX9LdnDWNBgiEbOXz_SFR89qApCdw-TVK_v6E87_BL-m3fg@mail.gmail.com>

On Mon, May 11, 2015 at 2:44 AM, Chris Angelico <rosuav at gmail.com> wrote:
> On Mon, May 11, 2015 at 5:12 AM, boB Stepp <robertvstepp at gmail.com> wrote:
>>> Common Python thought::  "We're all adults here."    If you want to override
>>> a builtin within your own namespace, who are we to stop you?
>>
>> I'm surprised that this thought has not been added to the "Zen Of
>> Python", as I see it as more and more recurrent as I continue my
>> studies. What I would like to comprehend is what is the essential
>> mindset of Python? That is, what do I need to understand, so that I am
>> no longer likely to be surprised by discovering new possibilities in
>> Python such as what the current thread is discussing?
>
> The Zen of Python is a static document, a historical artifact of a
> sort. But in terms of understanding the philosophy of Python, "we're
> all adults here" is a big part of it. Once you grok the notion that
> nothing can be prevented, you're freed from such considerations as:
>
> * Obfuscating, encrypting, or otherwise hiding your source code
> * Private members with restricted access
> * Strict type checking, to prevent someone passing in a wrong piece of data
> * Prevention of monkey-patching
>
> etc, etc, etc. In actual fact, anyone can bypass any restriction, in
> any language; and Python is just more open/honest about it than
> languages like C++; for instance, instead of having true private
> members where the compiler stops you from looking at or changing them,
> Python gives you single-underscore-named attributes, where nobody
> stops you from doing anything, but there's a general understanding
> that they're not governed by the usual compatibility rules, so
> upgrading a library might break your code. Happy with that? Go ahead
> then, use the internals.

Thanks, Chris, that helps a lot. It seems that beyond being a
programming language, Python has a well-established culture that
suggests how the language should be used. I am gathering that
understanding the language and embracing the existing culture are
needed to use Python in the way it is meant to be used, though the
language design allows *other* ways, too.

> Hakuna matata, what a wonderful phrase.

Indeed! And a good way to start my Monday morning.


-- 
boB


From ndbecker2 at gmail.com  Mon May 11 08:51:35 2015
From: ndbecker2 at gmail.com (Neal Becker)
Date: Mon, 11 May 2015 08:51:35 -0400
Subject: why would this print 'True'?
Message-ID: <miq8ko$q6a$1@ger.gmane.org>

from itertools import ifilter

if all (hasattr (b, 'test') for b in ifilter (lambda b: b < 10, [1,2,3,4])):
    print 'True'

same result using filter instead of ifilter.

hasattr (b, 'test') where b is 1, 2, 3... should all be False.  So why does 
this print True?

-- 
Those who fail to understand recursion are doomed to repeat it



From ndbecker2 at gmail.com  Mon May 11 08:58:43 2015
From: ndbecker2 at gmail.com (Neal Becker)
Date: Mon, 11 May 2015 08:58:43 -0400
Subject: why would this print 'True'?
References: <miq8ko$q6a$1@ger.gmane.org>
Message-ID: <miq923$343$1@ger.gmane.org>

Nevermind - I found the answer.  I was trying this in ipython with pylab:

http://stackoverflow.com/questions/7491951/python-builtin-all-with-generators

Neal Becker wrote:

> from itertools import ifilter
> 
> if all (hasattr (b, 'test') for b in ifilter (lambda b: b < 10,
> [1,2,3,4])):
>     print 'True'
> 
> same result using filter instead of ifilter.
> 
> hasattr (b, 'test') where b is 1, 2, 3... should all be False.  So why
> does this print True?
> 
-- 
Those who fail to understand recursion are doomed to repeat it



From davea at davea.name  Mon May 11 09:21:38 2015
From: davea at davea.name (Dave Angel)
Date: Mon, 11 May 2015 09:21:38 -0400
Subject: Feature Request: Reposition Execution
In-Reply-To: <5550a223$0$13013$c3e8da3$5496439d@news.astraweb.com>
References: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>
 <mailman.354.1431345441.12865.python-list@python.org>
 <5550a223$0$13013$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5550ACE2.3000608@davea.name>

On 05/11/2015 08:35 AM, Steven D'Aprano wrote:
> On Mon, 11 May 2015 09:57 pm, Dave Angel wrote:
>
>> On 05/11/2015 07:46 AM, Skybuck Flying wrote:
>>> Hello,
>>>
>>> Sometimes it can be handy to "interrupt/reset/reposition" a running
>>> script.
>>>
>>> For example something externally goes badly wrong.
>>>
>>
>> os.kill()
>>
>> then in your process, handle the exception, and do whatever you think is
>> worthwhile.
>
>
> Are you suggesting that the app sends itself a signal?
>
> Is that even allowed?
>

No idea if it's allowed.  I didn't notice his sample was multithreaded, 
as i grabbed on the "externally goes badly wrong".



-- 
DaveA


From antoon.pardon at rece.vub.ac.be  Mon May 11 09:27:50 2015
From: antoon.pardon at rece.vub.ac.be (Antoon Pardon)
Date: Mon, 11 May 2015 15:27:50 +0200
Subject: anomaly
In-Reply-To: <87fv73iavz.fsf@elektro.pacujo.net>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <87fv73iavz.fsf@elektro.pacujo.net>
Message-ID: <5550AE56.2070405@rece.vub.ac.be>

Op 11-05-15 om 13:58 schreef Marko Rauhamaa:
> Antoon Pardon <antoon.pardon at rece.vub.ac.be>:
>
>> Which is exactly the point! They were turned into keywords because the
>> developers didn't want to allow them being overridden. There is no a
>> priori reason why we should turn "True" into a keyword and allow "int"
>> in the builtins.
>>
>> We are only allowed to be adults, for as far as the developers let us.
>> They allow us to be adults with regards to "int" but they don't allow
>> us to be adults with regards to "True".
>>
>> Defending "int" being overridable by declating "We're all adults" is
>> being selective.
> I'm still failing to see the point. What problem are you having a
> difficulty solving?

The point is that all too often someone wants to defend a specific choice
the developers have made and cites some general rule or principle in support,
ignoring the fact that python breaks that rule/principle in other area's.

"We are all adults, we give you the freedom to break things or write
confusing code" and variantions is such a rule/principle, because often
enough changes in the language are introduced to make it easier to eliminate
bugs or are refused because they would be too bug prone.

-- 
Antoon Pardon



From mwilson at the-wire.com  Mon May 11 09:37:48 2015
From: mwilson at the-wire.com (Mel Wilson)
Date: Mon, 11 May 2015 13:37:48 +0000 (UTC)
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <mailman.345.1431329316.12865.python-list@python.org>
Message-ID: <miqbb8$o7l$1@dont-email.me>

On Sun, 10 May 2015 14:12:44 -0500, boB Stepp wrote:

> I have to admit being surprised by this, too. I am just now studying on
> how to write my own classes in Python, and have come to realize that
> doing this is *possible*, but the *surprise* to me is why the language
> design allowed this to actually be done.

Read Cory Doctorow lately on the War Against General Purpose Computing, 
where a bunch of people who don't really understand are trying to make it 
impossible for any computer to do something that is The Wrong Thing.  Or 
gently approach the relevant computing theory through Doug Hofstadter's 
_Goedel, Escher, Bach_, or some essays in _Metamagical Themas_.  
Generally speaking, making a computer totally incapable of doing Wrong 
Things leaves it incapable of doing anything at all.

	Mel.


From antoon.pardon at rece.vub.ac.be  Mon May 11 09:38:23 2015
From: antoon.pardon at rece.vub.ac.be (Antoon Pardon)
Date: Mon, 11 May 2015 15:38:23 +0200
Subject: anomaly
In-Reply-To: <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5550B0CF.208@rece.vub.ac.be>

Op 11-05-15 om 14:34 schreef Steven D'Aprano:

> On Mon, 11 May 2015 09:39 pm, Antoon Pardon wrote:
>
>> There is no
>> a priori reason why we should turn "True" into a keyword and allow
>> "int" in the builtins.
> Why should there be an *a priori* reason?

I don't say there should be an *a priori* reason, but one implies or
at least strongly suggests an *a priori* reason when one comes with
such general rule as /We are all adults, so we allow this kind of thing./

"We allow buitins to be overridden", doesn't sound as a very accurate
description of the underlining reason, when you know that things have
been removed from builtins and made a keyword in order to prevent them
from being overridden.



From marko at pacujo.net  Mon May 11 10:03:27 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 11 May 2015 17:03:27 +0300
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <87fv73iavz.fsf@elektro.pacujo.net>
 <mailman.362.1431350872.12865.python-list@python.org>
Message-ID: <87617zi53k.fsf@elektro.pacujo.net>

Antoon Pardon <antoon.pardon at rece.vub.ac.be>:

> The point is that all too often someone wants to defend a specific
> choice the developers have made and cites some general rule or
> principle in support, ignoring the fact that python breaks that
> rule/principle in other area's.

Granted, but you have set the trap for them by demanding a justification
when no justification was required. Every language has their cute
idiosyncrasies and arbitrary design choices.

Still, I have seen some glitches in the matrix as well:

   >>> setattr(3, "__str__", lambda: "hello")
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   AttributeError: 'int' object attribute '__str__' is read-only
   >>> setattr(3, "hello", lambda: "hello")
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   AttributeError: 'int' object has no attribute 'hello'


Bottom line, though, is that these corner cases in no way prevent you
from accomplishing your programming objective using Python.


Marko


From skip.montanaro at gmail.com  Mon May 11 10:04:05 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 11 May 2015 09:04:05 -0500
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <281c0f90-8353-4480-93d3-9c656505cc61@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <55502919$0$12997$c3e8da3$5496439d@news.astraweb.com>
 <281c0f90-8353-4480-93d3-9c656505cc61@googlegroups.com>
Message-ID: <CANc-5Uz1-RTS0GBTm0JKH2BxND731WaidHc44FPjJCbCVNgVFg@mail.gmail.com>

Don't CS departments still have a computer languages survey class? When I
was a graduate student at Iowa in the early 80s, we had one. (It was, as I
recall, an upper level undergrad course. I didn't get into CS until
graduate school, so went back to filled in some missing stuff.) I don't
recall all the languages we touched on, but ISTR there were five or six. I
know we hit Lisp (today, it would likely be Scheme), and probably APL
(today it would probably be Python+Pandas, MATLAB, R, or something similar).

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150511/2d2a5798/attachment.html>

From skip.montanaro at gmail.com  Mon May 11 10:12:10 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 11 May 2015 09:12:10 -0500
Subject: Calling a function is faster than not calling it?
In-Reply-To: <8Y_3x.668959$LI3.568865@fx24.am4>
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <fNP3x.641427$3i4.583649@fx29.am4>
 <555028d8$0$12997$c3e8da3$5496439d@news.astraweb.com>
 <8Y_3x.668959$LI3.568865@fx24.am4>
Message-ID: <CANc-5Uyz+DA5xmiEVcCe8R82ebgvDLpMYcYTNFs6+ffk8aDU5w@mail.gmail.com>

On Mon, May 11, 2015 at 4:50 AM, BartC <bc at freeuk.com> wrote:

> You just seem surprised that using eval() to do this is slower than a
> direct call.


Well, it is surprising. Most uses of eval() are to evaluate Python
expressions in string form. That I expect to be quite slow, given the
parsing+byte compilation overhead. I wouldn't expect eval()ing a code
object to be all that different than calling the function containing the
code object.

My guess (without looking in ceval.c) is that the code path through the VM
has been tweaked heavily over the years to try and speed it up. Except for
places where that path overlaps with the path for eval(code_object), I
doubt any attention has been paid to speeding up eval. I could certainly be
way off-base here. Perhaps the path taken through the interpreter for
eval(code_object) is more-or-less a subset of the path taken for a simple
function call.

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150511/7a6c961a/attachment.html>

From rosuav at gmail.com  Mon May 11 10:13:00 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 12 May 2015 00:13:00 +1000
Subject: anomaly
In-Reply-To: <5550B0CF.208@rece.vub.ac.be>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
Message-ID: <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>

On Mon, May 11, 2015 at 11:38 PM, Antoon Pardon
<antoon.pardon at rece.vub.ac.be> wrote:
> "We allow buitins to be overridden", doesn't sound as a very accurate
> description of the underlining reason, when you know that things have
> been removed from builtins and made a keyword in order to prevent them
> from being overridden.

There are principles, and then there are specific instances that go
against those principles. The overarching principle has its
justification; the violations have to have their own justifications.
As Steven said, there are no a priori reasons for most things - or to
put it another way, there are very few design decisions that come down
to a fundamental "this is right, this is wrong" - but there can be
strong and weak justifications for things.

Why does Python have most built-ins as simply looked-up names that can
be overridden? Because otherwise, there would be a veritable ton of
keywords:

>>> dir(builtins)
['ArithmeticError', 'AssertionError', 'AttributeError',
'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError',
'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError',
'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError',
'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError',
'Exception', 'False', 'FileExistsError', 'FileNotFoundError',
'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError',
'ImportError', 'ImportWarning', 'IndentationError', 'IndexError',
'InterruptedError', 'IsADirectoryError', 'KeyError',
'KeyboardInterrupt', 'LookupError', 'MemoryError', 'NameError',
'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError',
'OSError', 'OverflowError', 'PendingDeprecationWarning',
'PermissionError', 'ProcessLookupError', 'ReferenceError',
'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopIteration',
'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit',
'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError',
'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError',
'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning',
'ValueError', 'Warning', 'ZeroDivisionError', '__build_class__',
'__debug__', '__doc__', '__import__', '__loader__', '__name__',
'__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin',
'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod',
'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict',
'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter',
'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr',
'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance',
'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map',
'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord',
'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed',
'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str',
'sum', 'super', 'tuple', 'type', 'vars', 'zip']

in addition to these, which _are_ keywords:

>>> keyword.kwlist
['False', 'None', 'True', 'and', 'as', 'assert', 'break', 'class',
'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for',
'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal',
'not', 'or', 'pass', 'raise', 'return', 'try', 'while', 'with',
'yield']

Python 2 had 'print' as a keyword, and it was specifically turned into
a non-keyword in Python 3 to allow it to be overridden. It could have
been turned into a function while still being a keyword, but it
wasn't. Conversely, True and False became keywords, because there's no
practical reason to override them. [1] You may well want to shadow
'copyright' with your own program's copyright notice, given that the
built-in name is primarily there for interactive use. You might use
'input' to store the incoming text in a non-interactive program, or
'quit' in an interactive one to store a flag that becomes True when
the user wants to terminate. Very frequently, 'id' is used as a
database ID. None of these shadowings is a problem to the language;
chances are none of them will ever be a problem to your code either.
Having most of the built-in names *not* be keywords means that adding
new built-ins doesn't break code; code that used the name for some
other meaning will still work, but won't be able to use the new
feature. That's a good thing.

ChrisA

[1] http://thedailywtf.com/articles/What_Is_Truth_0x3f_ notwithstanding.


From dreamingforward at gmail.com  Mon May 11 10:23:06 2015
From: dreamingforward at gmail.com (zipher)
Date: Mon, 11 May 2015 07:23:06 -0700 (PDT)
Subject: anomaly
In-Reply-To: <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>

On Monday, May 11, 2015 at 1:11:26 AM UTC-5, Steven D'Aprano wrote:
> On Monday 11 May 2015 10:57, zipher wrote: 
> > I guess everyone expects this behavior since Python implemented this idea
> > of "everything is an object", but I think this branch of OOP (on the
> > branch of the Tree of Programming Languages) has to be chopped off.  The
> > idea of everything is an object is backwards (unless your in a LISP
> > machine).  Like I say, it's trying to be too pure and not practical.
> 
> Python is in production use in hundreds of thousands of organisations. It 
> has been heavily used for over twenty years, in everything from quick and 
> dirty one line scripts to hundred-thousand LOC applications. 

Yeah, so was COBOL.  Boom.

Mark


From invalid at invalid.invalid  Mon May 11 10:27:54 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Mon, 11 May 2015 14:27:54 +0000 (UTC)
Subject: Feature Request: Reposition Execution
References: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>
 <mailman.354.1431345441.12865.python-list@python.org>
 <5550a223$0$13013$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <miqe9a$r2r$1@reader1.panix.com>

On 2015-05-11, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
> On Mon, 11 May 2015 09:57 pm, Dave Angel wrote:
>
>> On 05/11/2015 07:46 AM, Skybuck Flying wrote:
>>> Hello,
>>>
>>> Sometimes it can be handy to "interrupt/reset/reposition" a running
>>> script.
>>>
>>> For example something externally goes badly wrong.
>>>
>> 
>> os.kill()
>> 
>> then in your process, handle the exception, and do whatever you think is
>> worthwhile.
>
>
> Are you suggesting that the app sends itself a signal?
>
> Is that even allowed?

Of course (at least on Unix/Linux/Posix systems).

And there's even a special case defined to make sending signals to
yourself easy: you just send them to PID 0.

>From "man 2 kill" on Linux:

DESCRIPTION

       The kill() system call can be used to send any signal to any
       process group or process.

       [...]

       If pid equals 0, then sig is sent to every process in the
       process group of the calling process.

And just to make sure I ran a little test, and it works exactly as
advertised:

---------------------------------testit.py--------------------------------
#!/usr/bin/python
import os, sys, time, threading, signal

def thread1():
    while True:
        sys.stdout.write("Hello %s\n" % time.time())
        time.sleep(1)
threading.Thread(target=thread1).start()
time.sleep(2)
os.kill(0,signal.SIGKILL)
---------------------------------------------------------------------------

$ ./testit.py
Hello 1431354383.19
Hello 1431354384.19
Killed
$ 


-- 
Grant Edwards               grant.b.edwards        Yow! Hello.  Just walk
                                  at               along and try NOT to think
                              gmail.com            about your INTESTINES being
                                                   almost FORTY YARDS LONG!!


From invalid at invalid.invalid  Mon May 11 10:35:45 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Mon, 11 May 2015 14:35:45 +0000 (UTC)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <55502919$0$12997$c3e8da3$5496439d@news.astraweb.com>
 <281c0f90-8353-4480-93d3-9c656505cc61@googlegroups.com>
 <mailman.364.1431353048.12865.python-list@python.org>
Message-ID: <miqeo1$r2r$2@reader1.panix.com>

On 2015-05-11, Skip Montanaro <skip.montanaro at gmail.com> wrote:

> Don't CS departments still have a computer languages survey class? When I
> was a graduate student at Iowa in the early 80s, we had one. (It was, as I
> recall, an upper level undergrad course. I didn't get into CS until
> graduate school, so went back to filled in some missing stuff.) I don't
> recall all the languages we touched on, but ISTR there were five or six. I
> know we hit Lisp (today, it would likely be Scheme), and probably APL
> (today it would probably be Python+Pandas, MATLAB, R, or something similar).

There was a similar class at both Iowa State and University of MN. You
learned a half-dozen languages in a single quarter.  IIRC, at ISU we
did Lisp, Prolog, APL, Snobol and a couple others.  The main
pedagogical language at the time was Pascal, but we also learned
FORTRAN, COBOL, BASIC, and a couple three assembly languages (PDP-11,
VAX, Z80, 6502).  If you were a computer enineering major instead of
computer science, you also leared a hardware description language.  At
the time it was AHPL.

More recent gruaduates only seem to know one language (Java or C++)
and are completely baffled by anything else.

And don't get me started on that damned noise they call music...

-- 
Grant Edwards               grant.b.edwards        Yow! !  Now I understand
                                  at               advanced MICROBIOLOGY and
                              gmail.com            th' new TAX REFORM laws!!


From skip.montanaro at gmail.com  Mon May 11 10:45:30 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 11 May 2015 09:45:30 -0500
Subject: anomaly
In-Reply-To: <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
Message-ID: <CANc-5Uzb_bZUFkrNC3azDzfAqzB7mJQss+i4FGZQjSXX=4vYcA@mail.gmail.com>

Steven> Python is in production use in hundreds of thousands of
organisations. It
Steven> has been heavily used for over twenty years, in everything
from quick and
Steven> dirty one line scripts to hundred-thousand LOC applications.

Mark> Yeah, so was COBOL.  Boom.

Your point being?

The software development landscape has changed so much since the days
when COBOL was king of data processing that its popularity then had
less to do with some sort of software natural selection than IBM's
utter domination of the computing landscape in the 1960s. If you
bought IBM's hardware (as almost everyone did *), you also got what
they offered in the way of software. I believe (though this is
certainly before my time in the industry) that basically meant FORTRAN
or COBOL (maybe APL? Wow, I've mentioned it twice in one day). In
contrast, software developers and project managers had plenty of
options available when they chose Python.

Skip

* Searching Google for "If it's blue, buy two" doesn't turn up any
obvious hits for this old aphorism (roughly meaning, "If you need to
buy computers, buying IBM is a safe bet", or more to the professional
bottom line of the person making the decision, "Nobody's going to fire
you if you buy an IBM mainframe and it turns out to be the wrong
choice"). Am I the only person who remembers it?


From tjreedy at udel.edu  Mon May 11 10:53:12 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 11 May 2015 10:53:12 -0400
Subject: anomaly
In-Reply-To: <CACac1F_EDsHyDFJR-H0MnBgqpmmfX-441B7c_nS6xk3h976-cw@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CACac1F_EDsHyDFJR-H0MnBgqpmmfX-441B7c_nS6xk3h976-cw@mail.gmail.com>
Message-ID: <miqfou$316$1@ger.gmane.org>

Further posts on this thread should delete pydev-list or 
gmane.comp.python.devel.  It was a mistake by the troll to ever post 
this there.

-- 
Terry Jan Reedy



From dreamingforward at gmail.com  Mon May 11 10:56:56 2015
From: dreamingforward at gmail.com (zipher)
Date: Mon, 11 May 2015 07:56:56 -0700 (PDT)
Subject: anomaly
In-Reply-To: <87617zi53k.fsf@elektro.pacujo.net>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <87fv73iavz.fsf@elektro.pacujo.net>
 <mailman.362.1431350872.12865.python-list@python.org>
 <87617zi53k.fsf@elektro.pacujo.net>
Message-ID: <052875ff-7879-4b2f-a00f-08e980e8c217@googlegroups.com>

On Monday, May 11, 2015 at 9:03:43 AM UTC-5, Marko Rauhamaa wrote:
> Antoon Pardon <antoon.pardon at rece.vub.ac.be>:
> 
> > The point is that all too often someone wants to defend a specific
> > choice the developers have made and cites some general rule or
> > principle in support, ignoring the fact that python breaks that
> > rule/principle in other area's.
> 
> Granted, but you have set the trap for them by demanding a justification
> when no justification was required. Every language has their cute
> idiosyncrasies and arbitrary design choices.

No.  Here's where I must disagree.  I think one can infer a goal for particular programming languages, even if it is subconscious.  For example, with LISP it could be "generality".  For C, it could be "staying as close to the machine as possible while maximizing the use to humans" -- contradiction that works because they've limited their architecture to VonNeumann (stackless) machines.

I think the subconscious goal of OOP languages is to create a data ecosystem, starting with a unified data model under the realization that ultimately:  all data relates to other data -- that my database of wind speed and direction from 2012 is relatable, by some finite number of hops, to your data on population growth in Chicago.  Call it the "seven degrees of data" and remember the exabytes of data out there.

Python is creating the perfect system for that because it has an interpreter environment with which to manipulate objects that could be retrieved on the net and sent back out.  It has docstrings so that your foreign object can self-document, and doctests, so that I can be confident that your code works as *I* expect.

There are reasons to have limits on programming freedom.  It puts order to chaos.  It guides the wily programmers into a particular train of thought.  You don't override "True" because you'd be breaking one of the [explicit] goals of the language:  readability.  If there were no constraints, life itself could not exist.

I don't think shadowing built-in types was a design choice but simply never got exercised because most people are used to handling such things, *subconsciously*, like C.

To Mr. Gatti, my point was not an insult, it is a theoretical postulate in the domain of Computer Science.  One that has not really been studied.  OOP is still far from it's goal, so the field is still answering questions within it.

Mark


From bc at freeuk.com  Mon May 11 11:01:49 2015
From: bc at freeuk.com (BartC)
Date: Mon, 11 May 2015 16:01:49 +0100
Subject: Calling a function is faster than not calling it?
In-Reply-To: <mailman.365.1431353538.12865.python-list@python.org>
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <fNP3x.641427$3i4.583649@fx29.am4>
 <555028d8$0$12997$c3e8da3$5496439d@news.astraweb.com>
 <8Y_3x.668959$LI3.568865@fx24.am4>
 <mailman.365.1431353538.12865.python-list@python.org>
Message-ID: <zv34x.521800$lV1.472319@fx05.am4>

On 11/05/2015 15:12, Skip Montanaro wrote:
>
> On Mon, May 11, 2015 at 4:50 AM, BartC <bc at freeuk.com
> <mailto:bc at freeuk.com>> wrote:
>
>     You just seem surprised that using eval() to do this is slower than
>     a direct call.
>
>
> Well, it is surprising. Most uses of eval() are to evaluate Python
> expressions in string form. That I expect to be quite slow, given the
> parsing+byte compilation overhead. I wouldn't expect eval()ing a code
> object to be all that different than calling the function containing the
> code object.

func() (when the call is itself inside a function) has this bytecode:

  11           0 LOAD_GLOBAL              0 (func)
               3 CALL_FUNCTION            0
               6 POP_TOP
               7 LOAD_CONST               0 (None)
              10 RETURN_VALUE


eval(code) (when this is also inside a function, and code is local) has 
this bytecode:

  16           6 LOAD_GLOBAL              0 (eval)
               9 LOAD_FAST                0 (code)
              12 CALL_FUNCTION            1
              15 POP_TOP
              16 LOAD_CONST               0 (None)
              19 RETURN_VALUE


So eval() seems to be just as much a function call as func() is. Except:

(1) It has an extra argument ('code'), in addition to any normal 
arguments of func (0 in this case)

(2) Before it can do whatever func() does, in has to /get there/ first. 
And that can code presumably has a similarly indirect path to return to 
the point to where eval() was called. So there is extra code inside eval 
to deal with this (as well as check whether a string has been passed). 
That code must be the extra overhead.

So it seems clear to me that the eval method has to do everything that 
calling func() has to do, and then some.

(That doesn't necessarily mean it would be slower; whatever goes on 
inside could well have been made less efficient for a direct call than 
for an indirect one via eval. But the results suggest it is slower using 
eval. And it isn't really surprising.)

-- 
Bartc


From dreamingforward at gmail.com  Mon May 11 11:11:05 2015
From: dreamingforward at gmail.com (zipher)
Date: Mon, 11 May 2015 08:11:05 -0700 (PDT)
Subject: anomaly
In-Reply-To: <mailman.367.1431355924.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
 <mailman.367.1431355924.12865.python-list@python.org>
Message-ID: <b05b09d6-9585-41d7-b573-b3c331681d89@googlegroups.com>

On Monday, May 11, 2015 at 9:52:16 AM UTC-5, Skip Montanaro wrote:
> Steven> Python is in production use in hundreds of thousands of
> organisations. It
> Steven> has been heavily used for over twenty years, in everything
> from quick and
> Steven> dirty one line scripts to hundred-thousand LOC applications.
> 
> Mark> Yeah, so was COBOL.  Boom.
> 
> Your point being?

That a billion lines of code doesn't equate to practicality. 

> The software development landscape has changed so much since the days
> when COBOL was king of data processing that its popularity then had
> less to do with some sort of software natural selection than IBM's
> utter domination of the computing landscape in the 1960s. 

Yes, and I would argue that the OOP landscape has changed since the 1990s (with the rise of very high level languages like Python).  The idea of "everything is an object" is an exploration of PL design that I'm claiming is not appropriate for whatever language will create this data ecosystem that I say is as inevitable as a single language being used on the Internet -- it's simply a chaotic attractor that is irresistible for it's sheer utility.

I also bought the idea of everything as an object, it has a unbeatable purity to it.  But we won't ever get to the point were OOP is like the purity of math because the greatest utility of OOP is working with real-world data.  And that real-world puts bounds on the otherwise abstract purity in which a language is theoretically capable.

Mark

P.S. As Mr. Reedy said, please remove pydev from the To: line.  This is the proper general rule when a discussion goes into abstractions, rather than practicality of development.


From nagarajuusstaffing at gmail.com  Mon May 11 11:12:29 2015
From: nagarajuusstaffing at gmail.com (nagaraju thoudoju)
Date: Mon, 11 May 2015 08:12:29 -0700 (PDT)
Subject: Immediate Hire: Sr. Java Software Engineer - Oklahoma - GC, US
 Citizens Onlt
Message-ID: <b4576228-1865-4ff5-b8d8-c7caa177c0c6@googlegroups.com>

Hi,
 
Hope you are doing well,
 
Please find the requirement below and let me know you interest on this position on  nagaraju at intsystech.com or feel free to call me on my
D:908-333-3534. 
 
Requirement:

Job Description - Sr. Java Software Engineer
Location: Edmond, Oklahoma
 
Responsibilities:
Provide applications development and enhancement functions; including analysis, design, coding, testing, and deployment of Java web applications and web services.
Work with business analysts and technical architects to formalize specifications for custom development.
Proactively work to solve issues while including the key functional and/or business stakeholders.
Provide analysis and prototyping of new custom solutions, or new requirements for existing custom solutions.
With limited supervision, develop and enhance custom solutions using approved technologies and platforms.
Document and perform unit, integration, and regression testing in support of new implementations and releases.
Skills:
Basic Qualifications
7+ years of object oriented application development experience using Java
Solid experience with web application development
Practical knowledge with web service and API development
Expertise in application servers including WebSphere, JBoss and Tomcat
Proficiency with the Spring Framework including Spring MVC and Spring Data
Strong understanding of object oriented concepts, web application and web service design, open source tools, and web application frameworks
Professional experience designing, developing and deploying software solutions
Nice to Have:
Financial Services Systems experience
Security Experience
Drools and jBPM knowledge
Understanding of LifeRay
Other attributes:
Attention to detail
Focus on quality of both code and user experience
Thrive in an entrepreneurial environment
Excellent oral and written communication skills
Strong analytical and problem solving skills
Inspire a culture of innovation and teamwork
Thanks & Best Regards....?

Raju

International Systems Technologies Inc.

10 Corporate place south.| Suite 203  |

Piscataway, NJ 08854  |

D:908-333-3540 |Fax:732-348-9533|

E-mail: nagaraju at intsystech.com   |

G-talk: nagarajuusstaffing at gmail.com

URL: www.intsystech.com


From skip.montanaro at gmail.com  Mon May 11 11:13:39 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 11 May 2015 10:13:39 -0500
Subject: Calling a function is faster than not calling it?
In-Reply-To: <zv34x.521800$lV1.472319@fx05.am4>
References: <554f2bb6$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <fNP3x.641427$3i4.583649@fx29.am4>
 <555028d8$0$12997$c3e8da3$5496439d@news.astraweb.com>
 <8Y_3x.668959$LI3.568865@fx24.am4>
 <mailman.365.1431353538.12865.python-list@python.org>
 <zv34x.521800$lV1.472319@fx05.am4>
Message-ID: <CANc-5UwN8Uzc2+=dZNXPpP+P5Jm4ssm_QPNM+6ZYynFt=doYBw@mail.gmail.com>

On Mon, May 11, 2015 at 10:01 AM, BartC <bc at freeuk.com> wrote:
> (1) It has an extra argument ('code'), in addition to any normal arguments
> of func (0 in this case)

Which might well push execution down the unoptimized code path. Also,
ISTR that Steven's original timeit runs tacked on a standalone "eval ;
..." to the function call case, so the LOAD_GLOBAL bytecode would have
been done in the function call case as well. He was, I think, doing
what he could to make the two cases as nearly identical as he could.

Skip


From dreamingforward at gmail.com  Mon May 11 11:22:02 2015
From: dreamingforward at gmail.com (zipher)
Date: Mon, 11 May 2015 08:22:02 -0700 (PDT)
Subject: code blocks
In-Reply-To: <mailman.340.1431315116.12865.python-list@python.org>
References: <mailman.49.1430633046.12865.python-list@python.org>
 <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>
 <mailman.340.1431315116.12865.python-list@python.org>
Message-ID: <ca2e1795-0eec-4584-ae38-6f6bd7f37585@googlegroups.com>

On Sunday, May 10, 2015 at 10:32:07 PM UTC-5, Ian wrote:
> On Sun, May 10, 2015 at 7:39 PM, zipher <dreamingforward at gmail.com> wrote:
> > Similarly, you'd want:
> >
> >>>> encode(codestr)
> >
> > to instantiate all objects in the codestr.  You can't do this with eval, because it doesn't allow assignment (eval(n=2) returns "InvalidSyntax").
> 
> Is exec what you're looking for?
> 
> >>> exec('n = 2')
> >>> print(n)
> 2

Ah, yeah, I guess that does it.  But (shame) it looks like you've gone past the BDFL.  Try:

>>> help(exec)
            ^
SyntaxError: invalid syntax 

Better

Mark


From tommyc168168 at gmail.com  Mon May 11 11:22:05 2015
From: tommyc168168 at gmail.com (Tommy C)
Date: Mon, 11 May 2015 08:22:05 -0700 (PDT)
Subject: How to properly apply OOP in the bouncing ball code
In-Reply-To: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>
References: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>
Message-ID: <e841eeeb-7a36-47fe-bd21-bfd7b53a9588@googlegroups.com>

Thanks for your help.

I have updated the code as follows, there are no more errors but the images will not move at all, as all the images are staying at the upper left corner. Please advice, thanks.


import sys, pygame

pygame.init()

size   = width,  height = 800, 600
black  = [0,0,0]
screen = pygame.display.set_mode(size)

class BALL:
    def __init__(self,image):
        self.ball     = pygame.image.load(image)
        self.ballrect = self.ball.get_rect()
        self.speed    = [2, 2]

    def control(self):
        ballmove = self.ballrect.move(self.speed)

        if ballmove.left < 0 or ballmove.right > width:
            self.speed[0] = -self.speed[0]

        if ballmove.top < 0  or ballmove.bottom > height:
            self.speed[1] = -self.speed[1]

    def settings(self):
        clock  = pygame.time.Clock()
        screen.fill(black)
        screen.blit(self.ball, self.ballrect)
        pygame.display.flip()
        clock.tick(60)
        while 1:
            for event in pygame.event.get():
                if event.type == pygame.QUIT: sys.exit()


bob = BALL("spongebob.png")
bob.control()
bob.settings()

patrick = BALL("patrick.jpg")
patrick.speed[0] = 5
patrick.speed[1] = 8
patrick.control()
patrick.settings()

jaws = BALL("jaws.jpg")
jaws.speed[0] = 1
jaws.speed[1] = 10
jaws.control()
jaws.settings()


On Friday, May 8, 2015 at 11:40:46 AM UTC-4, Tommy C wrote:
> I'm trying to apply OOP in this bouncing ball code in order to have multiple balls bouncing around the screen. The objective of this code is to create a method called settings, which controls all the settings for the screen and the bouncing behaviour of multiple balls, under the class Ball. However, I keep on getting errors related to attributes (e.g., speed). I'm new to OOP in Python so your help will be much appreciated. Thanks in advance.
> 
> 
> import pygame
> import random
> import sys
> 
> pygame.init()
> 
> class Ball:
>     def __init__(self, X, Y):
>         self.velocity = [1,1]
>         self.ball_image = pygame.image.load ("ball.jpg")
>         self.ball_boundary = self.ball_image.get_rect ()
>         self.black = [0,0,0]
>         self.width = 800
>         self.height = 600
>         self.num = 8
>         self.X = random.randint(0, self.width)
>         self.Y = random.randint(0, self.height)
> 
>     def settings(self):
>         #X = random.randint(0, self.width)
>         #Y = random.randint(0, self.height)
>         clock = pygame.time.Clock()
>         size = self.width, self.height
>         screen = pygame.display.set_mode(size)
>         ball_boundary = self.ball_image.get_rect()
>         speed = self.velocity
>         pic = self.ball_image
>         pygame.display.set_caption("Balls")
>         num_balls = self.num
>         ball_list = []
> 
>         for i in range(num_balls):
>            ball_list.append( Ball(random.randint(10, self.width-10),random.randint(10, self.height-10)) )
> 
>         while 1:
>             for event in pygame.event.get():
>                 if event.type == pygame.QUIT:
>                     sys.exit(0)
> 
>             screen.fill(self.black)
>             for balls in ball_list:
>                 if balls.ball_boundary.left < 0 or balls.ball_boundary.right > self.width:
>                     balls.speed[0] = -balls.speed[0]
>                 if balls.ball_boundary.top < 0 or balls.ball_boundary.bottom > self.height:
>                     balls.speed[1] = -balls.speed[1]
>                 balls.ball_boundary = balls.ball_boundary.move (self.velocity)
>                 screen.blit (balls.ball_image, balls.ball_boundary)
>             pygame.display.flip()
> 
> play = Ball(random.randint(0, 800), random.randint(0, 600))
> 
> play.settings()
> 
> 
> 
> 
> Message	File Name	Line	Position	
> Traceback				
>     <module>	C:\....\Multiple_balls_TC.py	63		
>     settings	C:\....\Multiple_balls_TC.py	56		
> AttributeError: Ball instance has no attribute 'speed'



From dreamingforward at gmail.com  Mon May 11 11:23:44 2015
From: dreamingforward at gmail.com (zipher)
Date: Mon, 11 May 2015 08:23:44 -0700 (PDT)
Subject: code blocks
In-Reply-To: <ca2e1795-0eec-4584-ae38-6f6bd7f37585@googlegroups.com>
References: <mailman.49.1430633046.12865.python-list@python.org>
 <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>
 <mailman.340.1431315116.12865.python-list@python.org>
 <ca2e1795-0eec-4584-ae38-6f6bd7f37585@googlegroups.com>
Message-ID: <1e507cbd-913e-4feb-aa3d-3a1ef676b17a@googlegroups.com>

On Monday, May 11, 2015 at 10:22:15 AM UTC-5, zipher wrote:
> Ah, yeah, I guess that does it.  But (shame) it looks like you've gone past the BDFL.  Try:
> [...]
> Better

Oops, omit word "better".  Sent before reading over it again...

m


From benjamin at python.org  Mon May 11 11:24:12 2015
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 11 May 2015 11:24:12 -0400
Subject: [RELEASE] Python 2.7.10 release candidate 1
Message-ID: <1431357852.505333.265716865.5DD10081@webmail.messagingengine.com>

It is my privilege to announce the first release candidate of 2.7.10,
the next bugfix release in the 2.7 series.

Downloads are at

   https://www.python.org/downloads/release/python-2710rc1/

The full changelog is at

   https://hg.python.org/cpython/raw-file/80ccce248ba2/Misc/NEWS

Please consider testing 2.7.10rc1 with your application and reporting
bugs to

   https://bugs.python.org

Regards,
Benjamin


From dreamingforward at gmail.com  Mon May 11 11:33:56 2015
From: dreamingforward at gmail.com (zipher)
Date: Mon, 11 May 2015 08:33:56 -0700 (PDT)
Subject: How to properly apply OOP in the bouncing ball code
In-Reply-To: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>
References: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>
Message-ID: <45313faa-d256-4fa4-9e37-4b8dd85e1681@googlegroups.com>

On Friday, May 8, 2015 at 10:40:46 AM UTC-5, Tommy C wrote:
> I'm trying to apply OOP in this bouncing ball code in order to have multiple balls bouncing around the screen. The objective of this code is to create a method called settings, which controls all the settings for the screen and the bouncing behaviour of multiple balls, under the class Ball. However, I keep on getting errors related to attributes (e.g., speed). I'm new to OOP in Python so your help will be much appreciated. Thanks in advance.

You are making a error that few in the programming community have caught up to.  OOP design for *data abstraction* is a completely different beast that OOP for *simulation*.  The confusion is around the use of the word "object" which both denotes a physical world item and a virtual one detached from reality.

I would say that Python might not be the right fit, but instead a language dedicated to simulation.

Mark


From invalid at invalid.invalid  Mon May 11 11:34:12 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Mon, 11 May 2015 15:34:12 +0000 (UTC)
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
 <mailman.367.1431355924.12865.python-list@python.org>
Message-ID: <miqi5k$id1$1@reader1.panix.com>

On 2015-05-11, Skip Montanaro <skip.montanaro at gmail.com> wrote:

>> Python is in production use in hundreds of thousands of
>> organisations. It has been heavily used for over twenty years, in
>> everything from quick and dirty one line scripts to hundred-thousand
>> LOC applications.
>
> Mark> Yeah, so was COBOL.  Boom.
>
> Your point being?

That Python, like COBOL, is an eminently practical language.

-- 
Grant Edwards               grant.b.edwards        Yow! YOU PICKED KARL
                                  at               MALDEN'S NOSE!!
                              gmail.com            


From dreamingforward at gmail.com  Mon May 11 11:39:33 2015
From: dreamingforward at gmail.com (zipher)
Date: Mon, 11 May 2015 08:39:33 -0700 (PDT)
Subject: anomaly
In-Reply-To: <miqi5k$id1$1@reader1.panix.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
 <mailman.367.1431355924.12865.python-list@python.org>
 <miqi5k$id1$1@reader1.panix.com>
Message-ID: <03e4e251-20b0-4a2f-968e-2d40d2b5dc1e@googlegroups.com>

On Monday, May 11, 2015 at 10:34:24 AM UTC-5, Grant Edwards wrote:
> On 2015-05-11, Skip Montanaro <skip.montanaro at gmail.com> wrote:
> >> Python is in production use in hundreds of thousands of
> >> organisations. It has been heavily used for over twenty years, in
> >> everything from quick and dirty one line scripts to hundred-thousand
> >> LOC applications.
> >
> > Mark> Yeah, so was COBOL.  Boom.
> >
> > Your point being?
> 
> That Python, like COBOL, is an eminently practical language.

LOL!  Good one.

mark


From rosuav at gmail.com  Mon May 11 11:40:23 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 12 May 2015 01:40:23 +1000
Subject: code blocks
In-Reply-To: <ca2e1795-0eec-4584-ae38-6f6bd7f37585@googlegroups.com>
References: <mailman.49.1430633046.12865.python-list@python.org>
 <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>
 <mailman.340.1431315116.12865.python-list@python.org>
 <ca2e1795-0eec-4584-ae38-6f6bd7f37585@googlegroups.com>
Message-ID: <CAPTjJmraBmENugJatA8Z6omu_pW94kd42jdYFD-fCt81RLQVdA@mail.gmail.com>

On Tue, May 12, 2015 at 1:22 AM, zipher <dreamingforward at gmail.com> wrote:
> Ah, yeah, I guess that does it.  But (shame) it looks like you've gone past the BDFL.  Try:
>
>>>> help(exec)
>             ^
> SyntaxError: invalid syntax
>

That's because, in the version of Python you're using, exec is a
keyword. You could switch to Python 3, where it's a function, or
request it by name. Though interestingly, my Py2 doesn't have any help
on exec:

>>> help('exec')
no documentation found for 'exec'

Not sure why that is.

ChrisA


From steve+comp.lang.python at pearwood.info  Mon May 11 11:44:39 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 12 May 2015 01:44:39 +1000
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <87fv73iavz.fsf@elektro.pacujo.net>
 <mailman.362.1431350872.12865.python-list@python.org>
Message-ID: <5550ce68$0$12990$c3e8da3$5496439d@news.astraweb.com>

On Mon, 11 May 2015 11:27 pm, Antoon Pardon wrote:

> The point is that all too often someone wants to defend a specific choice
> the developers have made and cites some general rule or principle in
> support, ignoring the fact that python breaks that rule/principle in other
> area's.


"It's a free country, you can do what you like."

"No I can't, I'm not allowed to kill people."

"Um, okay."


Just because there are exceptions to a rule doesn't mean it isn't a general
rule. A few exceptions are just exceptions, they don't invalidate the fact
that "consenting adults" is a basic design principle of Python.


-- 
Steven



From skip.montanaro at gmail.com  Mon May 11 11:48:32 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 11 May 2015 10:48:32 -0500
Subject: anomaly
In-Reply-To: <b05b09d6-9585-41d7-b573-b3c331681d89@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
 <mailman.367.1431355924.12865.python-list@python.org>
 <b05b09d6-9585-41d7-b573-b3c331681d89@googlegroups.com>
Message-ID: <CANc-5UyjDdNArUxE73fG4_Sx0o4ah53EVKGZZAGfWEzpoxUDFw@mail.gmail.com>

On Mon, May 11, 2015 at 10:11 AM, zipher <dreamingforward at gmail.com> wrote:
> I also bought the idea of everything as an object, it has a unbeatable purity to it.  But we won't ever get to the point were OOP is like the purity of math because the greatest utility of OOP is working with real-world data.  And that real-world puts bounds on the otherwise abstract purity in which a language is theoretically capable.

Did someone here say it would? Sure, OOP isn't as pure as math, but
most object-oriented languages aren't pure OO languages, either.
(Maybe Smalltalk?) In Python, when you want to manipulate bazillions
of numbers, you use numpy, pandas, etc. In C++, you code in the C
subset it (still) contains when you don't want objects.

The practicality side of things suggests that even though
everything-is-an-object isn't perfect, it may be good enough.
People/projects/companies generally can't afford to follow every
change that blows through their environment. That's why (for example),
COBOL lasted so long. In fact, I suspect you could still make a good
living writing COBOL, if you really wanted to. (Searching indeed.com
for "COBOL" in Chicago, IL gave me 81 hits.)

Python was never meant to be "pure". It has, by Guido's own admission,
borrowed ideas from many other languages. Very little in Python is
truly new, certainly not its object model. At the user level
everything appears to be an object, but not everything is under the
covers (e.g., numeric elements of array objects).

Skip


From steve+comp.lang.python at pearwood.info  Mon May 11 11:55:45 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 12 May 2015 01:55:45 +1000
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
Message-ID: <5550d102$0$12985$c3e8da3$5496439d@news.astraweb.com>

On Tue, 12 May 2015 12:23 am, zipher wrote:

> On Monday, May 11, 2015 at 1:11:26 AM UTC-5, Steven D'Aprano wrote:
>> On Monday 11 May 2015 10:57, zipher wrote:
>> > I guess everyone expects this behavior since Python implemented this
>> > idea of "everything is an object", but I think this branch of OOP (on
>> > the
>> > branch of the Tree of Programming Languages) has to be chopped off. 
>> > The idea of everything is an object is backwards (unless your in a LISP
>> > machine).  Like I say, it's trying to be too pure and not practical.
>> 
>> Python is in production use in hundreds of thousands of organisations. It
>> has been heavily used for over twenty years, in everything from quick and
>> dirty one line scripts to hundred-thousand LOC applications.
> 
> Yeah, so was COBOL.  Boom.

So *is* COBOL. (Except for the one-line scripts part.)

COBOL is nearly as old as Fortran, half a century old, and reports of its
death are grossly exaggerated. It might not be a nice language, or a
particularly modern language (even though it now has OOP features!) but,
like the cockroach, it's hard to kill.

It's a running gag in IT circles that "the imminent death of COBOL" is
predicted every year, and will continue to be predicted every year well
into the 2100s.



-- 
Steven



From __peter__ at web.de  Mon May 11 12:01:12 2015
From: __peter__ at web.de (Peter Otten)
Date: Mon, 11 May 2015 18:01:12 +0200
Subject: code blocks
References: <mailman.49.1430633046.12865.python-list@python.org>
 <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>
 <mailman.340.1431315116.12865.python-list@python.org>
 <ca2e1795-0eec-4584-ae38-6f6bd7f37585@googlegroups.com>
 <CAPTjJmraBmENugJatA8Z6omu_pW94kd42jdYFD-fCt81RLQVdA@mail.gmail.com>
Message-ID: <miqjoa$17d$1@ger.gmane.org>

Chris Angelico wrote:

> On Tue, May 12, 2015 at 1:22 AM, zipher <dreamingforward at gmail.com> wrote:
>> Ah, yeah, I guess that does it.  But (shame) it looks like you've gone
>> past the BDFL.  Try:
>>
>>>>> help(exec)
>>             ^
>> SyntaxError: invalid syntax
>>
> 
> That's because, in the version of Python you're using, exec is a
> keyword. You could switch to Python 3, where it's a function, or
> request it by name. Though interestingly, my Py2 doesn't have any help
> on exec:
> 
>>>> help('exec')
> no documentation found for 'exec'
> 
> Not sure why that is.

Path confusion? You may accidentally be importing Python 3's topics. 
Try

>>> from pydoc_data import topics
>>> topics.__file__
'/usr/lib/python2.7/pydoc_data/topics.pyc'
>>> "exec" in topics.topics
True
>>> help("exec")
The ``exec`` statement
[...]

>>> import sys
>>> sys.path.insert(0,  "/usr/lib/python3.4")
>>> del sys.modules["pydoc_data"]
>>> del sys.modules["pydoc_data.topics"]
>>> help("exec")
no documentation found for 'exec'




From rosuav at gmail.com  Mon May 11 12:07:53 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 12 May 2015 02:07:53 +1000
Subject: code blocks
In-Reply-To: <miqjoa$17d$1@ger.gmane.org>
References: <mailman.49.1430633046.12865.python-list@python.org>
 <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>
 <mailman.340.1431315116.12865.python-list@python.org>
 <ca2e1795-0eec-4584-ae38-6f6bd7f37585@googlegroups.com>
 <CAPTjJmraBmENugJatA8Z6omu_pW94kd42jdYFD-fCt81RLQVdA@mail.gmail.com>
 <miqjoa$17d$1@ger.gmane.org>
Message-ID: <CAPTjJmpMYodOf0PDc0ShtAiy-TWm=h3BCLtg7Ha+t2pswF9ZWQ@mail.gmail.com>

On Tue, May 12, 2015 at 2:01 AM, Peter Otten <__peter__ at web.de> wrote:
> Though interestingly, my Py2 doesn't have any help
>> on exec:
>>
>>>>> help('exec')
>> no documentation found for 'exec'
>>
>> Not sure why that is.
>
> Path confusion? You may accidentally be importing Python 3's topics.
> Try
>
>>>> from pydoc_data import topics
>>>> topics.__file__
> '/usr/lib/python2.7/pydoc_data/topics.pyc'
>>>> "exec" in topics.topics
> True

Peculiar.

$ python
Python 2.7.3 (default, Mar 13 2014, 11:03:55)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from pydoc_data import topics
>>> topics.__file__
'/usr/lib/python2.7/pydoc_data/topics.pyc'
>>> "exec" in topics.topics
False
>>> topics.topics.keys()
['conversions', 'debugger', 'attribute-access', 'augassign',
'numeric-types', 'context-managers', 'bitwise', 'global', 'numbers',
'customization', 'in', 'floating', 'integers', 'naming', 'if',
'binary', 'raise', 'for', 'typesmapping', 'subscriptions',
'specialnames', 'typesseq', 'dynamic-features', 'bltin-code-objects',
'continue', 'dict', 'bltin-type-objects', 'import', 'typesmethods',
'pass', 'atom-literals', 'slicings', 'function', 'typesseq-mutable',
'bltin-ellipsis-object', 'execmodel', 'return', 'exprlists', 'power',
'booleans', 'string-methods', 'assignment', 'callable-types', 'yield',
'lists', 'else', 'assert', 'formatstrings', 'objects', 'shifting',
'unary', 'compound', 'typesfunctions', 'imaginary', 'specialattrs',
'with', 'class', 'types', 'break', 'calls', 'try', 'identifiers',
'atom-identifiers', 'id-classes', 'bltin-null-object', 'while',
'attribute-references', 'del', 'truth', 'sequence-types',
'exceptions', 'comparisons', 'operator-summary', 'typesmodules',
'strings', 'lambda']

Whatever. I've made a few messes on this system, maybe I broke
something somewhere. In any case, help('exec') is what's needed for
help on keywords, even if one particular installation doesn't have one
particular keyword-help.

ChrisA


From breamoreboy at yahoo.co.uk  Mon May 11 12:12:20 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 11 May 2015 17:12:20 +0100
Subject: anomaly
In-Reply-To: <CANc-5UyjDdNArUxE73fG4_Sx0o4ah53EVKGZZAGfWEzpoxUDFw@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
 <mailman.367.1431355924.12865.python-list@python.org>
 <b05b09d6-9585-41d7-b573-b3c331681d89@googlegroups.com>
 <CANc-5UyjDdNArUxE73fG4_Sx0o4ah53EVKGZZAGfWEzpoxUDFw@mail.gmail.com>
Message-ID: <miqkd5$dvd$1@ger.gmane.org>

On 11/05/2015 16:48, Skip Montanaro wrote:
> On Mon, May 11, 2015 at 10:11 AM, zipher <dreamingforward at gmail.com> wrote:
>> I also bought the idea of everything as an object, it has a unbeatable purity to it.  But we won't ever get to the point were OOP is like the purity of math because the greatest utility of OOP is working with real-world data.  And that real-world puts bounds on the otherwise abstract purity in which a language is theoretically capable.
>
> Did someone here say it would? Sure, OOP isn't as pure as math, but
> most object-oriented languages aren't pure OO languages, either.
> (Maybe Smalltalk?) In Python, when you want to manipulate bazillions
> of numbers, you use numpy, pandas, etc. In C++, you code in the C
> subset it (still) contains when you don't want objects.
>
> The practicality side of things suggests that even though
> everything-is-an-object isn't perfect, it may be good enough.
> People/projects/companies generally can't afford to follow every
> change that blows through their environment. That's why (for example),
> COBOL lasted so long. In fact, I suspect you could still make a good
> living writing COBOL, if you really wanted to. (Searching indeed.com
> for "COBOL" in Chicago, IL gave me 81 hits.)
>
> Python was never meant to be "pure". It has, by Guido's own admission,
> borrowed ideas from many other languages. Very little in Python is
> truly new, certainly not its object model. At the user level
> everything appears to be an object, but not everything is under the
> covers (e.g., numeric elements of array objects).
>
> Skip
>

Are you aware that you're attempting to communicate with a known troll 
who thankfully has been absent for some years?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From dreamingforward at gmail.com  Mon May 11 12:17:50 2015
From: dreamingforward at gmail.com (zipher)
Date: Mon, 11 May 2015 09:17:50 -0700 (PDT)
Subject: anomaly
In-Reply-To: <5550ce68$0$12990$c3e8da3$5496439d@news.astraweb.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <87fv73iavz.fsf@elektro.pacujo.net>
 <mailman.362.1431350872.12865.python-list@python.org>
 <5550ce68$0$12990$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <f36041df-0e4d-4ae1-aced-d2e62e2fd7b7@googlegroups.com>

On Monday, May 11, 2015 at 10:44:51 AM UTC-5, Steven D'Aprano wrote:
> On Mon, 11 May 2015 11:27 pm, Antoon Pardon wrote:
> 
> > The point is that all too often someone wants to defend a specific choice
> > the developers have made and cites some general rule or principle in
> > support, ignoring the fact that python breaks that rule/principle in other
> > area's.
> 
> "It's a free country, you can do what you like."
> 
> "No I can't, I'm not allowed to kill people."
> 
> Just because there are exceptions to a rule doesn't mean it isn't a general
> rule. A few exceptions are just exceptions, they don't invalidate the fact
> that "consenting adults" is a basic design principle of Python.

The solution to this is to find an overarching maxim that encompasses both.  This was already solved (in your example) with the Age they called the Enlightenment (and perhaps arguably with Kant).  The solution was that your liberty extends to the point another's begins -- a more scalable maxim than "freedom".

With OOP, the solution is not to slide by having sloppy design goals, it's to find a noble purpose.  OOP by itself has no goal other than to try to make a system of re-usable objects.  By itself it's like having a bow and arrow with no target.  I'm suggesting there's a target:  a data ecosystem for the exabytes of data that exists with the Internet -- not simulating graphical objects on the screen or trying to re-make the real-world in the virtual world.   These are missteps that perhaps can be explored in languages specially tailored for it (Simula, POV-ray, Erlang, etc.).

Mark


From rustompmody at gmail.com  Mon May 11 12:29:03 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 11 May 2015 09:29:03 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <miqeo1$r2r$2@reader1.panix.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <55502919$0$12997$c3e8da3$5496439d@news.astraweb.com>
 <281c0f90-8353-4480-93d3-9c656505cc61@googlegroups.com>
 <mailman.364.1431353048.12865.python-list@python.org>
 <miqeo1$r2r$2@reader1.panix.com>
Message-ID: <a74be458-717e-4e02-b5cc-cb151a31ad0f@googlegroups.com>

On Monday, May 11, 2015 at 8:05:56 PM UTC+5:30, Grant Edwards wrote:
> On 2015-05-11, Skip Montanaro  wrote:
> 
> > Don't CS departments still have a computer languages survey class? When I
> > was a graduate student at Iowa in the early 80s, we had one. (It was, as I
> > recall, an upper level undergrad course. I didn't get into CS until
> > graduate school, so went back to filled in some missing stuff.) I don't
> > recall all the languages we touched on, but ISTR there were five or six. I
> > know we hit Lisp (today, it would likely be Scheme), and probably APL
> > (today it would probably be Python+Pandas, MATLAB, R, or something similar).
> 
> There was a similar class at both Iowa State and University of MN. You
> learned a half-dozen languages in a single quarter.  IIRC, at ISU we
> did Lisp, Prolog, APL, Snobol and a couple others.  The main
> pedagogical language at the time was Pascal, but we also learned
> FORTRAN, COBOL, BASIC, and a couple three assembly languages (PDP-11,
> VAX, Z80, 6502).  If you were a computer enineering major instead of
> computer science, you also leared a hardware description language.  At
> the time it was AHPL.

These survey of PLs courses are a travesty.
At best students learn nothing
At worst they imagine they learnt something.
IMHO if you dont get some feel for the language, something about its 'zen',
you should not be allowed to say you know anything about it.
Getting a grade for it ensures the exact opposite.

[I recently heard of a course whose exam papers had questions like:
"Name 7 keywords in C++
]


From steve+comp.lang.python at pearwood.info  Mon May 11 12:35:23 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 12 May 2015 02:35:23 +1000
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <mailman.345.1431329316.12865.python-list@python.org>
 <miqbb8$o7l$1@dont-email.me>
Message-ID: <5550da4d$0$12981$c3e8da3$5496439d@news.astraweb.com>

On Mon, 11 May 2015 11:37 pm, Mel Wilson wrote:

> On Sun, 10 May 2015 14:12:44 -0500, boB Stepp wrote:
> 
>> I have to admit being surprised by this, too. I am just now studying on
>> how to write my own classes in Python, and have come to realize that
>> doing this is *possible*, but the *surprise* to me is why the language
>> design allowed this to actually be done.
> 
> Read Cory Doctorow lately on the War Against General Purpose Computing,
> where a bunch of people who don't really understand are trying to make it
> impossible for any computer to do something that is The Wrong Thing.

I think you are conflating two different ideas of "the Wrong Thing".

One is a political view, driven almost entirely by a small subset of the
copyright industry. They are driven by fear of losing control, and greed.

But the other, the one Bob refers to, comes from the technical view that
writing correct code that does what you want is hard, and we need the help
of the compiler and programming language to do it.

Trivial and easy to make mistakes in coding shouldn't mean that your
computer gets taken over by criminals on the other side of the world and
used to send spam at your expense. Buffer overflows, integer overflow,
undefined behaviour in C, cross-site scripting attacks, SQL injection
attacks, the list goes on. Bugs in code are not only inconvenient and
expensive, they can literally kill.

The two ideas are completely unrelated.

The first group wants *outside parties* (the copyright cartels) to control
what you can do with your computer. They want to take away your control.

The second wants to give you tools which you can use to give you the control
you currently lack.

Right now, whatever computer you are running, there are probably anything up
to a couple of dozen ways that the NSA or criminal gangs can use to take
over your computer and use it against you. Some of the people reading this
message have already had their computer subverted and don't know it. (Some
know it, but don't care, out of either apathy, sheer irresponsibility, or
learned helplessness.)

Basically, human beings aren't clever enough to write bug-free code, and
those bugs mean that we lose control of our computers. (Sony once, and
infamously, actually used a Windows root kit to try to copy-protect some
music CDs.) We need better tools which extend our abilities to write
bug-free code, and those tools are new programming languages and code
analysis tools. They put us in control, they don't take it away.



-- 
Steven



From dreamingforward at gmail.com  Mon May 11 12:43:49 2015
From: dreamingforward at gmail.com (zipher)
Date: Mon, 11 May 2015 09:43:49 -0700 (PDT)
Subject: anomaly
In-Reply-To: <mailman.372.1431359322.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
 <mailman.367.1431355924.12865.python-list@python.org>
 <b05b09d6-9585-41d7-b573-b3c331681d89@googlegroups.com>
 <mailman.372.1431359322.12865.python-list@python.org>
Message-ID: <e6d39278-c569-4229-9713-94b426ae6e02@googlegroups.com>

On Monday, May 11, 2015 at 10:49:01 AM UTC-5, Skip Montanaro wrote:
> On Mon, May 11, 2015 at 10:11 AM, zipher <dreamingforward at gmail.com> wrote:
> > I also bought the idea of everything as an object, it has a unbeatable purity to it.  But we won't ever get to the point were OOP is like the purity of math because the greatest utility of OOP is working with real-world data.  And that real-world puts bounds on the otherwise abstract purity in which a language is theoretically capable.
> 
> Did someone here say it would? Sure, OOP isn't as pure as math, but
> most object-oriented languages aren't pure OO languages, either.

SKip, it's fine to have a loose definition of things in a language when you're not sure where you want it to go.  I think taking Python into the land of "everything is an object" was a neat exploration.  I even supported it.  But, now that I want to create a data ecosystem, sloppiness in design criteria or inadequacies in implementation gets exercised.  Perhaps it won't be Python's goals, but I would argue that it's the best direction for Python.

These aren't failings of the dev-team either -- no one's done this before.  There are some attempts like CORBA, but either they remained in proprietary domains or they simply weren't capable of scaling to every kind of data.  I've, in fact, had to invent a new data structure to do it, a data structure I'm implementing in Python and where the issue of the *nature* of OOP came to the fore.

For example, have you ever contemplated this:

class WeightedVertex(vertex_common(Vertex(dict))):
    """test class"""

That currently isn't valid Python code, but the idea is that I'm making composite objects that expand one upon the other -- another basic idea in the realm of theoretical OOP that this boneheaded community couldn't wrap it's head around.

Anyway, I'm also saying that it's better to have real design goals when such noble goals can be made, rather than be content with a sloppy flexibility.

> The practicality side of things suggests that even though
> everything-is-an-object isn't perfect, it may be good enough.
> People/projects/companies generally can't afford to follow every
> change that blows through their environment.

True.  Which is why theoretical CS exists -- to find deeper and better abstractions to encompass everything you want to do.  It started with variables, then linked lists, then trees, then graphs and I'm taking that trajectory to it's end point.  It's real Computer Science.  Most people here haven't seemed to understand the power of Graphs yet, or perhaps they just can't agree upon an API so leave out such a powerful data structure.  I'm currently refactoring my own implementation, so that it could be considered for inclusion in the collections module.

Mark

P.S. Sorry for the "boneheaded" insult -- I get called anything from a troll to a drug addict around here.


From dreamingforward at gmail.com  Mon May 11 12:51:41 2015
From: dreamingforward at gmail.com (zipher)
Date: Mon, 11 May 2015 09:51:41 -0700 (PDT)
Subject: code blocks
In-Reply-To: <mailman.374.1431360482.12865.python-list@python.org>
References: <mailman.49.1430633046.12865.python-list@python.org>
 <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>
 <mailman.340.1431315116.12865.python-list@python.org>
 <ca2e1795-0eec-4584-ae38-6f6bd7f37585@googlegroups.com>
 <CAPTjJmraBmENugJatA8Z6omu_pW94kd42jdYFD-fCt81RLQVdA@mail.gmail.com>
 <miqjoa$17d$1@ger.gmane.org>
 <mailman.374.1431360482.12865.python-list@python.org>
Message-ID: <470d5d47-b97d-41ed-86da-0f287398f1d6@googlegroups.com>

On Monday, May 11, 2015 at 11:08:14 AM UTC-5, Chris Angelico wrote:
> On Tue, May 12, 2015 at 2:01 AM, Peter Otten <__peter__ at web.de> wrote:
> > Though interestingly, my Py2 doesn't have any help
> >> on exec:
> >>
> >>>>> help('exec')
> >> no documentation found for 'exec'
> >>
> >> Not sure why that is.
> >
> > Path confusion? You may accidentally be importing Python 3's topics.
> > Try
> >
> >>>> from pydoc_data import topics
> >>>> topics.__file__
> > '/usr/lib/python2.7/pydoc_data/topics.pyc'
> >>>> "exec" in topics.topics
> > True
> 
> Peculiar.
> 
> $ python
> Python 2.7.3 (default, Mar 13 2014, 11:03:55)
> [GCC 4.7.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from pydoc_data import topics
> >>> topics.__file__
> '/usr/lib/python2.7/pydoc_data/topics.pyc'
> >>> "exec" in topics.topics
> False
> >>> topics.topics.keys()
> ['conversions', 'debugger', 'attribute-access', 'augassign',
> 'numeric-types', 'context-managers', 'bitwise', 'global', 'numbers',
> 'customization', 'in', 'floating', 'integers', 'naming', 'if',
> 'binary', 'raise', 'for', 'typesmapping', 'subscriptions',
> 'specialnames', 'typesseq', 'dynamic-features', 'bltin-code-objects',
> 'continue', 'dict', 'bltin-type-objects', 'import', 'typesmethods',
> 'pass', 'atom-literals', 'slicings', 'function', 'typesseq-mutable',
> 'bltin-ellipsis-object', 'execmodel', 'return', 'exprlists', 'power',
> 'booleans', 'string-methods', 'assignment', 'callable-types', 'yield',
> 'lists', 'else', 'assert', 'formatstrings', 'objects', 'shifting',
> 'unary', 'compound', 'typesfunctions', 'imaginary', 'specialattrs',
> 'with', 'class', 'types', 'break', 'calls', 'try', 'identifiers',
> 'atom-identifiers', 'id-classes', 'bltin-null-object', 'while',
> 'attribute-references', 'del', 'truth', 'sequence-types',
> 'exceptions', 'comparisons', 'operator-summary', 'typesmodules',
> 'strings', 'lambda']
> 
> Whatever. I've made a few messes on this system, maybe I broke
> something somewhere. In any case, help('exec') is what's needed for
> help on keywords, even if one particular installation doesn't have one
> particular keyword-help.
> 
> ChrisA

It's strange, help(eval) works but help(exec) creates a SyntaxError, even though they would seem to be the same semantically -- both imperative verbs.  You nailed it by calling "exec" a keyword, but still it's a strange distinction.

Mark


From __peter__ at web.de  Mon May 11 12:59:46 2015
From: __peter__ at web.de (Peter Otten)
Date: Mon, 11 May 2015 18:59:46 +0200
Subject: code blocks
References: <mailman.49.1430633046.12865.python-list@python.org>
 <66312d54-de5a-4015-99b5-41d332559e00@googlegroups.com>
 <mailman.340.1431315116.12865.python-list@python.org>
 <ca2e1795-0eec-4584-ae38-6f6bd7f37585@googlegroups.com>
 <CAPTjJmraBmENugJatA8Z6omu_pW94kd42jdYFD-fCt81RLQVdA@mail.gmail.com>
 <miqjoa$17d$1@ger.gmane.org>
 <CAPTjJmpMYodOf0PDc0ShtAiy-TWm=h3BCLtg7Ha+t2pswF9ZWQ@mail.gmail.com>
Message-ID: <miqn64$6ht$1@ger.gmane.org>

Chris Angelico wrote:

> On Tue, May 12, 2015 at 2:01 AM, Peter Otten <__peter__ at web.de> wrote:
>> Though interestingly, my Py2 doesn't have any help
>>> on exec:
>>>
>>>>>> help('exec')
>>> no documentation found for 'exec'
>>>
>>> Not sure why that is.
>>
>> Path confusion? You may accidentally be importing Python 3's topics.
>> Try
>>
>>>>> from pydoc_data import topics
>>>>> topics.__file__
>> '/usr/lib/python2.7/pydoc_data/topics.pyc'
>>>>> "exec" in topics.topics
>> True
> 
> Peculiar.
> 
> $ python
> Python 2.7.3 (default, Mar 13 2014, 11:03:55)
> [GCC 4.7.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from pydoc_data import topics
>>>> topics.__file__
> '/usr/lib/python2.7/pydoc_data/topics.pyc'
>>>> "exec" in topics.topics
> False
>>>> topics.topics.keys()
> ['conversions', 'debugger', 'attribute-access', 'augassign',
> 'numeric-types', 'context-managers', 'bitwise', 'global', 'numbers',
> 'customization', 'in', 'floating', 'integers', 'naming', 'if',
> 'binary', 'raise', 'for', 'typesmapping', 'subscriptions',
> 'specialnames', 'typesseq', 'dynamic-features', 'bltin-code-objects',
> 'continue', 'dict', 'bltin-type-objects', 'import', 'typesmethods',
> 'pass', 'atom-literals', 'slicings', 'function', 'typesseq-mutable',
> 'bltin-ellipsis-object', 'execmodel', 'return', 'exprlists', 'power',
> 'booleans', 'string-methods', 'assignment', 'callable-types', 'yield',
> 'lists', 'else', 'assert', 'formatstrings', 'objects', 'shifting',
> 'unary', 'compound', 'typesfunctions', 'imaginary', 'specialattrs',
> 'with', 'class', 'types', 'break', 'calls', 'try', 'identifiers',
> 'atom-identifiers', 'id-classes', 'bltin-null-object', 'while',
> 'attribute-references', 'del', 'truth', 'sequence-types',
> 'exceptions', 'comparisons', 'operator-summary', 'typesmodules',
> 'strings', 'lambda']

Note that "print" is missing, too.

> Whatever. I've made a few messes on this system, maybe I broke
> something somewhere. 

Probably.

> In any case, help('exec') is what's needed for
> help on keywords, even if one particular installation doesn't have one
> particular keyword-help.




From larry.martell at gmail.com  Mon May 11 13:37:31 2015
From: larry.martell at gmail.com (Larry Martell)
Date: Mon, 11 May 2015 13:37:31 -0400
Subject: Immediate Hire: Sr. Java Software Engineer - Oklahoma - GC, US
 Citizens Onlt
In-Reply-To: <b4576228-1865-4ff5-b8d8-c7caa177c0c6@googlegroups.com>
References: <b4576228-1865-4ff5-b8d8-c7caa177c0c6@googlegroups.com>
Message-ID: <CACwCsY4+jisUoTD5_H-gGjSQnp=DboA69PFfvOWrQjT5dKRYgA@mail.gmail.com>

I am only interested in work that I can do remotely from home. If you
have any opportunities like that, please contact me.

On Mon, May 11, 2015 at 11:12 AM, nagaraju thoudoju
<nagarajuusstaffing at gmail.com> wrote:
> Hi,
>
> Hope you are doing well,
>
> Please find the requirement below and let me know you interest on this position on  nagaraju at intsystech.com or feel free to call me on my
> D:908-333-3534.
>
> Requirement:
>
> Job Description - Sr. Java Software Engineer
> Location: Edmond, Oklahoma
>
> Responsibilities:
> Provide applications development and enhancement functions; including analysis, design, coding, testing, and deployment of Java web applications and web services.
> Work with business analysts and technical architects to formalize specifications for custom development.
> Proactively work to solve issues while including the key functional and/or business stakeholders.
> Provide analysis and prototyping of new custom solutions, or new requirements for existing custom solutions.
> With limited supervision, develop and enhance custom solutions using approved technologies and platforms.
> Document and perform unit, integration, and regression testing in support of new implementations and releases.
> Skills:
> Basic Qualifications
> 7+ years of object oriented application development experience using Java
> Solid experience with web application development
> Practical knowledge with web service and API development
> Expertise in application servers including WebSphere, JBoss and Tomcat
> Proficiency with the Spring Framework including Spring MVC and Spring Data
> Strong understanding of object oriented concepts, web application and web service design, open source tools, and web application frameworks
> Professional experience designing, developing and deploying software solutions
> Nice to Have:
> Financial Services Systems experience
> Security Experience
> Drools and jBPM knowledge
> Understanding of LifeRay
> Other attributes:
> Attention to detail
> Focus on quality of both code and user experience
> Thrive in an entrepreneurial environment
> Excellent oral and written communication skills
> Strong analytical and problem solving skills
> Inspire a culture of innovation and teamwork
> Thanks & Best Regards....?
>
> Raju
>
> International Systems Technologies Inc.
>
> 10 Corporate place south.| Suite 203  |
>
> Piscataway, NJ 08854  |
>
> D:908-333-3540 |Fax:732-348-9533|
>
> E-mail: nagaraju at intsystech.com   |
>
> G-talk: nagarajuusstaffing at gmail.com
>
> URL: www.intsystech.com
> --
> https://mail.python.org/mailman/listinfo/python-list


From motoom at xs4all.nl  Mon May 11 13:54:00 2015
From: motoom at xs4all.nl (Michiel Overtoom)
Date: Mon, 11 May 2015 19:54:00 +0200
Subject: Immediate Hire: Sr. Java Software Engineer - Oklahoma - GC,
 US Citizens Onlt
In-Reply-To: <b4576228-1865-4ff5-b8d8-c7caa177c0c6@googlegroups.com>
References: <b4576228-1865-4ff5-b8d8-c7caa177c0c6@googlegroups.com>
Message-ID: <FA8DDAB4-2AAE-45A9-9A56-FBCCB9A44A16@xs4all.nl>


On May 11, 2015, at 17:12, nagaraju thoudoju wrote:

> Job Description - Sr. Java Software Engineer


This is a Python mailinglist, not a Java one.

-- 
"You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes



From DLipman~NoSpam~ at Verizon.Net  Mon May 11 14:13:48 2015
From: DLipman~NoSpam~ at Verizon.Net (David H. Lipman)
Date: Mon, 11 May 2015 14:13:48 -0400
Subject: Immediate Hire: Sr. Java Software Engineer - Oklahoma - GC,
 US Citizens Onlt
In-Reply-To: <mailman.379.1431365858.12865.python-list@python.org>
References: <b4576228-1865-4ff5-b8d8-c7caa177c0c6@googlegroups.com>
 <mailman.379.1431365858.12865.python-list@python.org>
Message-ID: <I6WdnSj0WJnHbM3InZ2dnUU7-cOdnZ2d@giganews.com>

From: "Larry Martell" <larry.martell at gmail.com>

> I am only interested in work that I can do remotely from home. If you
> have any opportunities like that, please contact me.

Please do not engage spammers or quote spam.

-- 
Dave
Multi-AV Scanning Tool - http://multi-av.thespykiller.co.uk
http://www.pctipp.ch/downloads/dl/35905.asp


From skip.montanaro at gmail.com  Mon May 11 14:20:12 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 11 May 2015 13:20:12 -0500
Subject: Responding to Raju and friends at Intsystech...
Message-ID: <CANc-5UzDBsj6w25NDmkmij-EgOWTMU3KX1BHJoUHDjjs0Ep5tQ@mail.gmail.com>

> On Mon, May 11, 2015 at 11:12 AM, nagaraju thoudoju
> <nagarajuusstaffing at gmail.com> wrote:
>>
>> Please find the requirement below and let me know you interest on
>> this position....

So this Raju fellow wants to know our "interest on [sic] this
position," and he's not responded appropriately to anyone's pleas to
stop spamming comp.lang.python. Searching groups.google.com for
"intsystech" returned over 26,000 hits, so I doubt we are the only
people he's actively spamming. How about everybody reading this note
responds to his missive, telling him our interest. Be nice. Respond as
if you really are interested in something Intsystech might have to
offer. Be sure to leave python-list at python.org/comp.lang.python out of
your reply, but paste the following email addresses into your cc list
(which includes a few of Raju's fellow spammers):

  info at intsystech.com, hr at intsystech.com, suryaprakash at intsystech.com,
pavan.kumar at intsystech.com, kevin at intsystech.com,
manish.kumar at intsystech.com

There's no need to send an angry reply. We can kill them with
kindness.  In fact, it will waste more of their time if they think,
even for a moment, that your email represents legitimate interest in
their spam.  I think Larry Martell's response was was just about
right.  Raju posts about a Java position on-site in Edmond, OK, so
respond asking him about Python (or COBOL, or Perl, or C++) positions
in Anchorage AK, or Chicago IL, or London UK, or telecommuting
positions...

I don't think there will be any need to tell Raju and his associates
where you saw his message. It won't really matter, and it's clearly
everywhere at this point. I suspect it will take him a good long while
to sift through all the apparently valid responses which just waste
their time and make it nearly impossible for them to find the scant
valid responses to their spam.

Skip


From DLipman~NoSpam~ at Verizon.Net  Mon May 11 14:26:50 2015
From: DLipman~NoSpam~ at Verizon.Net (David H. Lipman)
Date: Mon, 11 May 2015 14:26:50 -0400
Subject: Responding to Raju and friends at Intsystech...
In-Reply-To: <mailman.381.1431368416.12865.python-list@python.org>
References: <mailman.381.1431368416.12865.python-list@python.org>
Message-ID: <5cmdnWX3XOf0ac3InZ2dnUU7-budnZ2d@giganews.com>

From: "Skip Montanaro" <skip.montanaro at gmail.com>

>> On Mon, May 11, 2015 at 11:12 AM, nagaraju thoudoju
>> <nagarajuusstaffing at gmail.com> wrote:
>>> 
>>> Please find the requirement below and let me know you interest on
>>> this position....
> 
> So this Raju fellow wants to know our "interest on [sic] this
> position," and he's not responded appropriately to anyone's pleas to
> stop spamming comp.lang.python. Searching groups.google.com for
> "intsystech" returned over 26,000 hits, so I doubt we are the only
> people he's actively spamming. How about everybody reading this note
> responds to his missive, telling him our interest. Be nice. Respond as
> if you really are interested in something Intsystech might have to
> offer. Be sure to leave python-list at python.org/comp.lang.python out of
> your reply, but paste the following email addresses into your cc list
> (which includes a few of Raju's fellow spammers):
> 
>   info at intsystech.com, hr at intsystech.com, suryaprakash at intsystech.com,
> pavan.kumar at intsystech.com, kevin at intsystech.com,
> manish.kumar at intsystech.com
> 
> There's no need to send an angry reply. We can kill them with
> kindness.  In fact, it will waste more of their time if they think,
> even for a moment, that your email represents legitimate interest in
> their spam.  I think Larry Martell's response was was just about
> right.  Raju posts about a Java position on-site in Edmond, OK, so
> respond asking him about Python (or COBOL, or Perl, or C++) positions
> in Anchorage AK, or Chicago IL, or London UK, or telecommuting
> positions...
> 
> I don't think there will be any need to tell Raju and his associates
> where you saw his message. It won't really matter, and it's clearly
> everywhere at this point. I suspect it will take him a good long while
> to sift through all the apparently valid responses which just waste
> their time and make it nearly impossible for them to find the scant
> valid responses to their spam.
> 
> Skip

abuse at 1and1.com
abuse at oneandone.net 
cabuse at tatacommunications.com
4755abuse at tatacommunications.com



-- 
Dave
Multi-AV Scanning Tool - http://multi-av.thespykiller.co.uk
http://www.pctipp.ch/downloads/dl/35905.asp


From beliavsky at aol.com  Mon May 11 15:01:57 2015
From: beliavsky at aol.com (beliavsky at aol.com)
Date: Mon, 11 May 2015 12:01:57 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mailman.333.1431308306.12865.python-list@python.org>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
Message-ID: <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>

On Sunday, May 10, 2015 at 9:38:38 PM UTC-4, Ian wrote:
> On Sun, May 10, 2015 at 3:16 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> > Scheme is my favorite language. I think, however, it is a pretty
> > advanced language and requires a pretty solid basis in programming and
> > computer science.
> >
> > Python, in contrast, is a great introductory programming language. Sure,
> > you *can* get quite advanced with it, too, but you can get quite a bit
> > of fun stuff done with just the basics.
> 
> MIT famously used Scheme in their introductory course for more than
> two decades. Although they switched to Python a few years ago, I don't
> think they did so because there was anything wrong with Scheme.
> Wikipedia informs me that Yale and Grinnell are still using Scheme for
> their introductory courses.

Yale has taken the unusual step of outsourcing its introductory CS class to Harvard, which uses C as the main language in its CS50 class.

http://yaledailynews.com/blog/2014/11/07/faculty-approve-cs50-for-yale/
Faculty approve CS50 for Yale

"Just under a month after announcing that Yale's computer science department was considering importing Harvard's most popular course, faculty voted to bring CS50 to Yale.

Following what Yale College Dean Jonathan Holloway described as a "long, healthy discussion," faculty at Thursday's monthly meeting voted overwhelmingly to approve CS50 as a class to be taught at Yale. Computer science department chair Joan Feigenbaum said that the next step for CS50 will be for Harvard to approve the sharing of CS50 with Yale. If the course earns approval, she noted, Yale will formally introduce the class in Fall 2015."



From fetchinson at googlemail.com  Mon May 11 16:31:53 2015
From: fetchinson at googlemail.com (Fetchinson .)
Date: Mon, 11 May 2015 22:31:53 +0200
Subject: smart scheduling for webapp tester
Message-ID: <CADjSo4TmFrL8wRKdMnLw_ox-DOvbz2mgytyYZopF1z-PoMK=6g@mail.gmail.com>

I'm looking into a robust solution for web application testing. While
selenium is great for the actual testing, I'm thinking of a scheduler
as the final piece in the pipeline. Let's say I have 4 websites that I
need to test periodically, A, B, C, D. I'd like to be able to define
things like "run the tests for site A once a day" or "run the tests
for site B twice a day indefinitely" or "run the tests for site C
twice a week between now and 2 months from now" or "run the tests for
site C once a week between May 25 and June 21".

What would be the right scheduling tool for this? I know about the
standard tools like cron, sched, etc, and first even wrote one myself
(which became of course unmanagable after a short while :)) but was
hoping a more sophisticated tool is available.

Any ideas?
Daniel


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown


From mwilson at the-wire.com  Mon May 11 16:48:16 2015
From: mwilson at the-wire.com (Mel Wilson)
Date: Mon, 11 May 2015 20:48:16 +0000 (UTC)
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <mailman.345.1431329316.12865.python-list@python.org>
 <miqbb8$o7l$1@dont-email.me>
 <5550da4d$0$12981$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mir4ie$4qc$1@dont-email.me>

On Tue, 12 May 2015 02:35:23 +1000, Steven D'Aprano wrote:

> On Mon, 11 May 2015 11:37 pm, Mel Wilson wrote:
> 
>> On Sun, 10 May 2015 14:12:44 -0500, boB Stepp wrote:
>> 
>>> I have to admit being surprised by this, too. I am just now studying
>>> on how to write my own classes in Python, and have come to realize
>>> that doing this is *possible*, but the *surprise* to me is why the
>>> language design allowed this to actually be done.
>> 
>> Read Cory Doctorow lately on the War Against General Purpose Computing,
>> where a bunch of people who don't really understand are trying to make
>> it impossible for any computer to do something that is The Wrong Thing.
> 
> I think you are conflating two different ideas of "the Wrong Thing".

I don't think so.  A formal solution to a problem, i.e. a solution coded 
as a computer program, is limited to the things that can be done using 
formal techniques.  Whether it's people trying to enact their social 
standards in code, or whether it's people trying to nail the door shut 
against everything they "don't expect", or "think is illogical", the 
limits will still be there.

	Mel.


From emile at fenx.com  Mon May 11 18:36:16 2015
From: emile at fenx.com (Emile van Sebille)
Date: Mon, 11 May 2015 15:36:16 -0700
Subject: anomaly
In-Reply-To: <miqi5k$id1$1@reader1.panix.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
 <mailman.367.1431355924.12865.python-list@python.org>
 <miqi5k$id1$1@reader1.panix.com>
Message-ID: <mirat6$trv$1@ger.gmane.org>

On 5/11/2015 8:34 AM, Grant Edwards wrote:

 > Yow! YOU PICKED KARL
 > MALDEN'S NOSE!!

I'd bet most people familiar with Karl Malden wouldn't have a
problem picking his from a selection of twenty random noses.  :)

Emile




From dreamingforward at gmail.com  Mon May 11 20:42:09 2015
From: dreamingforward at gmail.com (zipher)
Date: Mon, 11 May 2015 17:42:09 -0700 (PDT)
Subject: How to properly apply OOP in the bouncing ball code
In-Reply-To: <mailman.383.1431390292.12865.python-list@python.org>
References: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>
 <45313faa-d256-4fa4-9e37-4b8dd85e1681@googlegroups.com>
 <mailman.383.1431390292.12865.python-list@python.org>
Message-ID: <a25437cb-c525-41db-9bd4-c9759a982a3b@googlegroups.com>

On Monday, May 11, 2015 at 7:25:09 PM UTC-5, Dennis Lee Bieber wrote:
> On Mon, 11 May 2015 08:33:56 -0700 (PDT), zipher
> <dreamingforward at gmail.com> declaimed the following:
> >You are making a error that few in the programming community have caught up to.  OOP design for *data abstraction* is a completely different beast that OOP for *simulation*.  The confusion is around the use of the word "object" which both denotes a physical world item and a virtual one detached from reality.
> >
> >I would say that Python might not be the right fit, but instead a language dedicated to simulation.
> 
> 	The danger there is that a "language dedicated to simulation" might
> mean "discrete event" simulation -- which would be an even worse fit to a
> problem of particle motion simulation.

Huh?  VPython successfully models particle motion simulation with discrete events.  I don't see how you're going to update a variable non-discretely.  Tensors?  Good luck.

Mark


From steve+comp.lang.python at pearwood.info  Mon May 11 21:59:13 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 12 May 2015 11:59:13 +1000
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
 <mailman.367.1431355924.12865.python-list@python.org>
 <b05b09d6-9585-41d7-b573-b3c331681d89@googlegroups.com>
 <mailman.372.1431359322.12865.python-list@python.org>
 <e6d39278-c569-4229-9713-94b426ae6e02@googlegroups.com>
Message-ID: <55515e72$0$12994$c3e8da3$5496439d@news.astraweb.com>

On Tue, 12 May 2015 02:43 am, zipher wrote:

> For example, have you ever contemplated this:
> 
> class WeightedVertex(vertex_common(Vertex(dict))):
> """test class"""


No.

> That currently isn't valid Python code, 

Yes it is. You just have to arrange matters so that
vertex_common(Vertex(dict)) returns a type.


> but the idea is that I'm making 
> composite objects that expand one upon the other -- another basic idea in
> the realm of theoretical OOP that this boneheaded community couldn't wrap
> it's head around.

Yes, I'm sure that from where you are floating around in space way beyond
the stratosphere, we must all look like feeble-minded ants.

http://www.joelonsoftware.com/articles/fog0000000018.html

But at least we know what is valid Python and what isn't.



-- 
Steven



From steve+comp.lang.python at pearwood.info  Mon May 11 22:04:13 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 12 May 2015 12:04:13 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
Message-ID: <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>

On Tue, 12 May 2015 05:01 am, beliavsky at aol.com wrote:

> Yale has taken the unusual step of outsourcing its introductory CS class
> to Harvard, which uses C as the main language in its CS50 class.

And another generation of new programmers will be irreversibly damaged by
exposure to C...


-- 
Steven



From steve+comp.lang.python at pearwood.info  Mon May 11 22:18:45 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 12 May 2015 12:18:45 +1000
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <mailman.345.1431329316.12865.python-list@python.org>
 <miqbb8$o7l$1@dont-email.me>
 <5550da4d$0$12981$c3e8da3$5496439d@news.astraweb.com>
 <mir4ie$4qc$1@dont-email.me>
Message-ID: <55516306$0$13010$c3e8da3$5496439d@news.astraweb.com>

On Tue, 12 May 2015 06:48 am, Mel Wilson wrote:

> On Tue, 12 May 2015 02:35:23 +1000, Steven D'Aprano wrote:
> 
>> On Mon, 11 May 2015 11:37 pm, Mel Wilson wrote:
>> 
>>> On Sun, 10 May 2015 14:12:44 -0500, boB Stepp wrote:
>>> 
>>>> I have to admit being surprised by this, too. I am just now studying
>>>> on how to write my own classes in Python, and have come to realize
>>>> that doing this is *possible*, but the *surprise* to me is why the
>>>> language design allowed this to actually be done.
>>> 
>>> Read Cory Doctorow lately on the War Against General Purpose Computing,
>>> where a bunch of people who don't really understand are trying to make
>>> it impossible for any computer to do something that is The Wrong Thing.
>> 
>> I think you are conflating two different ideas of "the Wrong Thing".
> 
> I don't think so.  A formal solution to a problem, i.e. a solution coded
> as a computer program, is limited to the things that can be done using
> formal techniques.  Whether it's people trying to enact their social
> standards in code, or whether it's people trying to nail the door shut
> against everything they "don't expect", or "think is illogical", the
> limits will still be there.

Do you think that Python is part of Doctorow's war against general purpose
computing because it doesn't have a GOTO command?

Do you think that using Python reduces your control over your computer
because Python code has well-defined integer overflow behaviour?

When you have had a bug in your code, and Python has raised an exception
with a nice traceback and an informative error message, how many times have
you thought "I wish Python would just seg fault"?


-- 
Steven



From wuwei23 at gmail.com  Tue May 12 01:02:03 2015
From: wuwei23 at gmail.com (alex23)
Date: Tue, 12 May 2015 15:02:03 +1000
Subject: anomaly
In-Reply-To: <03e4e251-20b0-4a2f-968e-2d40d2b5dc1e@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
 <mailman.367.1431355924.12865.python-list@python.org>
 <miqi5k$id1$1@reader1.panix.com>
 <03e4e251-20b0-4a2f-968e-2d40d2b5dc1e@googlegroups.com>
Message-ID: <mis1e9$oq2$1@dont-email.me>

On 12/05/2015 1:39 AM, zipher wrote:
> On Monday, May 11, 2015 at 10:34:24 AM UTC-5, Grant Edwards wrote:
>> That Python, like COBOL, is an eminently practical language.
>
> LOL!  Good one.

I would make an incredibly substantial wager that you've never developed 
anything of note in either Python or COBOL.



From torriem at gmail.com  Tue May 12 01:04:10 2015
From: torriem at gmail.com (Michael Torrie)
Date: Mon, 11 May 2015 23:04:10 -0600
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <555189CA.8040202@gmail.com>

On 05/11/2015 08:04 PM, Steven D'Aprano wrote:
> On Tue, 12 May 2015 05:01 am, beliavsky at aol.com wrote:
> 
>> Yale has taken the unusual step of outsourcing its introductory CS class
>> to Harvard, which uses C as the main language in its CS50 class.
> 
> And another generation of new programmers will be irreversibly damaged by
> exposure to C...

How so?  Surely starting at first principles of a computer's operation
can't be all that bad.  In my program at uni, one of the very first
level courses was actually to build a simulated CPU from logic gates and
then program it in assembly.  C is just a step up from there.  I should
note they also had Java in the first year, and that certainly caused
irreversible damage.

The wonderfulness of LISP and Python can be appreciated just fine with a
solid background in how Von Neumann architecture actually functions. In
fact I appreciate the layers of abstraction even more after I understand
them.



From greg.ewing at canterbury.ac.nz  Tue May 12 01:37:16 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Tue, 12 May 2015 17:37:16 +1200
Subject: anomaly
In-Reply-To: <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <crdhsfFfhhfU1@mid.individual.net>

Steven D'Aprano wrote:
> With programming languages, the designer can take the same route as Pascal
> or Java, and define standard functions as keywords that cannot be shadowed
> or redefined.

Nit: Pascal's standard types and functions are not reserved
words, they're predeclared identifiers, much as in Python,
and as far as I know can be shadowed.

-- 
Greg


From rustompmody at gmail.com  Tue May 12 01:38:49 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 11 May 2015 22:38:49 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mailman.386.1431407062.12865.python-list@python.org>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
Message-ID: <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>

On Tuesday, May 12, 2015 at 10:34:32 AM UTC+5:30, Michael Torrie wrote:
> On 05/11/2015 08:04 PM, Steven D'Aprano wrote:
> > On Tue, 12 May 2015 05:01 am, beliavsky wrote:
> > 
> >> Yale has taken the unusual step of outsourcing its introductory CS class
> >> to Harvard, which uses C as the main language in its CS50 class.
> > 
> > And another generation of new programmers will be irreversibly damaged by
> > exposure to C...
> 
> How so?  Surely starting at first principles of a computer's operation
> can't be all that bad.  In my program at uni, one of the very first
> level courses was actually to build a simulated CPU from logic gates and
> then program it in assembly.

Thats true.
The intro to programming course needs to convey something beyond syntax
and minor details -- something like the 'Zen'

The difference between C/Lisp (I club them together) and python is that
the former are more heroic.

Like mountain climbing you can get a high, a thrill, even 'see God'? but
you can also break your back or worse.

Python is by contrast like a walk in the park.  If you find it interesting
(for reasons outside of python) you can get the job done. No epiphanies here

> C is just a step up from there.

which may be a step too much.
And I think its much more than one step. [How many links in the gcc toolchain?]

> I should note they also had Java in the first year, and that certainly caused
> irreversible damage.

A different question altogether.
What Joel Spolsky describes? is simply the fact that Java slides its
practitioners down the DIKW pyramid?

[My own record of the hell let lose by teaching too early C.??
The first written in 91 and rather widely cited at that time including
first edition of 'Code Complete'. Second is a toning down as I grow older!
]

To some extent good teaching can ameliorate.
Only to some extent since the whole purpose of such languages is to dumb down
programming.

[And lest pythonistas feel pleased with that, do consider whether what
Spolsky applies to Java in 2005 in 2015 applies to python]
> 
> The wonderfulness of LISP and Python can be appreciated just fine with a
> solid background in how Von Neumann architecture actually functions. In
> fact I appreciate the layers of abstraction even more after I understand
> them.

modulo the law of primacy

----------------------------------------
? Eric Raymond almost literally says this:
| Lisp is worth learning for the profound enlightenment experience you will 
| have when you finally get it; that experience will make you a better 
| programmer for the rest of your days, even if you never actually use Lisp 

? http://www.joelonsoftware.com/articles/ThePerilsofJavaSchools.html

? http://en.wikipedia.org/wiki/DIKW_Pyramid

? http://www.the-magus.in/Publications/chor.pdf
? http://blog.languager.org/2013/02/c-in-education-and-software-engineering.html


From rustompmody at gmail.com  Tue May 12 02:42:29 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 11 May 2015 23:42:29 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
Message-ID: <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>

On Tuesday, May 12, 2015 at 11:09:01 AM UTC+5:30, Rustom Mody wrote:

> The difference between C/Lisp (I club them together) and python is that
> the former are more heroic.
> 
> Like mountain climbing you can get a high, a thrill, even 'see God'? but
> you can also break your back or worse.
> 
> Python is by contrast like a walk in the park.  If you find it interesting
> (for reasons outside of python) you can get the job done. No epiphanies here

Just to be clear the "No epiphanies" is entirely positive.
Around 2002 I started teaching python.
Around the same time I started using an open editor rather than blackboard
to teach.
And for the most part python has been a trusted friend -- even if I dont
know the code I will write beforehand and the bugs and the debugging and so on.
[I remember one time getting completely screwed by regular expressions
And more recently some confusion re dunder methods. but these are the exception
 to the rule that mostly python is quite reliable.
]

The reason I am saying this (to OP):
Which language you choose may not matter too much.
But choose one you are sufficiently friendly with to start hacking with an
open editor in front of the class

Scheme?? I used scheme in the 80s -- ie before the age of projectable editor
So while *in principle* it may seem as reliable as python
In practice when I read racket today I find it far more forbidding than the PC
scheme manuals I read in the 80s

The other suggestion I would make: Use an interactive interpreter.

Even if I were using C today, I'd find a C interpreter.
And never mind that it messes up 10% of 'official' C semantics.
You'll breeze through the remaining 90% at 3? the rate

And related to that (and one reason a pure functional language is good for 
pedagogy): NO PRINT statement
It may seem trivial but beginning students have a real hard writing clean
structured code.  Tabooing prints helps get there faster
And working in the interpreter makes a print-taboo a viable option


From rosuav at gmail.com  Tue May 12 02:57:12 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 12 May 2015 16:57:12 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
Message-ID: <CAPTjJmp9K+CmhXRLYAJ_dP5XiGpikMYth-itt4MprzsK-yS5Hg@mail.gmail.com>

On Tue, May 12, 2015 at 4:42 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> And related to that (and one reason a pure functional language is good for
> pedagogy): NO PRINT statement
> It may seem trivial but beginning students have a real hard writing clean
> structured code.  Tabooing prints helps get there faster
> And working in the interpreter makes a print-taboo a viable option

I firmly disagree. Interactive work is well and good, but print (or
equivalent - console.log, log.info, werror, etc etc) is extremely
useful for learning about a larger application. You can play with
things at the terminal, but how can you find out exactly what happens
when you click this button? Ensuring that your application can be
imported, executed, and manipulated interactively, all without
breaking its primary purpose, is a LOT of extra work, and not
something I'd recommend to beginners. So learn about print, learn
about how to get info out of a running program. You'll be the better
programmer for it.

ChrisA


From rosuav at gmail.com  Tue May 12 04:09:12 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 12 May 2015 18:09:12 +1000
Subject: smart scheduling for webapp tester
In-Reply-To: <CADjSo4TmFrL8wRKdMnLw_ox-DOvbz2mgytyYZopF1z-PoMK=6g@mail.gmail.com>
References: <CADjSo4TmFrL8wRKdMnLw_ox-DOvbz2mgytyYZopF1z-PoMK=6g@mail.gmail.com>
Message-ID: <CAPTjJmqbjfSpV1AoZiVoeak-MxN8DWZ4ednVNjSAaq72++WbCg@mail.gmail.com>

On Tue, May 12, 2015 at 6:31 AM, Fetchinson . <fetchinson at googlemail.com> wrote:
> I'm looking into a robust solution for web application testing. While
> selenium is great for the actual testing, I'm thinking of a scheduler
> as the final piece in the pipeline. Let's say I have 4 websites that I
> need to test periodically, A, B, C, D. I'd like to be able to define
> things like "run the tests for site A once a day" or "run the tests
> for site B twice a day indefinitely" or "run the tests for site C
> twice a week between now and 2 months from now" or "run the tests for
> site C once a week between May 25 and June 21".
>
> What would be the right scheduling tool for this? I know about the
> standard tools like cron, sched, etc, and first even wrote one myself
> (which became of course unmanagable after a short while :)) but was
> hoping a more sophisticated tool is available.

I'm not sure what you're testing here, so I can't advise on specifics.
If you're testing your application code, it shouldn't need any
periodic testing at all, but if you're verifying an active database,
it may not be necessary to involve your application. Actually, I tend
never to verify database structures; anything that I would consider
testing can get coded as a constraint, so it's enforced by the
database before anything gets committed.

But if you really do need things on a scheduler, I would advise using
your OS's facilities (cron, or equivalent). No need to reinvent the
wheel, unless you want it to do something different. I've written
several simple task schedulers, but always because they do something
fundamentally different from a basic one (like my "Let Me Know", which
checks my calendar and shows me a tick-down until the next significant
event - 27 hours until I host Office Hours, at the moment). To simply
invoke a program every 4:00 UTC, use cron and save yourself the
trouble.

ChrisA


From rustompmody at gmail.com  Tue May 12 05:16:16 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 02:16:16 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mailman.387.1431413836.12865.python-list@python.org>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
Message-ID: <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>

On Tuesday, May 12, 2015 at 12:27:44 PM UTC+5:30, Chris Angelico wrote:
> On Tue, May 12, 2015 at 4:42 PM, Rustom Mody  wrote:
> > And related to that (and one reason a pure functional language is good for
> > pedagogy): NO PRINT statement
> > It may seem trivial but beginning students have a real hard writing clean
> > structured code.  Tabooing prints helps get there faster
> > And working in the interpreter makes a print-taboo a viable option
> 
> I firmly disagree.

Yes we know that!

As it happens you also disagree with ACM's latest CS curriculum:
https://www.acm.org/education/CS2013-final-report.pdf [pg 158]
Absolute basics of CS include functional programming; among which first point 
is 'effect-free programming'
[Note there is no mention or commitment to fancy functional programming 
*languages*, just the principles]

> Interactive work is well and good, but print (or
> equivalent - console.log, log.info, werror, etc etc) is extremely
> useful for learning about a larger application.

You are talking phd level (or maybe graduate level)
I am talking kindergarten

> You can play with things at the terminal,

Very important to play before you grow up

> but how can you find out exactly what happens
> when you click this button? Ensuring that your application can be
> imported, executed, and manipulated interactively, all without
> breaking its primary purpose, is a LOT of extra work, and not
> something I'd recommend to beginners. So learn about print, learn
> about how to get info out of a running program. You'll be the better
> programmer for it.

Maybe you should read up on Bloom's taxonomy
[ACM curriculum follows this but simplified from 5 to 3 levels -- familiarity,
usage, assessment]

In particular wrt print: You are not distinguishing
- learning the 'what' of print (familiarity)
from
- learning the how (usage)
and most important the "when not" (assessment)



From rustompmody at gmail.com  Tue May 12 05:33:42 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 02:33:42 -0700 (PDT)
Subject: Running cool and silent
Message-ID: <d5b29c73-b44d-4437-ad3b-6832164ed550@googlegroups.com>

For some reason my Dell laptop runs hot and noisy in linux but cool and
silent in Windows-8

Running

$ echo "min_power" | sudo tee /sys/class/scsi_host/host*/link_power_management_policy

makes the fan slow/stop.

But I am not sure what it does!!
[I dont want my laptop fried and its rather HOT out here right now :-) ]

So is this command safe?
Does someone know what it does? [Dont remember where I found it]


From rustompmody at gmail.com  Tue May 12 05:34:19 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 02:34:19 -0700 (PDT)
Subject: Running cool and silent
In-Reply-To: <d5b29c73-b44d-4437-ad3b-6832164ed550@googlegroups.com>
References: <d5b29c73-b44d-4437-ad3b-6832164ed550@googlegroups.com>
Message-ID: <24bc702a-36f0-4a71-b6ce-1b88fd92b2a7@googlegroups.com>

On Tuesday, May 12, 2015 at 3:03:53 PM UTC+5:30, Rustom Mody wrote:
> For some reason my Dell laptop runs hot and noisy in linux but cool and
> silent in Windows-8
> 
> Running
> 
> $ echo "min_power" | sudo tee /sys/class/scsi_host/host*/link_power_management_policy
> 
> makes the fan slow/stop.
> 
> But I am not sure what it does!!
> [I dont want my laptop fried and its rather HOT out here right now :-) ]
> 
> So is this command safe?
> Does someone know what it does? [Dont remember where I found it]

Whoops!!
Wrong list... Sorry


From antoon.pardon at rece.vub.ac.be  Tue May 12 07:34:07 2015
From: antoon.pardon at rece.vub.ac.be (Antoon Pardon)
Date: Tue, 12 May 2015 13:34:07 +0200
Subject: anomaly
In-Reply-To: <87617zi53k.fsf@elektro.pacujo.net>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <87fv73iavz.fsf@elektro.pacujo.net>
 <mailman.362.1431350872.12865.python-list@python.org>
 <87617zi53k.fsf@elektro.pacujo.net>
Message-ID: <5551E52F.5020804@rece.vub.ac.be>

Op 11-05-15 om 16:03 schreef Marko Rauhamaa:

> Antoon Pardon <antoon.pardon at rece.vub.ac.be>:
>
>> The point is that all too often someone wants to defend a specific
>> choice the developers have made and cites some general rule or
>> principle in support, ignoring the fact that python breaks that
>> rule/principle in other area's.
> Granted, but you have set the trap for them by demanding a justification
> when no justification was required. Every language has their cute
> idiosyncrasies and arbitrary design choices.

I did not. My first reaction was after someone made this kind of defence.

-- 
Antoon Pardon



From antoon.pardon at rece.vub.ac.be  Tue May 12 07:55:44 2015
From: antoon.pardon at rece.vub.ac.be (Antoon Pardon)
Date: Tue, 12 May 2015 13:55:44 +0200
Subject: anomaly
In-Reply-To: <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
Message-ID: <5551EA40.2000606@rece.vub.ac.be>

Op 11-05-15 om 16:13 schreef Chris Angelico:

> Why does Python have most built-ins as simply looked-up names that can
> be overridden? Because otherwise, there would be a veritable ton of
> keywords:

But that doesn't answer the question why the developers chose "True" to be a
keyword and "int" to be a looked-up name.

and pretending to justify that choice by stating that the python thought
is: We're all adults here, if you want to override a builtin, who are we
to stop you. That is disingenuous.



From breamoreboy at yahoo.co.uk  Tue May 12 07:56:13 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 12 May 2015 12:56:13 +0100
Subject: anomaly
In-Reply-To: <mis1e9$oq2$1@dont-email.me>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
 <mailman.367.1431355924.12865.python-list@python.org>
 <miqi5k$id1$1@reader1.panix.com>
 <03e4e251-20b0-4a2f-968e-2d40d2b5dc1e@googlegroups.com>
 <mis1e9$oq2$1@dont-email.me>
Message-ID: <mispou$cip$1@ger.gmane.org>

On 12/05/2015 06:02, alex23 wrote:
> On 12/05/2015 1:39 AM, zipher wrote:
>> On Monday, May 11, 2015 at 10:34:24 AM UTC-5, Grant Edwards wrote:
>>> That Python, like COBOL, is an eminently practical language.
>>
>> LOL!  Good one.
>
> I would make an incredibly substantial wager that you've never developed
> anything of note in either Python or COBOL.
>

Why waste the final seven words?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From fetchinson at googlemail.com  Tue May 12 08:00:12 2015
From: fetchinson at googlemail.com (Fetchinson .)
Date: Tue, 12 May 2015 14:00:12 +0200
Subject: smart scheduling for webapp tester
In-Reply-To: <CAPTjJmqbjfSpV1AoZiVoeak-MxN8DWZ4ednVNjSAaq72++WbCg@mail.gmail.com>
References: <CADjSo4TmFrL8wRKdMnLw_ox-DOvbz2mgytyYZopF1z-PoMK=6g@mail.gmail.com>
 <CAPTjJmqbjfSpV1AoZiVoeak-MxN8DWZ4ednVNjSAaq72++WbCg@mail.gmail.com>
Message-ID: <CADjSo4TpxtNmwZ6Nv_39nM1TaXpd-+zNgHZ0Sv1or+ogDHEyiQ@mail.gmail.com>

On 5/12/15, Chris Angelico <rosuav at gmail.com> wrote:
> On Tue, May 12, 2015 at 6:31 AM, Fetchinson . <fetchinson at googlemail.com>
> wrote:
>> I'm looking into a robust solution for web application testing. While
>> selenium is great for the actual testing, I'm thinking of a scheduler
>> as the final piece in the pipeline. Let's say I have 4 websites that I
>> need to test periodically, A, B, C, D. I'd like to be able to define
>> things like "run the tests for site A once a day" or "run the tests
>> for site B twice a day indefinitely" or "run the tests for site C
>> twice a week between now and 2 months from now" or "run the tests for
>> site C once a week between May 25 and June 21".
>>
>> What would be the right scheduling tool for this? I know about the
>> standard tools like cron, sched, etc, and first even wrote one myself
>> (which became of course unmanagable after a short while :)) but was
>> hoping a more sophisticated tool is available.
>
> I'm not sure what you're testing here, so I can't advise on specifics.
> If you're testing your application code, it shouldn't need any
> periodic testing at all, but if you're verifying an active database,
> it may not be necessary to involve your application. Actually, I tend
> never to verify database structures; anything that I would consider
> testing can get coded as a constraint, so it's enforced by the
> database before anything gets committed.

I'd like to test the application code. I do frequent live updates
which are mostly tested, but sometimes things go through which are
buggy and I don't detect them before pushing the change. I know, this
is bad practice, I should do my thorough testing first before going
live, but in my setup occasional bugs are not a big deal. So I find it
convenient to just push changes live fast before complicated and time
consuming testing (I do some tests of course but quick and dirty ones
only) and I'd like to catch all remaining bugs in an automated way by
periodically testing the live web application. Again, generally I
agree that this is bad practice, but my setup is an outlier and
certainly not generic (non-critical in house app with < 10 users).

> But if you really do need things on a scheduler, I would advise using
> your OS's facilities (cron, or equivalent). No need to reinvent the
> wheel, unless you want it to do something different. I've written
> several simple task schedulers, but always because they do something
> fundamentally different from a basic one (like my "Let Me Know", which
> checks my calendar and shows me a tick-down until the next significant
> event - 27 hours until I host Office Hours, at the moment). To simply
> invoke a program every 4:00 UTC, use cron and save yourself the
> trouble.

I might do this in the end and use cron indeed.

Cheers,
Daniel


> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown


From antoon.pardon at rece.vub.ac.be  Tue May 12 08:31:57 2015
From: antoon.pardon at rece.vub.ac.be (Antoon Pardon)
Date: Tue, 12 May 2015 14:31:57 +0200
Subject: anomaly
In-Reply-To: <5550ce68$0$12990$c3e8da3$5496439d@news.astraweb.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <87fv73iavz.fsf@elektro.pacujo.net>
 <mailman.362.1431350872.12865.python-list@python.org>
 <5550ce68$0$12990$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5551F2BD.3040802@rece.vub.ac.be>

Op 11-05-15 om 17:44 schreef Steven D'Aprano:
> On Mon, 11 May 2015 11:27 pm, Antoon Pardon wrote:
>
>> The point is that all too often someone wants to defend a specific choice
>> the developers have made and cites some general rule or principle in
>> support, ignoring the fact that python breaks that rule/principle in other
>> area's.
>
> "It's a free country, you can do what you like."
>
> "No I can't, I'm not allowed to kill people."
>
> "Um, okay."
>
>
> Just because there are exceptions to a rule doesn't mean it isn't a general
> rule. A few exceptions are just exceptions, they don't invalidate the fact
> that "consenting adults" is a basic design principle of Python.

There is a difference between a basic design principle and a justification
for a particular instance. Answering the latter with the first as if the
first is all there is to it, is being simplistic.

Especially when there are multiple general rules that tend to conflict
with each other and more than once people just pick a rule that supports
them and ignore the rest.

-- 
Antoon Pardon



From jimfreeze at gmail.com  Tue May 12 09:00:10 2015
From: jimfreeze at gmail.com (Jim Freeze)
Date: Tue, 12 May 2015 08:00:10 -0500
Subject: [ANN] OpenStack Collective Conference in Austin, TX, Aug 13-14, 2015
Message-ID: <CAP1mN_xhnf+g5XY_x26hNHLUmrjXOK2NtpLfp7sdShhMdyCJ_g@mail.gmail.com>

OpenStack Collective Conference (OSCC) is a two day, single track
conference that will take place in exciting Austin, Texas on August 13-14,
2015.

https://openstackcollective.com/

OpenStack Collective caters to Developers and Operators and will provide
attendees with two days of learning and community fun, a hackspace, access
to vendors, and opportunities to network with the best and brightest in the
OpenStack community.

OSCC is an OpenStack Dev Days event, but provides the best creature
comforts of any conference you will attend. The conference includes
breakfast, lunch, morning and afternoon snacks, and coffee and soda options
all day.

We look forward to making this an event that you will want to return to
again and again as an opportunity to learn more and solidify your work
between the OpenStack Design Summits.

We invite you to visit Austin this coming August to reunite with your
OpenStack community.

Proposals are being accepted now
https://openstackcollective.com/proposals/new

Super Early Bird Tickets are avaialble now
https://openstackcollective.com/register

Stay tuned by following us @oscconf on twitter

Contact us at organizers at openstackcollective.com

-- 
Dr. Jim Freeze, Ph.D.
OpenStack Collective Team Member
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150512/b258e7a4/attachment.html>

From cristiano.cortezia at gmail.com  Tue May 12 09:05:30 2015
From: cristiano.cortezia at gmail.com (Cristiano Cortezia)
Date: Tue, 12 May 2015 06:05:30 -0700 (PDT)
Subject: ANN: eGenix PyRun - One file Python Runtime 2.1.0
In-Reply-To: <mailman.355.1431345550.12865.python-list@python.org>
References: <mailman.355.1431345550.12865.python-list@python.org>
Message-ID: <d7783c66-6680-4771-8ebd-cdc966b4622c@googlegroups.com>

On Monday, May 11, 2015 at 8:59:22 AM UTC-3, eGenix Team: M.-A. Lemburg wrote:
> ________________________________________________________________________
> 
> ANNOUNCING
> 
>                  eGenix PyRun - One file Python Runtime
> 
>                             Version 2.1.0
> 
> 
>          An easy-to-use single file relocatable Python run-time -
>            available for Linux, Mac OS X and Unix platforms,
>                  with support for Python 2.6, 2.7 and
>                        **also for Python 3.4**.
> 
> 
> This announcement is also available on our web-site for online reading:
> http://www.egenix.com/company/news/eGenix-PyRun-2.1.0-GA.html
> 
> ________________________________________________________________________
> 
> INTRODUCTION
> 
> eGenix PyRun is our open source, one file, no installation version of
> Python, making the distribution of a Python interpreter to run based
> scripts and applications to Unix based systems as simple as copying a
> single file.
> 
> eGenix PyRun's executable only needs 11MB for Python 2 and 13MB for
> Python 3, but still supports most Python application and scripts - and
> it can be compressed to just 3-4MB using upx, if needed.
> 
> Compared to a regular Python installation of typically 100MB on disk,
> eGenix PyRun is ideal for applications and scripts that need to be
> distributed to several target machines, client installations or
> customers.
> 
> It makes "installing" Python on a Unix based system as simple as
> copying a single file.
> 
> eGenix has been using eGenix PyRun internally in the mxODBC Connect
> Server product since 2008 with great success and decided to make it
> available as a stand-alone open-source product.
> 
> We provide both the source archive to build your own eGenix PyRun, as
> well as pre-compiled binaries for Linux, FreeBSD and Mac OS X, as 32-
> and 64-bit versions. The binaries can be downloaded manually, or you
> can let our automatic install script install-pyrun take care of the
> installation: ./install-pyrun dir and you're done.
> 
> Please see the product page for more details:
> 
>     http://www.egenix.com/products/python/PyRun/
> 
> ________________________________________________________________________
> 
> NEWS
> 
> This major new release of eGenix PyRun 2.1 comes with the following new features and changes:
> 
> New Features
> ------------
> 
>  * Upgraded eGenix PyRun to work with and use Python 2.7.9 per
>    default.
> 
>  * Upgraded eGenix PyRun to work with and use Python 3.4.3 for Python
>    3 support.
> 
>  * Added support for setting rpath on the PyRun binary to make it easy
>    to ship external libraries together with PyRun, without having to
>    set the LD_LIBRARY_PATH variable.
> 
>  * Added special support for egenix-pyopenssl to load its OpenSSL libs
>    during startup.
> 
>  * PyRun allows to disable the new default HTTPS certificate
>    verification (PEP 476) by setting the env var PYRUN_HTTPSVERIFY to
>    0. Default is to apply certificate verification.
> 
>  * Add support for more Python command line options: -s and -3 are
>    ignored, -R raises an error explaining to use the PYTHONHASHSEED
>    env var instead, -B prevents writing of byte code files.This should
>    increase compatibility of PyRun with the standard Python command
>    line interface.
> 
> Enhancements / Changes
> ----------------------
> 
>  * Added patch for Python 2.7.9 and 3.4.3 to make ctypes work again
>    after changes to those versions of Python. See
>    https://bugs.python.org/issue23042 for details.
> 
>  * The PyRun -v command line switch will now be passed to Python and
>    supports setting the level using e.g. -vv for a verbose 2 level.
> 
>  * Disabled user site configurations in PyRun, since these are not
>    needed or wanted for typical PyRun uses and cost startup time.
> 
>  * Optimized sys.path setup a bit. PyRun won't check for a Python
>    build run and will not check lib/site-python on startup anymore.
> 
>  * PyRun for Python 3.4 is no longer compiled to always run in
>    optimized mode. This allows using asserts in tests again.
> 
> Fixes
> -----
> 
>  * Entering "license" in the interactive shell now returns the correct
>    URL for all supported Python versions.
> 
>  * Tilde expansion now works for most arguments of install-pyrun. This
>    wasn't working before due to a bug.
> 
> install-pyrun Quick Install Enhancements
> ---------------------------------------------
> 
> eGenix PyRun includes a shell script called install-pyrun, which
> greatly simplifies installation of PyRun. It works much like the
> virtualenv shell script used for creating new virtual environments
> (except that there's nothing virtual about PyRun environments).
> 
>     https://downloads.egenix.com/python/install-pyrun
> 
> With the script, an eGenix PyRun installation is as simple as running:
> 
>     ./install-pyrun targetdir
> 
> This will automatically detect the platform, download and install the
> right pyrun version into targetdir.
> 
> We have updated this script since the last release:
> 
>  * Updated install-pyrun to default to eGenix PyRun 2.1.0 and its
>    feature set.
> 
>  * Added -r/--requirements option which allows automatically
>    installing a set of required packages using a requirements file, so
>    you can easily set up a complete Python environment using a single
> 
>    ./install-pyrun -r requirements.txt targetdir
> 
>  * Updated install-pyrun to install pyrun 2.1.0, setuptools 15.2 and
>    pip 1.5.6 per default.
> 
> For a complete list of changes, please see the eGenix PyRun Changelog:
> 
>     http://www.egenix.com/products/python/PyRun/changelog.html
> 
> ________________________________________________________________________
> 
> LICENSE
> 
> eGenix PyRun is distributed under the eGenix.com Public License 1.1.0
> which is an Open Source license similar to the Python license. You can
> use eGenix PyRun in both commercial and non-commercial settings
> without fee or charge.
> 
> Please see our license page for more details:
> 
>     http://www.egenix.com/products/python/PyRun/license.html
> 
> The package comes with full source code.
> 
> ________________________________________________________________________
> 
> DOWNLOADS
> 
> The download archives and instructions for installing eGenix PyRun can
> be found at:
> 
>     http://www.egenix.com/products/python/PyRun/
> 
> As always, we are providing pre-built binaries for all common
> platforms: Windows 32/64-bit, Linux 32/64-bit, FreeBSD 32/64-bit, Mac
> OS X 32/64-bit. Source code archives are available for installation on
> other platforms, such as Solaris, AIX, HP-UX, etc.
> 
> _______________________________________________________________________
> 
> SUPPORT
> 
> Commercial support for this product is available from eGenix.com.
> Please see
> 
>     http://www.egenix.com/services/support/
> 
> for details about our support offerings.
> 
> ________________________________________________________________________
> 
> MORE INFORMATION
> 
> For more information about eGenix PyRun, licensing and download
> instructions, please visit our web-site:
> 
>     http://www.egenix.com/products/python/PyRun/
> 
> About Python (http://www.python.org/):
> 
>     Python is an object-oriented Open Source programming language
>     which runs on all modern platforms. By integrating ease-of-use,
>     clarity in coding, enterprise application connectivity and rapid
>     application design, Python establishes an ideal programming
>     platform for today's IT challenges.
> 
> About eGenix (http://www.egenix.com/):
> 
>     eGenix is a software project, consulting and product company
>     specializing in expert project services and professional quality
>     products for companies, Python users and developers.
> 
> Enjoy,
> -- 
> Marc-Andre Lemburg
> eGenix.com
> 
> Professional Python Services directly from the Source  (#1, May 11 2015)
> >>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
> >>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> 
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/

Really interesting. 

Should we consider this to be an improvement towards the slow startup problem ? 

It seems to have really decreased the number of failed attempt to open files (ENOENT's appearing with strace upon startup) due to the interpreter default behavior for module loading on startup. (with a simple hello world I got 72 ENOENT's against 307 from default python installation on my standard ubuntu machine).

This problem mainly shows up when slow access storages are used (say raspberry + sdcard). Any chances you can provide this tool as prebuilt arm binaries ?

My assumptions may be all wrong, but anyway thanks for sharing your work. :)


From skip.montanaro at gmail.com  Tue May 12 09:16:48 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 12 May 2015 08:16:48 -0500
Subject: Running cool and silent
In-Reply-To: <d5b29c73-b44d-4437-ad3b-6832164ed550@googlegroups.com>
References: <d5b29c73-b44d-4437-ad3b-6832164ed550@googlegroups.com>
Message-ID: <CANc-5UxJ_TtSu44bkd1RKVp=sMXnwnQUFYCpCVQhtugtKwZYfQ@mail.gmail.com>

On Tue, May 12, 2015 at 4:33 AM, Rustom Mody <rustompmody at gmail.com> wrote:
> $ echo "min_power" | sudo tee /sys/class/scsi_host/host*/link_power_management_policy
>
> makes the fan slow/stop.
>
> But I am not sure what it does!!

My guess is it lowers the clock speed. To bring this into the realm of
Python, here's something you might try:

python -m test.pystone
echo "min_power" | sudo tee
/sys/class/scsi_host/host*/link_power_management_policy
python -m test.pystone

My guess is that you will see the pystone value reduced after issuing
the min_power command.

Skip


From chrismeek4542 at gmail.com  Tue May 12 09:41:44 2015
From: chrismeek4542 at gmail.com (chrismeek4542 at gmail.com)
Date: Tue, 12 May 2015 06:41:44 -0700 (PDT)
Subject: MySQL connection over SSH
Message-ID: <3567e1cb-b23c-4d0b-a752-8edb4363bf14@googlegroups.com>

I am trying to connect to a remote MySQL Database over SSH. I am using paramiko and SQLAlchemy. I assume that my sqlalchemy engine is not going through the SSH tunnel. Here is what i have so far. Not sure where to go from here though.

import paramiko
from sqlalchemy import create_engine

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('host.com', port=port, username='usr', password='pass')

engine = create_engine('mysql+mysqldb://pass at host.com:3306/db')

I am getting this error:



From chrismeek4542 at gmail.com  Tue May 12 09:44:39 2015
From: chrismeek4542 at gmail.com (chrismeek4542 at gmail.com)
Date: Tue, 12 May 2015 06:44:39 -0700 (PDT)
Subject: MySQL connection over SSH
In-Reply-To: <3567e1cb-b23c-4d0b-a752-8edb4363bf14@googlegroups.com>
References: <3567e1cb-b23c-4d0b-a752-8edb4363bf14@googlegroups.com>
Message-ID: <e92927a3-b7c2-4939-94bf-310be298be21@googlegroups.com>

(2003, "Can't connect to MySQL server on 'mcsdev.croft-it.com' (60)")




From steve+comp.lang.python at pearwood.info  Tue May 12 09:56:20 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 12 May 2015 23:56:20 +1000
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
Message-ID: <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>

On Tue, 12 May 2015 09:55 pm, Antoon Pardon wrote:

> Op 11-05-15 om 16:13 schreef Chris Angelico:
> 
>> Why does Python have most built-ins as simply looked-up names that can
>> be overridden? Because otherwise, there would be a veritable ton of
>> keywords:
> 
> But that doesn't answer the question why the developers chose "True" to be
> a keyword and "int" to be a looked-up name.

No it doesn't. But then, nobody until now has *asked* that question, merely
commented on it as a fact.

If you go all the way back to Python 1.5, not only did True and False not
exist as built-ins, but None was not a keyword:


[steve at ando ~]$ python1.5
Python 1.5.2 (#1, Aug 27 2012, 09:09:18)  [GCC 4.1.2 20080704 (Red Hat
4.1.2-52)] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> None = 23
>>> [].sort() is None
0
>>> print None
23


The consensus among the core developers is:

* in general, the harm and inconvenience from accidentally
  shadowing built-ins is not great, and it usually easy to
  spot, debug and prevent;

* when it comes to built-in functions (e.g. sum, map, pow)
  and types (e.g. int, str, list) there are significant and
  important use-cases for allowing shadowing;

  (e.g. the built-in sum function shouldn't prevent other
  modules from providing their own sum function)

* but when it comes to None, True and False, there are no
  significant or important use-cases for shadowing (it is 
  almost always a bug, not a feature, to redefine None).

The general principle here is of consenting adults: Python allows you to
shadow built-ins because sometimes it is useful, and the good outweighs the
potential harm. There are a handful of exceptions to that general principle
(e.g. None, True, False, I can't think of any others) because in those
cases the harm (as tiny as it is) outweighs the good.



> and pretending to justify that choice by stating that the python thought
> is: We're all adults here, if you want to override a builtin, who are we
> to stop you. That is disingenuous.

No, it's quite sensible, given Python's stated position: it's not an
authoritarian B&D language like Pascal, nor is it a permissive "anything
goes" language like C. It takes a middle ground, protecting you from the
worst consequences (no seg faults), but otherwise you're trusted to look
after yourself.

To give an analogy, you can consent to having a boxing match, with rules, a
referee, and clear limits on what force is permissible, but you cannot
legally consent to a duel to the death.



-- 
Steven



From dreamingforward at gmail.com  Tue May 12 11:11:25 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 12 May 2015 08:11:25 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>

On Monday, May 11, 2015 at 9:04:24 PM UTC-5, Steven D'Aprano wrote:
> On Tue, 12 May 2015 05:01 am, beliavsky at aol.com wrote:
> 
> > Yale has taken the unusual step of outsourcing its introductory CS class
> > to Harvard, which uses C as the main language in its CS50 class.
> 
> And another generation of new programmers will be irreversibly damaged by
> exposure to C...

Come on, C is perfect for computer engineering students.  For CS students, it's of mixed value.


From dreamingforward at gmail.com  Tue May 12 11:18:00 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 12 May 2015 08:18:00 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
Message-ID: <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>

On Tuesday, May 12, 2015 at 4:16:31 AM UTC-5, Rustom Mody wrote:
> On Tuesday, May 12, 2015 at 12:27:44 PM UTC+5:30, Chris Angelico wrote:
> > On Tue, May 12, 2015 at 4:42 PM, Rustom Mody  wrote:
> > > And related to that (and one reason a pure functional language is good for
> > > pedagogy): NO PRINT statement
> > > It may seem trivial but beginning students have a real hard writing clean
> > > structured code.  Tabooing prints helps get there faster
> > > And working in the interpreter makes a print-taboo a viable option
> > 
> > I firmly disagree.
> 
> Yes we know that!
> 
> As it happens you also disagree with ACM's latest CS curriculum:

I/O is an essential part of computing in the West.  (I'll leave Symbolics out of of that category.)  It started with switches and lights, so what kind of bullshit is saying that you should get rid of PRINT?  ACM must have gotten confused or steamrolled by one of its members.

mark

Mark


From dreamingforward at gmail.com  Tue May 12 11:34:14 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 12 May 2015 08:34:14 -0700 (PDT)
Subject: anomaly
In-Reply-To: <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
 <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <369b3ce9-14c9-465e-afea-407058a1617a@googlegroups.com>

On Tuesday, May 12, 2015 at 8:56:32 AM UTC-5, Steven D'Aprano wrote:
> The consensus among the core developers is:
> * in general, the harm and inconvenience from accidentally
>   shadowing built-ins is not great, and it usually easy to
>   spot, debug and prevent;

Where is that the consensus?  Please cite examples.

> * when it comes to built-in functions (e.g. sum, map, pow)
>   and types (e.g. int, str, list) there are significant and
>   important use-cases for allowing shadowing;

Name one "significant and important" use case for shadowing built-in types.  Functions, I don't have a problem with, but types are more fundamental than functions.

> The general principle here is of consenting adults: Python allows you to
> shadow built-ins because sometimes it is useful, and the good outweighs the
> potential harm. 

I think you'll have to give examples, either from the developer community or some significant use cases to be believable.

Mark


From invalid at invalid.invalid  Tue May 12 11:36:17 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Tue, 12 May 2015 15:36:17 +0000 (UTC)
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <CAMjeLr-_jd2wAJJTZRO3+p58hzD+B3UiDyrkhz-Vw-=Lopmrmg@mail.gmail.com>
 <CAMjeLr9v3vVHmyThb=uTNTG6XJcWmebZE35=cmFognfsKj6-Sg@mail.gmail.com>
 <CAMjeLr-67ZJNF3N5nWuDNGRWjOY_B41++737TxC4V6DYFH1t5Q@mail.gmail.com>
 <mailman.330.1431303600.12865.python-list@python.org>
 <6f508ac2-0765-46b2-9408-955e7c811127@googlegroups.com>
 <55504803$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <a93604dd-74d6-42a8-b197-406e8fa15467@googlegroups.com>
 <mailman.367.1431355924.12865.python-list@python.org>
 <miqi5k$id1$1@reader1.panix.com>
 <03e4e251-20b0-4a2f-968e-2d40d2b5dc1e@googlegroups.com>
 <mis1e9$oq2$1@dont-email.me>
Message-ID: <mit6lg$jp8$1@reader1.panix.com>

On 2015-05-12, alex23 <wuwei23 at gmail.com> wrote:
> On 12/05/2015 1:39 AM, zipher wrote:
>> On Monday, May 11, 2015 at 10:34:24 AM UTC-5, Grant Edwards wrote:
>>> That Python, like COBOL, is an eminently practical language.
>>
>> LOL!  Good one.
>
> I would make an incredibly substantial wager that you've never developed 
> anything of note in either Python or COBOL.

I've write a lot of Python code over the past 15 years.  I only made
one attempt at a COBOL program.  It was using punch cards and a
card-reader connected to a remote mainframe via a 2400 baud link. That
was not fun.

I'm not sure what your point is, though.

COBOL may be an ugly, icky, language, but you must admit it is
_practical_.  It can be used to get real work done.

-- 
Grant Edwards               grant.b.edwards        Yow! Where does it go when
                                  at               you flush?
                              gmail.com            


From thierry.gayet at gmail.com  Tue May 12 11:39:49 2015
From: thierry.gayet at gmail.com (Thierry GAYET)
Date: Tue, 12 May 2015 08:39:49 -0700 (PDT)
Subject: problem while using os.walk with utf-8 characters
Message-ID: <ebf2c547-1ceb-40b0-92f1-32913ec14317@googlegroups.com>

Hi,

I'm using the os.walk function for parsing files from an external mass-storage such as usbkey.

When i try the following code, i have an error:

from __future__ import unicode_literals

import sys
reload(sys)
sys.setdefaultencoding('utf_8')

for dirname, dirnames, filenames in os.walk(os.path.join(massStorage, path)):
or
for dirname, dirnames, filenames in os.walk(unicode(path, 'utf-8')

And my error is:
'ascii' codec can't encode character u'\xe9' in position 58: ordinal not in range(128)

The wrong character is an accent (french one).

It doesn't work when i start it but after a reboot i need to restart the code to make it work properly without any error.

Any idea about the first fact related to the usf-8 support with external files that can be encoded with any charset (latin-1, utf-8, ... )

Secondly, why it can works after a restart of the python script ?

BR

Thierry GAYET





From rosuav at gmail.com  Tue May 12 11:43:26 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 13 May 2015 01:43:26 +1000
Subject: anomaly
In-Reply-To: <369b3ce9-14c9-465e-afea-407058a1617a@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
 <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
 <369b3ce9-14c9-465e-afea-407058a1617a@googlegroups.com>
Message-ID: <CAPTjJmot8Uu_vNF_j5tBzCi4vvYDHJjPVFVegUUMhhLNkcUm1Q@mail.gmail.com>

On Wed, May 13, 2015 at 1:34 AM, zipher <dreamingforward at gmail.com> wrote:
> On Tuesday, May 12, 2015 at 8:56:32 AM UTC-5, Steven D'Aprano wrote:
>> * when it comes to built-in functions (e.g. sum, map, pow)
>>   and types (e.g. int, str, list) there are significant and
>>   important use-cases for allowing shadowing;
>
> Name one "significant and important" use case for shadowing built-in types.  Functions, I don't have a problem with, but types are more fundamental than functions.
>

Please tell me what, precisely, is the difference between a type and a
function. Once you've settled that, please explain to me what the
built-in name 'int' is in all versions of Python.

ChrisA


From tjreedy at udel.edu  Tue May 12 11:45:31 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 May 2015 11:45:31 -0400
Subject: How to properly apply OOP in the bouncing ball code
In-Reply-To: <a25437cb-c525-41db-9bd4-c9759a982a3b@googlegroups.com>
References: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>
 <45313faa-d256-4fa4-9e37-4b8dd85e1681@googlegroups.com>
 <mailman.383.1431390292.12865.python-list@python.org>
 <a25437cb-c525-41db-9bd4-c9759a982a3b@googlegroups.com>
Message-ID: <mit772$p5g$2@ger.gmane.org>

On 5/11/2015 8:42 PM, zipher wrote:
> On Monday, May 11, 2015 at 7:25:09 PM UTC-5, Dennis Lee Bieber wrote:
>> On Mon, 11 May 2015 08:33:56 -0700 (PDT), zipher
>> <dreamingforward at gmail.com> declaimed the following:
>>> You are making a error that few in the programming community have caught up to.  OOP design for *data abstraction* is a completely different beast that OOP for *simulation*.  The confusion is around the use of the word "object" which both denotes a physical world item and a virtual one detached from reality.
>>>
>>> I would say that Python might not be the right fit, but instead a language dedicated to simulation.
>>
>> 	The danger there is that a "language dedicated to simulation" might
>> mean "discrete event" simulation -- which would be an even worse fit to a
>> problem of particle motion simulation.
>
> Huh?  VPython successfully models particle motion simulation with discrete events.  I don't see how you're going to update a variable non-discretely.  Tensors?  Good luck.
>
> Mark
>


-- 
Terry Jan Reedy



From breamoreboy at yahoo.co.uk  Tue May 12 12:04:01 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 12 May 2015 17:04:01 +0100
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
Message-ID: <mit89i$dk7$1@ger.gmane.org>

On 12/05/2015 07:42, Rustom Mody wrote:
>
> And related to that (and one reason a pure functional language is good for
> pedagogy): NO PRINT statement
> It may seem trivial but beginning students have a real hard writing clean
> structured code.  Tabooing prints helps get there faster
> And working in the interpreter makes a print-taboo a viable option
>

If students can't be taught to distinguish print from other parts of the 
language they should get themselves onto a course teaching semi-skilled 
light bulb fitting.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Tue May 12 12:05:37 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 13 May 2015 02:05:37 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <PRINT-20150512175315@ram.dialup.fu-berlin.de>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
 <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
 <PRINT-20150512175315@ram.dialup.fu-berlin.de>
Message-ID: <CAPTjJmo6oJ19AkK8fU8txgbAD7LChC93pUqKu8KUvkRHG-iymw@mail.gmail.com>

On Wed, May 13, 2015 at 1:53 AM, Stefan Ram <ram at zedat.fu-berlin.de> wrote:
> zipher <dreamingforward at gmail.com> writes:
>>so what kind of bullshit is saying that you should get rid of PRINT?
>
>   What might be reasonable is to be able to dissect a program
>   into functions, and have no effects in functions that are
>   there to calculate some value.
>
>   For example, when one calculates,
>
> y = sin( x )
>
>   , using a library function ?sin?, one does not want ?sin? to
>   print something.
>
>   Such library functions might even be used in code for a GUI
>   that has no console to print to at all.
>
>   In order to learn how to write such functions, one might
>   temporarily use the rule that PRINT is allowed only in the
>   main block (not in functions) or temporarily for debugging.

More generally, there are certain things which belong to the
application, and not to any library - things like:

* The console - print() and input(), and GUI windows
* Command-line arguments
* Program termination
* Working directory and root directory
* Credentials (uid/gid, etc)
* Logging configuration (destinations etc)

There are exceptions; argument parsing libraries often are granted
control over the console and program termination in addition to
reading sys.argv, but you would be somewhat surprised if having
"--no-check-ssl-certs" in sys.argv silently downgraded your HTTP
library's security settings. Control over application-wide settings
generally has to be in the hands of the application, not any one
library.

So if you're writing a library function, it probably shouldn't use
print()... but your application is most welcome to. You usually know
which one you're writing at any given time.

ChrisA


From gherron at digipen.edu  Tue May 12 12:07:29 2015
From: gherron at digipen.edu (Gary Herron)
Date: Tue, 12 May 2015 09:07:29 -0700
Subject: anomaly
In-Reply-To: <5551EA40.2000606@rece.vub.ac.be>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <5551EA40.2000606@rece.vub.ac.be>
Message-ID: <55522541.3020904@digipen.edu>

On 05/12/2015 04:55 AM, Antoon Pardon wrote:
> Op 11-05-15 om 16:13 schreef Chris Angelico:
>
>> Why does Python have most built-ins as simply looked-up names that can
>> be overridden? Because otherwise, there would be a veritable ton of
>> keywords:
> But that doesn't answer the question why the developers chose "True" to be a
> keyword and "int" to be a looked-up name.
>
> and pretending to justify that choice by stating that the python thought
> is: We're all adults here, if you want to override a builtin, who are we
> to stop you. That is disingenuous.
>

Bull.   Some design decisions were made with the knowledge that

  * they provide a freedom which may be useful but can be misused (e.g.,
    shadowing builtins), versus
  * they would be too disruptive of abusable (e.g. shadowing keywords)

Python tends to use the first category more than C family languages, and 
that's where the "We're all adults" argument applies.  You may argue 
about which category any particular feature ought to fall into, and in 
fact several things (shadowing None, True, and False) have changed 
category during the evolution of Python.  But to imply that the "adult" 
argument should drive *all* decisions is foolish. And disingenuous.

-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150512/99d6941e/attachment.html>

From breamoreboy at yahoo.co.uk  Tue May 12 12:19:19 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 12 May 2015 17:19:19 +0100
Subject: anomaly
In-Reply-To: <CAPTjJmot8Uu_vNF_j5tBzCi4vvYDHJjPVFVegUUMhhLNkcUm1Q@mail.gmail.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
 <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
 <369b3ce9-14c9-465e-afea-407058a1617a@googlegroups.com>
 <CAPTjJmot8Uu_vNF_j5tBzCi4vvYDHJjPVFVegUUMhhLNkcUm1Q@mail.gmail.com>
Message-ID: <mit968$v5k$1@ger.gmane.org>

On 12/05/2015 16:43, Chris Angelico wrote:
> On Wed, May 13, 2015 at 1:34 AM, zipher <dreamingforward at gmail.com> wrote:
>> On Tuesday, May 12, 2015 at 8:56:32 AM UTC-5, Steven D'Aprano wrote:
>>> * when it comes to built-in functions (e.g. sum, map, pow)
>>>    and types (e.g. int, str, list) there are significant and
>>>    important use-cases for allowing shadowing;
>>
>> Name one "significant and important" use case for shadowing built-in types.  Functions, I don't have a problem with, but types are more fundamental than functions.
>>
>
> Please tell me what, precisely, is the difference between a type and a
> function. Once you've settled that, please explain to me what the
> built-in name 'int' is in all versions of Python.
>
> ChrisA
>

Do we really have to feed this guy, he's worse than the RUE?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From skip.montanaro at gmail.com  Tue May 12 12:22:01 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 12 May 2015 11:22:01 -0500
Subject: anomaly
In-Reply-To: <369b3ce9-14c9-465e-afea-407058a1617a@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
 <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
 <369b3ce9-14c9-465e-afea-407058a1617a@googlegroups.com>
Message-ID: <CANc-5UzKBpwPo1V7c_YW_tOgzno2jLh03FwbnHe7B4Di3dMpXw@mail.gmail.com>

On Tue, May 12, 2015 at 10:34 AM, zipher <dreamingforward at gmail.com> wrote:
>> The general principle here is of consenting adults: Python allows you to
>> shadow built-ins because sometimes it is useful, and the good outweighs the
>> potential harm.
>
> I think you'll have to give examples, either from the developer community or some significant use cases to be believable.

http://docs.python-guide.org/en/latest/writing/style/#we-are-all-responsible-users
http://stackoverflow.com/questions/14168791/actual-implementation-of-private-variables-in-python-class

I'll leave it for you to rummage around in old Python posts for actual
posts by Guido where he has stated, "we're all adults here".

I did find this interesting blog post about encapsulation and
test-driven development:

https://jasonmbaker.wordpress.com/2009/01/08/enemies-of-test-driven-development-part-i-encapsulation/

I found the author's perspective interesting.

S


From oscar.j.benjamin at gmail.com  Tue May 12 12:53:23 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 12 May 2015 17:53:23 +0100
Subject: How to properly apply OOP in the bouncing ball code
In-Reply-To: <e841eeeb-7a36-47fe-bd21-bfd7b53a9588@googlegroups.com>
References: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>
 <e841eeeb-7a36-47fe-bd21-bfd7b53a9588@googlegroups.com>
Message-ID: <CAHVvXxRbNcuW6Cx7oVnP4rVmv6vQG7UpQguwWdvJH=rQoaR+kg@mail.gmail.com>

On 11 May 2015 at 16:22, Tommy C <tommyc168168 at gmail.com> wrote:
> Thanks for your help.
>
> I have updated the code as follows, there are no more errors but the images will not move at all, as all the images are staying at the upper left corner. Please advice, thanks.
>
>
> import sys, pygame
>
> pygame.init()
>
> size   = width,  height = 800, 600
> black  = [0,0,0]
> screen = pygame.display.set_mode(size)
>
> class BALL:
>     def __init__(self,image):
>         self.ball     = pygame.image.load(image)
>         self.ballrect = self.ball.get_rect()
>         self.speed    = [2, 2]
>
>     def control(self):
>         ballmove = self.ballrect.move(self.speed)
>
>         if ballmove.left < 0 or ballmove.right > width:
>             self.speed[0] = -self.speed[0]
>
>         if ballmove.top < 0  or ballmove.bottom > height:
>             self.speed[1] = -self.speed[1]

The function below should not be a method on a particular ball object
as it is the global event loop for pygame. This needs to be a top
level function that updates the animation for all balls.

>     def settings(self):
>         clock  = pygame.time.Clock()
>         screen.fill(black)
>         screen.blit(self.ball, self.ballrect)
>         pygame.display.flip()
>         clock.tick(60)
>         while 1:
>             for event in pygame.event.get():
>                 if event.type == pygame.QUIT: sys.exit()

In this loop you need to add code that will call the control() method
of each ball and then redraw the screen. At the moment you only draw
the objects once at the top of the settings() method. Then the loop
runs until you quit. So it should be something like:

# The functions below would make sense as methods of an "App" class.

def run():
    while 1:
        for event in pygame.event.get():
            if event.type = pygame.QUIT:
                sys.exit()
        update()
        draw()

def update():
    for ball in balls:
        ball.control()

def draw():
    screen.fill(black)
    for ball in balls:
        ball.draw()  # You need to add this method to BALL
    pygame.display.flip()

balls = [BALL("spongebob.png"), BALL("jaws.jpg")]

run()  # Actually runs the program's event loop.


--
Oscar


From rgaddi at technologyhighland.invalid  Tue May 12 13:03:45 2015
From: rgaddi at technologyhighland.invalid (Rob Gaddi)
Date: Tue, 12 May 2015 17:03:45 +0000 (UTC)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
Message-ID: <mitbph$2sq$3@dont-email.me>

On Tue, 12 May 2015 08:11:25 -0700, zipher wrote:

> On Monday, May 11, 2015 at 9:04:24 PM UTC-5, Steven D'Aprano wrote:
>> On Tue, 12 May 2015 05:01 am, beliavsky at aol.com wrote:
>> 
>> > Yale has taken the unusual step of outsourcing its introductory CS
>> > class to Harvard, which uses C as the main language in its CS50
>> > class.
>> 
>> And another generation of new programmers will be irreversibly damaged
>> by exposure to C...
> 
> Come on, C is perfect for computer engineering students.  For CS
> students, it's of mixed value.

And that's how you train CS students to write inefficient code that takes 
orders of magnitude longer to run than it should.

Is that a true array or a linked list?  "It's a high level language, 
that's just an implementation detail." Yes, but it's an implementation 
detail that determines whether even the simple act of looking up element 
n is O(1) or O(n).

C teaches you the language of the computer.  Understanding it allows you 
to grasp what your high-level code is actually doing, and why and when a 
list (array) is more efficient than a dict (hashtable).  Because you've 
written a dynamically resizing list, and learned the perils of having to 
realloc() as the size grows.  And you've written a hashtable, and 
understand the expense of the hashing function, and the tradeoffs between 
wasted memory and having to wade at O(n) pace through a linked list of 
collision candidates.

A firm grasp of C will make you a better programmer in any language, even 
if you haven't written a line of it in 20 years.  It's the ability to 
read a map.  A lack of C is the person blindly following their GPS and 
hoping for the best.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.


From rosuav at gmail.com  Tue May 12 13:15:49 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 13 May 2015 03:15:49 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mitbph$2sq$3@dont-email.me>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
Message-ID: <CAPTjJmpxDB5RUN-GCfWDLo9s=ui3X0Sw9h-KjvK-jkLrv9R8Ww@mail.gmail.com>

On Wed, May 13, 2015 at 3:03 AM, Rob Gaddi
<rgaddi at technologyhighland.invalid> wrote:
> A firm grasp of C will make you a better programmer in any language, even
> if you haven't written a line of it in 20 years.  It's the ability to
> read a map.  A lack of C is the person blindly following their GPS and
> hoping for the best.

That's as may be, but I would still not recommend it as a first
language. It's possible to explain algorithmic complexity with a high
level language; Python's data types generally have predictable costs
associated with them, although a lot of them aren't actually language
guarantees (I could imagine a Python implementation using an O(log n)
tree instead of an amortized O(1) hash table for its dict, if other
tradeoffs make it worthwhile); at any rate, you can always just
construct your own fundamental data structures if you want to teach
what a linked list is good for, or what a splay tree can do for you. C
knowledge is good, but first get to know the broader art of
programming, and then get to know C. Learning low-level languages too
soon will end up binding your mind to a particular CPU architecture.
That happened to me in a big way; I got to thinking about everything
in terms of how an 80x86 CPU would deal with it, without any
comprehension of some of the modern aspects of low-level programming
like pipeline management and cache locality. It was quite the
eye-opener when I started hand-optimizing my C compiler's output on a
more modern chip, and found that my improvements... uhh, added about
50% to the run time. That's when I stopped doing any assembly language
work, and just let the compiler do its magic :)

ChrisA


From rosuav at gmail.com  Tue May 12 13:26:29 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 13 May 2015 03:26:29 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <complexity-20150512191339@ram.dialup.fu-berlin.de>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
Message-ID: <CAPTjJmoaVbQNpmmY9ngzNMeJ_RoPFd8Q=0Xbuy8oNUX6AN_yjg@mail.gmail.com>

On Wed, May 13, 2015 at 3:15 AM, Stefan Ram <ram at zedat.fu-berlin.de> wrote:
> Rob Gaddi <rgaddi at technologyhighland.invalid> writes:
>>Is that a true array or a linked list?  "It's a high level language,
>>that's just an implementation detail." Yes, but it's an implementation
>>detail that determines whether even the simple act of looking up element
>>n is O(1) or O(n).
>
>   The complexity is given in the documentation of high-level languages.
>
>   For example, from the documentation of the Java standard library:
>
>       ?This implementation provides constant-time performance
>       for the basic operations (get and put),? (java.util.HashMap)
>
>   C++:
>
>       Section 23.2.1 specifies the complexity, such as ?compile time?,
>       ?constant?, ?linear? for container operations.
>
>   But the abstraction mechanisms (templates, polymorphism)
>   often allow one to change the implementation quite easily.

It isn't always given in the docs. Sometimes it's not even a promise;
back when MicroPython was debating the implementation of Unicode
strings, there was a lengthy discussion on python-dev about whether
it's okay for string subscripting to be O(n) instead of O(1), and the
final decision was that yes, that's an implementation detail. (UTF-8
internal string representation, so iterating over a string would still
yield characters in overall O(n), but iterating up to the string's
length and subscripting for each character would become O(n*n) on
uPy.)

ChrisA


From rustompmody at gmail.com  Tue May 12 13:35:38 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 10:35:38 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
 <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
Message-ID: <f84f7383-8fae-4b2d-b5d2-70285a46e584@googlegroups.com>

On Tuesday, May 12, 2015 at 8:48:13 PM UTC+5:30, zipher wrote:
> On Tuesday, May 12, 2015 at 4:16:31 AM UTC-5, Rustom Mody wrote:
> > On Tuesday, May 12, 2015 at 12:27:44 PM UTC+5:30, Chris Angelico wrote:
> > > On Tue, May 12, 2015 at 4:42 PM, Rustom Mody  wrote:
> > > > And related to that (and one reason a pure functional language is good for
> > > > pedagogy): NO PRINT statement
> > > > It may seem trivial but beginning students have a real hard writing clean
> > > > structured code.  Tabooing prints helps get there faster
> > > > And working in the interpreter makes a print-taboo a viable option
> > > 
> > > I firmly disagree.
> > 
> > Yes we know that!
> > 
> > As it happens you also disagree with ACM's latest CS curriculum:
> 
> I/O is an essential part of computing in the West.  (I'll leave Symbolics out of of that category.)  It started with switches and lights, so what kind of bullshit is saying that you should get rid of PRINT?  ACM must have gotten confused or steamrolled by one of its members.

In the West? WEST??
Did I hear that right?

My profound genuflections to the Mark-modified Michaelson-Morley result.


From rustompmody at gmail.com  Tue May 12 13:57:34 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 10:57:34 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <complexity-20150512191339@ram.dialup.fu-berlin.de>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
Message-ID: <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>

On Tuesday, May 12, 2015 at 10:45:39 PM UTC+5:30, Stefan Ram wrote:
> Rob Gaddi writes:
> >Is that a true array or a linked list?  "It's a high level language, 
> >that's just an implementation detail." Yes, but it's an implementation 
> >detail that determines whether even the simple act of looking up element 
> >n is O(1) or O(n).
> 
>   The complexity is given in the documentation of high-level languages.
> 
>   For example, from the documentation of the Java standard library:
> 
>       ?This implementation provides constant-time performance
>       for the basic operations (get and put),? (java.util.HashMap)
> 
>   C++:
> 
>       Section 23.2.1 specifies the complexity, such as ?compile time?,
>       ?constant?, ?linear? for container operations.
> 
>   But the abstraction mechanisms (templates, polymorphism)
>   often allow one to change the implementation quite easily.

This is regarded in some circles as one of the big open problems in CS
https://existentialtype.wordpress.com/2014/09/28/structure-and-efficiency-of-computer-programs/
[and the position paper therein]
viz how to integrate the elegance of high-level languages
with the precision of complexity analysis of machine language


From rgaddi at technologyhighland.invalid  Tue May 12 14:01:42 2015
From: rgaddi at technologyhighland.invalid (Rob Gaddi)
Date: Tue, 12 May 2015 18:01:42 +0000 (UTC)
Subject: Updating a package on PyPi, testing and etiquette
Message-ID: <mitf66$2sq$4@dont-email.me>

So I've got a package I put up on PyPi a while back (ctypes-bitfield, if 
it matters).  For version 0.2.6 I had access to some older versions of 
Python and was able to run my test suite on Python 2.6 and 3.0.

Well, I don't have them anymore.  I've got no access right now to 
anything older than 2.7 and 3.2, and my primary development environment 
is 3.4.  But I've also updated the package to support what I consider to 
be some really nice new functionality.  So, it comes down to two 
questions:

A) Is there any easy way to test against an older version of Python?  
Preferably without trying to install entire old environments and keep 
them nicely isolated from my actual doing work?  I'm running Ubuntu for 
what difference that makes.

B) If I can't manage that, what's the etiquette behind having later 
versions of a module break compatibility with older versions of Python.  
I've avoided using features I know are newer, like yield from and Enums, 
but I won't swear it'll work on 3.0 if I can't test it that way.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.


From ian.g.kelly at gmail.com  Tue May 12 14:07:47 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 12 May 2015 12:07:47 -0600
Subject: anomaly
In-Reply-To: <369b3ce9-14c9-465e-afea-407058a1617a@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
 <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
 <369b3ce9-14c9-465e-afea-407058a1617a@googlegroups.com>
Message-ID: <CALwzidn8KENRFKbx3=+o2j2e0fnuif73SOaSxox43i==d8e48g@mail.gmail.com>

On Tue, May 12, 2015 at 9:34 AM, zipher <dreamingforward at gmail.com> wrote:
>> * when it comes to built-in functions (e.g. sum, map, pow)
>>   and types (e.g. int, str, list) there are significant and
>>   important use-cases for allowing shadowing;
>
> Name one "significant and important" use case for shadowing built-in types.  Functions, I don't have a problem with, but types are more fundamental than functions.

try:
    str = unicode  # Use the Python 3 name.
except NameError:
    pass


From ned at nedbatchelder.com  Tue May 12 14:18:18 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 12 May 2015 11:18:18 -0700 (PDT)
Subject: Updating a package on PyPi, testing and etiquette
In-Reply-To: <mitf66$2sq$4@dont-email.me>
References: <mitf66$2sq$4@dont-email.me>
Message-ID: <0a7014a1-2320-4d22-ad14-c33adf755861@googlegroups.com>

On Tuesday, May 12, 2015 at 2:03:04 PM UTC-4, Rob Gaddi wrote:
> So I've got a package I put up on PyPi a while back (ctypes-bitfield, if 
> it matters).  For version 0.2.6 I had access to some older versions of 
> Python and was able to run my test suite on Python 2.6 and 3.0.
> 
> Well, I don't have them anymore.  I've got no access right now to 
> anything older than 2.7 and 3.2, and my primary development environment 
> is 3.4.  But I've also updated the package to support what I consider to 
> be some really nice new functionality.  So, it comes down to two 
> questions:
> 
> A) Is there any easy way to test against an older version of Python?  
> Preferably without trying to install entire old environments and keep 
> them nicely isolated from my actual doing work?  I'm running Ubuntu for 
> what difference that makes.

I've used pythonz (http://saghul.github.io/pythonz/) to install a
variety of Python versions.  It works really well for me.

> 
> B) If I can't manage that, what's the etiquette behind having later 
> versions of a module break compatibility with older versions of Python.  
> I've avoided using features I know are newer, like yield from and Enums, 
> but I won't swear it'll work on 3.0 if I can't test it that way.

No one supports 3.0 any more.  3.2 is kind of the oldest version
that is still reasonable, and many people are only aiming for 3.3
and 3.4.

Just because you are supporting 2.7 and 3.4 doesn't mean you need
to support 3.0 and 3.1.  2.x and 3.x are different languages. Just
as 2.4 is old and often not supported, 3.0 is old and often not
supported.

--Ned.


From breamoreboy at yahoo.co.uk  Tue May 12 14:22:33 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 12 May 2015 19:22:33 +0100
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <f84f7383-8fae-4b2d-b5d2-70285a46e584@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
 <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
 <f84f7383-8fae-4b2d-b5d2-70285a46e584@googlegroups.com>
Message-ID: <mitgda$9c5$1@ger.gmane.org>

On 12/05/2015 18:35, Rustom Mody wrote:
> On Tuesday, May 12, 2015 at 8:48:13 PM UTC+5:30, zipher wrote:
>>
>> I/O is an essential part of computing in the West.  (I'll leave Symbolics out of of that category.)  It started with switches and lights, so what kind of bullshit is saying that you should get rid of PRINT?  ACM must have gotten confused or steamrolled by one of its members.
>
> In the West? WEST??
> Did I hear that right?
>

I prefer this[1]

<quote>
Each object has to figure out how it will receive things from outside
of it.  Things it can't handle (a string sent to an int) just have to
be dropped to some other space, much like stderr does within the O.S.

There are probably many other very interesting examples, but the key
idea I'm working on (as noted in other messages), is a sort-of
universal language for the internet, a WebOS to be applied to a
universal data model.
</quote>

I'd guess that the concept is in the same boat as Python 2.8 or 
RickedPython, but I suspect that these two have far more chance of 
making it into the real world than "a sort-of universal language for the 
internet, a WebOS to be applied to a universal data model"

[1]https://mail.python.org/pipermail/python-ideas/2013-March/019979.html
-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From fomcl at yahoo.com  Tue May 12 14:36:19 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Tue, 12 May 2015 11:36:19 -0700
Subject: Updating a package on PyPi, testing and etiquette
Message-ID: <1431455779.60078.BPMail_high_carrier@web163806.mail.gq1.yahoo.com>


-----------------------------
On Tue, May 12, 2015 8:01 PM CEST Rob Gaddi wrote:

>So I've got a package I put up on PyPi a while back (ctypes-bitfield, if 
>it matters).  For version 0.2.6 I had access to some older versions of 
>Python and was able to run my test suite on Python 2.6 and 3.0.
>
>Well, I don't have them anymore.  I've got no access right now to 
>anything older than 2.7 and 3.2, and my primary development environment 
>is 3.4.  But I've also updated the package to support what I consider to 
>be some really nice new functionality.  So, it comes down to two 
>questions:
>
>A) Is there any easy way to test against an older version of Python?  
>Preferably without trying to install entire old environments and keep 
>them nicely isolated from my actual doing work?  I'm running Ubuntu for 
>what difference that makes.

I use tox, it's available on pypi. Works with virtualenv. There's also detox, which runs your unittests in parellel, but I have not used that.



From ned at nedbatchelder.com  Tue May 12 14:50:06 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 12 May 2015 11:50:06 -0700 (PDT)
Subject: Updating a package on PyPi, testing and etiquette
In-Reply-To: <mailman.409.1431456080.12865.python-list@python.org>
References: <mailman.409.1431456080.12865.python-list@python.org>
Message-ID: <33d9e2e7-1227-47ba-864e-c45898df9e57@googlegroups.com>

On Tuesday, May 12, 2015 at 2:41:51 PM UTC-4, Albert-Jan Roskam wrote:
> -----------------------------
> On Tue, May 12, 2015 8:01 PM CEST Rob Gaddi wrote:
> 
> >So I've got a package I put up on PyPi a while back (ctypes-bitfield, if 
> >it matters).  For version 0.2.6 I had access to some older versions of 
> >Python and was able to run my test suite on Python 2.6 and 3.0.
> >
> >Well, I don't have them anymore.  I've got no access right now to 
> >anything older than 2.7 and 3.2, and my primary development environment 
> >is 3.4.  But I've also updated the package to support what I consider to 
> >be some really nice new functionality.  So, it comes down to two 
> >questions:
> >
> >A) Is there any easy way to test against an older version of Python?  
> >Preferably without trying to install entire old environments and keep 
> >them nicely isolated from my actual doing work?  I'm running Ubuntu for 
> >what difference that makes.
> 
> I use tox, it's available on pypi. Works with virtualenv. There's also detox, which runs your unittests in parellel, but I have not used that.

Oh, yes, use tox.  I should have clarified: I use pythonz to
install different versions of Python, then tox to run my test
suite against a variety of those versions.

--Ned.


From zljubisicmob at gmail.com  Tue May 12 14:57:49 2015
From: zljubisicmob at gmail.com (zljubisicmob at gmail.com)
Date: Tue, 12 May 2015 11:57:49 -0700 (PDT)
Subject: SyntaxError: (unicode error) 'unicodeescape' codec can't decode
 bytes in position 2-3: truncated \UXXXXXXXX escape
In-Reply-To: <mailman.332.1431308009.12865.python-list@python.org>
References: <f4d02ce1-f528-4632-acb1-af667690a064@googlegroups.com>
 <mailman.260.1431113391.12865.python-list@python.org>
 <8360473a-45ac-4270-9bf3-81932da5f223@googlegroups.com>
 <554d768d$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <580ee0d6-a703-4da3-af2d-105589a1780f@googlegroups.com>
 <mailman.282.1431174340.12865.python-list@python.org>
 <d06f180f-bc6f-4610-b8cb-a6335ee5a2ce@googlegroups.com>
 <mailman.332.1431308009.12865.python-list@python.org>
Message-ID: <bff5e799-bde3-42fd-adf8-a9c55304999b@googlegroups.com>

I would say so as well.
Thanks to everyone who helped.

Regards and best wishes.


From zljubisicmob at gmail.com  Tue May 12 15:13:20 2015
From: zljubisicmob at gmail.com (zljubisicmob at gmail.com)
Date: Tue, 12 May 2015 12:13:20 -0700 (PDT)
Subject: Python file structure
Message-ID: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>

Hi, I have python file with the following structure:

import...

A = configparser.get(...) 
B = configparser.get(...)

Command line parameters parsing [they can change variable A or B]

Def usage()
	Print how to use script parameters

def main():
	...

if __name__ == "__main__":
    main()

If I find an error in command line parameters section I cannot call function usage() because it is not defined yet. 

I have few options here:
1.	Put definition of usage function before command line parameters parsing section
2.	Make parameters global and put them in the main function
3.	...maybe some other options...

The basic idea is that variables A and B should be accessible globally, but after they are read by configparser and/or changed by command line parameters parsing section, the variables should preserve their values throughout the whole program.

If I set them in the way described above, even the variables are not global I can read their values everywhere with no option to change them.
If I set them with global keyword I am risking changing their values afterwards. 

What approach do you suggest?
The goal is to have global variables that should preserve their values and at the same time no meter if they are defined by configparser and/or command line parameter to be able to print usage if there is a problem with a parameter?

Regards.


From rosuav at gmail.com  Tue May 12 15:29:48 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 13 May 2015 05:29:48 +1000
Subject: Python file structure
In-Reply-To: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
References: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
Message-ID: <CAPTjJmrtLoCGOAr+F1Q7zgzv3hUhvrVA3ADdaLe9mP9YSShi6Q@mail.gmail.com>

On Wed, May 13, 2015 at 5:13 AM,  <zljubisicmob at gmail.com> wrote:
> import...
>
> A = configparser.get(...)
> B = configparser.get(...)
>
> Command line parameters parsing [they can change variable A or B]
>
> Def usage()
>         Print how to use script parameters
>
> def main():
>         ...
>
> if __name__ == "__main__":
>     main()
>
> If I find an error in command line parameters section I cannot call function usage() because it is not defined yet.
>
> I have few options here:
> 1.      Put definition of usage function before command line parameters parsing section

I'd do this, unless there's a good reason not to. A simple usage
function probably doesn't have many dependencies, so it can logically
go high in the code. As a general rule, I like to organize code such
that things are defined higher up than they're used; it's not strictly
necessary (if they're used inside functions, the requirement is only
that they be defined before the function's called), but it helps with
clarity. That generally means that "def usage():" wants to go up above
any place where "usage()" occurs, but below the definitions of any
functions that usage() itself calls, and below the first assignments
to any global names it uses. It's not always possible, but when it is,
it tends to produce an easy-to-navigate source file.

ChrisA


From grantcmurphy at gmail.com  Tue May 12 15:46:35 2015
From: grantcmurphy at gmail.com (Grant Murphy)
Date: Tue, 12 May 2015 12:46:35 -0700
Subject: Suggestion: PEP for tracking vulnerable packages within PyPI
Message-ID: <CAHXGaxD+hj=N-UqzO=nepka0KJ7zbr+_VneuPRbs34G6NjZNZA@mail.gmail.com>

Hi,

When pulling in a dependency via pip it is currently difficult to reason about
whether there are any vulnerabilities associated with the package version you
are using. I think the Python package management infrastructure could be
extended to facilitate this capability reasonably easily. PyPI already
contains a lot of metadata around package owners and releases available.
Adding the ability to flag a release as having a vulnerability and CVE
associated with it seems like a reasonable addition to me.

Currently there are some projects that are trying to track this information [1],
however by including this type of information as a part of the main Python
infrastructure I think it would encourage better vulnerability management
practices within the community.

I'd like some feedback on how to move forward with this suggestion. Does
this seem like something that could be worth turning into a PEP?

1. https://github.com/victims/victims-cve-db

- Grant


From ned at nedbatchelder.com  Tue May 12 15:49:08 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 12 May 2015 12:49:08 -0700 (PDT)
Subject: Python file structure
In-Reply-To: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
References: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
Message-ID: <026e6357-917c-4d50-b70f-70903aa0e065@googlegroups.com>

On Tuesday, May 12, 2015 at 3:13:32 PM UTC-4, zljubi... at gmail.com wrote:
> Hi, I have python file with the following structure:
> 
> import...
> 
> A = configparser.get(...) 
> B = configparser.get(...)
> 
> Command line parameters parsing [they can change variable A or B]
> 
> Def usage()
> 	Print how to use script parameters
> 
> def main():
> 	...
> 
> if __name__ == "__main__":
>     main()
> 
> If I find an error in command line parameters section I cannot call function usage() because it is not defined yet. 
> 
> I have few options here:
> 1.	Put definition of usage function before command line parameters parsing section
> 2.	Make parameters global and put them in the main function
> 3.	...maybe some other options...
> 

I would put all of the code into a function some place.  Don't have
anything at the top level of the file except imports, function (and
class) definitions, and an "if __name__....." clause at the bottom.

If you need to use globals, assign them inside a parse_arguments
function that has a "global" statement in it.

This advice is consistent with Chris' "define things before they
are used."  It does it by defining everything before anything is
run.

As a side note, if you are going to have code at the top-level of
the file, then there's no point in the "if __name__..." clause.
That clause is designed to make a file both runnable and importable.
But your top-level code makes the file very difficult to import.

--Ned.


From marko at pacujo.net  Tue May 12 15:52:20 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 12 May 2015 22:52:20 +0300
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <mailman.405.1431450952.12865.python-list@python.org>
Message-ID: <878ucty3nv.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> That's as may be, but I would still not recommend [C] as a first
> language.

I think the field can be approached from many angles successfully. And
any approach will fail many students.

The nice thing about C is that your feet are firmly on the ground.
There's little magic. You can then abstract your concrete knowledge of C
to higher-level concepts.

In fact, going the other way could be harder. I'm thinking the lofty
abstractions like objects will remain sort of mysteries without an
experience with a low-level language.

That's why first-graders are not given education in abstract algebra or
category theory. They are first taught elementary arithmetics. The lofty
concepts are abstracted from the low-level concepts and not the other
way around. (I had a feeling in high-school that math was easy. In the
university, I had the opposite experience: I hadn't understood a thing
in high-school. However, the elementary "wrong" knowledge was a stepping
stone to the "correct" understanding.)


Marko


From ian.g.kelly at gmail.com  Tue May 12 15:54:21 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 12 May 2015 13:54:21 -0600
Subject: Python file structure
In-Reply-To: <CAPTjJmrtLoCGOAr+F1Q7zgzv3hUhvrVA3ADdaLe9mP9YSShi6Q@mail.gmail.com>
References: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
 <CAPTjJmrtLoCGOAr+F1Q7zgzv3hUhvrVA3ADdaLe9mP9YSShi6Q@mail.gmail.com>
Message-ID: <CALwzid=tsm9c6DWyQ+g72+uG_KM+eDQ2Be3Pgj5gj6sorhNGHw@mail.gmail.com>

On Tue, May 12, 2015 at 1:29 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Wed, May 13, 2015 at 5:13 AM,  <zljubisicmob at gmail.com> wrote:
>> If I find an error in command line parameters section I cannot call function usage() because it is not defined yet.
>>
>> I have few options here:
>> 1.      Put definition of usage function before command line parameters parsing section
>
> I'd do this, unless there's a good reason not to. A simple usage
> function probably doesn't have many dependencies, so it can logically
> go high in the code. As a general rule, I like to organize code such
> that things are defined higher up than they're used; it's not strictly
> necessary (if they're used inside functions, the requirement is only
> that they be defined before the function's called), but it helps with
> clarity. That generally means that "def usage():" wants to go up above
> any place where "usage()" occurs, but below the definitions of any
> functions that usage() itself calls, and below the first assignments
> to any global names it uses. It's not always possible, but when it is,
> it tends to produce an easy-to-navigate source file.

+1.

Also, I like to put command-line parsing inside the main function and
make that its *only* responsibility. The main function then calls the
real entry point of my script, which will be something more
specifically named. This also has the advantage that if some other
module needs to invoke my script, all it has to do is call the entry
point function which will be named something more suitable than
"main".


From zljubisicmob at gmail.com  Tue May 12 15:58:54 2015
From: zljubisicmob at gmail.com (zljubisicmob at gmail.com)
Date: Tue, 12 May 2015 12:58:54 -0700 (PDT)
Subject: Python file structure
In-Reply-To: <026e6357-917c-4d50-b70f-70903aa0e065@googlegroups.com>
References: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
 <026e6357-917c-4d50-b70f-70903aa0e065@googlegroups.com>
Message-ID: <05daad5f-b728-48ee-bf06-5e7374df936d@googlegroups.com>

On Tuesday, May 12, 2015 at 9:49:20 PM UTC+2, Ned Batchelder wrote:
> On Tuesday, May 12, 2015 at 3:13:32 PM UTC-4, zljubi... at gmail.com wrote:
> > Hi, I have python file with the following structure:
> > 
> > import...
> > 
> > A = configparser.get(...) 
> > B = configparser.get(...)
> > 
> > Command line parameters parsing [they can change variable A or B]
> > 
> > Def usage()
> > 	Print how to use script parameters
> > 
> > def main():
> > 	...
> > 
> > if __name__ == "__main__":
> >     main()
> > 
> > If I find an error in command line parameters section I cannot call function usage() because it is not defined yet. 
> > 
> > I have few options here:
> > 1.	Put definition of usage function before command line parameters parsing section
> > 2.	Make parameters global and put them in the main function
> > 3.	...maybe some other options...
> > 
> 
> I would put all of the code into a function some place.  Don't have
> anything at the top level of the file except imports, function (and
> class) definitions, and an "if __name__....." clause at the bottom.
> 
> If you need to use globals, assign them inside a parse_arguments
> function that has a "global" statement in it.
> 
> This advice is consistent with Chris' "define things before they
> are used."  It does it by defining everything before anything is
> run.
> 
> As a side note, if you are going to have code at the top-level of
> the file, then there's no point in the "if __name__..." clause.
> That clause is designed to make a file both runnable and importable.
> But your top-level code makes the file very difficult to import.
> 
> --Ned.

It makes sense. The only drawback is that variables are global so they could be changed anywhere in the program.
I also agree that it is more python approach.

Thanks to both of you.


From rosuav at gmail.com  Tue May 12 16:02:42 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 13 May 2015 06:02:42 +1000
Subject: Python file structure
In-Reply-To: <026e6357-917c-4d50-b70f-70903aa0e065@googlegroups.com>
References: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
 <026e6357-917c-4d50-b70f-70903aa0e065@googlegroups.com>
Message-ID: <CAPTjJmotQktSbEkp_NoFVmOC0oJQB98vLxFgp-ba0KfOX8VnSg@mail.gmail.com>

On Wed, May 13, 2015 at 5:49 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> I would put all of the code into a function some place.  Don't have
> anything at the top level of the file except imports, function (and
> class) definitions, and an "if __name__....." clause at the bottom.
>
> If you need to use globals, assign them inside a parse_arguments
> function that has a "global" statement in it.
>
> This advice is consistent with Chris' "define things before they
> are used."  It does it by defining everything before anything is
> run.

Consistent with, yes, but not for the reason you state. I'm talking
about code layout, not prevention of NameError. For instance, this
would fit your description, and wouldn't error out, but wouldn't fit
my ideal:

import sys

def main():
    if len(sys.argv) < 2: usage()
    # do stuff

def usage():
    print("USAGE: programname arguments")
    sys.exit(0)

if __name__ == '__main__': main()


I would shift the definition of usage() up above main(). Even though
both are executed before main() actually begins running, which
prevents NameError, it's harder to skim the file. Obviously this is an
ideal that can't always be attained (mutual references, for instance),
but where it can be done, it means that the first instance of any
token in a file is its definition - maybe in an import statement (so
"from X import *" violates the principle, but I think most people
avoid it anyway), or maybe in a def/class statement, or maybe a simple
assignment.

ChrisA


From rosuav at gmail.com  Tue May 12 16:07:31 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 13 May 2015 06:07:31 +1000
Subject: Python file structure
In-Reply-To: <CALwzid=tsm9c6DWyQ+g72+uG_KM+eDQ2Be3Pgj5gj6sorhNGHw@mail.gmail.com>
References: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
 <CAPTjJmrtLoCGOAr+F1Q7zgzv3hUhvrVA3ADdaLe9mP9YSShi6Q@mail.gmail.com>
 <CALwzid=tsm9c6DWyQ+g72+uG_KM+eDQ2Be3Pgj5gj6sorhNGHw@mail.gmail.com>
Message-ID: <CAPTjJmpDfdYZ1LXeHbHNTuv_L4vaNZXnz-yOffjc1sM8tAd1Uw@mail.gmail.com>

On Wed, May 13, 2015 at 5:54 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> Also, I like to put command-line parsing inside the main function and
> make that its *only* responsibility. The main function then calls the
> real entry point of my script, which will be something more
> specifically named. This also has the advantage that if some other
> module needs to invoke my script, all it has to do is call the entry
> point function which will be named something more suitable than
> "main".

That often makes sense, but sometimes doesn't. When it doesn't, you
can usually tell because your main function looks something like this:

def main():
    do_real_work(*sys.argv)
if __name__=="__main__": main()

A one-line function that's called from one place? In-line it.

if __name__ == "__main__":
    do_real_work(*sys.argv)

ChrisA


From grantcmurphy at gmail.com  Tue May 12 16:16:47 2015
From: grantcmurphy at gmail.com (Grant Murphy)
Date: Tue, 12 May 2015 13:16:47 -0700
Subject: Suggestion: PEP for tracking vulnerable Python packages
Message-ID: <CAHXGaxCS7ix-4-6-NFsCYHUYSzg80bK6RCaA9m-AqgEnd5y7cw@mail.gmail.com>

Hi,

When pulling in a dependency via pip it is currently difficult to reason about
whether there are any vulnerabilities associated with the package version you
are using. I think the Python package management infrastructure could be
extended to facilitate this capability reasonably easily. PyPI already
contains a lot of metadata around package owners and releases available.
Adding the ability to flag a release as having a vulnerability and CVE
associated with it seems like a reasonable addition to me.

Currently there are some projects that are trying to track this information [1],
however by including this type of information as a part of the Python
infrastructure I think it would encourage better vulnerability management
practices within the community.

I'd like some feedback on how to move forward with this suggestion. Does
this seem like something that could be worth turning into a PEP?

1. https://github.com/victims/victims-cve-db

- Grant


From grantcmurphy at gmail.com  Tue May 12 16:18:27 2015
From: grantcmurphy at gmail.com (Grant Murphy)
Date: Tue, 12 May 2015 13:18:27 -0700 (PDT)
Subject: Suggestion: PEP for tracking vulnerable Python packages
Message-ID: <1c8fe055-6277-4214-8be0-1d6aafe2eb6a@googlegroups.com>

Hi,

When pulling in a dependency via pip it is currently difficult to reason about
whether there are any vulnerabilities associated with the package version you
are using. I think the Python package management infrastructure could be
extended to facilitate this capability reasonably easily. PyPI already
contains a lot of metadata around package owners and releases available.
Adding the ability to flag a release as having a vulnerability and CVE
associated with it seems like a reasonable addition to me.

Currently there are some projects that are trying to track this information [1],
however by including this type of information as a part of the Python
infrastructure I think it would encourage better vulnerability management
practices within the community.

I'd like some feedback on how to move forward with this suggestion. Does
this seem like something that could be worth turning into a PEP?

1. https://github.com/victims/victims-cve-db

- Grant


From skybuck2000 at hotmail.com  Tue May 12 16:18:54 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Tue, 12 May 2015 22:18:54 +0200
Subject: Feature Request: Reposition Execution
In-Reply-To: <mailman.354.1431345441.12865.python-list@python.org>
References: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>
 <mailman.354.1431345441.12865.python-list@python.org>
Message-ID: <4568f$5552602d$5419aafe$61337@news.ziggo.nl>

"Dave Angel"  wrote in message 
news:mailman.354.1431345441.12865.python-list at python.org...

On 05/11/2015 07:46 AM, Skybuck Flying wrote:
> Hello,
>
> Sometimes it can be handy to "interrupt/reset/reposition" a running 
> script.
>
> For example something externally goes badly wrong.
>

"
os.kill()

then in your process, handle the exception, and do whatever you think is
worthwhile.
"

Thanks for suggestion, but I need a solution which can work in SikuliX as 
well.

Especially inside an observer handler that would be ideal.

So far os.kill() is not supported in SikuliX as far as I can tell.

Bye,
  Skybuck. 



From tjreedy at udel.edu  Tue May 12 16:23:42 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 May 2015 16:23:42 -0400
Subject: anomaly
In-Reply-To: <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
 <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mitngl$rp$1@ger.gmane.org>

On 5/12/2015 9:56 AM, Steven D'Aprano wrote:

> The consensus among the core developers is:
>
> * in general, the harm and inconvenience from accidentally
>    shadowing built-ins is not great, and it usually easy to
>    spot, debug and prevent;
>
> * when it comes to built-in functions (e.g. sum, map, pow)
>    and types (e.g. int, str, list) there are significant and
>    important use-cases for allowing shadowing;
>
>    (e.g. the built-in sum function shouldn't prevent other
>    modules from providing their own sum function)
>
> * but when it comes to None, True and False, there are no
>    significant or important use-cases for shadowing (it is
>    almost always a bug, not a feature, to redefine None).
>
> The general principle here is of consenting adults: Python allows you to
> shadow built-ins because sometimes it is useful, and the good outweighs the
> potential harm. There are a handful of exceptions to that general principle
> (e.g. None, True, False, I can't think of any others) because in those
> cases the harm (as tiny as it is) outweighs the good.

Having followed Python development for 18 years, I think this is a fair 
summary.



-- 
Terry Jan Reedy



From andres.riancho at gmail.com  Tue May 12 16:32:48 2015
From: andres.riancho at gmail.com (Andres Riancho)
Date: Tue, 12 May 2015 17:32:48 -0300
Subject: Suggestion: PEP for tracking vulnerable Python packages
In-Reply-To: <CAHXGaxCS7ix-4-6-NFsCYHUYSzg80bK6RCaA9m-AqgEnd5y7cw@mail.gmail.com>
References: <CAHXGaxCS7ix-4-6-NFsCYHUYSzg80bK6RCaA9m-AqgEnd5y7cw@mail.gmail.com>
Message-ID: <CA+1Rt64mJRT6hLPVvwPa=FkODVBgAfSD1JU3uTBSz==-UomB9g@mail.gmail.com>

Grant,

On Tue, May 12, 2015 at 5:16 PM, Grant Murphy <grantcmurphy at gmail.com> wrote:
> Hi,
>
> When pulling in a dependency via pip it is currently difficult to reason about
> whether there are any vulnerabilities associated with the package version you
> are using. I think the Python package management infrastructure could be
> extended to facilitate this capability reasonably easily. PyPI already
> contains a lot of metadata around package owners and releases available.
> Adding the ability to flag a release as having a vulnerability and CVE
> associated with it seems like a reasonable addition to me.
>
> Currently there are some projects that are trying to track this information [1],
> however by including this type of information as a part of the Python
> infrastructure I think it would encourage better vulnerability management
> practices within the community.
>
> I'd like some feedback on how to move forward with this suggestion. Does
> this seem like something that could be worth turning into a PEP?

I believe a PEP is not necessary, but it would be great to make this
information part of the package meta-data in pypi, and have "pip"
refuse to install a package that has known vulnerabilities. The user
could force the installation of a vulnerable package with
"--install-vulnerable package-name", but at least pypi / python
community is warning the dev.

> 1. https://github.com/victims/victims-cve-db
>
> - Grant
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Andr?s Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3


From davea at davea.name  Tue May 12 16:43:23 2015
From: davea at davea.name (Dave Angel)
Date: Tue, 12 May 2015 16:43:23 -0400
Subject: Python file structure
In-Reply-To: <05daad5f-b728-48ee-bf06-5e7374df936d@googlegroups.com>
References: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
 <026e6357-917c-4d50-b70f-70903aa0e065@googlegroups.com>
 <05daad5f-b728-48ee-bf06-5e7374df936d@googlegroups.com>
Message-ID: <555265EB.3080503@davea.name>

On 05/12/2015 03:58 PM, zljubisicmob at gmail.com wrote:
> On Tuesday, May 12, 2015 at 9:49:20 PM UTC+2, Ned Batchelder wrote:

>>
>> If you need to use globals, assign them inside a parse_arguments
>> function that has a "global" statement in it.
>>
>> This advice is consistent with Chris' "define things before they
>> are used."  It does it by defining everything before anything is
>> run.
>>
>> As a side note, if you are going to have code at the top-level of
>> the file, then there's no point in the "if __name__..." clause.
>> That clause is designed to make a file both runnable and importable.
>> But your top-level code makes the file very difficult to import.
>>
>> --Ned.
>
> It makes sense. The only drawback is that variables are global

only "if you need to use globals".  You can't have it both ways.  If 
they're needed, it's because you feel they must be changeable elsewhere 
in the program.  I try to avoid global variables, but make no such 
restraints on the use of global constants.  So for example, the argument 
parsing logic could very well export something as global, but it'd be 
all uppercase and anyone changing it subsequently would get their hand 
slapped by the linter.

> so they could be changed anywhere in the program.
> I also agree that it is more python approach.
>
> Thanks to both of you.
>


-- 
DaveA


From breamoreboy at yahoo.co.uk  Tue May 12 17:17:29 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 12 May 2015 22:17:29 +0100
Subject: Suggestion: PEP for tracking vulnerable packages within PyPI
In-Reply-To: <CAHXGaxD+hj=N-UqzO=nepka0KJ7zbr+_VneuPRbs34G6NjZNZA@mail.gmail.com>
References: <CAHXGaxD+hj=N-UqzO=nepka0KJ7zbr+_VneuPRbs34G6NjZNZA@mail.gmail.com>
Message-ID: <mitqlb$r6e$1@ger.gmane.org>

On 12/05/2015 20:46, Grant Murphy wrote:
> Hi,
>
> When pulling in a dependency via pip it is currently difficult to reason about
> whether there are any vulnerabilities associated with the package version you
> are using. I think the Python package management infrastructure could be
> extended to facilitate this capability reasonably easily. PyPI already
> contains a lot of metadata around package owners and releases available.
> Adding the ability to flag a release as having a vulnerability and CVE
> associated with it seems like a reasonable addition to me.
>
> Currently there are some projects that are trying to track this information [1],
> however by including this type of information as a part of the main Python
> infrastructure I think it would encourage better vulnerability management
> practices within the community.
>
> I'd like some feedback on how to move forward with this suggestion. Does
> this seem like something that could be worth turning into a PEP?
>
> 1. https://github.com/victims/victims-cve-db
>
> - Grant
>

It strikes me as a great idea.  As you've got the time to send three 
emails some 40 minutes apart saying the same thing, you must have the 
time to do the work that is involved, so please let us know what your 
plans are.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From mail at timgolden.me.uk  Tue May 12 17:23:40 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Tue, 12 May 2015 22:23:40 +0100
Subject: Suggestion: PEP for tracking vulnerable packages within PyPI
In-Reply-To: <mitqlb$r6e$1@ger.gmane.org>
References: <CAHXGaxD+hj=N-UqzO=nepka0KJ7zbr+_VneuPRbs34G6NjZNZA@mail.gmail.com>
 <mitqlb$r6e$1@ger.gmane.org>
Message-ID: <55526F5C.8050503@timgolden.me.uk>



On 12/05/2015 22:17, Mark Lawrence wrote:
> On 12/05/2015 20:46, Grant Murphy wrote:
>> Hi,
>>
>> When pulling in a dependency via pip it is currently difficult to
>> reason about
>> whether there are any vulnerabilities associated with the package
>> version you
>> are using. I think the Python package management infrastructure could be
>> extended to facilitate this capability reasonably easily. PyPI already
>> contains a lot of metadata around package owners and releases available.
>> Adding the ability to flag a release as having a vulnerability and CVE
>> associated with it seems like a reasonable addition to me.
>>
>> Currently there are some projects that are trying to track this
>> information [1],
>> however by including this type of information as a part of the main
>> Python
>> infrastructure I think it would encourage better vulnerability management
>> practices within the community.
>>
>> I'd like some feedback on how to move forward with this suggestion. Does
>> this seem like something that could be worth turning into a PEP?
>>
>> 1. https://github.com/victims/victims-cve-db
>>
>> - Grant
>>
>
> It strikes me as a great idea.  As you've got the time to send three
> emails some 40 minutes apart saying the same thing, you must have the
> time to do the work that is involved, so please let us know what your
> plans are.
>

Before you drown in your own snark, Mark, I'll just point out that the 
OP sent the later emails thinking that the earlier ones hadn't got 
through, since I was somewhere which didn't have internet access so 
couldn't approve the posts.

Still a tad impatient, I agree, but not the question-bomber you're 
suggesting.

TJG


From tjreedy at udel.edu  Tue May 12 17:34:32 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 12 May 2015 17:34:32 -0400
Subject: Python file structure
In-Reply-To: <026e6357-917c-4d50-b70f-70903aa0e065@googlegroups.com>
References: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
 <026e6357-917c-4d50-b70f-70903aa0e065@googlegroups.com>
Message-ID: <mitrlg$bbe$1@ger.gmane.org>

On 5/12/2015 3:49 PM, Ned Batchelder wrote:
> On Tuesday, May 12, 2015 at 3:13:32 PM UTC-4, zljubi... at gmail.com wrote:
>> Hi, I have python file with the following structure:
>>
>> import...
>>
>> A = configparser.get(...)
>> B = configparser.get(...)
>>
>> Command line parameters parsing [they can change variable A or B]
>>
>> Def usage()
>> 	Print how to use script parameters
>>
>> def main():
>> 	...
>>
>> if __name__ == "__main__":
>>      main()
>>
>> If I find an error in command line parameters section I cannot call function usage() because it is not defined yet.
>>
>> I have few options here:
>> 1.	Put definition of usage function before command line parameters parsing section
>> 2.	Make parameters global and put them in the main function
>> 3.	...maybe some other options...
>>
>
> I would put all of the code into a function some place.  Don't have
> anything at the top level of the file except imports, function (and
> class) definitions, and an "if __name__....." clause at the bottom.

I was about to suggest the same.  One advantage of 'write tests first' 
is that you force yourself to write testable code from the beginning, 
instead of refactoring later.

> If you need to use globals, assign them inside a parse_arguments
> function that has a "global" statement in it.

Better not to have mutable module globals if you can avoid it.  If you 
want application globals, put them in a separate module.

> As a side note, if you are going to have code at the top-level of
> the file, then there's no point in the "if __name__..." clause.
> That clause is designed to make a file both runnable and importable.
> But your top-level code makes the file very difficult to import.

And you need to import to test.

-- 
Terry Jan Reedy



From grantcmurphy at gmail.com  Tue May 12 17:56:59 2015
From: grantcmurphy at gmail.com (Grant Murphy)
Date: Tue, 12 May 2015 14:56:59 -0700
Subject: Suggestion: PEP for tracking vulnerable packages within PyPI
In-Reply-To: <mitqlb$r6e$1@ger.gmane.org>
References: <CAHXGaxD+hj=N-UqzO=nepka0KJ7zbr+_VneuPRbs34G6NjZNZA@mail.gmail.com>
 <mitqlb$r6e$1@ger.gmane.org>
Message-ID: <CAHXGaxBNDCPPCvAMjoYO088m7XDqVm=HY2zjn5Oks9y5-MPF5w@mail.gmail.com>

Ok so.. no PEP needed then..alright then... my plan now goes something
like this:

0. Send this email.
1. Unsubscribe from the python-list. (I don't enjoy the company of trolls).
2. Actually fix the problem and submit a PR.
3. Go have a beer.

Apologies for the multiple emails.  I can see how Mark needed to be a
jerk about it..

- Grant

On Tue, May 12, 2015 at 2:17 PM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> On 12/05/2015 20:46, Grant Murphy wrote:
>>
>> Hi,
>>
>> When pulling in a dependency via pip it is currently difficult to reason
>> about
>> whether there are any vulnerabilities associated with the package version
>> you
>> are using. I think the Python package management infrastructure could be
>> extended to facilitate this capability reasonably easily. PyPI already
>> contains a lot of metadata around package owners and releases available.
>> Adding the ability to flag a release as having a vulnerability and CVE
>> associated with it seems like a reasonable addition to me.
>>
>> Currently there are some projects that are trying to track this
>> information [1],
>> however by including this type of information as a part of the main Python
>> infrastructure I think it would encourage better vulnerability management
>> practices within the community.
>>
>> I'd like some feedback on how to move forward with this suggestion. Does
>> this seem like something that could be worth turning into a PEP?
>>
>> 1. https://github.com/victims/victims-cve-db
>>
>> - Grant
>>
>
> It strikes me as a great idea.  As you've got the time to send three emails
> some 40 minutes apart saying the same thing, you must have the time to do
> the work that is involved, so please let us know what your plans are.
>
> --
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
>
> Mark Lawrence
>
> --
> https://mail.python.org/mailman/listinfo/python-list


From dreamingforward at gmail.com  Tue May 12 18:00:05 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 12 May 2015 15:00:05 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
Message-ID: <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>

On Tuesday, May 12, 2015 at 12:57:48 PM UTC-5, Rustom Mody wrote:
> On Tuesday, May 12, 2015 at 10:45:39 PM UTC+5:30, Stefan Ram wrote:
> > Rob Gaddi writes:
> > >Is that a true array or a linked list?  "It's a high level language, 
> > >that's just an implementation detail." Yes, but it's an implementation 
> > >detail that determines whether even the simple act of looking up element 
> > >n is O(1) or O(n).
> > 
> >   The complexity is given in the documentation of high-level languages.
> > 
> >   For example, from the documentation of the Java standard library:
> > 
> >       ?This implementation provides constant-time performance
> >       for the basic operations (get and put),? (java.util.HashMap)
> > 
> >   C++:
> > 
> >       Section 23.2.1 specifies the complexity, such as ?compile time?,
> >       ?constant?, ?linear? for container operations.
> > 
> >   But the abstraction mechanisms (templates, polymorphism)
> >   often allow one to change the implementation quite easily.
> 
> This is regarded in some circles as one of the big open problems in CS
> https://existentialtype.wordpress.com/2014/09/28/structure-and-efficiency-of-computer-programs/
> [and the position paper therein]
> viz how to integrate the elegance of high-level languages
> with the precision of complexity analysis of machine language

The answer is you don't.  You *don't* because you can't.  And you *can't* because you don't know what you want to do yet.  

That is why you have very high-level languages that allow you to rapidly prototype ideas, test them, and then, depending all the other constraints, move them to lower-level language implementations.

So don't try it.  Everyone gets it wrong and now we have a plethora of languages which all do the same thing, without really knowing what they want as an overarching design or purpose.

Mark


From breamoreboy at yahoo.co.uk  Tue May 12 18:06:19 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 12 May 2015 23:06:19 +0100
Subject: Suggestion: PEP for tracking vulnerable packages within PyPI
In-Reply-To: <CAHXGaxBNDCPPCvAMjoYO088m7XDqVm=HY2zjn5Oks9y5-MPF5w@mail.gmail.com>
References: <CAHXGaxD+hj=N-UqzO=nepka0KJ7zbr+_VneuPRbs34G6NjZNZA@mail.gmail.com>
 <mitqlb$r6e$1@ger.gmane.org>
 <CAHXGaxBNDCPPCvAMjoYO088m7XDqVm=HY2zjn5Oks9y5-MPF5w@mail.gmail.com>
Message-ID: <mittgs$834$1@ger.gmane.org>

On 12/05/2015 22:56, Grant Murphy wrote:

Please don't top post here it's extremely irritating.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From marko at pacujo.net  Tue May 12 18:24:23 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 13 May 2015 01:24:23 +0300
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
Message-ID: <87y4ktwi20.fsf@elektro.pacujo.net>

zipher <dreamingforward at gmail.com>:

> That is why you have very high-level languages that allow you to
> rapidly prototype ideas, test them, and then, depending all the other
> constraints, move them to lower-level language implementations.

Finally an argument to tackle. That rapid prototyping role is often
mentioned as a strong point of high-level languages. However, I can't
remember personally doing that. Rather, you want to use the right
programming language for any given role.

Bash has really concise idioms as long as you stay within its comfort
zone. When you need finer-grained control, a bash program quickly
becomes very tacky. I have had to rewrite bash components in Python for
that reason. What is gained is clarity of expression at the cost of 2 to
5 times more code. And where performance is an issue, C does the job
nicely.

The end result is a system with bash, Python and C components that
interact through the regular linux IPC mechanisms.


Marko


From ben+python at benfinney.id.au  Tue May 12 18:35:02 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 13 May 2015 08:35:02 +1000
Subject: Updating a package on PyPi, testing and etiquette
References: <mitf66$2sq$4@dont-email.me>
Message-ID: <85fv71a0h5.fsf@benfinney.id.au>

Rob Gaddi <rgaddi at technologyhighland.invalid> writes:

> A) Is there any easy way to test against an older version of Python?

The convention today is to use Tox for testing one code base using
multiple Python impleemntations, in a single test run.

    <URL:https://pypi.python.org/pypi/tox>

> Preferably without trying to install entire old environments and keep 
> them nicely isolated from my actual doing work?

That's what Python virtualenv functionality is for, and Tox makes good
use of it. This is well-trod ground.

> B) If I can't manage that, what's the etiquette behind having later
> versions of a module break compatibility with older versions of
> Python.

Consult your user community, tell them the issue (maintaining obsolete
Python versions is a lot of effort), and incrementally drop support with
subsequent versions. Give your user community a chance to upgrade their
stuff, but also don't take on an unreasonable burden. Involve them in
the process.

-- 
 \     ?Don't be misled by the enormous flow of money into bad defacto |
  `\    standards for unsophisticated buyers using poor adaptations of |
_o__)                                     incomplete ideas.? ?Alan Kay |
Ben Finney



From flebber.crue at gmail.com  Tue May 12 18:43:49 2015
From: flebber.crue at gmail.com (Sayth Renshaw)
Date: Tue, 12 May 2015 15:43:49 -0700 (PDT)
Subject: itertools py3.4 - filter list using not equal - fails as bool
Message-ID: <05defef5-74aa-4a5d-b7e7-9b521512152c@googlegroups.com>

why can't I filter a list based on an itertools condition using dropwhile?

This is the docs and the example. https://docs.python.org/3/library/itertools.html#itertools.dropwhile

def less_than_10(x):
    return x < 10

itertools.takewhile(less_than_10, itertools.count()) =>
  0, 1, 2, 3, 4, 5, 6, 7, 8, 9

so I have a list I have created (converted from itertools). pm is permutations 


print(stats)
[(1, 2, 3), (1, 2, 4), (1, 2, 5), (1, 3, 2), (1, 3, 4), (1, 3, 5), (1, 4, 2), (1, 4, 3), (1, 4, 5), (1, 5, 2), (1, 5, 3), (1, 5, 4), (2, 1, 3), (2, 1, 4), (2, 1, 5), (2, 3, 1), (2, 3, 4), (2, 3, 5), (2, 4, 1), (2, 4, 3), (2, 4, 5), (2, 5, 1), (2, 5, 3), (2, 5, 4), (3, 1, 2), (3, 1, 4), (3, 1, 5), (3, 2, 1), (3, 2, 4), (3, 2, 5), (3, 4, 1), (3, 4, 2), (3, 4, 5), (3, 5, 1), (3, 5, 2), (3, 5, 4), (4, 1, 2), (4, 1, 3), (4, 1, 5), (4, 2, 1), (4, 2, 3), (4, 2, 5), (4, 3, 1), (4, 3, 2), (4, 3, 5), (4, 5, 1), (4, 5, 2), (4, 5, 3), (5, 1, 2), (5, 1, 3), (5, 1, 4), (5, 2, 1), (5, 2, 3), (5, 2, 4), (5, 3, 1), (5, 3, 2), (5, 3, 4), (5, 4, 1), (5, 4, 2), (5, 4, 3)]


I simply wanted to create an easy way to create summary stats of my stats list(poorly named). So in this case easy to check answers. so how many tuples in my list have a 1 in item[0] and how many don't. Then hoping to build on that for example how many have item[0] == 1 && (item[1] == 2 or item[1] == 4) etc.

I can achieve it via an else if but that would become ugly quick. 

for item in stats:
    if item[0] == 1: 
        nums += 1
    elif item[0] != 1:
        not_in += 1
    else:
        pass
    

myString = "i have {a} in and {b} so the percentage is {c}%".format(a=nums, b=not_in, c=(nums/(nums + not_in)))

I thought dropwhile from the docs appeared a good option but it returns bool.

answer = listb.extend(itertools.dropwhile(item[0] != 1, stats))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-130-70c93104d1c5> in <module>()
----> 1 answer = listb.extend(itertools.dropwhile(item[0] != 1, stats))

TypeError: 'bool' object is not callable

Think I am making this hard somewhere that it is actually easy. 

As an aside do I really need to convert the iterator object to a list to create and summarize iterator contents?

Currently I have created a function to achieve this as below.

def myGen(parcel):
    x = []
    [x.append(y) for y in parcel]
    return x

myPerm = pm(range(1,6),3)

stats = myGen(myPerm)


From python at mrabarnett.plus.com  Tue May 12 19:22:42 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 13 May 2015 00:22:42 +0100
Subject: itertools py3.4 - filter list using not equal - fails as bool
In-Reply-To: <05defef5-74aa-4a5d-b7e7-9b521512152c@googlegroups.com>
References: <05defef5-74aa-4a5d-b7e7-9b521512152c@googlegroups.com>
Message-ID: <55528B42.3070705@mrabarnett.plus.com>

On 2015-05-12 23:43, Sayth Renshaw wrote:
> why can't I filter a list based on an itertools condition using dropwhile?
>
> This is the docs and the example. https://docs.python.org/3/library/itertools.html#itertools.dropwhile
>
> def less_than_10(x):
>      return x < 10
>
> itertools.takewhile(less_than_10, itertools.count()) =>
>    0, 1, 2, 3, 4, 5, 6, 7, 8, 9
>
> so I have a list I have created (converted from itertools). pm is permutations
>
>
> print(stats)
> [(1, 2, 3), (1, 2, 4), (1, 2, 5), (1, 3, 2), (1, 3, 4), (1, 3, 5), (1, 4, 2), (1, 4, 3), (1, 4, 5), (1, 5, 2), (1, 5, 3), (1, 5, 4), (2, 1, 3), (2, 1, 4), (2, 1, 5), (2, 3, 1), (2, 3, 4), (2, 3, 5), (2, 4, 1), (2, 4, 3), (2, 4, 5), (2, 5, 1), (2, 5, 3), (2, 5, 4), (3, 1, 2), (3, 1, 4), (3, 1, 5), (3, 2, 1), (3, 2, 4), (3, 2, 5), (3, 4, 1), (3, 4, 2), (3, 4, 5), (3, 5, 1), (3, 5, 2), (3, 5, 4), (4, 1, 2), (4, 1, 3), (4, 1, 5), (4, 2, 1), (4, 2, 3), (4, 2, 5), (4, 3, 1), (4, 3, 2), (4, 3, 5), (4, 5, 1), (4, 5, 2), (4, 5, 3), (5, 1, 2), (5, 1, 3), (5, 1, 4), (5, 2, 1), (5, 2, 3), (5, 2, 4), (5, 3, 1), (5, 3, 2), (5, 3, 4), (5, 4, 1), (5, 4, 2), (5, 4, 3)]
>
>
> I simply wanted to create an easy way to create summary stats of my stats list(poorly named). So in this case easy to check answers. so how many tuples in my list have a 1 in item[0] and how many don't. Then hoping to build on that for example how many have item[0] == 1 && (item[1] == 2 or item[1] == 4) etc.
>
> I can achieve it via an else if but that would become ugly quick.
>
> for item in stats:
>      if item[0] == 1:
>          nums += 1
>      elif item[0] != 1:
>          not_in += 1
>      else:
>          pass
>
>
> myString = "i have {a} in and {b} so the percentage is {c}%".format(a=nums, b=not_in, c=(nums/(nums + not_in)))
>
> I thought dropwhile from the docs appeared a good option but it returns bool.
>
> answer = listb.extend(itertools.dropwhile(item[0] != 1, stats))
> ---------------------------------------------------------------------------
> TypeError                                 Traceback (most recent call last)
> <ipython-input-130-70c93104d1c5> in <module>()
> ----> 1 answer = listb.extend(itertools.dropwhile(item[0] != 1, stats))
>
> TypeError: 'bool' object is not callable
>
That evaluates:

     item[0] != 1

which returns True or False. Apparently, "item" happens to be bound to
something that can be subscripted.

It then calls:

     itertools.dropwhile(..., stats)

passing the bool as the first argument (the predicate) of dropwhile.

When it tries to test the first item from stats, it discovers that the
predicate is not callable; it's a bool.

Result: TypeError!

And another point: a list's "extend" method modifies the list and then
returns None, so "answer" will be None.

> Think I am making this hard somewhere that it is actually easy.
>
> As an aside do I really need to convert the iterator object to a list to create and summarize iterator contents?
>
> Currently I have created a function to achieve this as below.
>
> def myGen(parcel):
>      x = []

Don't use a list comprehension like this:

>      [x.append(y) for y in parcel]

Do this instead:

     for y in parcel:
         x.append(y)

or, alternatively, this:

     x.extend(parcel)

>      return x
>
> myPerm = pm(range(1,6),3)
>
> stats = myGen(myPerm)
>



From rgaddi at technologyhighland.invalid  Tue May 12 19:54:42 2015
From: rgaddi at technologyhighland.invalid (Rob Gaddi)
Date: Tue, 12 May 2015 23:54:42 +0000 (UTC)
Subject: Updating a package on PyPi, testing and etiquette
References: <mitf66$2sq$4@dont-email.me>
 <mailman.424.1431470123.12865.python-list@python.org>
Message-ID: <miu3s1$2sq$5@dont-email.me>

On Wed, 13 May 2015 08:35:02 +1000, Ben Finney wrote:

> Rob Gaddi <rgaddi at technologyhighland.invalid> writes:
> 
>> B) If I can't manage that, what's the etiquette behind having later
>> versions of a module break compatibility with older versions of Python.
> 
> Consult your user community, tell them the issue (maintaining obsolete
> Python versions is a lot of effort), and incrementally drop support with
> subsequent versions. Give your user community a chance to upgrade their
> stuff, but also don't take on an unreasonable burden. Involve them in
> the process.

The only user community that I know about for this is three people, all 
of whom have desks within 50 feet of mine, and all of whom have sworn a 
solemn vow to do all of the actual development we care about under Python 
3.4 or higher.  I just noted that there were a bunch of downloads from 
John Q. Internet, and didn't know how poor of form breaking ancient 
compatibility was seen as.  The answer, at least here on c.l.p, seems to 
be "not particularly".


-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.


From ben+python at benfinney.id.au  Tue May 12 20:08:05 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 13 May 2015 10:08:05 +1000
Subject: Updating a package on PyPi, testing and etiquette
References: <mitf66$2sq$4@dont-email.me>
 <mailman.424.1431470123.12865.python-list@python.org>
 <miu3s1$2sq$5@dont-email.me>
Message-ID: <857fsd9w62.fsf@benfinney.id.au>

Rob Gaddi <rgaddi at technologyhighland.invalid> writes:

> On Wed, 13 May 2015 08:35:02 +1000, Ben Finney wrote:
>
> > Rob Gaddi <rgaddi at technologyhighland.invalid> writes:
> > 
> >> B) If I can't manage that, what's the etiquette behind having later
> >> versions of a module break compatibility with older versions of
> >> Python.
> > 
> > Consult your user community, tell them the issue [?] Involve them in
> > the process.
>
> The only user community that I know about for this is three people [?]

Then, prior to questions of etiquette, you need to begin engaging with
your user community in the first place :-)

Seriously, if you have no good way of contacting your user community,
it's difficult to talk about the right way to treat with them. You might
first set up a forum (e.g. a mailing list) to discuss future releases
with them.

-- 
 \          ?If you get invited to your first orgy, don't just show up |
  `\             nude. That's a common mistake. You have to let nudity |
_o__)                                          ?happen.?? ?Jack Handey |
Ben Finney



From greg.ewing at canterbury.ac.nz  Tue May 12 20:19:19 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Wed, 13 May 2015 12:19:19 +1200
Subject: anomaly
In-Reply-To: <mailman.391.1431431746.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
Message-ID: <crfjk9F1ks8U1@mid.individual.net>

Antoon Pardon wrote:
> But that doesn't answer the question why the developers chose "True" to be a
> keyword and "int" to be a looked-up name.

Probably because True, False and None are very frequently
used constants. Making them keywords means that things
like 'while True:' don't incur the overhead of a name
lookup every time around the loop.

The same doesn't apply to other built-in names, which
are used much less frequently.

-- 
Greg


From john_ladasky at sbcglobal.net  Tue May 12 20:47:25 2015
From: john_ladasky at sbcglobal.net (John Ladasky)
Date: Tue, 12 May 2015 17:47:25 -0700 (PDT)
Subject: anomaly
In-Reply-To: <mailman.350.1431339365.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <mailman.350.1431339365.12865.python-list@python.org>
Message-ID: <ac141288-b7c7-49fb-8191-c5247b46a49e@googlegroups.com>

On Monday, May 11, 2015 at 3:16:16 AM UTC-7, Antoon Pardon wrote:

> Try overriding None, True or False in python3 and see what happens.

Much fun was able to be had in Python 2, though:

    Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
    [GCC 4.8.2] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> True = False
    >>> True
    False


Every politician's dream!



From rustompmody at gmail.com  Tue May 12 20:56:53 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 17:56:53 -0700 (PDT)
Subject: anomaly
In-Reply-To: <ac141288-b7c7-49fb-8191-c5247b46a49e@googlegroups.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <mailman.350.1431339365.12865.python-list@python.org>
 <ac141288-b7c7-49fb-8191-c5247b46a49e@googlegroups.com>
Message-ID: <7734399a-fe22-4aec-b75c-bcc9b74418dd@googlegroups.com>

On Wednesday, May 13, 2015 at 6:17:41 AM UTC+5:30, John Ladasky wrote:
> On Monday, May 11, 2015 at 3:16:16 AM UTC-7, Antoon Pardon wrote:
> 
> > Try overriding None, True or False in python3 and see what happens.
> 
> Much fun was able to be had in Python 2, though:
> 
>     Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
>     [GCC 4.8.2] on linux2
>     Type "help", "copyright", "credits" or "license" for more information.
>     >>> True = False
>     >>> True
>     False
> 
> 
> Every politician's dream!

I actually had a use case for this [Or I am a politician]
Scheme had two True-s #T was the proper true and plain T was the legacy from
Lisp which was deprecated.
To make sure students would use #T and not T I had sneaked in somewhere
[dont exactly remember where/how] a
(set! T nil) 
ie T = nil

As far as I was concerned it was moving from deprecation to all-out error
However 2 decades later and some of those (victims) still remember:
You made TRUE into FALSE!!


From steve+comp.lang.python at pearwood.info  Tue May 12 21:14:56 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 13 May 2015 11:14:56 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
 <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
 <PRINT-20150512175315@ram.dialup.fu-berlin.de>
 <mailman.400.1431446739.12865.python-list@python.org>
Message-ID: <5552a591$0$12979$c3e8da3$5496439d@news.astraweb.com>

On Wed, 13 May 2015 02:05 am, Chris Angelico wrote:

> So if you're writing a library function, it probably shouldn't use
> print()... but your application is most welcome to. You usually know
> which one you're writing at any given time.

You might be, but beginners are not.

I'm not sure I accept Rustom's fix for the problem (I think that his cure is
worse than the disease), but it is *hard* to get some beginners to use
return instead of print:

def add_twice(x, y):
    """Add twice y to x."""
    print x + 2*y


sort of thing.

Personally, I think that banning print is only useful if you wish to
encourage cargo-cult programming:

"Don't use print!" 
"Why not?"
"My CS lecture said not to use it! I dunno, maybe it has a virus or
something."


I'd rather give them exercises designed to show (rather than tell) the
differences between printing a result and returning a result, and how they
effect re-usability of software components and function chaining. Using a
procedural language is *perfect* for that, since you can highlight the
differences:

function foo(n:int): int;
  begin
    foo := n+1;
  end;

procedure foo(n:int);
  begin
    writeln(n+1);
  end;



-- 
Steven



From steve+comp.lang.python at pearwood.info  Tue May 12 21:23:00 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 13 May 2015 11:23:00 +1000
Subject: uPy Unicode [was Re: Instead of deciding between Python or Lisp blah
 blah blah]
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <mailman.406.1431451597.12865.python-list@python.org>
Message-ID: <5552a774$0$12993$c3e8da3$5496439d@news.astraweb.com>

On Wed, 13 May 2015 03:26 am, Chris Angelico wrote:

> back when MicroPython was debating the implementation of Unicode
> strings, there was a lengthy discussion on python-dev about whether
> it's okay for string subscripting to be O(n) instead of O(1), and the
> final decision was that yes, that's an implementation detail. (UTF-8
> internal string representation, so iterating over a string would still
> yield characters in overall O(n), but iterating up to the string's
> length and subscripting for each character would become O(n*n) on
> uPy.)

o_O

Got a link to that? I must have missed it.


-- 
Steven



From rustompmody at gmail.com  Tue May 12 22:02:35 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 19:02:35 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <5552a591$0$12979$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
 <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
 <PRINT-20150512175315@ram.dialup.fu-berlin.de>
 <mailman.400.1431446739.12865.python-list@python.org>
 <5552a591$0$12979$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <c1058d53-0ac1-40ae-bf7c-2611c3dd6863@googlegroups.com>

On Wednesday, May 13, 2015 at 6:45:07 AM UTC+5:30, Steven D'Aprano wrote:
> On Wed, 13 May 2015 02:05 am, Chris Angelico wrote:
> 
> > So if you're writing a library function, it probably shouldn't use
> > print()... but your application is most welcome to. You usually know
> > which one you're writing at any given time.
> 
> You might be, but beginners are not.
> 
> I'm not sure I accept Rustom's fix for the problem (I think that his cure is
> worse than the disease), but it is *hard* to get some beginners to use
> return instead of print:
> 
> def add_twice(x, y):
>     """Add twice y to x."""
>     print x + 2*y
> 
> 
> sort of thing.
> 
> Personally, I think that banning print is only useful if you wish to
> encourage cargo-cult programming:
> 
> "Don't use print!" 
> "Why not?"
> "My CS lecture said not to use it! I dunno, maybe it has a virus or
> something."

:-)

[And more real than you know --- every medicine has a side-effect]
Just to be be clear about the framing -- though it does not work too well
across Usenet --
I would suggest to TEACHERS to not use/show print TOO EARLY

Something like cigarettes: 
You dont ban children from smoking. You ban adults from setting up tobacco
shops near schools

> procedural language is *perfect* for that, since you can highlight the
> differences:
>  
> function foo(n:int): int;
>   begin
>     foo := n+1;
>   end;
>  
> procedure foo(n:int);
>   begin
>     writeln(n+1);
>   end; 

This cuts both ways
Python much more than Pascal can make the 1st possible because of data
structures in general being first class

Unfortunately python culture across the net is laissez faire about using the
2nd -- I am demoing SMTP protocol... Does print or no print really have anything
to do with it?

When you give the procedure foo vs function foo example, you are jumping across 
the levels of familiarity and usage to assessment/evaluation.

Shall I use an if or a while?
Shall I use a list or a dict?
Shall I use immutable + or mutable extend/append? 
:
:
And a dozen other more fundamental things to assess.

If I could count the number of times a beginner hangs an else onto a while...
Students dont 'Not get it' because they are asses but because they are at a
firehose.  The asses are the adults who stand by and keep turning on the pressure


From no.email at nospam.invalid  Tue May 12 22:16:53 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Tue, 12 May 2015 19:16:53 -0700
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <mailman.350.1431339365.12865.python-list@python.org>
 <ac141288-b7c7-49fb-8191-c5247b46a49e@googlegroups.com>
 <7734399a-fe22-4aec-b75c-bcc9b74418dd@googlegroups.com>
Message-ID: <87twvhz0fe.fsf@jester.gateway.sonic.net>

Rustom Mody <rustompmody at gmail.com> writes:
> You made TRUE into FALSE!!

The answer is: yes!  Haskell can do that.

   Prelude> let 2+2=5 in 2+2
   5


From steve+comp.lang.python at pearwood.info  Tue May 12 22:17:01 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 13 May 2015 12:17:01 +1000
Subject: Updating a package on PyPi, testing and etiquette
References: <mitf66$2sq$4@dont-email.me>
 <0a7014a1-2320-4d22-ad14-c33adf755861@googlegroups.com>
Message-ID: <5552b41e$0$12976$c3e8da3$5496439d@news.astraweb.com>

On Wed, 13 May 2015 04:18 am, Ned Batchelder wrote:

> On Tuesday, May 12, 2015 at 2:03:04 PM UTC-4, Rob Gaddi wrote:
[...]
>> B) If I can't manage that, what's the etiquette behind having later
>> versions of a module break compatibility with older versions of Python.
>> I've avoided using features I know are newer, like yield from and Enums,
>> but I won't swear it'll work on 3.0 if I can't test it that way.
> 
> No one supports 3.0 any more.  3.2 is kind of the oldest version
> that is still reasonable, and many people are only aiming for 3.3
> and 3.4.

No one *should* support 3.0. It is semi-officially "broken".

The first production-ready (i.e. "final") version of 3.0 was released on
December 3rd, 2008. It soon became obvious that it had serious problems,
and following a single point release (3.0.1) it reached official end of
life with the release of 3.1 on June 27th, 2009.

There's no outright prohibition on supporting 3.0, and if somebody wants to
use it they can (we can't stop you, it's FOSS), but unlike older versions
which have reached end of life in the fullness of time, 3.0 never had a
user-base actually relying on it in production. It would be a waste of time
to explicitly support it.

Anything written for 3.1 will probably work in 3.0, and if it doesn't, it's
probably a bug in 3.0. Either way, I wouldn't waste time fixing bugs that
only affected 3.0.


> Just because you are supporting 2.7 and 3.4 doesn't mean you need
> to support 3.0 and 3.1.  

I agree whole-heartily with that. 2.x and 3.x are a fork in the language,
and you can support older versions on one branch without necessarily doing
the same on the other branch.

# requires fixed-width font for best results

1.4->1.5->2.0->2.1->2.2->2.3->2.4->2.5->2.6->2.7
       \                           \
        \                           3.0->3.1->3.2->3.3->3.4
         1.6


(I am unsure whether what became 3.0 *literally* was forked from 2.4, I
suspect not, but 2.5 was intended as the first of the transition versions
for moving between 2.4 and what was then known as Python 3000, and 2.6 was
the first to explicitly include Python 3 features via the __future__
module.)


> 2.x and 3.x are different languages.  

I wish people wouldn't say that. They're different languages in the same
sense that American English and British English are different languages. I
prefer to emphasise the 98% similarities by calling them different dialects
than the 2% differences -- especially since it has turned out to be far
easier and more practical to write hybrid 2.x + 3.x code than anyone
imagined back in the Python 3000 days.



-- 
Steven



From steve+comp.lang.python at pearwood.info  Tue May 12 22:30:37 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 13 May 2015 12:30:37 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
Message-ID: <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>

On Wed, 13 May 2015 08:00 am, zipher wrote:

> Everyone gets it wrong and now we have a plethora of languages which all
> do the same thing, without really knowing what they want as an overarching
> design or purpose.


Why must a language be designed with some "overarching design or purpose"?

Why can't a language be designed with a *practical and concrete* need in
mind? As far as I know, only one language designed from theoretical first
principles has had any measure of mainstream success, Lisp, and that was
probably because there weren't that many decent alternatives at the time.


"I want to do numerical calculations" lead to Fortran.

"I want to control telescopes" lead to Forth.

"I want to teach beginners programming" lead to BASIC.

"I want to teach beginners good programming" lead to Pascal.

"I want to generate webpages on the fly" lead to PHP.

"I want to write interactive fiction and text-based games" lead to Inform.

And so forth.




-- 
Steven



From rustompmody at gmail.com  Tue May 12 22:31:34 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 19:31:34 -0700 (PDT)
Subject: anomaly
In-Reply-To: <87twvhz0fe.fsf@jester.gateway.sonic.net>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu>
 <mailman.350.1431339365.12865.python-list@python.org>
 <ac141288-b7c7-49fb-8191-c5247b46a49e@googlegroups.com>
 <7734399a-fe22-4aec-b75c-bcc9b74418dd@googlegroups.com>
 <87twvhz0fe.fsf@jester.gateway.sonic.net>
Message-ID: <e1fc200c-9da7-49dd-a313-70f2999f30dc@googlegroups.com>

On Wednesday, May 13, 2015 at 7:47:03 AM UTC+5:30, Paul Rubin wrote:
> Rustom Mody  writes:
> > You made TRUE into FALSE!!
> 
> The answer is: yes!  Haskell can do that.
> 
>    Prelude> let 2+2=5 in 2+2
>    5

:-)

And we come back to the OP.
It *looks* like this is some profound question about OO-philosophy
But name-resolution has happened before that and in an unexpected fashion

Like puzzling over a strange printf behavior in C
And not noticing that the compiler-call was
$ gcc -Dprintf=puts ...



From no.email at nospam.invalid  Tue May 12 22:38:12 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Tue, 12 May 2015 19:38:12 -0700
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87lhgtyzfv.fsf@jester.gateway.sonic.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
> As far as I know, only one language designed from theoretical first
> principles has had any measure of mainstream success, Lisp, 

APL was cool back in the day too.


From dreamingforward at gmail.com  Tue May 12 22:47:39 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 12 May 2015 19:47:39 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <5552a591$0$12979$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
 <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
 <PRINT-20150512175315@ram.dialup.fu-berlin.de>
 <mailman.400.1431446739.12865.python-list@python.org>
 <5552a591$0$12979$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <f3cc4eee-8e0d-4bd9-acd5-f08b3186b0c8@googlegroups.com>

On Tuesday, May 12, 2015 at 8:15:07 PM UTC-5, Steven D'Aprano wrote:
> On Wed, 13 May 2015 02:05 am, Chris Angelico wrote:
> 
> > So if you're writing a library function, it probably shouldn't use
> > print()... but your application is most welcome to. You usually know
> > which one you're writing at any given time.
> 
> You might be, but beginners are not.
> 
> I'm not sure I accept Rustom's fix for the problem (I think that his cure is
> worse than the disease), but it is *hard* to get some beginners to use
> return instead of print:
> 
> def add_twice(x, y):
>     """Add twice y to x."""
>     print x + 2*y
> 
> 
> sort of thing.
> 
> Personally, I think that banning print is only useful if you wish to
> encourage cargo-cult programming:
> 
> "Don't use print!" 
> "Why not?"
> "My CS lecture said not to use it! I dunno, maybe it has a virus or
> something."

No, no, no.  There's a very simple reason that you don't put extraneous I/O into your functions:  you want your functions to be as general as possible for the given focus for re-usability.  Otherwise, why write as a separate function?  It's called separability of domains.

See Hacking with the Tao:  http://wiki.hackerspaces.org/Hacking_with_the_Tao.

Mark


From dreamingforward at gmail.com  Tue May 12 23:08:56 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 12 May 2015 20:08:56 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mailman.408.1431454962.12865.python-list@python.org>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
 <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
 <f84f7383-8fae-4b2d-b5d2-70285a46e584@googlegroups.com>
 <mailman.408.1431454962.12865.python-list@python.org>
Message-ID: <3aa9c854-4a46-47b8-9742-5f3744557f6c@googlegroups.com>

On Tuesday, May 12, 2015 at 1:22:55 PM UTC-5, Mark Lawrence wrote:
> On 12/05/2015 18:35, Rustom Mody wrote:
> > On Tuesday, May 12, 2015 at 8:48:13 PM UTC+5:30, zipher wrote:
> >>
> >> I/O is an essential part of computing in the West.  (I'll leave Symbolics out of of that category.)  It started with switches and lights, so what kind of bullshit is saying that you should get rid of PRINT?  ACM must have gotten confused or steamrolled by one of its members.
> >
> > In the West? WEST??
> > Did I hear that right?
> >
> 
> I prefer this[1]
> [1]https://mail.python.org/pipermail/python-ideas/2013-March/019979.html
> Mark Lawrence

I was trying to differentiate models of computing.  Perhaps you don't know the concept.  I'll refer you to http://c2.com/cgi/wiki?ModelsOfComputation.  One might think that they ALL originated from Western Civ, but not quite.

Yes, I was sounding little too enthusiastic in that post.

Mark


From dreamingforward at gmail.com  Tue May 12 23:11:35 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 12 May 2015 20:11:35 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <87y4ktwi20.fsf@elektro.pacujo.net>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <87y4ktwi20.fsf@elektro.pacujo.net>
Message-ID: <20f6a95e-85d7-4b59-8b4d-3d2ae2626f2c@googlegroups.com>

On Tuesday, May 12, 2015 at 5:24:34 PM UTC-5, Marko Rauhamaa wrote:
> zipher <dreamingforward at gmail.com>:
> 
> > That is why you have very high-level languages that allow you to
> > rapidly prototype ideas, test them, and then, depending all the other
> > constraints, move them to lower-level language implementations.
> 
> Finally an argument to tackle. That rapid prototyping role is often
> mentioned as a strong point of high-level languages. However, I can't
> remember personally doing that. Rather, you want to use the right
> programming language for any given role.

I know.  That's because most people have fallen off the path (http://c2.com/cgi/wiki?OneTruePath).  You haven't done it because either others have done it for you (NumPy) or you simply aren't perfecting anything that needs to scale; i.e. you don't really need to minimize memory or CPU consumption because you're working with toy problems relative to the power of most hardware these days.

Mark


From dreamingforward at gmail.com  Tue May 12 23:27:50 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 12 May 2015 20:27:50 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <2474c9be-caae-4c66-8892-e31a0d8346ad@googlegroups.com>

On Tuesday, May 12, 2015 at 9:30:50 PM UTC-5, Steven D'Aprano wrote:
> On Wed, 13 May 2015 08:00 am, zipher wrote:
> 
> > Everyone gets it wrong and now we have a plethora of languages which all
> > do the same thing, without really knowing what they want as an overarching
> > design or purpose.
> 
> 
> Why must a language be designed with some "overarching design or purpose"?

Because it's considerable work.  Why re-invent the wheel?  After all, there are plenty of Turing-complete languages out there.  No, you make one because you have an implicit or explicit goal.

> Why can't a language be designed with a *practical and concrete* need in
> mind? As far as I know, only one language designed from theoretical first
> principles has had any measure of mainstream success, Lisp,

Yes, and that was a stellar achievement.  Many language makers still compare to such a gold standard.  Even Python.  Yet it has also misled us -- it is based on a fundamentally different model of computation.

An elegant model of computation, that many high-level languages seem to try to adopt, yet they fail because they can't re-produce the elegance without becoming LISP.  So why, then, are there other programming languages that specifically try NOT to be LISP?  

Because the model of computation for most hardware is iterative.  This is also why the design goal of "everything should be an object" needs to be revisited.  It doesn't align with any model of computation.  That's fine for small problems, but when you're trying to make a p2p data model that can scale to the Internet, it becomes an issue.
 
> "I want to do numerical calculations" lead to Fortran.
> 
> "I want to control telescopes" lead to Forth.
> 
> "I want to teach beginners programming" lead to BASIC.

You can stop there.  You are proving my point, that there are goals to a given programming language.  If those goals are strong or noble enough, it differentiates from existing languages.  "I want to do numerical calculations" led to Fortran, which led to C because people wanted to do such calculations on many platforms.  And C syntax had the more elegant syntax ultimately (I think the increment++ operator ultimately ran Fortran over, because it was directly implementable on CPU architectures in 1 clock cycle).

"I want to control telescopes" could morph into the more general goal of "I want to make a controller programming language" and Boom, you'd have an elegant programming language for all controllers.

"I want to teach beginners programming" started at BASIC, but then led to Pascal as the limitations of BASIC became clear, but then became Python.  These are all decent design goals for a language.  Python has superceded BASIC in almost every way.

Anyway, having a design goal guides the process and helps everyone who wants to contribute know how to do so.

Mark


From rustompmody at gmail.com  Tue May 12 23:35:16 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 20:35:16 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>

On Wednesday, May 13, 2015 at 8:00:50 AM UTC+5:30, Steven D'Aprano wrote:
> Why can't a language be designed with a *practical and concrete* need in
> mind? As far as I know, only one language designed from theoretical first
> principles has had any measure of mainstream success, Lisp, and that was
> probably because there weren't that many decent alternatives at the time.

How history U-turns!!
Lisp actually got every major/fundamental thing wrong
- variables scopes were dynamic by mistake
- lambdas were non-first class because the locution 'first-class' was still 8 
years in the future

And what it got right was more by fluke than by design

- gc because the machine word was too small for a refcount but could squeeze in a mark-bit
- Syntax: It was intended that a 'proper' (aka Algol-like) syntax was round
the corner.  The '1.5' in the lisp 1.5 manual was evidence of that interimness

To me the most mind-boggling aspect this u-turning of history is this interview 
with McCarthy: http://www.infoq.com/interviews/Steele-Interviews-John-McCarthy

Q: Who helped you with the early ideas Lisp?
McCarthy: Backus' Fortran taught me functional programming!!!!!


From dreamingforward at gmail.com  Tue May 12 23:39:46 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 12 May 2015 20:39:46 -0700 (PDT)
Subject: anomaly
In-Reply-To: <mailman.397.1431445410.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
 <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
 <369b3ce9-14c9-465e-afea-407058a1617a@googlegroups.com>
 <mailman.397.1431445410.12865.python-list@python.org>
Message-ID: <eccfea80-bdd4-4639-ba1b-06c1800d2d61@googlegroups.com>

On Tuesday, May 12, 2015 at 10:43:44 AM UTC-5, Chris Angelico wrote:
> On Wed, May 13, 2015 at 1:34 AM, zipher <dreamingforward at gmail.com> wrote:
> > Name one "significant and important" use case for shadowing built-in types.  Functions, I don't have a problem with, but types are more fundamental than functions.
>
> Please tell me what, precisely, is the difference between a type and a
> function.

A type is something that settles matters for the machine.  A function calculates on the machine.  You can have a type without a function, but you generally can't have a function without types.  If a distorted example like function f(): {} has an implicit return of an int in ANSI C.

> Once you've settled that, please explain to me what the
> built-in name 'int' is in all versions of Python.

'int' is a type that also has a function synonymous with it.  But they should not be considered the same.  They could, for example, be decoupled.  So that 'int' was the type and 'integer' was the function.

Mark



> ChrisA



From ian.g.kelly at gmail.com  Tue May 12 23:46:52 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 12 May 2015 21:46:52 -0600
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <20f6a95e-85d7-4b59-8b4d-3d2ae2626f2c@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <87y4ktwi20.fsf@elektro.pacujo.net>
 <20f6a95e-85d7-4b59-8b4d-3d2ae2626f2c@googlegroups.com>
Message-ID: <CALwzidmsiy1vLc1WCBK=JGpnV+47AsOXyA4s-Az7_emv_nKpNw@mail.gmail.com>

On Tue, May 12, 2015 at 9:11 PM, zipher <dreamingforward at gmail.com> wrote:
> I know.  That's because most people have fallen off the path (http://c2.com/cgi/wiki?OneTruePath).

You wrote that, didn't you? I recognize that combination of delusional
narcissism and curious obsession with Turing machines.

> You haven't done it because either others have done it for you (NumPy) or you simply aren't perfecting anything that needs to scale; i.e. you don't really need to minimize memory or CPU consumption because you're working with toy problems relative to the power of most hardware these days.

There is such a thing as over-optimization. Given unlimited programmer
time, sure, everything might be made to run using the minimum possible
time and space. Nobody has unlimited time, though. The job of the
programmer is to get the program to run "fast enough" for the needs of
the application. Getting it to run faster than it needs to is
generally a waste of the programmer's time that could be spent on more
valuable tasks.

Of course, I say this as somebody who works on a highly scaled
user-facing application that will never be "fast enough". :-)


From steve+comp.lang.python at pearwood.info  Tue May 12 23:58:46 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 13 May 2015 13:58:46 +1000
Subject: anomaly
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
 <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
 <369b3ce9-14c9-465e-afea-407058a1617a@googlegroups.com>
 <mailman.403.1431447729.12865.python-list@python.org>
Message-ID: <5552cbf8$0$2841$c3e8da3$76491128@news.astraweb.com>

On Wednesday 13 May 2015 02:22, Skip Montanaro wrote:

> I did find this interesting blog post about encapsulation and
> test-driven development:
> 
> https://jasonmbaker.wordpress.com/2009/01/08/enemies-of-test-driven-
development-part-i-encapsulation/
> 
> I found the author's perspective interesting.


Very nice!



-- 
Steve



From rosuav at gmail.com  Wed May 13 00:54:59 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 13 May 2015 14:54:59 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <5552a591$0$12979$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
 <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
 <PRINT-20150512175315@ram.dialup.fu-berlin.de>
 <mailman.400.1431446739.12865.python-list@python.org>
 <5552a591$0$12979$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmoKrc38OzjJ1sU6gJMMGO0bYZVT6c_yXw_Ebq1oKuLNMw@mail.gmail.com>

On Wed, May 13, 2015 at 11:14 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Wed, 13 May 2015 02:05 am, Chris Angelico wrote:
>
>> So if you're writing a library function, it probably shouldn't use
>> print()... but your application is most welcome to. You usually know
>> which one you're writing at any given time.
>
> You might be, but beginners are not.

I meant the generic "you" there. A beginner may well not know whether
to use / or //, whether it's better to use a list or a dict, etc etc
etc. That's what instructors are for. Make the distinction that
library functions shouldn't use print but application code can, and
then examples like this...

> I'm not sure I accept Rustom's fix for the problem (I think that his cure is
> worse than the disease), but it is *hard* to get some beginners to use
> return instead of print:
>
> def add_twice(x, y):
>     """Add twice y to x."""
>     print x + 2*y
>
>
> sort of thing.

... can be answered simply by explaining that "add_twice" ought to be
written as a library function rather than an application. It's the
exact same answer as this:

def find_all_whatevers(base_dir):
    whatevers = []
    os.chdir(base_dir)
    for fn in os.listdir():
        if fn is a whatever: whatevers.append(fn)
        if fn is a directory: whatevers.extend(find_all_whatevers(fn))
    return whatevers

The working directory belongs to the application, not to a library
function, so this shouldn't use os.chdir(), even though it does spare
you the effort of string manipulation. (Of course, a real-world
version of this should use os.walk(), but that's a complete rewrite.)

The precise boundary between reusable functions and the actual
application isn't always easy to draw, and there'll be times when you
refactor something across that line. That's not a problem. There are
still plenty of cases that are clear enough to use for explanation.

> Personally, I think that banning print is only useful if you wish to
> encourage cargo-cult programming:
>
> "Don't use print!"
> "Why not?"
> "My CS lecture said not to use it! I dunno, maybe it has a virus or
> something."

Agreed. There are VERY few features which should be utterly and
totally banned, and they're usually kept only for backward
compatibility with a time when people didn't know how dangerous they
were. In Python, the only one I can think of is Py2's input(), which
should be treated as XKCD 292 treats GOTO. (If you really *do* want to
take a string from the user and immediately eval it, just write it as
"eval(raw_input())" so everyone knows.) C has gets(), which is
similarly dangerous and has a similarly straight-forward replacement.
PHP has register_globals (or did until recently - it took a long time
to go from "default is on, we recommend you turn it off" through
"default is off, we recommend you don't turn it on" to finally "bad
feature is removed"). Beyond those, there's not much that doesn't have
at least _some_ reason for existing.

> I'd rather give them exercises designed to show (rather than tell) the
> differences between printing a result and returning a result, and how they
> effect re-usability of software components and function chaining. Using a
> procedural language is *perfect* for that, since you can highlight the
> differences:
>
> function foo(n:int): int;
>   begin
>     foo := n+1;
>   end;
>
> procedure foo(n:int);
>   begin
>     writeln(n+1);
>   end;

Yep. I'd also use clear function/procedure names to make it more
visible, and probably tie this in with loops to show how you can print
more than one thing but can only return one. (Generators are a more
advanced topic.) A few examples go a long way.

ChrisA


From rosuav at gmail.com  Wed May 13 01:13:32 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 13 May 2015 15:13:32 +1000
Subject: uPy Unicode [was Re: Instead of deciding between Python or Lisp
 blah blah blah]
In-Reply-To: <5552a774$0$12993$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <mailman.406.1431451597.12865.python-list@python.org>
 <5552a774$0$12993$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmpBRu-RnM2BpcGbThAdw-3ZtAEVO_UXY4WFcSiOnEW52w@mail.gmail.com>

On Wed, May 13, 2015 at 11:23 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Wed, 13 May 2015 03:26 am, Chris Angelico wrote:
>
>> back when MicroPython was debating the implementation of Unicode
>> strings, there was a lengthy discussion on python-dev about whether
>> it's okay for string subscripting to be O(n) instead of O(1), and the
>> final decision was that yes, that's an implementation detail. (UTF-8
>> internal string representation, so iterating over a string would still
>> yield characters in overall O(n), but iterating up to the string's
>> length and subscripting for each character would become O(n*n) on
>> uPy.)
>
> o_O
>
> Got a link to that? I must have missed it.

Linking to python-dev is a bit fiddly and/or unstable due to URL
changes, plus the discussion there was pretty long and rambly.
Probably the best I can do is point you to the tracker issue where I
opened the original question:

https://github.com/micropython/micropython/issues/657

(The biggest issue was that uPy was, at the time, fundamentally
incompatible with Python's stipulated semantics - imagine all the
problems of a narrow build of CPython <3.3, only more frequent because
it's actually UTF-8.)

It was finally decided, I think, that Python-the-language didn't
actually mandate O(1) indexing, meaning that a microcontroller (on
which strings aren't going to be gigantic anyway) is welcome to use a
UTF-8 internal representation, with "Hello, world"[4] required to scan
across and count non-continuation bytes to find the right character.
Whether or not uPy actually ended up accepting the requirements of
proper Unicode support I don't know, as I'm no longer involved with
the project.

ChrisA


From dieter at handshake.de  Wed May 13 01:55:23 2015
From: dieter at handshake.de (dieter)
Date: Wed, 13 May 2015 07:55:23 +0200
Subject: urllib2.urlopen error "socket.error: [Errno 104] Connection reset
 by peer"
References: <tencent_2B2ADFEE54D112343708D73B@qq.com>
Message-ID: <87r3ql9g38.fsf@handshake.de>

"Jia CHEN" <jia.j.chen at foxmail.com> writes:

> I have the error below when trying to download the html content of a webpage. I can open this webpage in a browser without any problem.

"Connection reset by peer" means that the other side (the HTTP server
in your case) has closed the connection.

It may have looked at the "User-Agent" request header to differentiate
between a browser request and an automated (script) request.

To work around this, you may provide a "User-Agent" header
to your "urllib2.Request" (see the documentation) letting
the "User-Agent" to look as if the request had been done by a browser
("Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
indicates a firefoy).



From rustompmody at gmail.com  Wed May 13 01:56:57 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 22:56:57 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mailman.428.1431492908.12865.python-list@python.org>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
 <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
 <PRINT-20150512175315@ram.dialup.fu-berlin.de>
 <mailman.400.1431446739.12865.python-list@python.org>
 <5552a591$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <mailman.428.1431492908.12865.python-list@python.org>
Message-ID: <b107c4b2-9da2-465a-b323-3e47bf26cb3e@googlegroups.com>

On Wednesday, May 13, 2015 at 10:25:18 AM UTC+5:30, Chris Angelico wrote:
> On Wed, May 13, 2015 at 11:14 AM, Steven D'Aprano wrote:
> > On Wed, 13 May 2015 02:05 am, Chris Angelico wrote:
> >
> >> So if you're writing a library function, it probably shouldn't use
> >> print()... but your application is most welcome to. You usually know
> >> which one you're writing at any given time.
> >
> > You might be, but beginners are not.
> 
> I meant the generic "you" there. A beginner may well not know whether
> to use / or //, whether it's better to use a list or a dict, etc etc
> etc. That's what instructors are for. Make the distinction that
> library functions shouldn't use print but application code can, and
> then examples like this...
> 
> > I'm not sure I accept Rustom's fix for the problem (I think that his cure is
> > worse than the disease), but it is *hard* to get some beginners to use
> > return instead of print:
> >
> > def add_twice(x, y):
> >     """Add twice y to x."""
> >     print x + 2*y
> >
> >
> > sort of thing.
> 
> ... can be answered simply by explaining that "add_twice" ought to be
> written as a library function rather than an application. It's the
> exact same answer as this:

And later

> Yep. I'd also use clear function/procedure names to make it more
> visible, and probably tie this in with loops to show how you can print
> more than one thing but can only return one. (Generators are a more
> advanced topic.)

[Structure of] Library function: basic
Generator: advanced

????

If you ask me that's sdrawkcab


From rosuav at gmail.com  Wed May 13 02:17:06 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 13 May 2015 16:17:06 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <b107c4b2-9da2-465a-b323-3e47bf26cb3e@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
 <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
 <PRINT-20150512175315@ram.dialup.fu-berlin.de>
 <mailman.400.1431446739.12865.python-list@python.org>
 <5552a591$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <mailman.428.1431492908.12865.python-list@python.org>
 <b107c4b2-9da2-465a-b323-3e47bf26cb3e@googlegroups.com>
Message-ID: <CAPTjJmqHR6zV5Nkq7VM2V9y98rD4ChuE+YVLuApu+0uTxENxTA@mail.gmail.com>

On Wed, May 13, 2015 at 3:56 PM, Rustom Mody <rustompmody at gmail.com> wrote:
>> Yep. I'd also use clear function/procedure names to make it more
>> visible, and probably tie this in with loops to show how you can print
>> more than one thing but can only return one. (Generators are a more
>> advanced topic.)
>
> [Structure of] Library function: basic
> Generator: advanced
>
> ????
>
> If you ask me that's sdrawkcab

You think that generators are less advanced than the notion of code reuse??

ChrisA


From rustompmody at gmail.com  Wed May 13 02:31:07 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 23:31:07 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mailman.432.1431497829.12865.python-list@python.org>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <mailman.386.1431407062.12865.python-list@python.org>
 <b68efd25-017a-4bd4-830b-54cc79ec853b@googlegroups.com>
 <60c01c07-aef9-4232-92cd-22e6c017fc9c@googlegroups.com>
 <mailman.387.1431413836.12865.python-list@python.org>
 <4628bce6-98d5-44b7-bb3b-bcb796c0df77@googlegroups.com>
 <d4559cef-69fb-4a62-a300-cde3a9e127d6@googlegroups.com>
 <PRINT-20150512175315@ram.dialup.fu-berlin.de>
 <mailman.400.1431446739.12865.python-list@python.org>
 <5552a591$0$12979$c3e8da3$5496439d@news.astraweb.com>
 <mailman.428.1431492908.12865.python-list@python.org>
 <b107c4b2-9da2-465a-b323-3e47bf26cb3e@googlegroups.com>
 <mailman.432.1431497829.12865.python-list@python.org>
Message-ID: <3725fd3d-d0d7-4d07-a18a-db838e316c9a@googlegroups.com>

On Wednesday, May 13, 2015 at 11:47:19 AM UTC+5:30, Chris Angelico wrote:
> On Wed, May 13, 2015 at 3:56 PM, Rustom Mody  wrote:
> >> Yep. I'd also use clear function/procedure names to make it more
> >> visible, and probably tie this in with loops to show how you can print
> >> more than one thing but can only return one. (Generators are a more
> >> advanced topic.)
> >
> > [Structure of] Library function: basic
> > Generator: advanced
> >
> > ????
> >
> > If you ask me that's sdrawkcab
> 
> You think that generators are less advanced than the notion of code reuse??
> 

Notion of code reuse -- easy
Writing reusable code -- hard
Certainly much harder than writing working useful generators.

Do remember that historical and pedagogical order dont necessarily match.
A C programmer would say integers (scalars) are easy, lists are hard.
Not a notion necessary or useful in python

Let's just close this (for now) Chris with this:

You are in all likelihood a better programmer than I.
It so happens I have directly or indirectly taught near to 2000 students in about 25 years.

Lets reopen this conversation when you reach ? that number, ok?


From rustompmody at gmail.com  Wed May 13 02:36:02 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 12 May 2015 23:36:02 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mailman.427.1431488855.12865.python-list@python.org>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <87y4ktwi20.fsf@elektro.pacujo.net>
 <20f6a95e-85d7-4b59-8b4d-3d2ae2626f2c@googlegroups.com>
 <mailman.427.1431488855.12865.python-list@python.org>
Message-ID: <b3b151c8-54a0-423b-8ddc-4679ba50e766@googlegroups.com>

On Wednesday, May 13, 2015 at 9:17:48 AM UTC+5:30, Ian wrote:
> On Tue, May 12, 2015 at 9:11 PM, zipher  wrote:
> > I know.  That's because most people have fallen off the path (http://c2.com/cgi/wiki?OneTruePath).
> 
> You wrote that, didn't you? I recognize that combination of delusional
> narcissism and curious obsession with Turing machines.

Interestingly saw this (CACM) the other day:

http://www.tomandmaria.com/Tom/Writing/CACMActuallyTuringDidNotInventTheComputer.pdf

non-delusional and in the opposite direction:

Turing's is a paper on mathematical logic. It describes a
thought experiment, like Schr?dinger's famous 1935 description of
a trapped cat shifting between life and death in response to the
behavior of a single atom. Schr?dinger was not trying to advance
the state of the art of feline euthanasia. Neither was Turing
proposing the construction of a new kind of calculating machine.
As the title of his paper suggested, Turing designed his
ingenious imaginary machines to address a question about the
fundamental limits of mathematical proof. They were structured
for simplicity, and had little in common 
with the approaches taken by people 
designing actual machines. 

Von Neumann's report said nothing explicitly about mathematical
logic. It described the architecture of an actual planned
computer and the technologies by which it could be realized, and
was written to guide the team that had already won a contract to
develop the EDVAC. Von Neumann does abstract away from details of
the hardware, both to focus instead on what we would now call
"architecture" and because the computer projects under way at the
Moore School were still classified in 1945. His letters from that
period are full of discussion of engineering details, such as
sketches of particular vacuum tube models and their performance
characteristic


From antoon.pardon at rece.vub.ac.be  Wed May 13 03:07:35 2015
From: antoon.pardon at rece.vub.ac.be (Antoon Pardon)
Date: Wed, 13 May 2015 09:07:35 +0200
Subject: anomaly
In-Reply-To: <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
 <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5552F837.20004@rece.vub.ac.be>

Op 12-05-15 om 15:56 schreef Steven D'Aprano:

> On Tue, 12 May 2015 09:55 pm, Antoon Pardon wrote:
>
>> Op 11-05-15 om 16:13 schreef Chris Angelico:
>>
>>> Why does Python have most built-ins as simply looked-up names that can
>>> be overridden? Because otherwise, there would be a veritable ton of
>>> keywords:
>> But that doesn't answer the question why the developers chose "True" to be
>> a keyword and "int" to be a looked-up name.
> No it doesn't. But then, nobody until now has *asked* that question, merely
> commented on it as a fact.

Whether the question was asked or not is unimportant. Somebody felt the
some need for justification, because otherwise there was no need to
mentions this fact. There are thousands of facts about python, so 
why mention this and not an other? Not because he wanted to mention
a fact but because he wanted to make a point and that point was bogus.

> If you go all the way back to Python 1.5, not only did True and False not
> exist as built-ins, but None was not a keyword:

This is completely irrelevant.
 

>> and pretending to justify that choice by stating that the python thought
>> is: We're all adults here, if you want to override a builtin, who are we
>> to stop you. That is disingenuous.
> No, it's quite sensible, given Python's stated position: it's not an
> authoritarian B&D language like Pascal, nor is it a permissive "anything
> goes" language like C.

You are arguing at the wrong level and thus not addressing my point.
I don't think it is useful to continue further.



From greg.ewing at canterbury.ac.nz  Wed May 13 03:19:29 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Wed, 13 May 2015 19:19:29 +1200
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <crgc83F71evU1@mid.individual.net>

Steven D'Aprano wrote:

> "I want to do numerical calculations" lead to Fortran.
> 
> "I want to control telescopes" lead to Forth.

I don't think those things led to their respective languages
in the same way. The notation that mathematicians use for
numerical calculations had a clear influence on the syntax
of Fortran. But there's nothing about controlling telescopes
that makes the structure of Forth particularly suited to it.

I think there *is* a design principle behind Forth: the
desire to keep everything as simple as possible. It's a
lot like Lisp in that respect.

-- 
Greg



From antoon.pardon at rece.vub.ac.be  Wed May 13 03:23:19 2015
From: antoon.pardon at rece.vub.ac.be (Antoon Pardon)
Date: Wed, 13 May 2015 09:23:19 +0200
Subject: anomaly
In-Reply-To: <crfjk9F1ks8U1@mid.individual.net>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
 <crfjk9F1ks8U1@mid.individual.net>
Message-ID: <5552FBE7.4070602@rece.vub.ac.be>

Op 13-05-15 om 02:19 schreef Gregory Ewing:
> Antoon Pardon wrote:
>> But that doesn't answer the question why the developers chose "True"
>> to be a
>> keyword and "int" to be a looked-up name.
>
> Probably because True, False and None are very frequently
> used constants.

I don't care about the specific answer. I care about how specific
answers are supported.

-- 
Antoon Pardon.




From auriocus at gmx.de  Wed May 13 03:27:19 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Wed, 13 May 2015 09:27:19 +0200
Subject: Feature Request: Reposition Execution
In-Reply-To: <4568f$5552602d$5419aafe$61337@news.ziggo.nl>
References: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>
 <mailman.354.1431345441.12865.python-list@python.org>
 <4568f$5552602d$5419aafe$61337@news.ziggo.nl>
Message-ID: <miuual$nek$1@dont-email.me>

Am 12.05.15 um 22:18 schrieb Skybuck Flying:
> Thanks for suggestion, but I need a solution which can work in SikuliX
> as well.

What the hell is that?

> Especially inside an observer handler that would be ideal.
>
> So far os.kill() is not supported in SikuliX as far as I can tell.

OK a quick look to the webpage sikulix.com suggests, that it is based on 
Java and Python runs on top of the JVM (Jython). So maybe there is a JVM 
based mechanism to do that.

A clean way to exit your script could be to raise an exception. It 
should propagate to the toplevel and halt your script. However it is not 
possible to back and resume the execution.

	Christian



From __peter__ at web.de  Wed May 13 03:40:05 2015
From: __peter__ at web.de (Peter Otten)
Date: Wed, 13 May 2015 09:40:05 +0200
Subject: itertools py3.4 - filter list using not equal - fails as bool
References: <05defef5-74aa-4a5d-b7e7-9b521512152c@googlegroups.com>
Message-ID: <miuv4l$ri0$1@ger.gmane.org>

Sayth Renshaw wrote:

> why can't I filter a list based on an itertools condition using dropwhile?
> 
> This is the docs and the example.
> https://docs.python.org/3/library/itertools.html#itertools.dropwhile
> 
> def less_than_10(x):
>     return x < 10
> 
> itertools.takewhile(less_than_10, itertools.count()) =>
>   0, 1, 2, 3, 4, 5, 6, 7, 8, 9

As the example demonstrates dropwhile() takes a function as its first 
argument. To apply it to a sequence of tuples you could write

>>> items
[(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3), (3, 1), (3, 2), (3, 3)]
>>> def first_is_one(t):
...     return t[0] == 1
... 
>>> list(itertools.dropwhile(first_is_one, items))
[(2, 1), (2, 2), (2, 3), (3, 1), (3, 2), (3, 3)]

However, if not all items where the predicate function evaluates to True are 
at the beginning of the sequence:

>>> def second_is_one(t):
...     return t[1] == 1
... 
>>> list(itertools.dropwhile(second_is_one, items))
[(1, 2), (1, 3), (2, 1), (2, 2), (2, 3), (3, 1), (3, 2), (3, 3)]

If you want only the items where t[1] != 1 you need filter() or 
itertools.filterfalse() (Python 2: itertools.ifilter or 
itertools.ifilterfalse)

>>> list(itertools.filterfalse(second_is_one, items))
[(1, 2), (1, 3), (2, 2), (2, 3), (3, 2), (3, 3)]

> so I have a list I have created (converted from itertools). pm is
> permutations
> 
> 
> print(stats)
> [(1, 2, 3), (1, 2, 4), (1, 2, 5), (1, 3, 2), (1, 3, 4), (1, 3, 5), (1, 4,
> [2), (1, 4, 3), (1, 4, 5), (1, 5, 2), (1, 5, 3), (1, 5, 4), (2, 1, 3), (2,
> [1, 4), (2, 1, 5), (2, 3, 1), (2, 3, 4), (2, 3, 5), (2, 4, 1), (2, 4, 3),
> [(2, 4, 5), (2, 5, 1), (2, 5, 3), (2, 5, 4), (3, 1, 2), (3, 1, 4), (3, 1,
> [5), (3, 2, 1), (3, 2, 4), (3, 2, 5), (3, 4, 1), (3, 4, 2), (3, 4, 5), (3,
> [5, 1), (3, 5, 2), (3, 5, 4), (4, 1, 2), (4, 1, 3), (4, 1, 5), (4, 2, 1),
> [(4, 2, 3), (4, 2, 5), (4, 3, 1), (4, 3, 2), (4, 3, 5), (4, 5, 1), (4, 5,
> [2), (4, 5, 3), (5, 1, 2), (5, 1, 3), (5, 1, 4), (5, 2, 1), (5, 2, 3), (5,
> [2, 4), (5, 3, 1), (5, 3, 2), (5, 3, 4), (5, 4, 1), (5, 4, 2), (5, 4, 3)]
> 
> 
> I simply wanted to create an easy way to create summary stats of my stats
> list(poorly named). So in this case easy to check answers. so how many
> tuples in my list have a 1 in item[0] and how many don't. Then hoping to
> build on that for example how many have item[0] == 1 && (item[1] == 2 or
> item[1] == 4) etc.
> 
> I can achieve it via an else if but that would become ugly quick.
> 
> for item in stats:
>     if item[0] == 1:
>         nums += 1
>     elif item[0] != 1:
>         not_in += 1
>     else:
>         pass

Why would this become ugly? You can reshuffle it a bit to make it more 
general:

>>> def count(items):
...     t = f = 0
...     for item in items:
...         if item:
...             t += 1
...         else:
...             f += 1
...     return f, t
... 
>>> count(t[1] == 1 for t in stats)
(48, 12)
>>> count(t[0] == 1 for t in stats)
(48, 12)
>>> count(t[0] in (1, 2) for t in stats)
(36, 24)
>>> count(sum(t) == 6 for t in stats)
(54, 6)

Alternatively collections.Counter() supports an arbitrary number of bins...

>>> import collections
>>> freq = collections.Counter(t[1] for t in stats)
>>> freq
Counter({1: 12, 2: 12, 3: 12, 4: 12, 5: 12})

...but you can easily reduce them:

>>> freq = collections.Counter(t[1] == 1 for t in stats)
>>> freq
Counter({False: 48, True: 12})
>>> one = freq[True]
>>> total = sum(freq.values())
>>> print("{} of {} ({}%) have t[1] == 1".format(one, total, one/total*100))
12 of 60 (20.0%) have t[1] == 1





From greg.ewing at canterbury.ac.nz  Wed May 13 04:20:55 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Wed, 13 May 2015 20:20:55 +1200
Subject: How to properly apply OOP in the bouncing ball code
In-Reply-To: <mailman.398.1431445807.12865.python-list@python.org>
References: <009ceef8-066d-4d92-a6c9-761e86584e75@googlegroups.com>
 <45313faa-d256-4fa4-9e37-4b8dd85e1681@googlegroups.com>
 <mailman.383.1431390292.12865.python-list@python.org>
 <a25437cb-c525-41db-9bd4-c9759a982a3b@googlegroups.com>
 <mailman.398.1431445807.12865.python-list@python.org>
Message-ID: <crgfr9F7updU1@mid.individual.net>

Terry Reedy wrote:
> On 5/11/2015 8:42 PM, zipher wrote:
>> Huh?  VPython successfully models particle motion simulation with 
>> discrete events.

That's a discrete approximation to a continuous process.
The term "discrete event simulation" is usually used to
mean modelling a process that is inherently discrete,
e.g. sending data packets over a network.

-- 
Greg


From subhabrata.banerji at gmail.com  Wed May 13 04:27:01 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Wed, 13 May 2015 01:27:01 -0700 (PDT)
Subject: Integrating Python Code with Django
Message-ID: <698df135-ac0c-4d6b-9550-c15dd4f05a6d@googlegroups.com>

Dear Group,

I am trying to learn Django. My initial exercise seems fine. I want to create 
an API with REST on Django for an interactive Python code. REST framework on Django I am understanding more or less. 

I am looking for a simple example to start with. I am using Python 2.7+ on MS-Windows 7 Professional. 

If any one may kindly suggest. 

Regards,
Subhabrata Banerjee. 



From info at egenix.com  Wed May 13 04:58:06 2015
From: info at egenix.com (eGenix Team: M.-A. Lemburg)
Date: Wed, 13 May 2015 10:58:06 +0200
Subject: ANN: eGenix mxODBC Connect 2.1.3 - Remote Python Database Interface
Message-ID: <5553121E.2050108@egenix.com>

________________________________________________________________________

ANNOUNCING
                      eGenix.com mxODBC Connect

                  Remote Python Database Interface

                            Version 2.1.3


 mxODBC Connect is our commercially supported client-server product for
       connecting Python applications to relational databases
                 in a truly platform independent way.


This announcement is also available on our web-site for online reading:
http://www.egenix.com/company/news/eGenix-mxODBC-Connect-2.1.3-GA.html

________________________________________________________________________

INTRODUCTION

The mxODBC Connect Database Interface for Python allows users to
easily connect Python applications to all major databases on the
market today in a highly portable, convenient and secure way.

Python Database Connectivity the Easy Way
-----------------------------------------

Unlike our mxODBC Python extension, mxODBC Connect is designed as
client-server application, so you no longer need to find production
quality database drivers for all the platforms you target with your
Python application.

Instead, you use an easy to install royalty-free Python client library
which connects directly to the mxODBC Connect database server over the
network.

This makes mxODBC Connect a great basis for writing cross-platform
multi-tier database applications and utilities in Python, especially
if you run applications that need to communicate with databases such
as MS SQL Server and MS Access, Oracle Database, IBM DB2 and Informix,
Sybase ASE and Sybase Anywhere, MySQL, PostgreSQL, SAP MaxDB and many
more, that run on Windows or Linux machines.

Ideal for Database Driven Client Applications
---------------------------------------------

By removing the need to install and configure ODBC drivers on the
client side and dealing with complicated network setups for each set
of drivers, mxODBC Connect greatly simplifies deployment of database
driven client applications, while at the same time making the network
communication between client and database server more efficient and
more secure.

For more information, please have a look at the mxODBC Connect product
page, in particular, the full list of available features.

For more information, please see the product page:

    http://www.egenix.com/products/python/mxODBCConnect/

________________________________________________________________________

NEWS

mxODBC Connect 2.1.3 is a patch level release of our successful mxODBC
Connect database product. It includes these enhancements and fixes:

Security Enhancements
---------------------

 * Updated included OpenSSL libraries to 1.0.1m. See
   https://www.openssl.org/news/secadv_20150319.txt for a complete
   list of changes. Among other security fixes, this addresses the
   FREAK Attack (CVE-2015-0204).

 * Upgraded default RSA key length for demo certificates to 2048 bits.

mxODBC Connect Enhancements
---------------------------

 * Upgraded eGenix PyRun used for mxODBC Connect Server on Linux to
   2.1.0.

 * Upgraded the Python version used for mxODBC Connect Server to
   2.7.9.

 * Fixed a bug causing pip uninstall not to remove .pyc/.pyo files

 * Resolved an intermittent error related to hash seeds which
   sometimes caused prebuilt archives to not install correctly. Thanks
   to Albert-Jan Roskam for reporting this.

mxODBC API Enhancements
-----------------------

 * Upgraded the mxODBC Connect Server to mxODBC 3.3.2:

   https://cms.egenix.com/company/news/eGenix-mxODBC-3.3.2-GA.html

MS SQL Server
-------------

 * Fixed an "ODBC driver sent negative string size" error when using
   empty strings or None with output parameters for SQL Server ODBC
   drivers.

 * Clarified that due to the way the SQL Server ODBC driver sends
   data, mixing output parameters and output result sets is not
   possible. A work-around for this is to send back output parameters
   as additional result set.

SAP Sybase ASE
--------------

 * Added a work-around for the Sybase ASE ODBC driver which has
   problems with BIGINT columns. These are now supported.

 * Fixed a possible "ODBC driver sent negative string size" error when
   using empty strings or None with output parameters for Sybase ASE
   ODBC drivers.

Fixes
-----

 * Fixed the handling of None as default value for output parameters
   in e.g. stored procedures to use VARCHAR binding rather than CHAR
   binding. The latter caused padding with some database backends.

 * Changed cursor.colcount to be determined on-demand rather than
   right after the prepare step of statement execution.

For the full set of changes, including those of the 2.1 series of
mxODBC Connect, please check the mxODBC Connect change log:

    http://www.egenix.com/products/python/mxODBCConnect/changelog.html

________________________________________________________________________

UPGRADING

You are encouraged to upgrade to this latest mxODBC Connect release.
When upgrading, please always upgrade both the server and the client
installations to the same version - even for patch level releases.

We will give out 20% discount coupons for upgrade purchases going from
mxODBC Connect Server 1.x to 2.1 and 50% coupons for upgrades from
mxODBC Connect Server 2.x to 2.1. Please contact the eGenix.com Sales
Team (sales at egenix.com) with your existing license serials for
details.

Users of our stand-alone mxODBC product will have to purchase new
licenses from our online shop in order to use mxODBC Connect.

You can request free 30-day evaluation licenses by visiting our
web-site or writing to sales at egenix.com, stating your name (or the
name of the company) and the number of eval licenses that you need.

    http://www.egenix.com/products/python/mxODBCConnect/#Evaluation

________________________________________________________________________

DOWNLOADS

The download archives as well as instructions for installation and
configuration of the product can be found on the product page:

    http://www.egenix.com/products/python/mxODBCConnect/

If you want to try the package, jump straight to the download
instructions:

    https://cms.egenix.com/products/python/mxODBCConnect/#Download

Fully functional evaluation licenses for the mxODBC Connect Server are
available free of charge:

    http://www.egenix.com/products/python/mxODBCConnect/#Evaluation

The mxODBC Connect Client is always free of charge.

_______________________________________________________________________

SUPPORT

Commercial support for this product is available from eGenix.com.

Please see

    http://www.egenix.com/services/support/

for details about our support offerings.

_______________________________________________________________________

INFORMATION

About Python (http://www.python.org/):

    Python is an object-oriented Open Source programming language
    which runs on all modern platforms. By integrating ease-of-use,
    clarity in coding, enterprise application connectivity and rapid
    application design, Python establishes an ideal programming
    platform for today's IT challenges.

About eGenix (http://www.egenix.com/):

    eGenix is a software project, consulting and product company
    focusing on expert project services and professional quality
    products for companies, Python users and developers.

Enjoy,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 13 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From steve+comp.lang.python at pearwood.info  Wed May 13 05:07:37 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 13 May 2015 19:07:37 +1000
Subject: Feature Request: Reposition Execution
References: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>
 <mailman.354.1431345441.12865.python-list@python.org>
 <4568f$5552602d$5419aafe$61337@news.ziggo.nl> <miuual$nek$1@dont-email.me>
Message-ID: <5553145b$0$11119$c3e8da3@news.astraweb.com>

On Wednesday 13 May 2015 17:27, Christian Gollwitzer wrote:

> A clean way to exit your script could be to raise an exception. It
> should propagate to the toplevel and halt your script. However it is not
> possible to back and resume the execution.

while True:
    try:
        run_script()  # May raise TryAgain
        break
    except TryAgain:
        pass


If you prefer to only retry a finite number of times:


for i in range(10):
    try:
        run_script()  # May raise TryAgain
        break
    except TryAgain:
        pass
else:
    # break skips past the for...else block
    raise GiveUpError('too many failures')





-- 
Steve



From kylotan at gmail.com  Wed May 13 06:23:43 2015
From: kylotan at gmail.com (Ben Sizer)
Date: Wed, 13 May 2015 03:23:43 -0700 (PDT)
Subject: ImportError with pickle (Python 2.7.9),
 possibly platform dependent
In-Reply-To: <mailman.9.1430482176.3347.python-list@python.org>
References: <494551ca-532f-4d4d-aff0-a3932416c8f4@googlegroups.com>
 <mailman.108.1430354695.3680.python-list@python.org>
 <20a5c7bf-2163-4b7a-8495-30ce23239903@googlegroups.com>
 <mailman.9.1430482176.3347.python-list@python.org>
Message-ID: <c46f6933-6a3f-4e13-8396-946e675c35e7@googlegroups.com>

On Friday, 1 May 2015 13:09:48 UTC+1, Chris Angelico  wrote:
> 
> Cool! That's part way. So, can you simply stuff OMDBMap into
> sys.modules prior to loading? It might be a bit of a hack, but it
> should work for testing, at least. Conversely, you could change the
> dump script to import via the name my_wsgi_app to make it consistent.
> 
> ChrisA

Sorry for not coming back to this sooner.

In our case we are probably just going to work with a different file format because this was something like the 3rd time that the way pickle works caused our loading to fail for some reason or other. I think the hoops we need to jump through to ensure that the dumping and loading environments match up are too high relative to the cost of switching to JSON or similar, especially if we might need to load the data from something other than Python in future (god forbid).

-- 
Ben Sizer



From kylotan at gmail.com  Wed May 13 06:27:12 2015
From: kylotan at gmail.com (Ben Sizer)
Date: Wed, 13 May 2015 03:27:12 -0700 (PDT)
Subject: ImportError with pickle (Python 2.7.9),
 possibly platform dependent
In-Reply-To: <mailman.11.1430483669.3347.python-list@python.org>
References: <494551ca-532f-4d4d-aff0-a3932416c8f4@googlegroups.com>
 <mailman.108.1430354695.3680.python-list@python.org>
 <20a5c7bf-2163-4b7a-8495-30ce23239903@googlegroups.com>
 <mailman.11.1430483669.3347.python-list@python.org>
Message-ID: <a4cc8273-0e0f-4eec-adc3-4edd321343dc@googlegroups.com>

On Friday, 1 May 2015 13:34:41 UTC+1, Peter Otten  wrote:
> Ben Sizer wrote:
> 
> > So... I don't know how to fix this, but I do now know why it fails, and I
> > have a satisfactory answer for why it is acting differently on the Linux
> > server (and that is just because that is the only one running under WSGI).
> > Two out of three isn't bad!
> 
> How about moving OMDBMap.py into the parent folder of my_wsgi_app.__file__ 
> or any other folder in sys.path?

That might work, but wouldn't be practical for us because in some configurations my_wsgi_app doesn't exist at all (as it is an artifact of running under mod_wsgi environment) and when it does, it it at the top of the hierarchy - so the rest of the app wouldn't be access modules there.

It could be put into sys.path somewhere else... but that is starting to break the project structure just to satisfy pickle. Instead, we'll just use a different format in future.

-- 
Ben Sizer


From mal at egenix.com  Wed May 13 08:43:15 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 13 May 2015 14:43:15 +0200
Subject: ANN: eGenix PyRun - One file Python Runtime 2.1.0
In-Reply-To: <d7783c66-6680-4771-8ebd-cdc966b4622c@googlegroups.com>
References: <mailman.355.1431345550.12865.python-list@python.org>
 <d7783c66-6680-4771-8ebd-cdc966b4622c@googlegroups.com>
Message-ID: <555346E3.4030400@egenix.com>

On 12.05.2015 15:05, Cristiano Cortezia wrote:
> On Monday, May 11, 2015 at 8:59:22 AM UTC-3, eGenix Team: M.-A. Lemburg wrote:
>> ________________________________________________________________________
>>
>> ANNOUNCING
>>
>>                  eGenix PyRun - One file Python Runtime
>>
>>                             Version 2.1.0
>>
>>
>>          An easy-to-use single file relocatable Python run-time -
>>            available for Linux, Mac OS X and Unix platforms,
>>                  with support for Python 2.6, 2.7 and
>>                        **also for Python 3.4**.
> 
> Really interesting. 
> 
> Should we consider this to be an improvement towards the slow startup problem ? 
> 
> It seems to have really decreased the number of failed attempt to open files (ENOENT's appearing with strace upon startup) due to the interpreter default behavior for module loading on startup. (with a simple hello world I got 72 ENOENT's against 307 from default python installation on my standard ubuntu machine).
> 
> This problem mainly shows up when slow access storages are used (say raspberry + sdcard). Any chances you can provide this tool as prebuilt arm binaries ?
> 
> My assumptions may be all wrong, but anyway thanks for sharing your work. :)

We have been providing binaries for Raspberry Pis for quite
some time, but without official support, so they are not listed
on the product page.

If you run install-pyrun on a Raspi, this should install PyRun
for you. I'm not sure whether it'll work on the new Raspi 2.
We'll probably add one to our build farm in a few weeks.

These are the direct download links if you want to give it
a try:

https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs2-linux-armv6l.tgz
https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs2-linux-armv6l.tgz.asc
https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs2-linux-armv6l.tgz.md5
https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs2-linux-armv6l.tgz.sha1
https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs4-linux-armv6l.tgz
https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs4-linux-armv6l.tgz.asc
https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs4-linux-armv6l.tgz.md5
https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs4-linux-armv6l.tgz.sha1
https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py3.4_ucs4-linux-armv6l.tgz
https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py3.4_ucs4-linux-armv6l.tgz.asc
https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py3.4_ucs4-linux-armv6l.tgz.md5
https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py3.4_ucs4-linux-armv6l.tgz.sha1

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 13 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2015-05-13: Released mxODBC Connect 2.1.3 ...     http://egenix.com/go75
2015-05-11: Released eGenix PyRun 2.1.0 ...       http://egenix.com/go74
2015-05-25: PyWaw Summit 2015, Warsaw, Poland ...          12 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From mal at egenix.com  Wed May 13 09:11:41 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 13 May 2015 15:11:41 +0200
Subject: Encrypt python files
In-Reply-To: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>
Message-ID: <55534D8D.9010602@egenix.com>



On 06.05.2015 08:37, Palpandi wrote:
> Hi,
> 
> What are the ways to encrypt python files?

This talk shows some usable ways of doing that:

https://www.youtube.com/watch?v=tKXpMVQIH2Y&index=92&list=PL8uoeex94UhEomMao7wuOrOGuj3jxJYlz

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 13 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2015-05-13: Released mxODBC Connect 2.1.3 ...     http://egenix.com/go75
2015-05-11: Released eGenix PyRun 2.1.0 ...       http://egenix.com/go74
2015-05-25: PyWaw Summit 2015, Warsaw, Poland ...          12 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From andrew at acooke.org  Wed May 13 09:25:01 2015
From: andrew at acooke.org (andrew cooke)
Date: Wed, 13 May 2015 06:25:01 -0700 (PDT)
Subject: Basic misunderstanding on object creation
Message-ID: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>


Hi,

The following code worked on Python 3.2, but no longer works in 3.4.  Did something change, or have I always been doing something dumb?

(I realise the code is pointless as is - it's the simplest example I can give of a problem I am seeing with more complex code).

>>> class Foo:
...     def __new__(cls, *args, **kargs):
...         print('new', args, kargs)
...         super().__new__(cls, *args, **kargs)
... 
>>> class Bar(Foo):
...     def __init__(self, a):
...         print('init', a)
... 
>>> Bar(1)
new (1,) {}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in __new__
TypeError: object() takes no parameters

What I was expecting to happen (and what happens in 3.2) is that the object.__new__ method passes the argument to the __init__ of the subclass.

Any help appreciated.

Thanks,
Andrew


From bc at freeuk.com  Wed May 13 09:48:00 2015
From: bc at freeuk.com (BartC)
Date: Wed, 13 May 2015 14:48:00 +0100
Subject: Encrypt python files
In-Reply-To: <mii4tu$olm$7@dont-email.me>
References: <686d08c9-adb7-4f0b-af8d-4d37d9ad4c0e@googlegroups.com>	<72bd42d6-6926-44ce-bd6f-956ce40afe08@googlegroups.com>
 <mii4tu$olm$7@dont-email.me>
Message-ID: <kCI4x.479029$%35.64327@fx37.am4>

On 08/05/2015 11:59, Denis McMahon wrote:
> On Wed, 06 May 2015 00:23:39 -0700, Palpandi wrote:

>> No, I just want to hide the scripts from others.

> As the decryption method is always available to anyone who has legitimate
> access to execute the code, it's impossible to hide the code at that
> point.

Execute yes, but not necessarily allow access to the source if that is 
the problem.

> Example - if I give you an encrypted binary to run on your system, it
> either has to be unencryptable using tools you already have, or using a
> built in unencrypter, both of which you have access to and can use to
> unencrypt the encrypted executable code.

It can certainly be made harder than simply distributing source code. By 
using a proprietary byte-code for example, or code designed to run on 
some virtual machine. The tools run execute it will need to be provided, 
but that's a long way from being able to reconstruct source code (and if 
that was achieved, you would likely end up with some source code that 
corresponds to the interpreter, not the application).

But it depends on what secrets the OP is trying to hide. Just proving 
.pyc files might be enough, and is minimal inconvenience to the user.

-- 
Bartc


From Cecil at decebal.nl  Wed May 13 09:53:28 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 13 May 2015 15:53:28 +0200
Subject: Testing build
Message-ID: <8738307fdz.fsf@Equus.decebal.nl>

To be clear I did not built Python yet, I used the versions installed
on my system. (3.4.1 and 2.78)

After looking at:
    https://docs.python.org/devguide/

I executed first:
    python3 -m test -j3
which gave:
    a1: [6.435562750993995, 6.446765303000575, 6.405053696988034]
    a2: [4.437405036995187, 4.101042214999325, 4.112324360001367]
    a3: [4.096015618008096, 4.123215412007994, 4.108890180999879]
    [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52, 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88, 96, 98, 102, 105, 109, 110, 114, 118, 121]
    [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52, 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88, 96, 98, 102, 105, 109, 110, 114, 118, 121]
    [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52, 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88, 96, 98, 102, 105, 109, 110, 114, 118, 121]

What is the meaning of this?

After this I executed:
    python2 -m  test.regrtest -j3
which gave:
    a1: [3.996155023574829, 3.9603538513183594, 3.9782469272613525]
    a2: [4.345172166824341, 4.30767297744751, 4.409227132797241]
    a3: [4.328500032424927, 4.3080408573150635, 4.324347019195557]
    [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52, 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88, 96, 98, 102, 105, 109, 110, 114, 118, 121]
    [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52, 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88, 96, 98, 102, 105, 109, 110, 114, 118, 121]
    [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52, 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88, 96, 98, 102, 105, 109, 110, 114, 118, 121]
    /usr/bin/python2: No module named test.regrtest

What is happening here?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From flebber.crue at gmail.com  Wed May 13 09:58:31 2015
From: flebber.crue at gmail.com (Sayth Renshaw)
Date: Wed, 13 May 2015 06:58:31 -0700 (PDT)
Subject: itertools py3.4 - filter list using not equal - fails as bool
In-Reply-To: <mailman.435.1431502814.12865.python-list@python.org>
References: <05defef5-74aa-4a5d-b7e7-9b521512152c@googlegroups.com>
 <mailman.435.1431502814.12865.python-list@python.org>
Message-ID: <4a6fb545-15e1-450f-bb0a-ee8ad97296ee@googlegroups.com>

Thank You for the explanations.

I found this counter implementation is really cool and easily adaptable to more solutions.

Thanks
> Alternatively collections.Counter() supports an arbitrary number of bins...
> 
> >>> import collections
> >>> freq = collections.Counter(t[1] for t in stats)
> >>> freq
> Counter({1: 12, 2: 12, 3: 12, 4: 12, 5: 12})
> 
> ...but you can easily reduce them:
> 
> >>> freq = collections.Counter(t[1] == 1 for t in stats)
> >>> freq
> Counter({False: 48, True: 12})
> >>> one = freq[True]
> >>> total = sum(freq.values())
> >>> print("{} of {} ({}%) have t[1] == 1".format(one, total, one/total*100))
> 12 of 60 (20.0%) have t[1] == 1



From cristiano.cortezia at gmail.com  Wed May 13 10:09:00 2015
From: cristiano.cortezia at gmail.com (Cristiano Cortezia)
Date: Wed, 13 May 2015 11:09:00 -0300
Subject: ANN: eGenix PyRun - One file Python Runtime 2.1.0
In-Reply-To: <555346E3.4030400@egenix.com>
References: <mailman.355.1431345550.12865.python-list@python.org>
 <d7783c66-6680-4771-8ebd-cdc966b4622c@googlegroups.com>
 <555346E3.4030400@egenix.com>
Message-ID: <CAHH7gRKWvm+GY84tuaarLELerwZeBuwk=5jBTdZdksWENw33bw@mail.gmail.com>

Well I gave it a try, and it seems my assumptions were *somehow* true.
Here is what I got when running one of my apps in single shot mode (load,
run, terminate):

*default python distribution*
total time 9.022s
ENOENT's count 7377

*pyrun*
total time 8.455s
ENOENT's count 3064

So, it indeed failed much less to open files, but I guess this didn't make
that much difference after all (500ms).
Perhaps it is because this app has some external dependencies (22 to be
precise) not bundled on pyrun that had to be scanned by the interpreter
anyway. If by any means we could bundle them all the same way, than it
could bring a much higher performance gain. But I guess it is not really
safe-feasible.

Thanks


2015-05-13 9:43 GMT-03:00 M.-A. Lemburg <mal at egenix.com>:

> On 12.05.2015 15:05, Cristiano Cortezia wrote:
> > On Monday, May 11, 2015 at 8:59:22 AM UTC-3, eGenix Team: M.-A. Lemburg
> wrote:
> >> ________________________________________________________________________
> >>
> >> ANNOUNCING
> >>
> >>                  eGenix PyRun - One file Python Runtime
> >>
> >>                             Version 2.1.0
> >>
> >>
> >>          An easy-to-use single file relocatable Python run-time -
> >>            available for Linux, Mac OS X and Unix platforms,
> >>                  with support for Python 2.6, 2.7 and
> >>                        **also for Python 3.4**.
> >
> > Really interesting.
> >
> > Should we consider this to be an improvement towards the slow startup
> problem ?
> >
> > It seems to have really decreased the number of failed attempt to open
> files (ENOENT's appearing with strace upon startup) due to the interpreter
> default behavior for module loading on startup. (with a simple hello world
> I got 72 ENOENT's against 307 from default python installation on my
> standard ubuntu machine).
> >
> > This problem mainly shows up when slow access storages are used (say
> raspberry + sdcard). Any chances you can provide this tool as prebuilt arm
> binaries ?
> >
> > My assumptions may be all wrong, but anyway thanks for sharing your
> work. :)
>
> We have been providing binaries for Raspberry Pis for quite
> some time, but without official support, so they are not listed
> on the product page.
>
> If you run install-pyrun on a Raspi, this should install PyRun
> for you. I'm not sure whether it'll work on the new Raspi 2.
> We'll probably add one to our build farm in a few weeks.
>
> These are the direct download links if you want to give it
> a try:
>
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs2-linux-armv6l.tgz
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs2-linux-armv6l.tgz.asc
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs2-linux-armv6l.tgz.md5
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs2-linux-armv6l.tgz.sha1
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs4-linux-armv6l.tgz
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs4-linux-armv6l.tgz.asc
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs4-linux-armv6l.tgz.md5
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py2.7_ucs4-linux-armv6l.tgz.sha1
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py3.4_ucs4-linux-armv6l.tgz
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py3.4_ucs4-linux-armv6l.tgz.asc
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py3.4_ucs4-linux-armv6l.tgz.md5
>
> https://downloads.egenix.com/python/egenix-pyrun-2.1.0-py3.4_ucs4-linux-armv6l.tgz.sha1
>
> Thanks,
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, May 13 2015)
> >>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
> >>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2015-05-13: Released mxODBC Connect 2.1.3 ...     http://egenix.com/go75
> 2015-05-11: Released eGenix PyRun 2.1.0 ...       http://egenix.com/go74
> 2015-05-25: PyWaw Summit 2015, Warsaw, Poland ...          12 days to go
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150513/fb004b3d/attachment.html>

From rosuav at gmail.com  Wed May 13 10:12:50 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 14 May 2015 00:12:50 +1000
Subject: Testing build
In-Reply-To: <8738307fdz.fsf@Equus.decebal.nl>
References: <8738307fdz.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmpjjyvp12OwpmBtfepWF2RN-_Nh5deeJhDTxZ0pAsjK9g@mail.gmail.com>

On Wed, May 13, 2015 at 11:53 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> To be clear I did not built Python yet, I used the versions installed
> on my system. (3.4.1 and 2.78)
>
> After looking at:
>     https://docs.python.org/devguide/
>
> I executed first:
>     python3 -m test -j3
> which gave:
>     a1: [6.435562750993995, 6.446765303000575, 6.405053696988034]
>     a2: [4.437405036995187, 4.101042214999325, 4.112324360001367]
>     a3: [4.096015618008096, 4.123215412007994, 4.108890180999879]
>     [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52, 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88, 96, 98, 102, 105, 109, 110, 114, 118, 121]
>     [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52, 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88, 96, 98, 102, 105, 109, 110, 114, 118, 121]
>     [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52, 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88, 96, 98, 102, 105, 109, 110, 114, 118, 121]
>
> What is the meaning of this?

Do you have a directory named 'test' or a file named 'test.py' in the
current directory? If so, it'll be loading that up instead of the
standard library 'test' module.

ChrisA


From nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de  Wed May 13 10:24:27 2015
From: nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de (Thomas Rachel)
Date: Wed, 13 May 2015 16:24:27 +0200
Subject: Basic misunderstanding on object creation
In-Reply-To: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
Message-ID: <mivmr0$1mb$1@r01.glglgl.de>

Am 13.05.2015 um 15:25 schrieb andrew cooke:

>>>> class Foo:
> ...     def __new__(cls, *args, **kargs):
> ...         print('new', args, kargs)
> ...         super().__new__(cls, *args, **kargs)

> new (1,) {}
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "<stdin>", line 4, in __new__
> TypeError: object() takes no parameters

object's __new__() dosn't take any parameters. So call it without arguments:

class Foo:
     def __new__(cls, *args, **kargs):
         print('new', args, kargs)
         super().__new__(cls)

(at least if we know that we inherit from object. Might be that this one 
doesn't work very good with multiple inheritance...)


Thomas


From mal at egenix.com  Wed May 13 10:29:58 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 13 May 2015 16:29:58 +0200
Subject: ANN: eGenix PyRun - One file Python Runtime 2.1.0
In-Reply-To: <CAHH7gRKWvm+GY84tuaarLELerwZeBuwk=5jBTdZdksWENw33bw@mail.gmail.com>
References: <mailman.355.1431345550.12865.python-list@python.org>	<d7783c66-6680-4771-8ebd-cdc966b4622c@googlegroups.com>	<555346E3.4030400@egenix.com>
 <CAHH7gRKWvm+GY84tuaarLELerwZeBuwk=5jBTdZdksWENw33bw@mail.gmail.com>
Message-ID: <55535FE6.2090007@egenix.com>

On 13.05.2015 16:09, Cristiano Cortezia wrote:
> Well I gave it a try, and it seems my assumptions were *somehow* true.
> Here is what I got when running one of my apps in single shot mode (load,
> run, terminate):
> 
> *default python distribution*
> total time 9.022s
> ENOENT's count 7377
> 
> *pyrun*
> total time 8.455s
> ENOENT's count 3064
> 
> So, it indeed failed much less to open files, but I guess this didn't make
> that much difference after all (500ms).

PyRun has the advantage of being able to read the byte code
directly from the binary (using memory mapping). However,
it still needs to run the same startup machinery as Python
itself.

Note that startup time for Python was a lot worse before
Python used the same approach as PyRun to compile in the
parsed sysconfig data.

> Perhaps it is because this app has some external dependencies (22 to be
> precise) not bundled on pyrun that had to be scanned by the interpreter
> anyway. If by any means we could bundle them all the same way, than it
> could bring a much higher performance gain. But I guess it is not really
> safe-feasible.

It's certainly possible to use the pyrun build system to create
bundles with more packages and tools included.

The one we're shipping has most of the stdlib included,
but leaves all the application code to reside on the
sys.path or in a ZIP archive.

In one of the next releases we'll probably add a tool to bundle
complete applications together with pyrun, perhaps even by
recompiling it to include the application byte code files
right in the binary like we do for the stdlib.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 13 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2015-05-13: Released mxODBC Connect 2.1.3 ...     http://egenix.com/go75
2015-05-11: Released eGenix PyRun 2.1.0 ...       http://egenix.com/go74
2015-05-25: PyWaw Summit 2015, Warsaw, Poland ...          12 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From cristiano.cortezia at gmail.com  Wed May 13 10:34:22 2015
From: cristiano.cortezia at gmail.com (Cristiano Cortezia)
Date: Wed, 13 May 2015 11:34:22 -0300
Subject: ANN: eGenix PyRun - One file Python Runtime 2.1.0
In-Reply-To: <55535FE6.2090007@egenix.com>
References: <mailman.355.1431345550.12865.python-list@python.org>
 <d7783c66-6680-4771-8ebd-cdc966b4622c@googlegroups.com>
 <555346E3.4030400@egenix.com>
 <CAHH7gRKWvm+GY84tuaarLELerwZeBuwk=5jBTdZdksWENw33bw@mail.gmail.com>
 <55535FE6.2090007@egenix.com>
Message-ID: <CAHH7gRJuN+CEpPAGym=a3UQi9Ho6=_Hj5jsA7VFvgJ7sYSq3kw@mail.gmail.com>

>
> In one of the next releases we'll probably add a tool to bundle
> complete applications together with pyrun, perhaps even by
> recompiling it to include the application byte code files
> right in the binary like we do for the stdlib.


Well, that would be simply awesome. Looking forward to it.

PS: you guys should definitely advertise this work on the embedded software
community.


2015-05-13 11:29 GMT-03:00 M.-A. Lemburg <mal at egenix.com>:

> On 13.05.2015 16:09, Cristiano Cortezia wrote:
> > Well I gave it a try, and it seems my assumptions were *somehow* true.
> > Here is what I got when running one of my apps in single shot mode (load,
> > run, terminate):
> >
> > *default python distribution*
> > total time 9.022s
> > ENOENT's count 7377
> >
> > *pyrun*
> > total time 8.455s
> > ENOENT's count 3064
> >
> > So, it indeed failed much less to open files, but I guess this didn't
> make
> > that much difference after all (500ms).
>
> PyRun has the advantage of being able to read the byte code
> directly from the binary (using memory mapping). However,
> it still needs to run the same startup machinery as Python
> itself.
>
> Note that startup time for Python was a lot worse before
> Python used the same approach as PyRun to compile in the
> parsed sysconfig data.
>
> > Perhaps it is because this app has some external dependencies (22 to be
> > precise) not bundled on pyrun that had to be scanned by the interpreter
> > anyway. If by any means we could bundle them all the same way, than it
> > could bring a much higher performance gain. But I guess it is not really
> > safe-feasible.
>
> It's certainly possible to use the pyrun build system to create
> bundles with more packages and tools included.
>
> The one we're shipping has most of the stdlib included,
> but leaves all the application code to reside on the
> sys.path or in a ZIP archive.
>
> In one of the next releases we'll probably add a tool to bundle
> complete applications together with pyrun, perhaps even by
> recompiling it to include the application byte code files
> right in the binary like we do for the stdlib.
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, May 13 2015)
> >>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
> >>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2015-05-13: Released mxODBC Connect 2.1.3 ...     http://egenix.com/go75
> 2015-05-11: Released eGenix PyRun 2.1.0 ...       http://egenix.com/go74
> 2015-05-25: PyWaw Summit 2015, Warsaw, Poland ...          12 days to go
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150513/e846abdb/attachment.html>

From andrew at acooke.org  Wed May 13 10:42:15 2015
From: andrew at acooke.org (andrew cooke)
Date: Wed, 13 May 2015 07:42:15 -0700 (PDT)
Subject: Basic misunderstanding on object creation
In-Reply-To: <mivmr0$1mb$1@r01.glglgl.de>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mivmr0$1mb$1@r01.glglgl.de>
Message-ID: <06301cb8-39a6-410b-9fad-afe6bd4d3217@googlegroups.com>

On Wednesday, 13 May 2015 11:36:12 UTC-3, Thomas Rachel  wrote:
> Am 13.05.2015 um 15:25 schrieb andrew cooke:
> 
> >>>> class Foo:
> > ...     def __new__(cls, *args, **kargs):
> > ...         print('new', args, kargs)
> > ...         super().__new__(cls, *args, **kargs)
> 
> > new (1,) {}
> > Traceback (most recent call last):
> >    File "<stdin>", line 1, in <module>
> >    File "<stdin>", line 4, in __new__
> > TypeError: object() takes no parameters
> 
> object's __new__() dosn't take any parameters. So call it without arguments:
> 
> class Foo:
>      def __new__(cls, *args, **kargs):
>          print('new', args, kargs)
>          super().__new__(cls)
> 
> (at least if we know that we inherit from object. Might be that this one 
> doesn't work very good with multiple inheritance...)
> 
> 
> Thomas

But then nothing will be passed to __init__ on the subclass.

Andrew


From dreamingforward at gmail.com  Wed May 13 10:44:28 2015
From: dreamingforward at gmail.com (zipher)
Date: Wed, 13 May 2015 07:44:28 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
Message-ID: <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>

On Tuesday, May 12, 2015 at 10:35:29 PM UTC-5, Rustom Mody wrote:
> On Wednesday, May 13, 2015 at 8:00:50 AM UTC+5:30, Steven D'Aprano wrote:
> > Why can't a language be designed with a *practical and concrete* need in
> > mind? As far as I know, only one language designed from theoretical first
> > principles has had any measure of mainstream success, Lisp, and that was
> > probably because there weren't that many decent alternatives at the time.
> 
> How history U-turns!!
> Lisp actually got every major/fundamental thing wrong
> - variables scopes were dynamic by mistake
> - lambdas were non-first class because the locution 'first-class' was still 8 
> years in the future

I think you're confused.  LISP doesn't have variables.  It's a lambda calculus with an entirely different model computation than other programming languages which use variables all the time.  To the extent that it DOES have variables, it's to accomidate those coming over from iterative programming.

And the idea of lambdas were already encoded by the use of special expressions, set-off by parenthesis.  So they practically *defined* the concept of lambdas.  

Mark


From andrew at acooke.org  Wed May 13 10:45:31 2015
From: andrew at acooke.org (andrew cooke)
Date: Wed, 13 May 2015 07:45:31 -0700 (PDT)
Subject: Basic misunderstanding on object creation
In-Reply-To: <06301cb8-39a6-410b-9fad-afe6bd4d3217@googlegroups.com>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mivmr0$1mb$1@r01.glglgl.de>
 <06301cb8-39a6-410b-9fad-afe6bd4d3217@googlegroups.com>
Message-ID: <e09c84fe-609c-4794-a117-fc72c8ccf3ad@googlegroups.com>

> But then nothing will be passed to __init__ on the subclass.
> 
> Andrew

>>> class Foo:
...     def __new__(cls, *args, **kargs):
...         print('new', args, kargs)
...         super().__new__(cls)
... 
>>> class Bar(Foo):
...     def __init__(self, a):
...         print('init', a)
... 
>>> Bar(1)
new (1,) {}

no "init" is printed.


From dreamingforward at gmail.com  Wed May 13 10:49:31 2015
From: dreamingforward at gmail.com (zipher)
Date: Wed, 13 May 2015 07:49:31 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mailman.427.1431488855.12865.python-list@python.org>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <87y4ktwi20.fsf@elektro.pacujo.net>
 <20f6a95e-85d7-4b59-8b4d-3d2ae2626f2c@googlegroups.com>
 <mailman.427.1431488855.12865.python-list@python.org>
Message-ID: <eb476e26-bd1f-43e6-aa7f-db4bcff280ba@googlegroups.com>

On Tuesday, May 12, 2015 at 10:47:48 PM UTC-5, Ian wrote:
> On Tue, May 12, 2015 at 9:11 PM, zipher <dreamingforward at gmail.com> wrote:
> > I know.  That's because most people have fallen off the path (http://c2.com/cgi/wiki?OneTruePath).
> 
> You wrote that, didn't you? I recognize that combination of delusional
> narcissism and curious obsession with Turing machines.

Damn straight.  But to call the reference to Turing machines delusional, means you've already been captured by the first enemy on the Path.  (cf. Javascript!)

> > You haven't done it because either others have done it for you (NumPy) or you simply aren't perfecting anything that needs to scale; i.e. you don't really need to minimize memory or CPU consumption because you're working with toy problems relative to the power of most hardware these days.
> 
> There is such a thing as over-optimization. Given unlimited programmer
> time, sure, everything might be made to run using the minimum possible
> time and space.

You don't need to teach me these basics.  If you had read the document, you would have seen that I already had assimilated the lessons of [Master] Jon Bentley.  I know about over-optimization, however, if you had followed my other posts, you would also know that I'm attempting something that will require some interest in the underlying hardware at some point.  Or, perhaps let's say, it's just my personal (unique?) preference not to create bloatware.

mark


From ian.g.kelly at gmail.com  Wed May 13 10:54:28 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 13 May 2015 08:54:28 -0600
Subject: Basic misunderstanding on object creation
In-Reply-To: <06301cb8-39a6-410b-9fad-afe6bd4d3217@googlegroups.com>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mivmr0$1mb$1@r01.glglgl.de>
 <06301cb8-39a6-410b-9fad-afe6bd4d3217@googlegroups.com>
Message-ID: <CALwzidmQNVSO52ZV6WvzLYbTzRUT0TUOQKE3ig8_NMqhOYsLNg@mail.gmail.com>

On Wed, May 13, 2015 at 8:42 AM, andrew cooke <andrew at acooke.org> wrote:
> On Wednesday, 13 May 2015 11:36:12 UTC-3, Thomas Rachel  wrote:
>> Am 13.05.2015 um 15:25 schrieb andrew cooke:
>>
>> >>>> class Foo:
>> > ...     def __new__(cls, *args, **kargs):
>> > ...         print('new', args, kargs)
>> > ...         super().__new__(cls, *args, **kargs)
>>
>> > new (1,) {}
>> > Traceback (most recent call last):
>> >    File "<stdin>", line 1, in <module>
>> >    File "<stdin>", line 4, in __new__
>> > TypeError: object() takes no parameters
>>
>> object's __new__() dosn't take any parameters. So call it without arguments:
>>
>> class Foo:
>>      def __new__(cls, *args, **kargs):
>>          print('new', args, kargs)
>>          super().__new__(cls)
>>
>> (at least if we know that we inherit from object. Might be that this one
>> doesn't work very good with multiple inheritance...)
>>
>>
>> Thomas
>
> But then nothing will be passed to __init__ on the subclass.

__init__ is not called by __new__. In the object construction, __new__
is called, and *then* __init__ is called on the result.


From ian.g.kelly at gmail.com  Wed May 13 10:55:19 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 13 May 2015 08:55:19 -0600
Subject: Basic misunderstanding on object creation
In-Reply-To: <e09c84fe-609c-4794-a117-fc72c8ccf3ad@googlegroups.com>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mivmr0$1mb$1@r01.glglgl.de>
 <06301cb8-39a6-410b-9fad-afe6bd4d3217@googlegroups.com>
 <e09c84fe-609c-4794-a117-fc72c8ccf3ad@googlegroups.com>
Message-ID: <CALwzidk01oXUb1jvAOHBB1b_5tw13X8iYgqoEOVk=+66zYacpw@mail.gmail.com>

On Wed, May 13, 2015 at 8:45 AM, andrew cooke <andrew at acooke.org> wrote:
>>>> class Foo:
> ...     def __new__(cls, *args, **kargs):
> ...         print('new', args, kargs)
> ...         super().__new__(cls)
> ...
>>>> class Bar(Foo):
> ...     def __init__(self, a):
> ...         print('init', a)
> ...
>>>> Bar(1)
> new (1,) {}
>
> no "init" is printed.

You're not returning anything from Foo.__new__, so the result of the
constructor is None.  None.__init__ does nothing.


From __peter__ at web.de  Wed May 13 10:55:31 2015
From: __peter__ at web.de (Peter Otten)
Date: Wed, 13 May 2015 16:55:31 +0200
Subject: Basic misunderstanding on object creation
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mivmr0$1mb$1@r01.glglgl.de>
 <06301cb8-39a6-410b-9fad-afe6bd4d3217@googlegroups.com>
 <e09c84fe-609c-4794-a117-fc72c8ccf3ad@googlegroups.com>
Message-ID: <mivol3$grm$1@ger.gmane.org>

andrew cooke wrote:

>> But then nothing will be passed to __init__ on the subclass.
>> 
>> Andrew
> 
>>>> class Foo:
> ...     def __new__(cls, *args, **kargs):
> ...         print('new', args, kargs)
> ...         super().__new__(cls)
> ...
>>>> class Bar(Foo):
> ...     def __init__(self, a):
> ...         print('init', a)
> ...
>>>> Bar(1)
> new (1,) {}
> 
> no "init" is printed.


That's because yor __new__() returns None. Try

$ cat new_demo.py
class Foo:
    def __new__(cls, *args, **kargs):
        print('new', cls, args, kargs)
        return super().__new__(cls)

class Bar(Foo):
    def __init__(self, a):
        print('init', a)

bar = Bar(42)
$ python3 new_demo.py 
new <class '__main__.Bar'> (42,) {}
init 42
$ 




From Cecil at decebal.nl  Wed May 13 11:18:44 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 13 May 2015 17:18:44 +0200
Subject: Testing build
References: <8738307fdz.fsf@Equus.decebal.nl>
 <mailman.443.1431526372.12865.python-list@python.org>
Message-ID: <87sib05wvf.fsf@Equus.decebal.nl>

Op Wednesday 13 May 2015 16:12 CEST schreef Chris Angelico:

> On Wed, May 13, 2015 at 11:53 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> To be clear I did not built Python yet, I used the versions
>> installed on my system. (3.4.1 and 2.78)
>>
>> After looking at:
>> https://docs.python.org/devguide/
>>
>> I executed first:
>> python3 -m test -j3
>> which gave:
>> a1: [6.435562750993995, 6.446765303000575, 6.405053696988034]
>> a2: [4.437405036995187, 4.101042214999325, 4.112324360001367]
>> a3: [4.096015618008096, 4.123215412007994, 4.108890180999879]
>> [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52,
>> 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88,
>> 96, 98, 102, 105, 109, 110, 114, 118, 121]
>> [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52,
>> 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88,
>> 96, 98, 102, 105, 109, 110, 114, 118, 121]
>> [1, 5, 6, 9, 11, 13, 19, 20, 28, 30, 34, 38, 39, 40, 42, 49, 52,
>> 53, 57, 58, 62, 63, 66, 68, 72, 73, 74, 76, 78, 82, 83, 85, 86, 88,
>> 96, 98, 102, 105, 109, 110, 114, 118, 121]
>>
>> What is the meaning of this?
>
> Do you have a directory named 'test' or a file named 'test.py' in
> the current directory? If so, it'll be loading that up instead of
> the standard library 'test' module.

Oops, that was the problem. Moved and problem solved.

With:
    python3 -m test -j3
I now get:
    Traceback (most recent call last):
      File "/usr/lib64/python3.4/runpy.py", line 170, in _run_module_as_main
        "__main__", mod_spec)
      File "/usr/lib64/python3.4/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/usr/lib64/python3.4/_import_failed/test.py", line 21, in <module>
        Please file a bug on the Novell Bugzilla.""")
    ImportError: Module '{}' is not installed.
    It is supposed to be part of python3 distribution, but missing from failed import map.
    Please file a bug on the Novell Bugzilla.

With:
    python2 -m  test.regrtest -j3
I now get:
    .
    .
    .
    test_xpickle -- skipping backwards compat tests.
    Use 'regrtest.py -u xpickle' to run them.
    test_wait4
    test_xrange
    test_zipfile64
    test_zipfile64 skipped -- test requires loads of disk-space bytes and a long time to run
    test_zipimport
    test_xmlrpc
    test_zipimport_support
    test_zlib
    test_zipfile
    354 tests OK.
    3 tests failed:
        test_distutils test_gdb test_poplib
    1 test altered the execution environment:
        test_site
    41 tests skipped:
        test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
        test_cd test_cl test_codecmaps_cn test_codecmaps_hk
        test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
        test_dbm test_dl test_gdbm test_gl test_idle test_imageop
        test_imgfile test_kqueue test_linuxaudiodev test_macos
        test_macostools test_msilib test_ossaudiodev test_py3kwarn
        test_scriptpackages test_smtpnet test_socketserver test_startfile
        test_sunaudiodev test_timeout test_tk test_tools test_ttk_guionly
        test_urllib2net test_urllibnet test_winreg test_winsound
        test_zipfile64
    4 skips unexpected on linux2:
        test_dbm test_gdbm test_idle test_tools

This is on openSUSE 13.2.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From dreamingforward at gmail.com  Wed May 13 11:19:06 2015
From: dreamingforward at gmail.com (zipher)
Date: Wed, 13 May 2015 08:19:06 -0700 (PDT)
Subject: anomaly
In-Reply-To: <mailman.402.1431447571.12865.python-list@python.org>
References: <CAMjeLr--RfnfgWc1hdOcHiEBpq3QwC7SZk-m14wM65JtpbgMLQ@mail.gmail.com>
 <mailman.318.1431276220.12865.python-list@python.org>
 <c681b3ce-2727-4aa9-ad8a-369c662395a3@googlegroups.com>
 <554F9525.5040101@digipen.edu> <5550815E.5080600@rece.vub.ac.be>
 <miq0vh$tdg$1@ger.gmane.org>
 <mailman.352.1431344390.12865.python-list@python.org>
 <5550a1d4$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <5550B0CF.208@rece.vub.ac.be>
 <CAPTjJmpHqXJz-DsJHxy9Vc8eg7uOVY6QCVuV1KHy8wZrc=EH2g@mail.gmail.com>
 <mailman.391.1431431746.12865.python-list@python.org>
 <55520685$0$12984$c3e8da3$5496439d@news.astraweb.com>
 <369b3ce9-14c9-465e-afea-407058a1617a@googlegroups.com>
 <CAPTjJmot8Uu_vNF_j5tBzCi4vvYDHJjPVFVegUUMhhLNkcUm1Q@mail.gmail.com>
 <mailman.402.1431447571.12865.python-list@python.org>
Message-ID: <ef3da345-db5f-4369-b27b-bed7883154f5@googlegroups.com>

[Mr. Lawrence spaketh:]
> Do we really have to feed this guy, he's worse than the RUE?

While it may seem like an impossible goal, is it unworthy?  Is there something *better* for high-level, interpreted languages to be good for?

The truth is, that all the theory is worked out, as well as the heavy math that will be required to visualize it (thanks VPython!).  So it's only a matter of people being interested.  I, personally, give up on many occasions, not because of *its* impossibility but the stubborness of People`s.

mark


From ian.g.kelly at gmail.com  Wed May 13 11:26:23 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 13 May 2015 09:26:23 -0600
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
Message-ID: <CALwzidnjKrvReSVHamT=gtdSL_WFFUc7oZFqXzxVk1y+cm0MMg@mail.gmail.com>

I don't know why I'm replying to this...

On Wed, May 13, 2015 at 8:44 AM, zipher <dreamingforward at gmail.com> wrote:
> On Tuesday, May 12, 2015 at 10:35:29 PM UTC-5, Rustom Mody wrote:
>> How history U-turns!!
>> Lisp actually got every major/fundamental thing wrong
>> - variables scopes were dynamic by mistake
>> - lambdas were non-first class because the locution 'first-class' was still 8
>> years in the future
>
> I think you're confused.  LISP doesn't have variables.

Yes, it does.

> It's a lambda calculus

No, it isn't. Lambda calculus is a formal system of mathematics. LISP
is a programming language. It may draw inspiration and borrow notation
from lambda calculus, but these are different things with different
uses and purposes.

> with an entirely different model computation than other programming languages which use variables all the time.  To the extent that it DOES have variables, it's to accomidate those coming over from iterative programming.

What is "iterative programming"? If you mean "writing programs that
work iteratively", then this describes both functional and procedural
styles alike.

The opposite of "iterative programming" would then be a style where
the program can't ever repeat anything. That would be a very limited
language and would *not* be equivalent to a Turing machine.

> And the idea of lambdas were already encoded by the use of special expressions, set-off by parenthesis.  So they practically *defined* the concept of lambdas.

LISP is also the reason why we're cursed with the terrible name
"lambda" for anonymous functions rather than something more mnemonic
(like "function").


From rosuav at gmail.com  Wed May 13 11:45:38 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 14 May 2015 01:45:38 +1000
Subject: Testing build
In-Reply-To: <87sib05wvf.fsf@Equus.decebal.nl>
References: <8738307fdz.fsf@Equus.decebal.nl>
 <mailman.443.1431526372.12865.python-list@python.org>
 <87sib05wvf.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmqFE=LZB6hPQWpD=++vjEeBiX7USWSCHLuUXFsp8D=shA@mail.gmail.com>

On Thu, May 14, 2015 at 1:18 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> With:
>     python3 -m test -j3
> I now get:
>     Traceback (most recent call last):
>       File "/usr/lib64/python3.4/runpy.py", line 170, in _run_module_as_main
>         "__main__", mod_spec)
>       File "/usr/lib64/python3.4/runpy.py", line 85, in _run_code
>         exec(code, run_globals)
>       File "/usr/lib64/python3.4/_import_failed/test.py", line 21, in <module>
>         Please file a bug on the Novell Bugzilla.""")
>     ImportError: Module '{}' is not installed.
>     It is supposed to be part of python3 distribution, but missing from failed import map.
>     Please file a bug on the Novell Bugzilla.

... wat?

I'm having trouble parsing the grammar of that, but I'm guessing
something isn't properly set up to allow you to run the test suite on
Python 3. You could try using your package manager (is that zypper?)
to install more Python packages, though I'm not sure what you'd go
for. Is there a "python3-all" package, perhaps? Otherwise, try a
"python3-dev" package, or similar.

In any case, you shouldn't have this problem on a Py3 that you build
from source. What you're looking at is a cut-down Python standard
library that assumes that most people won't need to run the test suite
- which is a reasonable assumption, but one you're now breaking. :)

> With:
>     python2 -m  test.regrtest -j3
> I now get:
>     354 tests OK.
>     3 tests failed:
>         test_distutils test_gdb test_poplib
>     4 skips unexpected on linux2:
>         test_dbm test_gdbm test_idle test_tools
>
> This is on openSUSE 13.2.

This looks like a normal test run. You're seeing some failures, which
should have been run verbosely further above; the unexpected skips
might mean that you need to tell the test runner that it's allowed to
use certain resources, but otherwise they're probably not a big deal.

ChrisA


From andrew at acooke.org  Wed May 13 11:52:30 2015
From: andrew at acooke.org (andrew cooke)
Date: Wed, 13 May 2015 08:52:30 -0700 (PDT)
Subject: Basic misunderstanding on object creation
In-Reply-To: <mailman.448.1431528969.12865.python-list@python.org>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mivmr0$1mb$1@r01.glglgl.de>
 <06301cb8-39a6-410b-9fad-afe6bd4d3217@googlegroups.com>
 <e09c84fe-609c-4794-a117-fc72c8ccf3ad@googlegroups.com>
 <mailman.448.1431528969.12865.python-list@python.org>
Message-ID: <66066418-2b01-44a2-a551-1d822af53a3b@googlegroups.com>

On Wednesday, 13 May 2015 11:56:21 UTC-3, Ian  wrote:
> On Wed, May 13, 2015 at 8:45 AM, andrew cooke <andrew at acooke.org> wrote:
> >>>> class Foo:
> > ...     def __new__(cls, *args, **kargs):
> > ...         print('new', args, kargs)
> > ...         super().__new__(cls)
> > ...
> >>>> class Bar(Foo):
> > ...     def __init__(self, a):
> > ...         print('init', a)
> > ...
> >>>> Bar(1)
> > new (1,) {}
> >
> > no "init" is printed.
> 
> You're not returning anything from Foo.__new__, so the result of the
> constructor is None.  None.__init__ does nothing.

ah, you're right, thanks.  that was a typo.


more generally, it seems that the error is:

  (1) __new__ is called and then __init__ from some other, external code

  (2) in 3.2 anything passed to object's __new__ was silently discarded.

the following code works in 3.2 and 3.4:

class Foo:
    def __new__(cls, *args, **kargs):
        print("new", args, kargs)
        return super().__new__(cls)

class Bar(Foo): 
    def __init__(self, *args, **kargs):
        print("init", args, kargs)

Bar(1)

(while my original code didn't).

thanks everyone,
andrew


From tjreedy at udel.edu  Wed May 13 12:36:09 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 13 May 2015 12:36:09 -0400
Subject: Basic misunderstanding on object creation
In-Reply-To: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
Message-ID: <mivui1$vjt$1@ger.gmane.org>

On 5/13/2015 9:25 AM, andrew cooke wrote:

> The following code worked on Python 3.2, but no longer works in 3.4.

Bugfixes break code that depends on buggy behavior. See
https://bugs.python.org/issue1683368
Your code also fails in 2.7.9 if you inherit Foo from object.
The exact error messages changed for 3.4 in
https://bugs.python.org/issue7963

 > Did something change,

Obviously yes.

 > or have I always been doing something dumb?

You were depending on behavior of object that Guido decided was buggy.

I found the tracker issue by looking for 'object' in the Core and 
Builtins sections of the changelog one can access from What's New, first 
paragraph (using Highlight All in Firefox).

>>>> class Foo:
> ...     def __new__(cls, *args, **kargs):
> ...         print('new', args, kargs)
> ...         super().__new__(cls, *args, **kargs)
> ...
>>>> class Bar(Foo):
> ...     def __init__(self, a):
> ...         print('init', a)
> ...
>>>> Bar(1)
> new (1,) {}
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "<stdin>", line 4, in __new__
> TypeError: object() takes no parameters
>
> What I was expecting to happen (and what happens in 3.2) is that the object.__new__ method passes the argument to the __init__ of the subclass.

-- 
Terry Jan Reedy



From breamoreboy at yahoo.co.uk  Wed May 13 12:38:27 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 May 2015 17:38:27 +0100
Subject: Basic misunderstanding on object creation
In-Reply-To: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
Message-ID: <mivum5$203$1@ger.gmane.org>

On 13/05/2015 14:25, andrew cooke wrote:
>
> Hi,
>
> The following code worked on Python 3.2, but no longer works in 3.4.  Did something change, or have I always been doing something dumb?
>
> (I realise the code is pointless as is - it's the simplest example I can give of a problem I am seeing with more complex code).
>
>>>> class Foo:
> ...     def __new__(cls, *args, **kargs):
> ...         print('new', args, kargs)
> ...         super().__new__(cls, *args, **kargs)
> ...
>>>> class Bar(Foo):
> ...     def __init__(self, a):
> ...         print('init', a)
> ...
>>>> Bar(1)
> new (1,) {}
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "<stdin>", line 4, in __new__
> TypeError: object() takes no parameters
>
> What I was expecting to happen (and what happens in 3.2) is that the object.__new__ method passes the argument to the __init__ of the subclass.
>
> Any help appreciated.
>
> Thanks,
> Andrew
>

I'm completely convinced that I've seen a change go through on the bug 
tracker that impacts on this area, but many months if not years ago. 
Unfortunately searching the bug tracker for super, __new__, __init__ and 
so on gets a lot of hits, leaving my Mk1 eyeballs overworked.  At least 
I've tried but sorry, had to give up :(

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From tjreedy at udel.edu  Wed May 13 13:05:54 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 13 May 2015 13:05:54 -0400
Subject: Basic misunderstanding on object creation
In-Reply-To: <mivum5$203$1@ger.gmane.org>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mivum5$203$1@ger.gmane.org>
Message-ID: <mj009r$tso$1@ger.gmane.org>

On 5/13/2015 12:38 PM, Mark Lawrence wrote:

> I'm completely convinced that I've seen a change go through on the bug
> tracker that impacts on this area, but many months if not years ago.
> Unfortunately searching the bug tracker for super, __new__, __init__ and
> so on gets a lot of hits, leaving my Mk1 eyeballs overworked.  At least
> I've tried but sorry, had to give up :(

 From my post, sent while you were composing this. (It took me awhile to 
find it.)

Bugfixes break code that depends on buggy behavior. See
https://bugs.python.org/issue1683368
Your code also fails in 2.7.9 if you inherit Foo from object.
The exact error messages changed for 3.4 in
https://bugs.python.org/issue7963

(Found by searching https://docs.python.org/3.4/whatsnew/changelog.html
for 'object'. selecting 'Highlight All', and scanning Core and Builtins 
sections. I believe the change went into some 2.7.x release, as well as 
3.3.0a1)
-- 
Terry Jan Reedy



From tjreedy at udel.edu  Wed May 13 13:13:39 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 13 May 2015 13:13:39 -0400
Subject: Basic misunderstanding on object creation
In-Reply-To: <mivui1$vjt$1@ger.gmane.org>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mivui1$vjt$1@ger.gmane.org>
Message-ID: <mj00ob$5vn$1@ger.gmane.org>

On 5/13/2015 12:36 PM, Terry Reedy wrote:
> On 5/13/2015 9:25 AM, andrew cooke wrote:
>
>> The following code worked on Python 3.2, but no longer works in 3.4.
>
> Bugfixes break code that depends on buggy behavior. See
> https://bugs.python.org/issue1683368
> Your code also fails in 2.7.9 if you inherit Foo from object.
> The exact error messages changed for 3.4 in
> https://bugs.python.org/issue7963
>
>  > Did something change,
>
> Obviously yes.
>
>  > or have I always been doing something dumb?
>
> You were depending on behavior of object that Guido decided was buggy.
>
> I found the tracker issue by looking for 'object' in the Core and
> Builtins sections of the changelog one can access from What's New, first
> paragraph (using Highlight All in Firefox).
>
>>>>> class Foo:
>> ...     def __new__(cls, *args, **kargs):
>> ...         print('new', args, kargs)
>> ...         super().__new__(cls, *args, **kargs)

I forgot to memtion that it is intended that one remove arguments that 
are specific to a class before passing them on to super. All arguments 
should be consumed (removed) somewhere in the chain before reaching 
object.  The bug that was fixed was not raising when not all args were 
consumed. As other showed, your code works properly in all versions when 
you do so by only passing on cls.

-- 
Terry Jan Reedy



From Cecil at decebal.nl  Wed May 13 13:15:23 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 13 May 2015 19:15:23 +0200
Subject: Testing build
References: <8738307fdz.fsf@Equus.decebal.nl>
 <mailman.443.1431526372.12865.python-list@python.org>
 <87sib05wvf.fsf@Equus.decebal.nl>
 <mailman.450.1431531941.12865.python-list@python.org>
Message-ID: <87k2wc5rh0.fsf@Equus.decebal.nl>

Op Wednesday 13 May 2015 17:45 CEST schreef Chris Angelico:

> On Thu, May 14, 2015 at 1:18 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> With: python3 -m test -j3 I now get: Traceback (most recent call
>> last): File "/usr/lib64/python3.4/runpy.py", line 170, in
>> _run_module_as_main "__main__", mod_spec) File
>> "/usr/lib64/python3.4/runpy.py", line 85, in _run_code exec(code,
>> run_globals) File "/usr/lib64/python3.4/_import_failed/test.py",
>> line 21, in <module> Please file a bug on the Novell Bugzilla.""")
>> ImportError: Module '{}' is not installed. It is supposed to be
>> part of python3 distribution, but missing from failed import map.
>> Please file a bug on the Novell Bugzilla.
>
> ... wat?
>
> I'm having trouble parsing the grammar of that, but I'm guessing
> something isn't properly set up to allow you to run the test suite
> on Python 3. You could try using your package manager (is that
> zypper?) to install more Python packages, though I'm not sure what
> you'd go for. Is there a "python3-all" package, perhaps? Otherwise,
> try a "python3-dev" package, or similar.

I needed python3-testsuite. For python 3 that is needed, for python 2
not.

Now I get:
    .
    .
    .
    [381/388/4] test_xmlrpc_net
    test_xmlrpc_net skipped -- Use of the 'network' resource not enabled
    [382/388/4] test_wait4
    [383/388/4] test_zipfile64
    test_zipfile64 skipped -- test requires loads of disk-space bytes and a long time to run
    [384/388/4] test_zipimport
    [385/388/4] test_zipimport_support
    [386/388/4] test_zlib
    [387/388/4] test_xmlrpc
    [388/388/4] test_zipfile
    355 tests OK.
    4 tests failed:
        test_compileall test_dbm_dumb test_pydoc test_site
    29 tests skipped:
        test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp
        test_codecmaps_kr test_codecmaps_tw test_curses test_dbm
        test_dbm_gnu test_dbm_ndbm test_devpoll test_gdb test_idle
        test_kqueue test_msilib test_ossaudiodev test_shelve test_smtpnet
        test_socketserver test_startfile test_timeout test_tk test_tools
        test_ttk_guionly test_urllib2net test_urllibnet test_winreg
        test_winsound test_xmlrpc_net test_zipfile64

I will look into it later, but it does not look to bad.


> In any case, you shouldn't have this problem on a Py3 that you build
> from source. What you're looking at is a cut-down Python standard
> library that assumes that most people won't need to run the test
> suite - which is a reasonable assumption, but one you're now
> breaking. :)

Sometimes I do things a little differently. ;-)

>> With:
>> python2 -m  test.regrtest -j3
>> I now get:
>> 354 tests OK.
>> 3 tests failed:
>> test_distutils test_gdb test_poplib
>> 4 skips unexpected on linux2:
>> test_dbm test_gdbm test_idle test_tools
>>
>> This is on openSUSE 13.2.
>
> This looks like a normal test run. You're seeing some failures,
> which should have been run verbosely further above; the unexpected
> skips might mean that you need to tell the test runner that it's
> allowed to use certain resources, but otherwise they're probably not
> a big deal.

I will look into this also later.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From breamoreboy at yahoo.co.uk  Wed May 13 13:43:59 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 May 2015 18:43:59 +0100
Subject: Basic misunderstanding on object creation
In-Reply-To: <mj009r$tso$1@ger.gmane.org>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mivum5$203$1@ger.gmane.org> <mj009r$tso$1@ger.gmane.org>
Message-ID: <mj02h1$i5$1@ger.gmane.org>

On 13/05/2015 18:05, Terry Reedy wrote:
> On 5/13/2015 12:38 PM, Mark Lawrence wrote:
>
>> I'm completely convinced that I've seen a change go through on the bug
>> tracker that impacts on this area, but many months if not years ago.
>> Unfortunately searching the bug tracker for super, __new__, __init__ and
>> so on gets a lot of hits, leaving my Mk1 eyeballs overworked.  At least
>> I've tried but sorry, had to give up :(
>
>  From my post, sent while you were composing this. (It took me awhile to
> find it.)
>
> Bugfixes break code that depends on buggy behavior. See
> https://bugs.python.org/issue1683368
> Your code also fails in 2.7.9 if you inherit Foo from object.
> The exact error messages changed for 3.4 in
> https://bugs.python.org/issue7963
>
> (Found by searching https://docs.python.org/3.4/whatsnew/changelog.html
> for 'object'. selecting 'Highlight All', and scanning Core and Builtins
> sections. I believe the change went into some 2.7.x release, as well as
> 3.3.0a1)

Yes, I saw your email from the bug tracker roughly five seconds after I 
hit the SEND key for mine above :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From invalid at invalid.invalid  Wed May 13 13:56:09 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Wed, 13 May 2015 17:56:09 +0000 (UTC)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <87y4ktwi20.fsf@elektro.pacujo.net>
Message-ID: <mj037p$6tp$1@reader1.panix.com>

On 2015-05-12, Marko Rauhamaa <marko at pacujo.net> wrote:
> zipher <dreamingforward at gmail.com>:
>
>> That is why you have very high-level languages that allow you to
>> rapidly prototype ideas, test them, and then, depending all the other
>> constraints, move them to lower-level language implementations.
>
> Finally an argument to tackle. That rapid prototyping role is often
> mentioned as a strong point of high-level languages. However, I can't
> remember personally doing that.

I've done that on several occasions: develop and debug an algorithm or
protocol using Python, then re-write it in C once I'm happy with the
way it works.  Those instances are not for desktop use, though.  They
final app has to run on an embedded system that has zero chance of
ever supporting Python.

-- 
Grant Edwards               grant.b.edwards        Yow! YOU PICKED KARL
                                  at               MALDEN'S NOSE!!
                              gmail.com            


From dreamingforward at gmail.com  Wed May 13 14:07:07 2015
From: dreamingforward at gmail.com (zipher)
Date: Wed, 13 May 2015 11:07:07 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mailman.449.1431530832.12865.python-list@python.org>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
 <mailman.449.1431530832.12865.python-list@python.org>
Message-ID: <71ee2dbb-27a3-45ad-b903-bffca6d8923f@googlegroups.com>

On Wednesday, May 13, 2015 at 10:27:23 AM UTC-5, Ian wrote:
> I don't know why I'm replying to this...

Because you're trying to get an answer to a question that even Academia hasn't answered or understood.

> On Wed, May 13, 2015 at 8:44 AM, zipher <dreamingforward at gmail.com> wrote:
> > On Tuesday, May 12, 2015 at 10:35:29 PM UTC-5, Rustom Mody wrote:
> >> How history U-turns!!
> >> Lisp actually got every major/fundamental thing wrong
> >> - variables scopes were dynamic by mistake
> >> - lambdas were non-first class because the locution 'first-class' was still 8
> >> years in the future
> >
> > I think you're confused.  LISP doesn't have variables.
> 
> Yes, it does.

No, Common LISP does, but as the website says Common LISP is a "multi-paradigm" langauge.   It's trying to be everything to everybody, just like Python tried to do in the other direction, making "everything an object".  Python was trying to be too pure, while LISP was trying to be too universal:  be everything to everyone -- you might say "batteries included".

True LISP, doesn't need a 'let' statement for example.  To understand true LISP you have to understand the modus operandi of the "lambda the ultimate" crowd.  Very few do from academic computer science.  MIT understands it.  You think you understand it, but you don't.

It's only abstractions, like math.  It's purpose is to output a final result, that is all.  It's not at all to make commercial applications.  It's rather like Asimov's computer in the Last Question.  It's a whole different model of computation.  Instead of a Turing Tape or VonNeumann stream, you have hierarchies of expressions all evaluating... 

...well I would say all at the same time, but since I have to constrain my description to a common set of reality that is shared with you, then I'd say "on the stack frame".  It's why they had specialized machines to run true LISP.  
 
> > It's a lambda calculus
> 
> No, it isn't. Lambda calculus is a formal system of mathematics. LISP
> is a programming language. It may draw inspiration and borrow notation
> from lambda calculus, but these are different things with different
> uses and purposes.

Again, you're speaking of "Common LISP".  Such a lisp also wants to do OOP, but it's like lipstick on a pig.

> > with an entirely different model computation than other programming languages which use variables all the time.  To the extent that it DOES have variables, it's to accommodate those coming over from iterative programming.
> 
> What is "iterative programming"? If you mean "writing programs that
> work iteratively", then this describes both functional and procedural
> styles alike.

Yes, and LISP is neither.  Although LISP is a functional style, that is only by appearance.  It's completely different from Haskell, which I would describe as a true functional language.  The difference is how the program is lexed in the mind or on the machine.  But that's too difficult to explain on this thread.

> The opposite of "iterative programming" would then be a style where
> the program can't ever repeat anything. That would be a very limited
> language and would *not* be equivalent to a Turing machine.


The "opposite" of iterative programming is recursive programming.  It's not limited, except that it has an entirely different relationship to data, one orthogonal to iterative computation.

> > And the idea of lambdas were already encoded by the use of special expressions, set-off by parenthesis.  So they practically *defined* the concept of lambdas.
> 
> LISP is also the reason why we're cursed with the terrible name
> "lambda" for anonymous functions rather than something more mnemonic
> (like "function").

No, you haven't understood, padawan.  Lambda *is* the function, not it's definition.  Perhaps you will understand what I mean by that, perhaps you won't.  It's subtle.

Mark


From python.list at tim.thechases.com  Wed May 13 14:34:22 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Wed, 13 May 2015 13:34:22 -0500
Subject: Python file structure
In-Reply-To: <CAPTjJmpDfdYZ1LXeHbHNTuv_L4vaNZXnz-yOffjc1sM8tAd1Uw@mail.gmail.com>
References: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
 <CAPTjJmrtLoCGOAr+F1Q7zgzv3hUhvrVA3ADdaLe9mP9YSShi6Q@mail.gmail.com>
 <CALwzid=tsm9c6DWyQ+g72+uG_KM+eDQ2Be3Pgj5gj6sorhNGHw@mail.gmail.com>
 <CAPTjJmpDfdYZ1LXeHbHNTuv_L4vaNZXnz-yOffjc1sM8tAd1Uw@mail.gmail.com>
Message-ID: <20150513133422.78617956@bigbox.christie.dr>

On 2015-05-13 06:07, Chris Angelico wrote:
> On Wed, May 13, 2015 at 5:54 AM, Ian Kelly <ian.g.kelly at gmail.com>
> wrote:
> > Also, I like to put command-line parsing inside the main function
> > and make that its *only* responsibility. The main function then
> > calls the real entry point of my script, which will be something
> > more specifically named. This also has the advantage that if some
> > other module needs to invoke my script, all it has to do is call
> > the entry point function which will be named something more
> > suitable than "main".
> 
> That often makes sense, but sometimes doesn't. When it doesn't, you
> can usually tell because your main function looks something like
> this:
> 
> def main():
>     do_real_work(*sys.argv)
> if __name__=="__main__": main()
> 
> A one-line function that's called from one place? In-line it.
> 
> if __name__ == "__main__":
>     do_real_work(*sys.argv)

Usually mine look something like

  def do_real_work(options, args):
    ...
  def main():
    parser = [optparse,argparse,docopt]....
    options, args = parser.parse_args()
    do_real_work(options, args)
  if __name__ == "__main__":
    main()

since my real-work function usually relies on configuration
(sometimes this also includes a config-file or environment variables
being munged into some "options" data structure).

-tkc



  


From andrew at acooke.org  Wed May 13 14:42:45 2015
From: andrew at acooke.org (andrew cooke)
Date: Wed, 13 May 2015 11:42:45 -0700 (PDT)
Subject: Basic misunderstanding on object creation
In-Reply-To: <mailman.451.1431534990.12865.python-list@python.org>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mailman.451.1431534990.12865.python-list@python.org>
Message-ID: <624b0ba1-3d8a-44fe-a774-69d2d87e2cd0@googlegroups.com>

On Wednesday, 13 May 2015 13:37:23 UTC-3, Terry Reedy  wrote:
> On 5/13/2015 9:25 AM, andrew cooke wrote:
> 
> > The following code worked on Python 3.2, but no longer works in 3.4.
> 
> Bugfixes break code that depends on buggy behavior. See
> https://bugs.python.org/issue1683368
> Your code also fails in 2.7.9 if you inherit Foo from object.
> The exact error messages changed for 3.4 in
> https://bugs.python.org/issue7963
> 
>  > Did something change,
> 
> Obviously yes.

thanks, but why does someone on this group always have to be a dick and make some smart-assed comment like this?

>  > or have I always been doing something dumb?
> 
> You were depending on behavior of object that Guido decided was buggy.
> 
> I found the tracker issue by looking for 'object' in the Core and 
> Builtins sections of the changelog one can access from What's New, first 
> paragraph (using Highlight All in Firefox).
> 
> >>>> class Foo:
> > ...     def __new__(cls, *args, **kargs):
> > ...         print('new', args, kargs)
> > ...         super().__new__(cls, *args, **kargs)
> > ...
> >>>> class Bar(Foo):
> > ...     def __init__(self, a):
> > ...         print('init', a)
> > ...
> >>>> Bar(1)
> > new (1,) {}
> > Traceback (most recent call last):
> >    File "<stdin>", line 1, in <module>
> >    File "<stdin>", line 4, in __new__
> > TypeError: object() takes no parameters
> >
> > What I was expecting to happen (and what happens in 3.2) is that the object.__new__ method passes the argument to the __init__ of the subclass.
> 
> -- 
> Terry Jan Reedy



From tjreedy at udel.edu  Wed May 13 15:17:53 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 13 May 2015 15:17:53 -0400
Subject: Basic misunderstanding on object creation
In-Reply-To: <624b0ba1-3d8a-44fe-a774-69d2d87e2cd0@googlegroups.com>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mailman.451.1431534990.12865.python-list@python.org>
 <624b0ba1-3d8a-44fe-a774-69d2d87e2cd0@googlegroups.com>
Message-ID: <mj081a$2e9$1@ger.gmane.org>

On 5/13/2015 2:42 PM, andrew cooke wrote:
> On Wednesday, 13 May 2015 13:37:23 UTC-3, Terry Reedy  wrote:
>> On 5/13/2015 9:25 AM, andrew cooke wrote:
>>
>>> The following code worked on Python 3.2, but no longer works in 3.4.
>>
>> Bugfixes break code that depends on buggy behavior. See
>> https://bugs.python.org/issue1683368
>> Your code also fails in 2.7.9 if you inherit Foo from object.
>> The exact error messages changed for 3.4 in
>> https://bugs.python.org/issue7963
>>
>>   > Did something change,
>>
>> Obviously yes.
>
> thanks, but why does someone on this group always have to be a dick and make some smart-assed comment like this?

To remind people to ask the question they want answered.

>>   > or have I always been doing something dumb?
>>
>> You were depending on behavior of object that Guido decided was buggy.
>>
>> I found the tracker issue by looking for 'object' in the Core and
>> Builtins sections of the changelog one can access from What's New, first
>> paragraph (using Highlight All in Firefox).

I spent 15 minutes digging around to find the answer that no one else 
gave you, and all your total response is to ignore that, not thank me, 
and call me a dick?  Whose is the dick here?

Being punished for answering questions is one reason I have mostly 
stopped.  I will try to remember not to waste time responding to you again.

-- 
Terry Jan Reedy



From bc at freeuk.com  Wed May 13 15:36:34 2015
From: bc at freeuk.com (BartC)
Date: Wed, 13 May 2015 20:36:34 +0100
Subject: Building CPython
Message-ID: <7JN4x.37133$Q41.15375@fx25.am4>

I'm interested in playing with the CPython sources. I need to be able to 
build under Windows, but don't want to use make files (which rarely work 
properly), nor do a 6GB installation of Visual Studio Express which is 
what seems to be needed (I'm hopeless with complicated IDEs anyway).

Is it possible to do this by using mingw-gcc to compile the .c files of 
the Python sources one by one, or is it one of those complicated 
projects where some of the source is generated as it goes along?

I thought I'd start with the source file containing Py_Main and continue 
from there, but some modules compile and some don't, obscure errors that 
I don't want to investigate unless it's going to be worthwhile (ie. 
eventually ending up with a python.exe that can run simple .py programs).

-- 
Bartc


From p-santoro at sbcglobal.net  Wed May 13 15:43:21 2015
From: p-santoro at sbcglobal.net (Peter)
Date: Wed, 13 May 2015 15:43:21 -0400
Subject: use of subprocess module inside generator
Message-ID: <5553A959.1070305@sbcglobal.net>


I'm using Python 3.4.3 on Windows 7 (with latest patches) to develop a 
sqlcmd module for accessing SQL Server (via Microsoft's sqlcmd.exe).  My 
goal is to develop a 100% Python 3 module that's easy to use, flexible, 
and by design shifts the majority of future SQL Server Python database 
access maintenance to Microsoft.  So far, I've successfully and quickly 
converted a few of my Python pyodbc applications/tools to use this new 
module.  However, I recently ran into an apparent Python issue which 
cost me a few hours to diagnose and work around.

I'm hoping that someone might know what the root cause of my issue was. 
  Perhaps I've hit a bug/restriction with Python generators?

My original generator function looked like this:

def _raw_data(cl, stdout, *, opath=None, timeout=timeout):
     stdout = subprocess.check_output(cl, universal_newlines=True, 
timeout=timeout)

     if opath is None:
         for line in stdout.splitlines():
             yield line.strip()
     else:
         with open(opath) as f:
             for line in f:
                 yield line.strip()

The above function appeared to work fine, if the command line directed 
sqlcmd.exe to send its output to stdout.  However, if the command line 
directed sqlcmd.exe to send its output to a file, 
subprocess.check_output would never be called when next was called on 
the returned generator.  I verified this behavior with print statements 
inside my code, as well as, inside the subprocess module.

My work around was to simply move the call to subprocess.check_output 
outside of the generator function (see below) to its caller (a 
non-generator function).  With this minor change, everything appears to 
work as expected.  OK, so am I missing something here?

def _raw_data(stdout, *, opath=None):
     if opath is None:
         for line in stdout.splitlines():
             yield line.strip()
     else:
         with open(opath) as f:
             for line in f:
                 yield line.strip()

Thank you in advance for your assistance.

Peter Santoro


From breamoreboy at yahoo.co.uk  Wed May 13 15:45:54 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 May 2015 20:45:54 +0100
Subject: Basic misunderstanding on object creation
In-Reply-To: <624b0ba1-3d8a-44fe-a774-69d2d87e2cd0@googlegroups.com>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mailman.451.1431534990.12865.python-list@python.org>
 <624b0ba1-3d8a-44fe-a774-69d2d87e2cd0@googlegroups.com>
Message-ID: <mj09lk$sma$1@ger.gmane.org>

On 13/05/2015 19:42, andrew cooke wrote:
> On Wednesday, 13 May 2015 13:37:23 UTC-3, Terry Reedy  wrote:
>> On 5/13/2015 9:25 AM, andrew cooke wrote:
>>
>>> The following code worked on Python 3.2, but no longer works in 3.4.
>>
>> Bugfixes break code that depends on buggy behavior. See
>> https://bugs.python.org/issue1683368
>> Your code also fails in 2.7.9 if you inherit Foo from object.
>> The exact error messages changed for 3.4 in
>> https://bugs.python.org/issue7963
>>
>>   > Did something change,
>>
>> Obviously yes.
>
> thanks, but why does someone on this group always have to be a dick and make some smart-assed comment like this?
>

Please don't bother asking again after that response, especially to 
somebody with the stature of Terry Reedy.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From marko at pacujo.net  Wed May 13 15:50:20 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 13 May 2015 22:50:20 +0300
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
 <mailman.449.1431530832.12865.python-list@python.org>
Message-ID: <87egmkw937.fsf@elektro.pacujo.net>

Ian Kelly <ian.g.kelly at gmail.com>:

> LISP is also the reason why we're cursed with the terrible name
> "lambda" for anonymous functions rather than something more mnemonic
> (like "function").

The only terrible aspect of "lambda" is how difficult it is to type.

BTW, Common Lisp actually has an operator called "function": <URL:
http://www.lispworks.com/documentation/HyperSpec/Body/s_fn.htm>.


Marko


From breamoreboy at yahoo.co.uk  Wed May 13 15:58:13 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 May 2015 20:58:13 +0100
Subject: Building CPython
In-Reply-To: <7JN4x.37133$Q41.15375@fx25.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>
Message-ID: <mj0acn$9ae$1@ger.gmane.org>

On 13/05/2015 20:36, BartC wrote:
> I'm interested in playing with the CPython sources. I need to be able to
> build under Windows, but don't want to use make files (which rarely work
> properly), nor do a 6GB installation of Visual Studio Express which is
> what seems to be needed (I'm hopeless with complicated IDEs anyway).
>
> Is it possible to do this by using mingw-gcc to compile the .c files of
> the Python sources one by one, or is it one of those complicated
> projects where some of the source is generated as it goes along?
>
> I thought I'd start with the source file containing Py_Main and continue
> from there, but some modules compile and some don't, obscure errors that
> I don't want to investigate unless it's going to be worthwhile (ie.
> eventually ending up with a python.exe that can run simple .py programs).
>

Before you spend too much time on the mingw route, check out the 
outstanding issues for it on the bug tracker.  Then you'll maybe realise 
that using the supported VS setup is far easier.  Everything you need is 
in the PCBuild directory, including pcbuild.sln for use with VS and 
build.bat for command line use.  Details here 
https://docs.python.org/devguide/setup.html#windows

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From ned at nedbatchelder.com  Wed May 13 16:33:38 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 13 May 2015 13:33:38 -0700 (PDT)
Subject: Basic misunderstanding on object creation
In-Reply-To: <mailman.459.1431546366.12865.python-list@python.org>
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mailman.451.1431534990.12865.python-list@python.org>
 <624b0ba1-3d8a-44fe-a774-69d2d87e2cd0@googlegroups.com>
 <mailman.459.1431546366.12865.python-list@python.org>
Message-ID: <9bf061a8-03f9-47ed-9774-cc21d5fec8ed@googlegroups.com>

On Wednesday, May 13, 2015 at 3:46:16 PM UTC-4, Mark Lawrence wrote:
> On 13/05/2015 19:42, andrew cooke wrote:
> > On Wednesday, 13 May 2015 13:37:23 UTC-3, Terry Reedy  wrote:
> >> On 5/13/2015 9:25 AM, andrew cooke wrote:
> >>
> >>> The following code worked on Python 3.2, but no longer works in 3.4.
> >>
> >> Bugfixes break code that depends on buggy behavior. See
> >> https://bugs.python.org/issue1683368
> >> Your code also fails in 2.7.9 if you inherit Foo from object.
> >> The exact error messages changed for 3.4 in
> >> https://bugs.python.org/issue7963
> >>
> >>   > Did something change,
> >>
> >> Obviously yes.
> >
> > thanks, but why does someone on this group always have to be a dick and make some smart-assed comment like this?
> >
> 
> Please don't bother asking again after that response, especially to 
> somebody with the stature of Terry Reedy.
> 

To my ears, "Obviously yes" is a pedantic smart-ass comment.  It added
nothing to the answer, other than to point out that the question was
obvious. Essentially, it said, "you are silly."  I don't think it's
unreasonable to expect the regulars here to be able to answer politely.

I certainly don't think Andrew's behavior in this thread is cause for
asking him not to return.  I also don't think "people with stature"
should be allowed to behave in ways that others would not be allowed.
And how are newcomers expected to identify "people with stature"?

Email can be a harsh medium, lacking social cues that can soften
interchanges in other arenas.  We all have to try hard to be kind to
each other.  And yes, I expect more of the regulars than I do of
people new to the list.  The regulars set the tone.  They are the
hosts here, and should be doing what they can to make new people
feel welcome.

--Ned.

> -- 
> My fellow Pythonistas, ask not what our language can do for you, ask
> what you can do for our language.
> 
> Mark Lawrence



From ian.g.kelly at gmail.com  Wed May 13 17:38:28 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 13 May 2015 15:38:28 -0600
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <71ee2dbb-27a3-45ad-b903-bffca6d8923f@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
 <mailman.449.1431530832.12865.python-list@python.org>
 <71ee2dbb-27a3-45ad-b903-bffca6d8923f@googlegroups.com>
Message-ID: <CALwzidmqY1Ks+Qjh-jzXkHCyZkxCV8M+0vGp0B_pYP3WmN09Gw@mail.gmail.com>

On Wed, May 13, 2015 at 12:07 PM, zipher <dreamingforward at gmail.com> wrote:
> On Wednesday, May 13, 2015 at 10:27:23 AM UTC-5, Ian wrote:
>> I don't know why I'm replying to this...
>
> Because you're trying to get an answer to a question that even Academia hasn't answered or understood.
>
>> On Wed, May 13, 2015 at 8:44 AM, zipher <dreamingforward at gmail.com> wrote:
>> > On Tuesday, May 12, 2015 at 10:35:29 PM UTC-5, Rustom Mody wrote:
>> >> How history U-turns!!
>> >> Lisp actually got every major/fundamental thing wrong
>> >> - variables scopes were dynamic by mistake
>> >> - lambdas were non-first class because the locution 'first-class' was still 8
>> >> years in the future
>> >
>> > I think you're confused.  LISP doesn't have variables.
>>
>> Yes, it does.
>
> No, Common LISP does, but as the website says Common LISP is a "multi-paradigm" langauge.   It's trying to be everything to everybody, just like Python tried to do in the other direction, making "everything an object".  Python was trying to be too pure, while LISP was trying to be too universal:  be everything to everyone -- you might say "batteries included".
>
> True LISP, doesn't need a 'let' statement for example.  To understand true LISP you have to understand the modus operandi of the "lambda the ultimate" crowd.  Very few do from academic computer science.  MIT understands it.  You think you understand it, but you don't.

By "true LISP" are you referring to the original specification by John
McCarthy? Here's an example lambda S-expression from McCarthy's
original paper:

(LABEL, SUBST, (LAMBDA, (X, Y, Z), (COND ((ATOM, Z), (COND, (EQ, Y,
Z), X), ((QUOTE, T), Z))), ((QUOTE, T), (CONS, (SUBST, X, Y, (CAR Z)),
(SUBST, X, Y, (CDR, Z)))))))

Ugh, what a mess. But ignoring that, tell us how many variables you
see there. I'll give you a hint: I count more than two.

> It's only abstractions, like math.  It's purpose is to output a final result, that is all.  It's not at all to make commercial applications.  It's rather like Asimov's computer in the Last Question.  It's a whole different model of computation.  Instead of a Turing Tape or VonNeumann stream, you have hierarchies of expressions all evaluating...
>
> ...well I would say all at the same time, but since I have to constrain my description to a common set of reality that is shared with you, then I'd say "on the stack frame".  It's why they had specialized machines to run true LISP.

Sure. Lisp machines never, ever ran computer graphics applications. Or
medical image processing. Nope, never, because that would be
commercial and dirty and a hideous perversion of the idol of computer
science created by the prophet McCarthy. Text editors? Heavens to
Betsy, now you're just trying to shock me, aren't you!

>> > with an entirely different model computation than other programming languages which use variables all the time.  To the extent that it DOES have variables, it's to accommodate those coming over from iterative programming.
>>
>> What is "iterative programming"? If you mean "writing programs that
>> work iteratively", then this describes both functional and procedural
>> styles alike.
>
> Yes, and LISP is neither.  Although LISP is a functional style, that is only by appearance.  It's completely different from Haskell, which I would describe as a true functional language.  The difference is how the program is lexed in the mind or on the machine.  But that's too difficult to explain on this thread.

And Fermat had a truly marvelous proof, which you would think
wonderful, if only he had enough room in that infamous margin.

>> The opposite of "iterative programming" would then be a style where
>> the program can't ever repeat anything. That would be a very limited
>> language and would *not* be equivalent to a Turing machine.
>
>
> The "opposite" of iterative programming is recursive programming.  It's not limited, except that it has an entirely different relationship to data, one orthogonal to iterative computation.

Iteration is a type of recursion. Specifically, it's the type of
recursion that doesn't require keeping a stack of values from each
higher-up repetition in the evaluation. Also known as "linear
recursion" or "tail recursion".

Often people use the word "iteration" to mean a syntactic construct
that repeats itself without explicit reference to a function and
"recursion" to mean a syntactic construct where a function explicitly
repeats itself, but from a theoretical standpoint this is all just
syntax. The two constructs are fundamentally the same.

>
>> > And the idea of lambdas were already encoded by the use of special expressions, set-off by parenthesis.  So they practically *defined* the concept of lambdas.
>>
>> LISP is also the reason why we're cursed with the terrible name
>> "lambda" for anonymous functions rather than something more mnemonic
>> (like "function").
>
> No, you haven't understood, padawan.

*plonk*


From breamoreboy at yahoo.co.uk  Wed May 13 17:55:21 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 13 May 2015 22:55:21 +0100
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <CALwzidmqY1Ks+Qjh-jzXkHCyZkxCV8M+0vGp0B_pYP3WmN09Gw@mail.gmail.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
 <mailman.449.1431530832.12865.python-list@python.org>
 <71ee2dbb-27a3-45ad-b903-bffca6d8923f@googlegroups.com>
 <CALwzidmqY1Ks+Qjh-jzXkHCyZkxCV8M+0vGp0B_pYP3WmN09Gw@mail.gmail.com>
Message-ID: <mj0h8b$pbn$1@ger.gmane.org>

On 13/05/2015 22:38, Ian Kelly wrote:
> On Wed, May 13, 2015 at 12:07 PM, zipher <dreamingforward at gmail.com> wrote:
>>
>> Yes, and LISP is neither.  Although LISP is a functional style, that is only by appearance.  It's completely different from Haskell, which I would describe as a true functional language.  The difference is how the program is lexed in the mind or on the machine.  But that's too difficult to explain on this thread.
>
> And Fermat had a truly marvelous proof, which you would think
> wonderful, if only he had enough room in that infamous margin.
>

The RUE also has a marvellous proof that the PEP 393 FSR is complete 
nonsense, but as it's so obvious to himself he's decided its not worth 
sharing with anybody else.  What with that and this thread I therefore 
conclude that both the RUE and zipher suffer from a very severe case of 
Emperor's New Clothes Syndrome.

>>
>> No, you haven't understood, padawan.
>
> *plonk*
>

You took your time over that :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From gherron at digipen.edu  Wed May 13 17:58:37 2015
From: gherron at digipen.edu (Gary Herron)
Date: Wed, 13 May 2015 14:58:37 -0700
Subject: use of subprocess module inside generator
In-Reply-To: <5553A959.1070305@sbcglobal.net>
References: <5553A959.1070305@sbcglobal.net>
Message-ID: <5553C90D.1010406@digipen.edu>

On 05/13/2015 12:43 PM, Peter wrote:
>
> I'm using Python 3.4.3 on Windows 7 (with latest patches) to develop a 
> sqlcmd module for accessing SQL Server (via Microsoft's sqlcmd.exe).  
> My goal is to develop a 100% Python 3 module that's easy to use, 
> flexible, and by design shifts the majority of future SQL Server 
> Python database access maintenance to Microsoft.  So far, I've 
> successfully and quickly converted a few of my Python pyodbc 
> applications/tools to use this new module.  However, I recently ran 
> into an apparent Python issue which cost me a few hours to diagnose 
> and work around.

I doubt that you've hit a bug -- we've all done this kind of thing many 
times, and there's certainly no restriction on making procedure calls 
within a generator -- so that's probably not the problem either.

My guess is that you've misinterpreted the failure of the original 
code.  I don't know how that might be, but I do spot one oddity in your 
original code which may be responsible.  See below ...

>
> I'm hoping that someone might know what the root cause of my issue 
> was.  Perhaps I've hit a bug/restriction with Python generators?
>
> My original generator function looked like this:
>
> def _raw_data(cl, stdout, *, opath=None, timeout=timeout):
>     stdout = subprocess.check_output(cl, universal_newlines=True, 
> timeout=timeout)

This seems muddled -- you pass in a parameter, stdout, only to 
immediately overwrite its value with the output of check_output. What 
was in stdout originally, and more importantly, did you expect the newly 
assigned value from check_output to be returned to the calling 
procedure?  If so, that's your bug, because parameters in function calls 
don't work that way.

This makes sense with your workaround, since the assignment to stdout is 
preserved when done outside the function.

I hope that helps.

Gary Herron



>
>     if opath is None:
>         for line in stdout.splitlines():
>             yield line.strip()
>     else:
>         with open(opath) as f:
>             for line in f:
>                 yield line.strip()
>
> The above function appeared to work fine, if the command line directed 
> sqlcmd.exe to send its output to stdout.  However, if the command line 
> directed sqlcmd.exe to send its output to a file, 
> subprocess.check_output would never be called when next was called on 
> the returned generator.  I verified this behavior with print 
> statements inside my code, as well as, inside the subprocess module.
>
> My work around was to simply move the call to subprocess.check_output 
> outside of the generator function (see below) to its caller (a 
> non-generator function).  With this minor change, everything appears 
> to work as expected.  OK, so am I missing something here?
>
> def _raw_data(stdout, *, opath=None):
>     if opath is None:
>         for line in stdout.splitlines():
>             yield line.strip()
>     else:
>         with open(opath) as f:
>             for line in f:
>                 yield line.strip()
>
> Thank you in advance for your assistance.
>
> Peter Santoro


-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418



From tjreedy at udel.edu  Wed May 13 18:34:47 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 13 May 2015 18:34:47 -0400
Subject: Building CPython
In-Reply-To: <7JN4x.37133$Q41.15375@fx25.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>
Message-ID: <mj0jif$uje$1@ger.gmane.org>

On 5/13/2015 3:36 PM, BartC wrote:
> I'm interested in playing with the CPython sources. I need to be able to
> build under Windows, but don't want to use make files (which rarely work
> properly), nor do a 6GB installation of Visual Studio Express which is
> what seems to be needed (I'm hopeless with complicated IDEs anyway).

Once installed hg or tortoisehg (I use this) and VSE are installed and 
repository cloned, are half done. At command prompt, with top directory 
of repository as current directory enter
tools\scripts\external.bat
Double-clicking file in Explorer does not work.
Usually only needs to be done once per branch after x.y.0 release as 
dependencies are usually not updated for bugfix releases.

Then in same directory enter
pcbuild\python.sln
or double click in Explorer or open VSE and open this file.
Hit F7, wait until get line like
========== Build: 1 succeeded, 0 failed, 24 up-to-date, 1 skipped,
hit F5, pin python_d to taskbar (optional, but handy), and go.

And read devguide.

-- 
Terry Jan Reedy



From dreamingforward at gmail.com  Wed May 13 19:16:08 2015
From: dreamingforward at gmail.com (zipher)
Date: Wed, 13 May 2015 16:16:08 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mailman.461.1431553152.12865.python-list@python.org>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
 <mailman.449.1431530832.12865.python-list@python.org>
 <71ee2dbb-27a3-45ad-b903-bffca6d8923f@googlegroups.com>
 <mailman.461.1431553152.12865.python-list@python.org>
Message-ID: <2aecbb14-86a4-42c7-b40c-bfa93ce83185@googlegroups.com>

On Wednesday, May 13, 2015 at 4:39:52 PM UTC-5, Ian wrote:
> On Wed, May 13, 2015 at 12:07 PM, zipher <dreamingforward at gmail.com> wrote:
> > On Wednesday, May 13, 2015 at 10:27:23 AM UTC-5, Ian wrote:
> >> I don't know why I'm replying to this...
> >
> > Because you're trying to get an answer to a question that even Academia hasn't answered or understood.
> >
> >> On Wed, May 13, 2015 at 8:44 AM, zipher <dreamingforward at gmail.com> wrote:
> >> > On Tuesday, May 12, 2015 at 10:35:29 PM UTC-5, Rustom Mody wrote:
> >> >> How history U-turns!!
> >> >> Lisp actually got every major/fundamental thing wrong
> >> >> - variables scopes were dynamic by mistake
> >> >> - lambdas were non-first class because the locution 'first-class' was still 8
> >> >> years in the future
> >> >
> >> > I think you're confused.  LISP doesn't have variables.
> >>
> >> Yes, it does.
> >
> > No, Common LISP does, but as the website says Common LISP is a "multi-paradigm" langauge.   It's trying to be everything to everybody, just like Python tried to do in the other direction, making "everything an object".  Python was trying to be too pure, while LISP was trying to be too universal:  be everything to everyone -- you might say "batteries included".
> >
> > True LISP, doesn't need a 'let' statement for example.  To understand true LISP you have to understand the modus operandi of the "lambda the ultimate" crowd.  Very few do from academic computer science.  MIT understands it.  You think you understand it, but you don't.
> 
> By "true LISP" are you referring to the original specification by John
> McCarthy? Here's an example lambda S-expression from McCarthy's
> original paper:
> 
> (LABEL, SUBST, (LAMBDA, (X, Y, Z), (COND ((ATOM, Z), (COND, (EQ, Y,
> Z), X), ((QUOTE, T), Z))), ((QUOTE, T), (CONS, (SUBST, X, Y, (CAR Z)),
> (SUBST, X, Y, (CDR, Z)))))))
> 
> Ugh, what a mess. But ignoring that, tell us how many variables you
> see there. I'll give you a hint: I count more than two.

LoL, it's an interesting example you've thrown up there.  It's more than just a mess though.  The problem is the use of the quote.  I was hoping you wouldn't include it.  The problem is that its a compromise to interact with your mental-visual lexer.  It's like Javascript which runs on no virtual machine anyone's heard of.  It's too magical.
 
> Sure. Lisp machines never, ever ran computer graphics applications. Or
> medical image processing. Nope, never, because that would be
> commercial and dirty and a hideous perversion of the idol of computer
> science created by the prophet McCarthy. Text editors? Heavens to
> Betsy, now you're just trying to shock me, aren't you!

CLISP =/= LISP. 
 
> > Yes, and LISP is neither.  Although LISP is a functional style, that is only by appearance.  It's completely different from Haskell, which I would describe as a true functional language.  The difference is how the program is lexed in the mind or on the machine.  But that's too difficult to explain on this thread.
> 
> And Fermat had a truly marvelous proof, which you would think
> wonderful, if only he had enough room in that infamous margin.

Ha, you know, I actually think I found the proof he alleges.  You have to visualize the problem geometrically.

> Iteration is a type of recursion. Specifically, it's the type of
> recursion that doesn't require keeping a stack of values from each
> higher-up repetition in the evaluation.  
> Also known as "linear
> recursion" or "tail recursion".

You are so confused, it boggles.  Can anyone else assert this?

Mark


From lab at pacbell.net  Wed May 13 19:24:30 2015
From: lab at pacbell.net (20/20 Lab)
Date: Wed, 13 May 2015 16:24:30 -0700
Subject: Looking for direction
Message-ID: <5553DD2E.2080600@pacbell.net>

I'm a beginner to python.  Reading here and there.  Written a couple of 
short and simple programs to make life easier around the office.

That being said, I'm not even sure what I need to ask for. I've never 
worked with external data before.

I have a LARGE csv file that I need to process.  110+ columns, 72k 
rows.  I managed to write enough to reduce it to a few hundred rows, and 
the five columns I'm interested in.

Now is were I have my problem:

myList = [ [123, "XXX", "Item", "Qty", "Noise"],
            [72976, "YYY", "Item", "Qty", "Noise"],
            [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]

Basically, I need to check for rows with duplicate accounts row[0] and 
staff (row[1]), and if so, remove that row, and add it's Qty to the 
original row. I really dont have a clue how to go about this.  The 
number of rows change based on which run it is, so I couldnt even get 
away with using hundreds of compare loops.

If someone could point me to some documentation on the functions I would 
need, or a tutorial it would be a great help.

Thank you.


From breamoreboy at yahoo.co.uk  Wed May 13 19:36:17 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 14 May 2015 00:36:17 +0100
Subject: Looking for direction
In-Reply-To: <5553DD2E.2080600@pacbell.net>
References: <5553DD2E.2080600@pacbell.net>
Message-ID: <mj0n5k$kfm$1@ger.gmane.org>

On 14/05/2015 00:24, 20/20 Lab wrote:
> I'm a beginner to python.  Reading here and there.  Written a couple of
> short and simple programs to make life easier around the office.

Welcome :)

>
> That being said, I'm not even sure what I need to ask for. I've never
> worked with external data before.
>
> I have a LARGE csv file that I need to process.  110+ columns, 72k
> rows.  I managed to write enough to reduce it to a few hundred rows, and
> the five columns I'm interested in.
>
> Now is were I have my problem:
>
> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
>             [72976, "YYY", "Item", "Qty", "Noise"],
>             [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
>
> Basically, I need to check for rows with duplicate accounts row[0] and
> staff (row[1]), and if so, remove that row, and add it's Qty to the
> original row. I really dont have a clue how to go about this.  The
> number of rows change based on which run it is, so I couldnt even get
> away with using hundreds of compare loops.
>
> If someone could point me to some documentation on the functions I would
> need, or a tutorial it would be a great help.
>
> Thank you.

Check this out http://pandas.pydata.org/

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From p-santoro at sbcglobal.net  Wed May 13 19:54:43 2015
From: p-santoro at sbcglobal.net (Peter)
Date: Wed, 13 May 2015 19:54:43 -0400
Subject: use of subprocess module inside generator
Message-ID: <5553E443.3040507@sbcglobal.net>

Gary,

Thank you for the response.  I made a small typo in my original post, 
which you correctly picked up.  My original generator actually did not 
have the stdout parameter (see below).  Only the new generator has this 
parameter, as it's now being passed into the generator from the caller's 
execution of subprocess.check_output.

def _raw_data(cl, *, opath=None, timeout=timeout):
     stdout = subprocess.check_output(cl, universal_newlines=True, 
timeout=timeout)

     if opath is None:
         for line in stdout.splitlines():
             yield line.strip()
     else:
         with open(opath) as f:
             for line in f:
                 yield line.strip()


Peter


From ethan at stoneleaf.us  Wed May 13 20:06:55 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 13 May 2015 17:06:55 -0700
Subject: Looking for direction
In-Reply-To: <5553DD2E.2080600@pacbell.net>
References: <5553DD2E.2080600@pacbell.net>
Message-ID: <5553E71F.8090402@stoneleaf.us>

On 05/13/2015 04:24 PM, 20/20 Lab wrote:
> I'm a beginner to python.  Reading here and there.  Written a couple of
> short and simple programs to make life easier around the office.
>
> That being said, I'm not even sure what I need to ask for. I've never
> worked with external data before.
>
> I have a LARGE csv file that I need to process.  110+ columns, 72k
> rows.  I managed to write enough to reduce it to a few hundred rows, and
> the five columns I'm interested in.
>
> Now is were I have my problem:
>
> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
>             [72976, "YYY", "Item", "Qty", "Noise"],
>             [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
>
> Basically, I need to check for rows with duplicate accounts row[0] and
> staff (row[1]), and if so, remove that row, and add it's Qty to the
> original row. I really dont have a clue how to go about this.  The
> number of rows change based on which run it is, so I couldnt even get
> away with using hundreds of compare loops.
>
> If someone could point me to some documentation on the functions I would
> need, or a tutorial it would be a great help.

You could try using a dictionary, combining when needed:

# untested
data = {}
for row in all_rows:
   key = row[0], row[1]
   if key in data:
     item, qty, noise = data[key]
     qty += row[3]
   else:
     item, qty, noise = row[2:]
   data[key] = item, qty, noise

for (account, staff), (item, qty, noise) in data.items():
   do_stuff_with(account, staff, item, qty, noise)

At the end, data should have what you want.  It won't, however, be in 
the same order, so hopefully that's not an issue for you.

--
~Ethan~


From davea at davea.name  Wed May 13 20:07:07 2015
From: davea at davea.name (Dave Angel)
Date: Wed, 13 May 2015 20:07:07 -0400
Subject: Looking for direction
In-Reply-To: <5553DD2E.2080600@pacbell.net>
References: <5553DD2E.2080600@pacbell.net>
Message-ID: <5553E72B.5000309@davea.name>

On 05/13/2015 07:24 PM, 20/20 Lab wrote:
> I'm a beginner to python.  Reading here and there.  Written a couple of
> short and simple programs to make life easier around the office.
>
Welcome to Python, and to this mailing list.

> That being said, I'm not even sure what I need to ask for. I've never
> worked with external data before.
>
> I have a LARGE csv file that I need to process.  110+ columns, 72k
> rows.

That's not very large at all.

>  I managed to write enough to reduce it to a few hundred rows, and
> the five columns I'm interested in.

>
> Now is were I have my problem:
>
> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
>             [72976, "YYY", "Item", "Qty", "Noise"],
>             [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
>

It'd probably be useful to identify names for your columns, even if it's 
just in a comment.  Guessing from the paragraph below, I figure the 
first two columns are "account" & "staff"

> Basically, I need to check for rows with duplicate accounts row[0] and
> staff (row[1]), and if so, remove that row, and add it's Qty to the
> original row.

And which column is that supposed to be?  Shouldn't there be a number 
there, rather than a string?

> I really dont have a clue how to go about this.  The
> number of rows change based on which run it is, so I couldnt even get
> away with using hundreds of compare loops.
>
> If someone could point me to some documentation on the functions I would
> need, or a tutorial it would be a great help.
>

Is the order significant?  Do you have to preserve the order that the 
accounts appear?  I'll assume not.

Have you studied dictionaries?  Seems to me the way to handle the 
problem is to read in a row, create a dictionary with key of (account, 
staff), and data of the rest of the line.

Each time you read a row, you check if the key is already in the 
dictionary.  If not, add it.  If it's already there, merge the data as 
you say.

Then when you're done, turn the dict back into a list of lists.

-- 
DaveA


From ben+python at benfinney.id.au  Wed May 13 20:15:12 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 14 May 2015 10:15:12 +1000
Subject: Looking for direction
References: <5553DD2E.2080600@pacbell.net>
Message-ID: <85pp648167.fsf@benfinney.id.au>

20/20 Lab <lab at pacbell.net> writes:

> I'm a beginner to python. Reading here and there. Written a couple of
> short and simple programs to make life easier around the office.

Welcome, and congratulations on self-educating to this point.

> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
>            [72976, "YYY", "Item", "Qty", "Noise"],
>            [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
>
> Basically, I need to check for rows with duplicate accounts row[0] and
> staff (row[1]), and if so, remove that row, and add it's Qty to the
> original row. I really dont have a clue how to go about this.

You might benefit from doing some simple study of algorithms, with
exercises so you can test your knowledge and learn new ways of thinking
about basic algorithms.

I say that because what will be most helpful in the situation you're
facing is to *already* have learned to think about already-solved
algorithms like a toolkit. And the only way to get that toolkit is to do
some study, rather than solving each problem in the real world as it
comes along.

In this case, you are stuck IMO because the process you want to perform
on the data needs to be formally expressed. Here's an attempt:

    For each unique pair (?account_nr?, ?staff_name?):
        Sum all the ?qty? values as ?total_qty?
        Emit a record (?account_nr?, ?staff_name?, ?total_qty?)

Once expressed that way, it becomes clear to me that the requirements as
stated have a gap. What becomes of ?item?, ?noise?, etc. values for the
same (?account_nr?, ?staff_name?) pair? Are they simply discarded as
uninteresting? If not discarded, how are they processed to make a single
record for that (?account_nr?, ?staff_name?) pair?

You don't have to respond to me with the answers. But you will need to
deal with that issue, and probably others. The advantage of formally
stating the process you want is to debug it before even writing a line
of code to solve it.

Here is a course on problem solving with algorithms that uses Python
<URL:http://interactivepython.org/runestone/static/pythonds/index.html>.

Good hunting!

-- 
 \        ?We cannot solve our problems with the same thinking we used |
  `\                           when we created them.? ?Albert Einstein |
_o__)                                                                  |
Ben Finney



From python at mrabarnett.plus.com  Wed May 13 20:41:23 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 14 May 2015 01:41:23 +0100
Subject: Looking for direction
In-Reply-To: <5553E71F.8090402@stoneleaf.us>
References: <5553DD2E.2080600@pacbell.net> <5553E71F.8090402@stoneleaf.us>
Message-ID: <5553EF33.4080101@mrabarnett.plus.com>

On 2015-05-14 01:06, Ethan Furman wrote:
> On 05/13/2015 04:24 PM, 20/20 Lab wrote:
>> I'm a beginner to python.  Reading here and there.  Written a couple of
>> short and simple programs to make life easier around the office.
>>
>> That being said, I'm not even sure what I need to ask for. I've never
>> worked with external data before.
>>
>> I have a LARGE csv file that I need to process.  110+ columns, 72k
>> rows.  I managed to write enough to reduce it to a few hundred rows, and
>> the five columns I'm interested in.
>>
>> Now is were I have my problem:
>>
>> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
>>             [72976, "YYY", "Item", "Qty", "Noise"],
>>             [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
>>
>> Basically, I need to check for rows with duplicate accounts row[0] and
>> staff (row[1]), and if so, remove that row, and add it's Qty to the
>> original row. I really dont have a clue how to go about this.  The
>> number of rows change based on which run it is, so I couldnt even get
>> away with using hundreds of compare loops.
>>
>> If someone could point me to some documentation on the functions I would
>> need, or a tutorial it would be a great help.
>
> You could try using a dictionary, combining when needed:
>
> # untested
> data = {}
> for row in all_rows:
>     key = row[0], row[1]
>     if key in data:
>       item, qty, noise = data[key]
>       qty += row[3]
>     else:
>       item, qty, noise = row[2:]
>     data[key] = item, qty, noise
>
> for (account, staff), (item, qty, noise) in data.items():
>     do_stuff_with(account, staff, item, qty, noise)
>
> At the end, data should have what you want.  It won't, however, be in
> the same order, so hopefully that's not an issue for you.
>
Starting from that, if the order matters, you can do it this way:

data = {}
order = {}
for index, row in enumerate(all_rows):
     key = row[0], row[1]
     if key in data:
         item, qty, noise = data[key]
         qty += row[3]
     else:
         item, qty, noise = row[2:]
     data[key] = item, qty, noise
     order.setdefault(key, index)

merged_rows = [(account, staff, item, qty, noise) for (account, staff), 
(item, qty, noise) in data.items()]

def original_order(row):
     key = row[0], row[1]
     return order[key]

merged_rows.sort(key=original_order)



From steve+comp.lang.python at pearwood.info  Wed May 13 21:07:40 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 14 May 2015 11:07:40 +1000
Subject: Basic misunderstanding on object creation
References: <25ba3a96-21ee-4a83-b7c1-8ac60508d30c@googlegroups.com>
 <mailman.451.1431534990.12865.python-list@python.org>
 <624b0ba1-3d8a-44fe-a774-69d2d87e2cd0@googlegroups.com>
 <mailman.459.1431546366.12865.python-list@python.org>
 <9bf061a8-03f9-47ed-9774-cc21d5fec8ed@googlegroups.com>
Message-ID: <5553f55d$0$12996$c3e8da3$5496439d@news.astraweb.com>

On Thu, 14 May 2015 06:33 am, Ned Batchelder wrote:

> On Wednesday, May 13, 2015 at 3:46:16 PM UTC-4, Mark Lawrence wrote:
>> On 13/05/2015 19:42, andrew cooke wrote:
>> > On Wednesday, 13 May 2015 13:37:23 UTC-3, Terry Reedy  wrote:
>> >> On 5/13/2015 9:25 AM, andrew cooke wrote:
[...]
>> >>   > Did something change,
>> >>
>> >> Obviously yes.
>> >
>> > thanks, but why does someone on this group always have to be a dick and
>> > make some smart-assed comment like this?
>> 
>> Please don't bother asking again after that response, especially to
>> somebody with the stature of Terry Reedy.
>> 
> 
> To my ears, "Obviously yes" is a pedantic smart-ass comment.  It added
> nothing to the answer, other than to point out that the question was
> obvious. Essentially, it said, "you are silly."  I don't think it's
> unreasonable to expect the regulars here to be able to answer politely.

I don't think "Obviously yes" is necessarily intended to be rude, although
it can be read that way. "Obviously yes" goes without saying, just as "Did
something change?" goes without saying. Obviously something changed, and
just as obviously it should be obvious that it changed.

(Ooh, recursive obviousness...)


In face to face communication, a response like "Obviously yes" could be said
with a gentle tone and a smile, or it could be said with a tone that
implies an unstated "dumbarse" following. In email, it is very hard to
avoid coming across as the second without the explicit use of a smiley.


[puts on amateur headology hat to explain the sociology happening here]

I think what some people may be missing is that Andrew's question "Has
something changed?" isn't an actual question about whether something has
changed. It's more of a rhetorical question inviting validation:

"I can see something has changed, but I'm unwilling to come right out and
say so because I might be wrong, or because I'm not assertive, or because
I've been taught not to jump to conclusions (especially on technical
forums), and so I'm posing this as a question and looking for confirmation
of what I can actually see with my own eyes."

In which case, the answer being looked for is along the lines "Correct,
things changed in version <whatever>". An answer like "Obviously yes"
implies "dumb insolence" from an authority figure:

(1) It's dumb insolence because we're expected to understand the question as
a rhetorical device, not an actual question. Of course Andrew can see that
something has changed, he's not an idiot. Since Terry isn't an idiot,
presumably he knows that too, so for him to take the question literally
implies that he sees Andrew as an idiot.

(2) Terry is the authority figure here because Andrew is asking for help.
That makes Andrew the low status individual in this interaction, and Terry
is giving the help, which gives him the power in the interaction.

(This is especially annoying because normally dumb insolence is one of the
few ways that people of low status can fight back against those of high
status: by following the letter of the given instructions and treating
statements and questions literally.)

Note that many of the interactions above are based on getting into the other
person's head and inferring their state of mind. Andrew assumes that Terry
knows that his question wasn't intended literally; Andrew assumes that if
Terry does take it literally, it is because Terry intends the answer to be
a put down. (Terry confirms that with his response of "To remind people to
ask the question they want answered".) But such inferences aren't reliable
in the absence of non-verbal clues such as body language and tone of voice.


> I certainly don't think Andrew's behavior in this thread is cause for
> asking him not to return.  I also don't think "people with stature"
> should be allowed to behave in ways that others would not be allowed.
> And how are newcomers expected to identify "people with stature"?

Well *obviously* they're the ones acting like dicks without being called out
on it :-)


> Email can be a harsh medium, lacking social cues that can soften
> interchanges in other arenas.  We all have to try hard to be kind to
> each other.  And yes, I expect more of the regulars than I do of
> people new to the list.  The regulars set the tone.  They are the
> hosts here, and should be doing what they can to make new people
> feel welcome.

I agree with that. It takes two to be offended: one to give offense, and one
to take it. Nobody can force you to be offended. Either Tim or Terry could
have refused to take offense.

Instead of asking a rhetorical question about people acting like dicks,
Andrew could have thought "Terry's response sounds a bit dickish, but he
probably didn't mean it to come across as rude"; instead of defending his
choice of words in a patronising tone Terry could have said "Hmmm, yes my
response did make me sound a bit rude, but that wasn't intended".

(Unless of course it was intended to be rude. But perhaps that is too
obvious to need saying.) *wink*




-- 
Steven



From davea at davea.name  Wed May 13 21:12:53 2015
From: davea at davea.name (Dave Angel)
Date: Wed, 13 May 2015 21:12:53 -0400
Subject: Looking for direction
In-Reply-To: <5553E72B.5000309@davea.name>
References: <5553DD2E.2080600@pacbell.net> <5553E72B.5000309@davea.name>
Message-ID: <5553F695.9040903@davea.name>

On 05/13/2015 08:45 PM, 20/20 Lab wrote:>

You accidentally replied to me, rather than the mailing list.  Please 
use reply-list, or if your mailer can't handle that, do a Reply-All, and 
remove the parts you don't want.

 >
 > On 05/13/2015 05:07 PM, Dave Angel wrote:
 >> On 05/13/2015 07:24 PM, 20/20 Lab wrote:
 >>> I'm a beginner to python.  Reading here and there.  Written a couple of
 >>> short and simple programs to make life easier around the office.
 >>>
 >> Welcome to Python, and to this mailing list.
 >>
 >>> That being said, I'm not even sure what I need to ask for. I've never
 >>> worked with external data before.
 >>>
 >>> I have a LARGE csv file that I need to process.  110+ columns, 72k
 >>> rows.
 >>
 >> That's not very large at all.
 >>
 > In the grand scheme, I guess not.  However I'm currently doing this
 > whole process using office.  So it can be a bit daunting.

I'm not familiar with the "office" operating system.

 >>>  I managed to write enough to reduce it to a few hundred rows, and
 >>> the five columns I'm interested in.
 >>
 >>>
 >>> Now is were I have my problem:
 >>>
 >>> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
 >>>             [72976, "YYY", "Item", "Qty", "Noise"],
 >>>             [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
 >>>
 >>
 >> It'd probably be useful to identify names for your columns, even if
 >> it's just in a comment.  Guessing from the paragraph below, I figure
 >> the first two columns are "account" & "staff"
 >
 > The columns that I pull are Account, Staff, Item Sold, Quantity sold,
 > and notes about the sale (notes arent particularly needed, but the
 > higher ups would like them in the report)
 >>
 >>> Basically, I need to check for rows with duplicate accounts row[0] and
 >>> staff (row[1]), and if so, remove that row, and add it's Qty to the
 >>> original row.
 >>
 >> And which column is that supposed to be?  Shouldn't there be a number
 >> there, rather than a string?
 >>
 >>> I really dont have a clue how to go about this.  The
 >>> number of rows change based on which run it is, so I couldnt even get
 >>> away with using hundreds of compare loops.
 >>>
 >>> If someone could point me to some documentation on the functions I 
would
 >>> need, or a tutorial it would be a great help.
 >>>
 >>
 >> Is the order significant?  Do you have to preserve the order that the
 >> accounts appear?  I'll assume not.
 >>
 >> Have you studied dictionaries?  Seems to me the way to handle the
 >> problem is to read in a row, create a dictionary with key of (account,
 >> staff), and data of the rest of the line.
 >>
 >> Each time you read a row, you check if the key is already in the
 >> dictionary.  If not, add it.  If it's already there, merge the data as
 >> you say.
 >>
 >> Then when you're done, turn the dict back into a list of lists.
 >>
 > The order is irrelevant.  No, I've not really studied dictionaries, but
 > a few people have mentioned it.  I'll have to read up on them and, more
 > importantly, their applications.  Seems that they are more versatile
 > then I thought.
 >
 > Thank you.

You have to realize that a tuple can be used as a key, in your case a 
tuple of Account and Staff.

You'll have to decide how you're going to merge the ItemSold, 
QuantitySold, and notes.

-- 
DaveA


-- 
DaveA


From steve+comp.lang.python at pearwood.info  Wed May 13 21:23:09 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 14 May 2015 11:23:09 +1000
Subject: Looking for direction
References: <mailman.465.1431559626.12865.python-list@python.org>
Message-ID: <5553f8fe$0$13012$c3e8da3$5496439d@news.astraweb.com>

On Thu, 14 May 2015 09:24 am, 20/20 Lab wrote:

> I'm a beginner to python.  Reading here and there.  Written a couple of
> short and simple programs to make life easier around the office.
> 
> That being said, I'm not even sure what I need to ask for. I've never
> worked with external data before.
> 
> I have a LARGE csv file that I need to process.  110+ columns, 72k
> rows.  I managed to write enough to reduce it to a few hundred rows, and
> the five columns I'm interested in.

That's not large. Large is millions of rows, or tens of millions if you have
enough memory. What's large to you and me is usually small to the computer.

You should use the csv module for handling the CSV file, if you aren't
already doing so. Do you need a url to the docs?


> Now is were I have my problem:
> 
> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
>             [72976, "YYY", "Item", "Qty", "Noise"],
>             [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
> 
> Basically, I need to check for rows with duplicate accounts row[0] and
> staff (row[1]), and if so, remove that row, and add it's Qty to the
> original row. I really dont have a clue how to go about this.

Is the order of the rows important? If not, the problem is simpler.


processed = {}  # hold the processed data in a dict

for row in myList:
    account, staff = row[0:2]
    key = (account, staff)  # Put them in a tuple.
    if key in processed:
        # We've already seen this combination.
        processed[key][3] += row[3]  # Add the quantities.
    else:
        # Never seen this combination before.
        processed[key] = row

newlist = list(processed.values())


Does that help?



-- 
Steven



From steve+comp.lang.python at pearwood.info  Wed May 13 21:40:29 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 14 May 2015 11:40:29 +1000
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
 <mailman.449.1431530832.12865.python-list@python.org>
 <71ee2dbb-27a3-45ad-b903-bffca6d8923f@googlegroups.com>
Message-ID: <5553fd0e$0$12995$c3e8da3$5496439d@news.astraweb.com>

On Thu, 14 May 2015 04:07 am, zipher wrote:

> On Wednesday, May 13, 2015 at 10:27:23 AM UTC-5, Ian wrote:
>> I don't know why I'm replying to this...
> 
> Because you're trying to get an answer to a question that even Academia
> hasn't answered or understood.
> 
>> On Wed, May 13, 2015 at 8:44 AM, zipher <dreamingforward at gmail.com>
>> wrote:
>> > On Tuesday, May 12, 2015 at 10:35:29 PM UTC-5, Rustom Mody wrote:
>> >> How history U-turns!!
>> >> Lisp actually got every major/fundamental thing wrong
>> >> - variables scopes were dynamic by mistake
>> >> - lambdas were non-first class because the locution 'first-class' was
>> >> still 8 years in the future
>> >
>> > I think you're confused.  LISP doesn't have variables.
>> 
>> Yes, it does.
> 
> No, Common LISP does, but as the website says Common LISP is a
> "multi-paradigm" langauge.   It's trying to be everything to everybody,
> just like Python tried to do in the other direction, making "everything an
> object".  Python was trying to be too pure, while LISP was trying to be
> too universal:  be everything to everyone -- you might say "batteries
> included".
> 
> True LISP [...]

Ah, the "No True Scotsman" fallacy. You know that Lisp compiler you're
using? That's not *true* Lisp, because *true* Lisp doesn't have [insert
feature here].

If you really want to justify your claim that Lisp has no variables, you can
start by answering two questions:

(1) What is the definition of "variable" you are using?

(2) What *objective* methods of distinguishing a "True" Lisp from an
false/fake/faux/counterfit Lisp are there?


> It's only abstractions, like math.  It's purpose is to output a final
> result, that is all.  It's not at all to make commercial applications.

Your suggestion that the people and companies that created Lisp machines
weren't interested in commercial applications for Lisp is very naive.


[...]
> Again, you're speaking of "Common LISP".  Such a lisp also wants to do
> OOP, but it's like lipstick on a pig.

You really love that metaphor, don't you? You keep using it, and variations
like "a pig made of lipstick". I suggest you don't understand the metaphor.

"Lipstick on a pig" is not a metaphor for combining two things which are
unrelated, or that shouldn't go together. It is a metaphor for the
impossibility of turning something of low value (a pig) into high value
just by applying a few cosmetic changes.

"Lipstick on a pig" would describe giving a wax and polish to a lemon of a
car. It might be all shiny and clean, but it still won't go. It might
describe putting a coarse, rough, uncultured thug in a suit and tie, or
putting a weak and under-powered text editor like Windows Notepad in a
fancy GUI -- it's still weak and underpowered, Aero interface or not.

So by your metaphor, Lisp is the pig (i.e. a useless, weak, unpleasant and
disagreeable programming language) and OOP is a mere cosmetic change that
doesn't change the fact that Lisp is useless and unpleasant. Is that the
message you want to give?


>> > with an entirely different model computation than other programming
>> > languages which use variables all the time.  To the extent that it DOES
>> > have variables, it's to accommodate those coming over from iterative
>> > programming.
>> 
>> What is "iterative programming"? If you mean "writing programs that
>> work iteratively", then this describes both functional and procedural
>> styles alike.
> 
> Yes, and LISP is neither.  Although LISP is a functional style, that is
> only by appearance.  It's completely different from Haskell, which I would
> describe as a true functional language.  The difference is how the program
> is lexed in the mind or on the machine.  But that's too difficult to
> explain on this thread.

Just because something is difficult to explain doesn't make it correct. It's
difficult to explain how the Atlantic Ocean merely looks, tastes and feels
like salt water while actually being filled with strawberry yoghurt. That
doesn't make it true.

Something which is difficult to explain might be deeply profound, but is far
more likely to be rubbish. Saying that Lisp isn't actually functional but
merely is so "only by appearance" is as profound as saying that the
Atlantic isn't filled with water, it only appears to be filled with water.


>> The opposite of "iterative programming" would then be a style where
>> the program can't ever repeat anything. That would be a very limited
>> language and would *not* be equivalent to a Turing machine.
> 
> 
> The "opposite" of iterative programming is recursive programming.

No it isn't. Iteration and recursion are, in a very deep sense, actually the
same thing, they're certainly not opposites. That is a fact which genuinely
is profound.


> It's 
> not limited, except that it has an entirely different relationship to
> data, one orthogonal to iterative computation.

That's *not even wrong*.


>> > And the idea of lambdas were already encoded by the use of special
>> > expressions, set-off by parenthesis.  So they practically *defined* the
>> > concept of lambdas.
>> 
>> LISP is also the reason why we're cursed with the terrible name
>> "lambda" for anonymous functions rather than something more mnemonic
>> (like "function").
> 
> No, you haven't understood, padawan.  Lambda *is* the function, not it's
> definition.  Perhaps you will understand what I mean by that, perhaps you
> won't.  It's subtle.

Subtle like a kick to the head.

Mark, you seem to be labouring under the delusion that we don't agree with
you because we "boneheads" don't understand what you are talking about.
That's wrong. We understand what you are talking about. We don't agree with
you because half of your thesis is wrong and the other half is not even
wrong.

If you wish to change our minds, you're going to have to demonstrate
objective, verifiable facts and not just allude to how your ideas are too
subtle and clever for us.



-- 
Steven



From rosuav at gmail.com  Wed May 13 22:03:49 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 14 May 2015 12:03:49 +1000
Subject: Python file structure
In-Reply-To: <20150513133422.78617956@bigbox.christie.dr>
References: <f25aa9d4-4025-457d-8072-5327c98db1bd@googlegroups.com>
 <CAPTjJmrtLoCGOAr+F1Q7zgzv3hUhvrVA3ADdaLe9mP9YSShi6Q@mail.gmail.com>
 <CALwzid=tsm9c6DWyQ+g72+uG_KM+eDQ2Be3Pgj5gj6sorhNGHw@mail.gmail.com>
 <CAPTjJmpDfdYZ1LXeHbHNTuv_L4vaNZXnz-yOffjc1sM8tAd1Uw@mail.gmail.com>
 <20150513133422.78617956@bigbox.christie.dr>
Message-ID: <CAPTjJmogc=bOZhqEMnuNuVWTxwrxOy2-zPemj6erqE_Xadt_zA@mail.gmail.com>

On Thu, May 14, 2015 at 4:34 AM, Tim Chase
<python.list at tim.thechases.com> wrote:
> Usually mine look something like
>
>   def do_real_work(options, args):
>     ...
>   def main():
>     parser = [optparse,argparse,docopt]....
>     options, args = parser.parse_args()
>     do_real_work(options, args)
>   if __name__ == "__main__":
>     main()
>
> since my real-work function usually relies on configuration
> (sometimes this also includes a config-file or environment variables
> being munged into some "options" data structure).
>

Sure. I rather dislike the whole duplication that that entails,
though, so I try to have the functions themselves do their own
argparse config. To that end, I put together a new project on PyPI,
but have now deprecated it in favour of Clize; the upshot is that my
main function becomes trivial again:

https://github.com/Rosuav/LetMeKnow/blob/master/letmeknow.py

@command
def await(...):
    """argparse config comes from here"""

if __name__ == "__main__":
    clize.run(commands)

So it comes and goes a bit. If there's real content in your main(),
then by all means, separate it out from do_real_work; but if the work
is all done elsewhere, not much point with main().

ChrisA


From breamoreboy at yahoo.co.uk  Wed May 13 22:35:55 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 14 May 2015 03:35:55 +0100
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <5553fd0e$0$12995$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
 <mailman.449.1431530832.12865.python-list@python.org>
 <71ee2dbb-27a3-45ad-b903-bffca6d8923f@googlegroups.com>
 <5553fd0e$0$12995$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mj11md$c5n$1@ger.gmane.org>

On 14/05/2015 02:40, Steven D'Aprano wrote:
> On Thu, 14 May 2015 04:07 am, zipher wrote:
>
>>
>> No, you haven't understood, padawan.  Lambda *is* the function, not it's
>> definition.  Perhaps you will understand what I mean by that, perhaps you
>> won't.  It's subtle.
>
> Subtle like a kick to the head.
>
> Mark, you seem to be labouring under the delusion that we don't agree with
> you because we "boneheads" don't understand what you are talking about.
> That's wrong. We understand what you are talking about. We don't agree with
> you because half of your thesis is wrong and the other half is not even
> wrong.
>
> If you wish to change our minds, you're going to have to demonstrate
> objective, verifiable facts and not just allude to how your ideas are too
> subtle and clever for us.
>

 From the very first drivel that he posted on python ideas just over two 
years ago, he's shown that he's incapable of any logical thought 
relating to computing, in just the same way that the RUE has never 
posted anything logical about the FSR.  Please can we stop feeding him.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From kaiser.yann at gmail.com  Thu May 14 01:43:00 2015
From: kaiser.yann at gmail.com (Yann Kaiser)
Date: Thu, 14 May 2015 05:43:00 +0000
Subject: Clize 3.0b1: An argument parser that draws a CLI from your
 function sigature
In-Reply-To: <CANUJvPWSO8bW3mNh1_wCH6qcoGtUE=9zJ71acdga8d0vViZZ8Q@mail.gmail.com>
References: <CANUJvPWSO8bW3mNh1_wCH6qcoGtUE=9zJ71acdga8d0vViZZ8Q@mail.gmail.com>
Message-ID: <CANUJvPWsppJfxqQfntLT9VkmZV=X8WDw8D+BjG9U0ovXOV7hmw@mail.gmail.com>

And... 3.0 is released! :-)

Feel free to contact me or reply should you encounter any issues!

http://clize.readthedocs.org/en/3.0/releases.html#v3-0

On Mon, 27 Apr 2015 at 02:02 Yann Kaiser <kaiser.yann at gmail.com> wrote:

> Hello everyone!
>
> After a few years in development, I am proud to say Clize is landing its
> feet again and is now in beta for an upcoming release.
>
> You can try it out using    pip install --user clize=3.0b1    and you can
> browse the docs at    https://clize.readthedocs.org/
>
> For those who'd like an explanation before clicking links, here's Clize's
> 5-bullet point explanation:
>
> * Command-line interfaces are created by passing functions to `clize.run`.
> * Parameter types are deduced from the functions' parameters.
> * A ``--help`` message is generated from your docstrings. (Why does this
> still need to be a bullet point?)
> * Decorators can be used to reuse functionality across functions.
> * Clize can be extended with new parameter behavior.
>
> I am primarily looking for feedback on the documentation contents, and I
> need to source a FAQ page, but any feedback is welcome!
>
> If you are into that sort of thing, I've also just set up a G+ community
> for it at: https://plus.google.com/communities/101146333300650079362
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150514/062bb7d3/attachment.html>

From steve+comp.lang.python at pearwood.info  Thu May 14 11:45:28 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 15 May 2015 01:45:28 +1000
Subject: Survey -- Move To Trash function in Python?
Message-ID: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>

I'd like to do a little survey, and get a quick show of hands.

How many people have written GUI or text-based applications or scripts where
a "Move file to trash" function would be useful?

Would you like to see that in the standard library, even if it meant that
the library had feature-freeze and could gain no more functionality?


-- 
Steven



From invalid at invalid.invalid  Thu May 14 11:49:52 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 14 May 2015 15:49:52 +0000 (UTC)
Subject: Survey -- Move To Trash function in Python?
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mj2g70$eij$1@reader1.panix.com>

On 2015-05-14, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:

> I'd like to do a little survey, and get a quick show of hands.
>
> How many people have written GUI or text-based applications or
> scripts where a "Move file to trash" function would be useful?

How would you even define what "move to trash" means in a standard
way?

Even withing the limited context of something like a mail client, it's
meaning varies depending on mail store format or which IMAP server
you're talking to.

Or are you assuming the only thing that can be "moved to trash" is a
file?

> Would you like to see that in the standard library, even if it meant that
> the library had feature-freeze and could gain no more functionality?

-- 
Grant Edwards               grant.b.edwards        Yow! I'm having fun
                                  at               HITCHHIKING to CINCINNATI
                              gmail.com            or FAR ROCKAWAY!!


From bc at freeuk.com  Thu May 14 11:51:29 2015
From: bc at freeuk.com (BartC)
Date: Thu, 14 May 2015 16:51:29 +0100
Subject: Building CPython
In-Reply-To: <mailman.464.1431556504.12865.python-list@python.org>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
Message-ID: <6w35x.645690$I97.19867@fx31.am4>

On 13/05/2015 23:34, Terry Reedy wrote:
> On 5/13/2015 3:36 PM, BartC wrote:
>> I'm interested in playing with the CPython sources. I need to be able to
>> build under Windows, but don't want to use make files (which rarely work
>> properly), nor do a 6GB installation of Visual Studio Express which is
>> what seems to be needed (I'm hopeless with complicated IDEs anyway).
>
> Once installed hg or tortoisehg (I use this) and VSE are installed and
> repository cloned, are half done. At command prompt, with top directory
> of repository as current directory enter
> tools\scripts\external.bat
> Double-clicking file in Explorer does not work.
> Usually only needs to be done once per branch after x.y.0 release as
> dependencies are usually not updated for bugfix releases.
>
> Then in same directory enter
> pcbuild\python.sln
> or double click in Explorer or open VSE and open this file.
> Hit F7, wait until get line like
> ========== Build: 1 succeeded, 0 failed, 24 up-to-date, 1 skipped,
> hit F5, pin python_d to taskbar (optional, but handy), and go.

OK, the answer seems to be No then - you can't just trivially compile 
the C modules that comprise the sources with the nearest compiler to 
hand. So much for C's famous portability!

(Actually, I think you already lost me on your first line.)

That's a shame because I wanted to tinker with the main dispatcher loop 
to try and find out what exactly is making it slow. Nothing that seems 
obvious at first sight. (The comments even talk about improving branch 
prediction on certain architectures, even though the performance is a 
couple of magnitudes away from that kind of optimisation being relevant.)

Perhaps I was hoping there were some options turned on by default which, 
if disabled, would suddenly double the speed of simple benchmarks. Now I 
won't be able to find out...

-- 
Bartc



From lab at pacbell.net  Thu May 14 11:58:48 2015
From: lab at pacbell.net (20/20 Lab)
Date: Thu, 14 May 2015 08:58:48 -0700
Subject: Looking for direction
In-Reply-To: <5553F695.9040903@davea.name>
References: <5553DD2E.2080600@pacbell.net> <5553E72B.5000309@davea.name>
 <5553F695.9040903@davea.name>
Message-ID: <5554C638.5070305@pacbell.net>



On 05/13/2015 06:12 PM, Dave Angel wrote:
> On 05/13/2015 08:45 PM, 20/20 Lab wrote:>
>
> You accidentally replied to me, rather than the mailing list. Please 
> use reply-list, or if your mailer can't handle that, do a Reply-All, 
> and remove the parts you don't want.
>
> >
> > On 05/13/2015 05:07 PM, Dave Angel wrote:
> >> On 05/13/2015 07:24 PM, 20/20 Lab wrote:
> >>> I'm a beginner to python.  Reading here and there. Written a 
> couple of
> >>> short and simple programs to make life easier around the office.
> >>>
> >> Welcome to Python, and to this mailing list.
> >>
> >>> That being said, I'm not even sure what I need to ask for. I've never
> >>> worked with external data before.
> >>>
> >>> I have a LARGE csv file that I need to process.  110+ columns, 72k
> >>> rows.
> >>
> >> That's not very large at all.
> >>
> > In the grand scheme, I guess not.  However I'm currently doing this
> > whole process using office.  So it can be a bit daunting.
>
> I'm not familiar with the "office" operating system.
>
> >>>  I managed to write enough to reduce it to a few hundred rows, and
> >>> the five columns I'm interested in.
> >>
> >>>
> >>> Now is were I have my problem:
> >>>
> >>> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
> >>>             [72976, "YYY", "Item", "Qty", "Noise"],
> >>>             [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
> >>>
> >>
> >> It'd probably be useful to identify names for your columns, even if
> >> it's just in a comment.  Guessing from the paragraph below, I figure
> >> the first two columns are "account" & "staff"
> >
> > The columns that I pull are Account, Staff, Item Sold, Quantity sold,
> > and notes about the sale (notes arent particularly needed, but the
> > higher ups would like them in the report)
> >>
> >>> Basically, I need to check for rows with duplicate accounts row[0] 
> and
> >>> staff (row[1]), and if so, remove that row, and add it's Qty to the
> >>> original row.
> >>
> >> And which column is that supposed to be?  Shouldn't there be a number
> >> there, rather than a string?
> >>
> >>> I really dont have a clue how to go about this.  The
> >>> number of rows change based on which run it is, so I couldnt even get
> >>> away with using hundreds of compare loops.
> >>>
> >>> If someone could point me to some documentation on the functions I 
> would
> >>> need, or a tutorial it would be a great help.
> >>>
> >>
> >> Is the order significant?  Do you have to preserve the order that the
> >> accounts appear?  I'll assume not.
> >>
> >> Have you studied dictionaries?  Seems to me the way to handle the
> >> problem is to read in a row, create a dictionary with key of (account,
> >> staff), and data of the rest of the line.
> >>
> >> Each time you read a row, you check if the key is already in the
> >> dictionary.  If not, add it.  If it's already there, merge the data as
> >> you say.
> >>
> >> Then when you're done, turn the dict back into a list of lists.
> >>
> > The order is irrelevant.  No, I've not really studied dictionaries, but
> > a few people have mentioned it.  I'll have to read up on them and, more
> > importantly, their applications.  Seems that they are more versatile
> > then I thought.
> >
> > Thank you.
>
> You have to realize that a tuple can be used as a key, in your case a 
> tuple of Account and Staff.
>
> You'll have to decide how you're going to merge the ItemSold, 
> QuantitySold, and notes.
>
Tells you how often I actually talk in mailing lists.  My apologies, and 
thank you again.


From rosuav at gmail.com  Thu May 14 11:59:00 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 15 May 2015 01:59:00 +1000
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <mj2g70$eij$1@reader1.panix.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <mj2g70$eij$1@reader1.panix.com>
Message-ID: <CAPTjJmpjk6WB2h_qq3uOv1CpYoXhe6cS_Rp9sPCOU6gGu1qt8g@mail.gmail.com>

On Fri, May 15, 2015 at 1:49 AM, Grant Edwards <invalid at invalid.invalid> wrote:
> On 2015-05-14, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
>
>> I'd like to do a little survey, and get a quick show of hands.
>>
>> How many people have written GUI or text-based applications or
>> scripts where a "Move file to trash" function would be useful?
>
> How would you even define what "move to trash" means in a standard
> way?
>
> Even withing the limited context of something like a mail client, it's
> meaning varies depending on mail store format or which IMAP server
> you're talking to.
>
> Or are you assuming the only thing that can be "moved to trash" is a
> file?

AIUI this is specifically about files.

I have never used such a feature, and generally found them to be more
annoying than useful. The times when I want to mark a file for
deletion either now or at some undefined time in the future (at the
file system's discretion) are approximately zero; either I want the
file, or I can delete it immediately. Given that actually-removed
files can usually be undeleted if their disk space hasn't been
reclaimed, I don't think trash cans really help much.

Plus, I think it's a UI insanity when a console program dumps
something in the trash, which to me has always felt like a GUI file
manager tool.

That said, though, I think a standardized API for trashing files is
not a bad thing; and it's not something that will need huge API
changes, so the costs of feature-freeze would be low.

ChrisA


From rosuav at gmail.com  Thu May 14 12:09:29 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 15 May 2015 02:09:29 +1000
Subject: Building CPython
In-Reply-To: <6w35x.645690$I97.19867@fx31.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
Message-ID: <CAPTjJmrwLrPcwh20xc+osrpmAXT7fuDdr4+NG7krZxyFkfQETw@mail.gmail.com>

On Fri, May 15, 2015 at 1:51 AM, BartC <bc at freeuk.com> wrote:
> OK, the answer seems to be No then - you can't just trivially compile the C
> modules that comprise the sources with the nearest compiler to hand. So much
> for C's famous portability!
>
> (Actually, I think you already lost me on your first line.)
>
> That's a shame because I wanted to tinker with the main dispatcher loop to
> try and find out what exactly is making it slow. Nothing that seems obvious
> at first sight. (The comments even talk about improving branch prediction on
> certain architectures, even though the performance is a couple of magnitudes
> away from that kind of optimisation being relevant.)

C's portability isn't really sufficient for building a huge project,
so what you generally end up with is a huge slab of common code that
doesn't change from platform to platform, plus a (relatively) tiny
section of platform-specific code, such as makefiles/project files,
linker definitions, and so on. When you start hacking on CPython, you
don't generally have to consider which platform you're aiming at, as
long as you're building on one of the ones that's well supported;
trying to port Python to a new compiler on a known OS is actually
about as much work as porting it to a known compiler on a new OS. (I
know this, because I've attempted both - using mingw on Windows, and
gcc on OS/2. It's a big job either way.)

If you want to just quickly play around with CPython's sources, I
would strongly recommend getting yourself a Linux box. Either spin up
some actual hardware with actual Linux, or grab a virtualization
engine like VMWare, VirtualBox, etc, etc, and installing into a VM.
With a Debian-based Linux (Debian, Ubuntu, Mint, etc), you should
simply be able to:

sudo apt-get build-dep python3

to get all the build dependencies for Python 3; that, plus the source
code, should be enough to get you a'building. Similar simplicities are
available for other Linux distros, but I'll let someone else recommend
them.

Even when you have all the appropriate build tools, Windows can be at
times a pain for building software on. The general philosophy of
Windows is that you should normally be taking ready-made binaries; the
general philosophy of Linux is that it's perfectly normal to spin up
your own binaries from the distributed source code. It's not
impossible to build on Windows, by any means, but be prepared for
extra hassles and less support from the OS than you might like.

ChrisA


From lab at pacbell.net  Thu May 14 12:23:11 2015
From: lab at pacbell.net (20/20 Lab)
Date: Thu, 14 May 2015 09:23:11 -0700
Subject: Looking for direction
In-Reply-To: <5553F695.9040903@davea.name>
References: <5553DD2E.2080600@pacbell.net> <5553E72B.5000309@davea.name>
 <5553F695.9040903@davea.name>
Message-ID: <5554CBEF.5090309@pacbell.net>



On 05/13/2015 06:12 PM, Dave Angel wrote:
> On 05/13/2015 08:45 PM, 20/20 Lab wrote:>
>
> You accidentally replied to me, rather than the mailing list. Please 
> use reply-list, or if your mailer can't handle that, do a Reply-All, 
> and remove the parts you don't want.
>
...and now that you mention it.  I appear to have done that with all of 
my replies yesterday.

My deepest apologies for that.



From marko at pacujo.net  Thu May 14 12:29:40 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 14 May 2015 19:29:40 +0300
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
Message-ID: <874mnfunpn.fsf@elektro.pacujo.net>

BartC <bc at freeuk.com>:

> That's a shame because I wanted to tinker with the main dispatcher
> loop to try and find out what exactly is making it slow. Nothing that
> seems obvious at first sight.

My guess is the main culprit is attribute lookup in two ways:

 * Each object attribute reference involves a dictionary lookup.

 * Each method call involves *two* dictionary lookups plus an object
   creation.

Tell us what you find out.


Marko


From lab at pacbell.net  Thu May 14 12:57:48 2015
From: lab at pacbell.net (20/20 Lab)
Date: Thu, 14 May 2015 09:57:48 -0700
Subject: Looking for direction
In-Reply-To: <5553f8fe$0$13012$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.465.1431559626.12865.python-list@python.org>
 <5553f8fe$0$13012$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5554D40C.9090505@pacbell.net>



On 05/13/2015 06:23 PM, Steven D'Aprano wrote:
> On Thu, 14 May 2015 09:24 am, 20/20 Lab wrote:
>
>> I'm a beginner to python.  Reading here and there.  Written a couple of
>> short and simple programs to make life easier around the office.
>>
>> That being said, I'm not even sure what I need to ask for. I've never
>> worked with external data before.
>>
>> I have a LARGE csv file that I need to process.  110+ columns, 72k
>> rows.  I managed to write enough to reduce it to a few hundred rows, and
>> the five columns I'm interested in.
> That's not large. Large is millions of rows, or tens of millions if you have
> enough memory. What's large to you and me is usually small to the computer.
>
> You should use the csv module for handling the CSV file, if you aren't
> already doing so. Do you need a url to the docs?
>
I actually stumbled across the csv module after coding enough to make a 
list of lists.  So that is more the reason I approached the list;  
Nothing like spending hours (or days) coding something that already 
exists and just dont know about.
>> Now is were I have my problem:
>>
>> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
>>              [72976, "YYY", "Item", "Qty", "Noise"],
>>              [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
>>
>> Basically, I need to check for rows with duplicate accounts row[0] and
>> staff (row[1]), and if so, remove that row, and add it's Qty to the
>> original row. I really dont have a clue how to go about this.
> Is the order of the rows important? If not, the problem is simpler.
>
>
> processed = {}  # hold the processed data in a dict
>
> for row in myList:
>      account, staff = row[0:2]
>      key = (account, staff)  # Put them in a tuple.
>      if key in processed:
>          # We've already seen this combination.
>          processed[key][3] += row[3]  # Add the quantities.
>      else:
>          # Never seen this combination before.
>          processed[key] = row
>
> newlist = list(processed.values())
>
>
> Does that help?
>
>
>
It does, immensely.  I'll make this work.  Thank you again for the link 
from yesterday and apologies for hitting the wrong reply button.  I'll 
have to study more on the usage and implementations of dictionaries and 
tuples.


From bc at freeuk.com  Thu May 14 13:02:11 2015
From: bc at freeuk.com (BartC)
Date: Thu, 14 May 2015 18:02:11 +0100
Subject: Building CPython
In-Reply-To: <mailman.4.1431619772.17265.python-list@python.org>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <mailman.4.1431619772.17265.python-list@python.org>
Message-ID: <oy45x.458028$zE7.268573@fx40.am4>

On 14/05/2015 17:09, Chris Angelico wrote:
> On Fri, May 15, 2015 at 1:51 AM, BartC <bc at freeuk.com> wrote:
>> OK, the answer seems to be No then - you can't just trivially compile the C
>> modules that comprise the sources with the nearest compiler to hand. So much
>> for C's famous portability!
>>
>> (Actually, I think you already lost me on your first line.)

> If you want to just quickly play around with CPython's sources, I
> would strongly recommend getting yourself a Linux box. Either spin up
> some actual hardware with actual Linux, or grab a virtualization
> engine like VMWare, VirtualBox, etc, etc, and installing into a VM.
> With a Debian-based Linux (Debian, Ubuntu, Mint, etc), you should
> simply be able to:
>
> sudo apt-get build-dep python3

Actually I had VirtualBox with Ubuntu, but I don't know my way around 
Linux and preferred doing things under Windows (and with all my own tools).

But it's now building under Ubuntu.

(Well, I'm not sure what it's doing exactly; the instructions said type 
make, then make test, then make install, and it's still doing make test.

I hope there's a quicker way of re-building an executable after a minor 
source file change, otherwise doing any sort of development is going to 
be impractical.)

-- 
Bartc


From fomcl at yahoo.com  Thu May 14 13:08:21 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Thu, 14 May 2015 10:08:21 -0700
Subject: Looking for direction
Message-ID: <1431623301.67617.BPMail_high_carrier@web163803.mail.gq1.yahoo.com>


-----------------------------
On Thu, May 14, 2015 3:35 PM CEST Dennis Lee Bieber wrote:

>On Wed, 13 May 2015 16:24:30 -0700, 20/20 Lab <lab at pacbell.net> declaimed
>the following:
>
>>Now is were I have my problem:
>>
>>myList = [ [123, "XXX", "Item", "Qty", "Noise"],
>>            [72976, "YYY", "Item", "Qty", "Noise"],
>>            [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
>>
>>Basically, I need to check for rows with duplicate accounts row[0] and 
>>staff (row[1]), and if so, remove that row, and add it's Qty to the 
>>original row. I really dont have a clue how to go about this.  The 
>>number of rows change based on which run it is, so I couldnt even get 
>>away with using hundreds of compare loops.
>>
>>If someone could point me to some documentation on the functions I would 
>>need, or a tutorial it would be a great help.
>>
>
>	This appears to be a matter of algorithm development -- there won't be
>an pre-made "function" for it. The closest would be the summing functions
>(control break http://en.wikipedia.org/wiki/Control_break ) of a report
>writer application.
>
>	The short gist would be:
>
>	SORT the data by the account field
>	Initialize sum using first record
>	loop
>		read next record
>		if end of data
>			output sum record
>			exit
>		if record is same account as sum
>			add quantity to sum
>		else
>			output sum record
>			reset sum to the new record
>
>	Granted -- loading the data into an SQL capable database would make
>this simple...
>
>	select account, sum(quantity) from table
>	order by account

You could also use pandas. Read the data in a DataFrame, create a groupby object, use the sum() and the first() methods.

http://pandas.pydata.org/pandas-docs/version/0.15.2/groupby.html




From davea at davea.name  Thu May 14 13:10:54 2015
From: davea at davea.name (Dave Angel)
Date: Thu, 14 May 2015 13:10:54 -0400
Subject: Building CPython
In-Reply-To: <oy45x.458028$zE7.268573@fx40.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <mailman.4.1431619772.17265.python-list@python.org>
 <oy45x.458028$zE7.268573@fx40.am4>
Message-ID: <5554D71E.1050806@davea.name>

On 05/14/2015 01:02 PM, BartC wrote:
> On 14/05/2015 17:09, Chris Angelico wrote:
>> On Fri, May 15, 2015 at 1:51 AM, BartC <bc at freeuk.com> wrote:
>>> OK, the answer seems to be No then - you can't just trivially compile
>>> the C
>>> modules that comprise the sources with the nearest compiler to hand.
>>> So much
>>> for C's famous portability!
>>>
>>> (Actually, I think you already lost me on your first line.)
>
>> If you want to just quickly play around with CPython's sources, I
>> would strongly recommend getting yourself a Linux box. Either spin up
>> some actual hardware with actual Linux, or grab a virtualization
>> engine like VMWare, VirtualBox, etc, etc, and installing into a VM.
>> With a Debian-based Linux (Debian, Ubuntu, Mint, etc), you should
>> simply be able to:
>>
>> sudo apt-get build-dep python3
>
> Actually I had VirtualBox with Ubuntu, but I don't know my way around
> Linux and preferred doing things under Windows (and with all my own tools).
>
> But it's now building under Ubuntu.
>
> (Well, I'm not sure what it's doing exactly; the instructions said type
> make, then make test, then make install, and it's still doing make test.
>
> I hope there's a quicker way of re-building an executable after a minor
> source file change, otherwise doing any sort of development is going to
> be impractical.)
>

That's what make is good for.  It compares the datestamps of the source 
files against the obj files (etc.) and recompiles only when the source 
is newer.  (It's more complex, but that's the idea)

-- 
DaveA


From rosuav at gmail.com  Thu May 14 13:11:07 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 15 May 2015 03:11:07 +1000
Subject: Building CPython
In-Reply-To: <oy45x.458028$zE7.268573@fx40.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <mailman.4.1431619772.17265.python-list@python.org>
 <oy45x.458028$zE7.268573@fx40.am4>
Message-ID: <CAPTjJmq7amGAYn63C_w2uHp0AGNPxzQHVngQ5bnXrJrVxg5nXg@mail.gmail.com>

On Fri, May 15, 2015 at 3:02 AM, BartC <bc at freeuk.com> wrote:
> Actually I had VirtualBox with Ubuntu, but I don't know my way around Linux
> and preferred doing things under Windows (and with all my own tools).
>
> But it's now building under Ubuntu.
>
> (Well, I'm not sure what it's doing exactly; the instructions said type
> make, then make test, then make install, and it's still doing make test.
>
> I hope there's a quicker way of re-building an executable after a minor
> source file change, otherwise doing any sort of development is going to be
> impractical.)

The whole point of 'make' is to rebuild only the parts that need to be
rebuilt (either they've changed, or they depend on something that was
changed). Sometimes practically everything needs to be rebuilt, if you
do some really fundamental change, but generally not.

The three parts to the build process are:

1) make - actually generate an executable. Takes ages the first time,
will be a lot quicker if you haven't changed much.
2) make test - run the entire test suite. Takes just as long every
time, but most of it won't have changed.
3) make install (needs root access, so probably 'sudo make install') -
install this as your primary build of Python.

When you start tinkering, I suggest just running make; rerunning the
test suite isn't necessary till you're all done, and even then it's
only important for making sure that your change hasn't broken anything
anywhere else. Test your actual changes by simply running the
freshly-built Python - most likely that'll be "./python". Working that
way is fairly quick - you can tweak some C code, see the results, and
go back and tweak some more, all without pausing for a sword fight.
But once you go and rerun the full test suite, well... that's when
it's time for some:

https://xkcd.com/303/

ChrisA


From python.list at tim.thechases.com  Thu May 14 13:17:59 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Thu, 14 May 2015 12:17:59 -0500
Subject: Looking for direction
In-Reply-To: <5554D40C.9090505@pacbell.net>
References: <mailman.465.1431559626.12865.python-list@python.org>
 <5553f8fe$0$13012$c3e8da3$5496439d@news.astraweb.com>
 <5554D40C.9090505@pacbell.net>
Message-ID: <20150514121759.73f98b76@bigbox.christie.dr>

On 2015-05-14 09:57, 20/20 Lab wrote:
> On 05/13/2015 06:23 PM, Steven D'Aprano wrote:
>>> I have a LARGE csv file that I need to process.  110+ columns,
>>> 72k rows.  I managed to write enough to reduce it to a few
>>> hundred rows, and the five columns I'm interested in.
> I actually stumbled across the csv module after coding enough to
> make a list of lists.  So that is more the reason I approached the
> list; Nothing like spending hours (or days) coding something that
> already exists and just dont know about.
>>> Now is were I have my problem:
>>>
>>> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
>>>              [72976, "YYY", "Item", "Qty", "Noise"],
>>>              [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
>>>
>>> Basically, I need to check for rows with duplicate accounts
>>> row[0] and staff (row[1]), and if so, remove that row, and add
>>> it's Qty to the original row. I really dont have a clue how to
>>> go about this.
>>
>> processed = {}  # hold the processed data in a dict
>>
>> for row in myList:
>>      account, staff = row[0:2]
>>      key = (account, staff)  # Put them in a tuple.
>>      if key in processed:
>>          # We've already seen this combination.
>>          processed[key][3] += row[3]  # Add the quantities.
>>      else:
>>          # Never seen this combination before.
>>          processed[key] = row
>>
>> newlist = list(processed.values())
>>
> It does, immensely.  I'll make this work.  Thank you again for the
> link from yesterday and apologies for hitting the wrong reply
> button.  I'll have to study more on the usage and implementations
> of dictionaries and tuples.

In processing the initial CSV file, I suspect that using a
csv.DictReader would make the code a bit cleaner.  Additionally,
as you're processing through the initial file, unless you need
the intermediate data, you should be able to do it in one pass.
Something like

  HEADER_ACCOUNT = "account"
  HEADER_STAFF = "staff"
  HEADER_QTY = "Qty"

  processed = {}
  with open("data.csv") as f:
    reader = csv.DictReader(f)
    for row in reader:
      if should_process_row(row):
        account = row[HEADER_ACCOUNT]
        staff = row[HEADER_STAFF]
        qty = row[HEADER_QTY]
        try:
          row[HEADER_QTY] = qty = int(qty)
        except Exception:
          # not a numeric quantity?
          continue
        # from Steven's code
        key = (account, staff)
        if key in processed:
          processed[key][HEADER_QTY] += qty
        else:
          processed[key][HEADER_QTY] = row
  so_something_with(processed.values())
          
I find that using names is a lot clearer than using arbitrary
indexing.  Barring that, using indexes-as-constants still would
add further clarity.

-tkc




.


From bc at freeuk.com  Thu May 14 13:32:17 2015
From: bc at freeuk.com (BartC)
Date: Thu, 14 May 2015 18:32:17 +0100
Subject: Building CPython
In-Reply-To: <mailman.8.1431623477.17265.python-list@python.org>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <mailman.4.1431619772.17265.python-list@python.org>
 <oy45x.458028$zE7.268573@fx40.am4>
 <mailman.8.1431623477.17265.python-list@python.org>
Message-ID: <B_45x.572799$LW6.82017@fx09.am4>

On 14/05/2015 18:11, Chris Angelico wrote:
> On Fri, May 15, 2015 at 3:02 AM, BartC <bc at freeuk.com> wrote:

>> I hope there's a quicker way of re-building an executable after a minor
>> source file change, otherwise doing any sort of development is going to be
>> impractical.)
>
> The whole point of 'make' is to rebuild only the parts that need to be
> rebuilt (either they've changed, or they depend on something that was
> changed). Sometimes practically everything needs to be rebuilt, if you
> do some really fundamental change, but generally not.
>
> The three parts to the build process are:
>
> 1) make - actually generate an executable. Takes ages the first time,
> will be a lot quicker if you haven't changed much.
> 2) make test - run the entire test suite. Takes just as long every
> time, but most of it won't have changed.
> 3) make install (needs root access, so probably 'sudo make install') -
> install this as your primary build of Python.
>
> When you start tinkering, I suggest just running make; rerunning the
> test suite isn't necessary till you're all done, and even then it's
> only important for making sure that your change hasn't broken anything
> anywhere else. Test your actual changes by simply running the
> freshly-built Python - most likely that'll be "./python".

OK, thanks. I didn't even know where the executable was put! Now I don't 
need 'make install', while 'make test' I won't bother with any more.

Making a small change and typing 'make' took 5 seconds, which is 
reasonable enough (although I had to use the copy of the source in 
Windows to find where the main.c file I needed was located).

Now Python 3.4.3 says "Bart's Python".


-- 
Bartc


From DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk  Thu May 14 13:32:19 2015
From: DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk (Dave Farrance)
Date: Thu, 14 May 2015 18:32:19 +0100
Subject: Survey -- Move To Trash function in Python?
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <lfm9la1dlhd2mnothbbbm7cphlc0mjsjtb@4ax.com>

Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:

>I'd like to do a little survey, and get a quick show of hands.
>
>How many people have written GUI or text-based applications or scripts where
>a "Move file to trash" function would be useful?
>
>Would you like to see that in the standard library, even if it meant that
>the library had feature-freeze and could gain no more functionality?

It's bad enough when things are filesystem-dependent but this is
OS-dependent or even desktop-version-dependent in the case of Linux
distros, so not easy.

E.g. in the case of KDE4, the command-line is:
$ kioclient move <filename> trash:/

...and for KDE3 it was:
$ kfmclient  move <filename> trash:/


From rosuav at gmail.com  Thu May 14 13:45:09 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 15 May 2015 03:45:09 +1000
Subject: Building CPython
In-Reply-To: <B_45x.572799$LW6.82017@fx09.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <mailman.4.1431619772.17265.python-list@python.org>
 <oy45x.458028$zE7.268573@fx40.am4>
 <mailman.8.1431623477.17265.python-list@python.org>
 <B_45x.572799$LW6.82017@fx09.am4>
Message-ID: <CAPTjJmpFiThb-_3jPy9YmbpFsCqzdeXZ5s3RgWqMW7EPdfAMpQ@mail.gmail.com>

On Fri, May 15, 2015 at 3:32 AM, BartC <bc at freeuk.com> wrote:
> OK, thanks. I didn't even know where the executable was put! Now I don't
> need 'make install', while 'make test' I won't bother with any more.
>
> Making a small change and typing 'make' took 5 seconds, which is reasonable
> enough (although I had to use the copy of the source in Windows to find
> where the main.c file I needed was located).
>
> Now Python 3.4.3 says "Bart's Python".

Haha. I don't usually bother rebranding my builds of things; though
that's partly because normally I'm making very few changes, and all in
the hope that they'll be accepted upstream anyway.

Incidentally, a quick 'make' can range anywhere from a fraction of a
second to quite a long time, depending mainly on the speed of your
hard drive and the performance of your disk cache. On my Linux, a
"null make" (ie when literally nothing has changed - just rerunning
make) takes about half a second, and that's dealing with a module
that's failing to build. When you rebuild lots of times all at once,
you'll pretty much be working in RAM the whole time, assuming you have
enough of it available.

ChrisA


From billy.earney at gmail.com  Thu May 14 14:06:09 2015
From: billy.earney at gmail.com (Billy Earney)
Date: Thu, 14 May 2015 13:06:09 -0500
Subject: a python pitfall
Message-ID: <CAB1ii-cFpbmdjQfRA_J3TBUZ+Gx8e8vLvqTLo2R-f0ygkykV1A@mail.gmail.com>

Hello friends:

I saw the following example at
http://nafiulis.me/potential-pythonic-pitfalls.html#using-mutable-default-arguments
and
did not believe the output produced and had to try it for myself....

def foo(a,b,c=[]):
    c.append(a)
    c.append(b)
    print(c)

foo(1,1)
foo(1,1)
foo(1,1)

produces:
[1, 1]
[1, 1, 1, 1]
[1, 1, 1, 1, 1, 1]

One would expect the following output:
[1, 1]
[1, 1]
[1, 1]

Doesn't this valid the zen of python: "Explicit is better than implicit."  ?

Thanks!
Billy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150514/4fad8403/attachment.html>

From invalid at invalid.invalid  Thu May 14 14:11:24 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 14 May 2015 18:11:24 +0000 (UTC)
Subject: Survey -- Move To Trash function in Python?
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <lfm9la1dlhd2mnothbbbm7cphlc0mjsjtb@4ax.com>
Message-ID: <mj2ogc$6j$1@reader1.panix.com>

On 2015-05-14, Dave Farrance <DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk> wrote:
> Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
>
>>I'd like to do a little survey, and get a quick show of hands.
>>
>>How many people have written GUI or text-based applications or scripts where
>>a "Move file to trash" function would be useful?
>>
>>Would you like to see that in the standard library, even if it meant that
>>the library had feature-freeze and could gain no more functionality?
>
> It's bad enough when things are filesystem-dependent but this is
> OS-dependent or even desktop-version-dependent in the case of Linux
> distros, so not easy.

Or even file-manager dependent.  I think some desktops support
multiple file-manager (at least XFCE always used to) -- and there's
probably no requirement that they all handle "trash" the same way. 

For this to work "right" you're going to need to be able to figure out
which desktop you're using: Linux systems often have multiple desktops
available.  Most of mine only have one, but some of them have as many
as three.

Then you might have to determine which file-manager you're using. 
Some of my desktops use file browsers that don't have any concept of
"trash".  And some of them don't have any file-browsers or "trash"
folders at all.

Sounds pretty nasty...

-- 
Grant Edwards               grant.b.edwards        Yow! MMM-MM!!  So THIS is
                                  at               BIO-NEBULATION!
                              gmail.com            


From ethan at stoneleaf.us  Thu May 14 14:29:19 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 14 May 2015 11:29:19 -0700
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5554E97F.9000402@stoneleaf.us>

On 05/14/2015 08:45 AM, Steven D'Aprano wrote:
> I'd like to do a little survey, and get a quick show of hands.
>
> How many people have written GUI or text-based applications or scripts where
> a "Move file to trash" function would be useful?

Never.

> Would you like to see that in the standard library, even if it meant that
> the library had feature-freeze and could gain no more functionality?

Nope.

--
~Ethan~



From kwpolska at gmail.com  Thu May 14 14:43:11 2015
From: kwpolska at gmail.com (Chris Warrick)
Date: Thu, 14 May 2015 20:43:11 +0200
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <mj2ogc$6j$1@reader1.panix.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <lfm9la1dlhd2mnothbbbm7cphlc0mjsjtb@4ax.com>
 <mj2ogc$6j$1@reader1.panix.com>
Message-ID: <CAMw+j7KE+bw_YSjRg4Fa9S24q66jOO=EC4ZCKmZ+Q=N_refYnQ@mail.gmail.com>

On Thu, May 14, 2015 at 8:11 PM, Grant Edwards <invalid at invalid.invalid> wrote:
> On 2015-05-14, Dave Farrance <DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk> wrote:
>> Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
>>
>>>I'd like to do a little survey, and get a quick show of hands.
>>>
>>>How many people have written GUI or text-based applications or scripts where
>>>a "Move file to trash" function would be useful?
>>>
>>>Would you like to see that in the standard library, even if it meant that
>>>the library had feature-freeze and could gain no more functionality?
>>
>> It's bad enough when things are filesystem-dependent but this is
>> OS-dependent or even desktop-version-dependent in the case of Linux
>> distros, so not easy.
>
> Or even file-manager dependent.  I think some desktops support
> multiple file-manager (at least XFCE always used to) -- and there's
> probably no requirement that they all handle "trash" the same way.

Actually, there is. There are actual STANDARDS in Linux desktops. One
of them is the Trash Specification:
http://standards.freedesktop.org/trash-spec/trashspec-1.0.html

This spec is implemented by Xfce, KDE, GNOME, PCManFM and probably many others.

And if you are looking for a mostly-compliant Python library/app (and
a shameless plug): https://pypi.python.org/pypi/trashman/1.5.0

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16


From dreamingforward at gmail.com  Thu May 14 14:48:05 2015
From: dreamingforward at gmail.com (zipher)
Date: Thu, 14 May 2015 11:48:05 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <5553fd0e$0$12995$c3e8da3$5496439d@news.astraweb.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
 <mailman.449.1431530832.12865.python-list@python.org>
 <71ee2dbb-27a3-45ad-b903-bffca6d8923f@googlegroups.com>
 <5553fd0e$0$12995$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <a81a81f9-a5aa-4760-aa8c-e910d05eae48@googlegroups.com>

> > No, Common LISP does, but as the website says Common LISP is a
> > "multi-paradigm" langauge.   It's trying to be everything to everybody,
> > just like Python tried to do in the other direction, making "everything an
> > object".  Python was trying to be too pure, while LISP was trying to be
> > too universal:  be everything to everyone -- you might say "batteries
> > included".
> > 
> > True LISP [...]
> 
> Ah, the "No True Scotsman" fallacy. You know that Lisp compiler you're
> using? That's not *true* Lisp, because *true* Lisp doesn't have [insert
> feature here].

While schools use the term "variable" and there are letters like x, y, z.  These should not be considered variables in any sense that you know.  They (generally) are not assigned to.  They are placeholders holding state.  They don't change once the program starts running.
 
> > Again, you're speaking of "Common LISP".  Such a lisp also wants to do
> > OOP, but it's like lipstick on a pig.
> 
> You really love that metaphor, don't you? You keep using it, and variations
> like "a pig made of lipstick". I suggest you don't understand the metaphor.

My apologies.  I thought better after I posted the message.  It's more like putting lipstick on a *boy*.  LISP, indeed, is not a pig.

> > Yes, and LISP is neither.  Although LISP is a functional style, that is
> > only by appearance.  It's completely different from Haskell, which I would
> > describe as a true functional language.  The difference is how the program
> > is lexed in the mind or on the machine.  But that's too difficult to
> > explain on this thread.
> 
> Just because something is difficult to explain doesn't make it correct. It's
> difficult to explain how the Atlantic Ocean merely looks, tastes and feels
> like salt water while actually being filled with strawberry yoghurt. That
> doesn't make it true.
> 
> Something which is difficult to explain might be deeply profound, but is far
> more likely to be rubbish.

That's all fine and good, Steven. But the academically and politically correct thing to do in that case, is withhold judgement.  Perhaps you don't know what to do with yourself if I'm right, so your remaining recourse is to defences rather than inquiry.

> > The "opposite" of iterative programming is recursive programming.
> 
> No it isn't. Iteration and recursion are, in a very deep sense, actually the
> same thing, they're certainly not opposites. That is a fact which genuinely
> is profound.

My God, you and Lawrence have gone off the deep end.  What you're telling me is that you've never programmed something on digital hardware.  That, fucking C, you've never even counted on your FINGERS!!

> >> > And the idea of lambdas were already encoded by the use of special
> >> > expressions, set-off by parenthesis.  So they practically *defined* the
> >> > concept of lambdas.
> >> 
> >> LISP is also the reason why we're cursed with the terrible name
> >> "lambda" for anonymous functions rather than something more mnemonic
> >> (like "function").
> > 
> > No, you haven't understood, padawan.  Lambda *is* the function, not it's
> > definition.  Perhaps you will understand what I mean by that, perhaps you
> > won't.  It's subtle.
> 
> Subtle like a kick to the head.

Well, at least you got the kick.  It's only be understanding your error, can you then proceed to a greater truth.  

More concretely, programming in an iterative language is like programming f(x, y, z...), where f is the idea your wanting to implement and x,y,z are inputs.  Since the are input by user interaction, they are VARIABLE.  A type of interactive program (not a Turing tape, either I might add).

A program in LISP is simply f, or more to their nomemclature: lambda.  It's like the difference of saying f(x) = x + 1, or saying f = /x.x+1 (where "/" is the Greek lambda character). 

Do you see the subtle difference?  They both appear to have variables, but they are not the same.

To repeat myself from two years ago:  Here endeth the lesson. 

You may be seated.

> If you wish to change our minds, you're going to have to demonstrate
> objective, verifiable facts and not just allude to how your ideas are too
> subtle and clever for us.

Hopefully that helps.  But if I thought my ideas were too subtle or clever for you I wouldn't keep responding to you. 

Mark


From tjreedy at udel.edu  Thu May 14 14:50:43 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 14 May 2015 14:50:43 -0400
Subject: Building CPython
In-Reply-To: <CAPTjJmq7amGAYn63C_w2uHp0AGNPxzQHVngQ5bnXrJrVxg5nXg@mail.gmail.com>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <mailman.4.1431619772.17265.python-list@python.org>
 <oy45x.458028$zE7.268573@fx40.am4>
 <CAPTjJmq7amGAYn63C_w2uHp0AGNPxzQHVngQ5bnXrJrVxg5nXg@mail.gmail.com>
Message-ID: <mj2qqd$2t3$1@ger.gmane.org>

On 5/14/2015 1:11 PM, Chris Angelico wrote:

> 2) make test - run the entire test suite. Takes just as long every
> time, but most of it won't have changed.

The test runner has an option, -jn, to run tests in n processes instead 
of just 1.  On my 6 core pentium, -j5 cuts time to almost exactly 1/5th 
of otherwise.  -j10 seems faster but have not times it.  I suspect that 
'make test' does not use -j option.


-- 
Terry Jan Reedy



From skip.montanaro at gmail.com  Thu May 14 14:55:23 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Thu, 14 May 2015 13:55:23 -0500
Subject: a python pitfall
In-Reply-To: <CAB1ii-cFpbmdjQfRA_J3TBUZ+Gx8e8vLvqTLo2R-f0ygkykV1A@mail.gmail.com>
References: <CAB1ii-cFpbmdjQfRA_J3TBUZ+Gx8e8vLvqTLo2R-f0ygkykV1A@mail.gmail.com>
Message-ID: <CANc-5UzBJ6BW7PTBayGKj7mGC==jCF=OOrHWivom6t3mNnbQvA@mail.gmail.com>

> I saw the following example at
> http://nafiulis.me/potential-pythonic-pitfalls.html#using-mutable-default-arguments
> and did not believe the output produced and had to try it for myself....

Pylint (and perhaps other Python "linters") would, I think, warn you
that you were using a mutable object as a default. It's more obvious
that the list is being reused if you call foo with different values
each time.

Skip


From ethan at stoneleaf.us  Thu May 14 15:32:59 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 14 May 2015 12:32:59 -0700
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <CAMw+j7KE+bw_YSjRg4Fa9S24q66jOO=EC4ZCKmZ+Q=N_refYnQ@mail.gmail.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <lfm9la1dlhd2mnothbbbm7cphlc0mjsjtb@4ax.com> <mj2ogc$6j$1@reader1.panix.com>
 <CAMw+j7KE+bw_YSjRg4Fa9S24q66jOO=EC4ZCKmZ+Q=N_refYnQ@mail.gmail.com>
Message-ID: <5554F86B.3040009@stoneleaf.us>

On 05/14/2015 11:43 AM, Chris Warrick wrote:

> And if you are looking for a mostly-compliant Python library/app (and
> a shameless plug): https://pypi.python.org/pypi/trashman/1.5.0

The docs listed link to Package Builder.  How is that related to TrashMan?

--
~Ethan~



From ian.g.kelly at gmail.com  Thu May 14 15:48:32 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 14 May 2015 13:48:32 -0600
Subject: a python pitfall
In-Reply-To: <CAB1ii-cFpbmdjQfRA_J3TBUZ+Gx8e8vLvqTLo2R-f0ygkykV1A@mail.gmail.com>
References: <CAB1ii-cFpbmdjQfRA_J3TBUZ+Gx8e8vLvqTLo2R-f0ygkykV1A@mail.gmail.com>
Message-ID: <CALwzidktkonq7cW0QJcnerakmEW9ySxz+ph+jwynFZdmcLWnng@mail.gmail.com>

On Thu, May 14, 2015 at 12:06 PM, Billy Earney <billy.earney at gmail.com> wrote:
> Hello friends:
>
> I saw the following example at
> http://nafiulis.me/potential-pythonic-pitfalls.html#using-mutable-default-arguments
> and did not believe the output produced and had to try it for myself....

See also https://docs.python.org/3/faq/programming.html#why-are-default-values-shared-between-objects


From breamoreboy at yahoo.co.uk  Thu May 14 16:12:35 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 14 May 2015 21:12:35 +0100
Subject: a python pitfall
In-Reply-To: <CAB1ii-cFpbmdjQfRA_J3TBUZ+Gx8e8vLvqTLo2R-f0ygkykV1A@mail.gmail.com>
References: <CAB1ii-cFpbmdjQfRA_J3TBUZ+Gx8e8vLvqTLo2R-f0ygkykV1A@mail.gmail.com>
Message-ID: <mj2vjl$loj$1@ger.gmane.org>

On 14/05/2015 19:06, Billy Earney wrote:
> Hello friends:
>
> I saw the following example at
> http://nafiulis.me/potential-pythonic-pitfalls.html#using-mutable-default-arguments

Thanks for this link, the title "Engineering Fantasy" could have been 
made for the troll who's just arrived back here after a couple of years 
being absent.

> and
> did not believe the output produced and had to try it for myself....
>
> def foo(a,b,c=[]):
>      c.append(a)
>      c.append(b)
>      print(c)
>
> foo(1,1)
> foo(1,1)
> foo(1,1)
>
> produces:
> [1, 1]
> [1, 1, 1, 1]
> [1, 1, 1, 1, 1, 1]
>
> One would expect the following output:
> [1, 1]
> [1, 1]
> [1, 1]
>

One wouldn't expect the above because one knows better :)  This comes up 
every few months, has probably come up every few months since Python was 
first let loose on the world, will probably keep coming up every few 
months, and will cease coming up when Python is no longer used as 
something better has taken its place or the planet no longer exists, 
whichever comes first.  The most recent I recall was just six days ago 
http://comments.gmane.org/gmane.comp.python.general/776290

> Doesn't this valid the zen of python: "Explicit is better than implicit."  ?
>
> Thanks!
> Billy
>

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From bc at freeuk.com  Thu May 14 17:55:42 2015
From: bc at freeuk.com (BartC)
Date: Thu, 14 May 2015 22:55:42 +0100
Subject: Building CPython
In-Reply-To: <874mnfunpn.fsf@elektro.pacujo.net>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net>
Message-ID: <zR85x.441429$Zf4.246168@fx22.am4>

On 14/05/2015 17:29, Marko Rauhamaa wrote:
> BartC <bc at freeuk.com>:
>
>> That's a shame because I wanted to tinker with the main dispatcher
>> loop to try and find out what exactly is making it slow. Nothing that
>> seems obvious at first sight.
>
> My guess is the main culprit is attribute lookup in two ways:
>
>   * Each object attribute reference involves a dictionary lookup.
>
>   * Each method call involves *two* dictionary lookups plus an object
>     creation.
>
> Tell us what you find out.

I'm just starting but I can tell you that it isn't because debug mode 
(Py_DEBUG defined) was left on by mistake!

What is interesting however is that on the very simple test I'm doing (a 
while loop incrementing a variable up to 100 million), the timings under 
Windows are:

Python 2.5       9.2 seconds
Python 3.1      13.1
Python 3.4.3    17.0
Python 3.4.3    14.3 (under Ubuntu on same machine, using the version
                      I built today)

That's quite a big range!

PyPy does it in 0.7 seconds. The same program within my own *simple* 
bytecode interpreter (not for Python) has a fastest time of 1.5 seconds 
but makes use of ASM. A version 100% in (gcc) C can manage 2 seconds.

So far I haven't find anything that explains the discrepancy (the 
languages are different, mine is simpler, but the Python code isn't 
doing anything that complicated, only LOAD_FASTs and such, and LOAD_FAST 
is apparently just an array access.

But the nearly 2:1 difference between new and old Python versions is 
also intriguing.

def whiletest():
	i=0
	while i<=100000000:
		i=i+1

whiletest()

-- 
Bartc


From python at mrabarnett.plus.com  Thu May 14 18:19:24 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 14 May 2015 23:19:24 +0100
Subject: Building CPython
In-Reply-To: <zR85x.441429$Zf4.246168@fx22.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net> <zR85x.441429$Zf4.246168@fx22.am4>
Message-ID: <55551F6C.60803@mrabarnett.plus.com>

On 2015-05-14 22:55, BartC wrote:
> On 14/05/2015 17:29, Marko Rauhamaa wrote:
>> BartC <bc at freeuk.com>:
>>
>>> That's a shame because I wanted to tinker with the main dispatcher
>>> loop to try and find out what exactly is making it slow. Nothing that
>>> seems obvious at first sight.
>>
>> My guess is the main culprit is attribute lookup in two ways:
>>
>>   * Each object attribute reference involves a dictionary lookup.
>>
>>   * Each method call involves *two* dictionary lookups plus an object
>>     creation.
>>
>> Tell us what you find out.
>
> I'm just starting but I can tell you that it isn't because debug mode
> (Py_DEBUG defined) was left on by mistake!
>
> What is interesting however is that on the very simple test I'm doing (a
> while loop incrementing a variable up to 100 million), the timings under
> Windows are:
>
> Python 2.5       9.2 seconds
> Python 3.1      13.1
> Python 3.4.3    17.0
> Python 3.4.3    14.3 (under Ubuntu on same machine, using the version
>                        I built today)
>
> That's quite a big range!
>
> PyPy does it in 0.7 seconds. The same program within my own *simple*
> bytecode interpreter (not for Python) has a fastest time of 1.5 seconds
> but makes use of ASM. A version 100% in (gcc) C can manage 2 seconds.
>
> So far I haven't find anything that explains the discrepancy (the
> languages are different, mine is simpler, but the Python code isn't
> doing anything that complicated, only LOAD_FASTs and such, and LOAD_FAST
> is apparently just an array access.
>
> But the nearly 2:1 difference between new and old Python versions is
> also intriguing.
>
> def whiletest():
> 	i=0
> 	while i<=100000000:
> 		i=i+1
>
> whiletest()
>
Python 2.x has int and long; Python 3 has int, which is the old 'long'.
Try Python 2 with longs.



From bc at freeuk.com  Thu May 14 20:50:35 2015
From: bc at freeuk.com (BartC)
Date: Fri, 15 May 2015 01:50:35 +0100
Subject: Building CPython
In-Reply-To: <zR85x.441429$Zf4.246168@fx22.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net> <zR85x.441429$Zf4.246168@fx22.am4>
Message-ID: <xpb5x.597872$X95.548374@fx10.am4>

On 14/05/2015 22:55, BartC wrote:

 > def whiletest():
 >      i=0
 >      while i<=100000000:
 >          i=i+1
 >
 > whiletest()
 >

> Python 2.5       9.2 seconds
> Python 3.1      13.1
> Python 3.4.3    17.0
> Python 3.4.3    14.3 (under Ubuntu on same machine, using the version
>                       I built today)
>
> That's quite a big range!
>
> PyPy does it in 0.7 seconds. The same program within my own *simple*
> bytecode interpreter (not for Python) has a fastest time of 1.5 seconds
> but makes use of ASM. A version 100% in (gcc) C can manage 2 seconds.

(Actually my ASM-aided code took 0.5 seconds (some crucial section had 
been commented out). Faster than PyPy and just using simple brute-force 
methods.)

> So far I haven't find anything that explains the discrepancy (the
> languages are different, mine is simpler, but the Python code isn't
> doing anything that complicated, only LOAD_FASTs and such, and LOAD_FAST
> is apparently just an array access.

It turns out the LOAD_FASTs were the simple byte-codes!

I'm just starting to find out just how much of a big complicated mess 
this project really is. I wouldn't be surprised if there aren't many 
people who actually understand it all, and that would explain why no-one 
seems to have had much luck in getting the speed up (if anything, it's 
getting slower).

I still have no idea yet exactly what an object comprises. I know that 
even an innocuous-looking LOAD_CONST actually loads a tuple from a table 
(complete with flag testing and bounds checks, although taking those out 
made no discernible difference).

It appears to be those "<=" and "+" operations in the code above where 
much of the time is spent. When I trace out the execution paths a bit 
more, I'll have a better picture of how many lines of C code are 
involved in each iteration.

-- 
Bartc


From steve+comp.lang.python at pearwood.info  Thu May 14 21:37:30 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 15 May 2015 11:37:30 +1000
Subject: Survey -- Move To Trash function in Python?
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <mj2g70$eij$1@reader1.panix.com>
Message-ID: <55554ddb$0$12995$c3e8da3$5496439d@news.astraweb.com>

On Fri, 15 May 2015 01:49 am, Grant Edwards wrote:

> On 2015-05-14, Steven D'Aprano <steve+comp.lang.python at pearwood.info>
> wrote:
> 
>> I'd like to do a little survey, and get a quick show of hands.
>>
>> How many people have written GUI or text-based applications or
>> scripts where a "Move file to trash" function would be useful?
> 
> How would you even define what "move to trash" means in a standard
> way?

There are standard locations for the trash on Linux, Windows and OS X.

I presume that there are standard locations on the other main OSes, such as
Android, iOS, any Unix which supports the Free Desktop Standard(?), etc.
Anything else can either remain unsupported, or can use a caller-defined
location.


> Even withing the limited context of something like a mail client, it's
> meaning varies depending on mail store format or which IMAP server
> you're talking to.
> 
> Or are you assuming the only thing that can be "moved to trash" is a
> file?

I'm specifically referring to files, hence "move FILE to trash".


-- 
Steven



From rosuav at gmail.com  Thu May 14 21:53:05 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 15 May 2015 11:53:05 +1000
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <55554ddb$0$12995$c3e8da3$5496439d@news.astraweb.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <mj2g70$eij$1@reader1.panix.com>
 <55554ddb$0$12995$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmokNo3e1=LEBBnC_6zQmtP=Smh7bXm-uCsGEew4sKZ52Q@mail.gmail.com>

On Fri, May 15, 2015 at 11:37 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Fri, 15 May 2015 01:49 am, Grant Edwards wrote:
>
>> On 2015-05-14, Steven D'Aprano <steve+comp.lang.python at pearwood.info>
>> wrote:
>>
>>> I'd like to do a little survey, and get a quick show of hands.
>>>
>>> How many people have written GUI or text-based applications or
>>> scripts where a "Move file to trash" function would be useful?
>>
>> How would you even define what "move to trash" means in a standard
>> way?
>
> There are standard locations for the trash on Linux, Windows and OS X.
>
> I presume that there are standard locations on the other main OSes, such as
> Android, iOS, any Unix which supports the Free Desktop Standard(?), etc.
> Anything else can either remain unsupported, or can use a caller-defined
> location.
>

Point to note: Inclusion in the Python standard library would cement
its *API*, but not its OS and desktop support. A call like
trash.retrieve("oops_i_deleted_this") will either succeed or raise an
error; it can change from raising to succeeding for a given platform
without breaking anyone's code. So it wouldn't be a problem to start
out supporting only the most major platforms (or not even all of
them), and expand later.

Much as I detest the operation itself, I do think this is something
worth papering over. The multiprocessing module papers over a whole
lot of platform quirks, and all we have left is "make sure your main
module is importable if you use this on Windows"; a trashing module
could deal with all the differing ways of trashing things, and leave
us with "trashing may not work on certain file systems". (Though when
it comes to the bikeshedding phase, I'm sure there'll be some who say
"if it can't be trashed, just hard delete it", and others who say "if
it can't be trashed, raise an exception". And neither is truly wrong.)

ChrisA


From steve+comp.lang.python at pearwood.info  Thu May 14 22:03:06 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 15 May 2015 12:03:06 +1000
Subject: a python pitfall
References: <mailman.11.1431626772.17265.python-list@python.org>
Message-ID: <555553db$0$12985$c3e8da3$5496439d@news.astraweb.com>

On Fri, 15 May 2015 04:06 am, Billy Earney wrote:

> Hello friends:
> 
> I saw the following example at
>
http://nafiulis.me/potential-pythonic-pitfalls.html#using-mutable-default-arguments
> and
> did not believe the output produced and had to try it for myself....
> 
> def foo(a,b,c=[]):
>     c.append(a)
>     c.append(b)
>     print(c)
> 
> foo(1,1)
> foo(1,1)
> foo(1,1)
> 
> produces:
> [1, 1]
> [1, 1, 1, 1]
> [1, 1, 1, 1, 1, 1]
> 
> One would expect the following output:
> [1, 1]
> [1, 1]
> [1, 1]

Really? Why would you expect that?

The *body* of the function is executed every time the function runs. The
*definition* of the function is executed once, and only once, when the
function is first defined. Setting the default value of a parameter is part
of the definition, not the body, and so it happens once only, not every
time. It truly would be surprising if the function recalculated the default
value every time.

Since the default value for c is a list, you are mutating the *same list*
each time.

This question comes up repeatedly. It's a FAQ, and the most recent time was
less than a week ago:

https://mail.python.org/pipermail/python-list/2015-May/703043.html

See, in particular, my response:

https://mail.python.org/pipermail/python-list/2015-May/703065.html


> Doesn't this valid the zen of python: "Explicit is better than implicit." 
> ?

I think you mean "invalidate".

No, this has nothing to do with explicitness or implicitness. Regardless of
whether Python uses "early binding" or "late binding" of the default
parameter, it is still equally explicit. The problem occurs because people
assume one binding model (which is entirely inconsistent with everything
else in Python) and get it wrong: people assume that function defaults are
late binding, but they are not.


-- 
Steven



From steve+comp.lang.python at pearwood.info  Thu May 14 22:03:19 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 15 May 2015 12:03:19 +1000
Subject: Survey -- Move To Trash function in Python?
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <lfm9la1dlhd2mnothbbbm7cphlc0mjsjtb@4ax.com>
Message-ID: <555553e8$0$12985$c3e8da3$5496439d@news.astraweb.com>

On Fri, 15 May 2015 03:32 am, Dave Farrance wrote:

> Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
> 
>>I'd like to do a little survey, and get a quick show of hands.
>>
>>How many people have written GUI or text-based applications or scripts
>>where a "Move file to trash" function would be useful?
>>
>>Would you like to see that in the standard library, even if it meant that
>>the library had feature-freeze and could gain no more functionality?
> 
> It's bad enough when things are filesystem-dependent but this is
> OS-dependent or even desktop-version-dependent in the case of Linux
> distros, so not easy.
> 
> E.g. in the case of KDE4, the command-line is:
> $ kioclient move <filename> trash:/
> 
> ...and for KDE3 it was:
> $ kfmclient  move <filename> trash:/


I'm not sure if you intend that as a vote in favour or against the idea :-)


The idea is that the library will hide that complexity from you, so your
python code will just say:

import shutil
shutil.move_to_trash(filename)


and it will work on any supported OS, filesystem and desktop.



-- 
Steven



From tjreedy at udel.edu  Thu May 14 22:27:00 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 14 May 2015 22:27:00 -0400
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <555553e8$0$12985$c3e8da3$5496439d@news.astraweb.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <lfm9la1dlhd2mnothbbbm7cphlc0mjsjtb@4ax.com>
 <555553e8$0$12985$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mj3lhu$6t2$1@ger.gmane.org>

On 5/14/2015 10:03 PM, Steven D'Aprano wrote:

> The idea is that the library will hide that complexity from you, so your
> python code will just say:
>
> import shutil
> shutil.move_to_trash(filename)

Since 'trash' is (or is used as) a verb, shutil.trash(filename)
seems sufficient.

> and it will work on any supported OS, filesystem and desktop.

This seems like a plausible enhancement.

-- 
Terry Jan Reedy



From steve+comp.lang.python at pearwood.info  Thu May 14 22:28:28 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 15 May 2015 12:28:28 +1000
Subject: Survey -- Move To Trash function in Python?
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <mj2g70$eij$1@reader1.panix.com>
 <mailman.3.1431619149.17265.python-list@python.org>
Message-ID: <555559cc$0$12986$c3e8da3$5496439d@news.astraweb.com>

On Fri, 15 May 2015 01:59 am, Chris Angelico wrote:

> On Fri, May 15, 2015 at 1:49 AM, Grant Edwards <invalid at invalid.invalid>
> wrote:
>> On 2015-05-14, Steven D'Aprano <steve+comp.lang.python at pearwood.info>
>> wrote:
>>
>>> I'd like to do a little survey, and get a quick show of hands.
>>>
>>> How many people have written GUI or text-based applications or
>>> scripts where a "Move file to trash" function would be useful?
[...]
> I have never used such a feature, and generally found them to be more
> annoying than useful. The times when I want to mark a file for
> deletion either now or at some undefined time in the future (at the
> file system's discretion) are approximately zero; either I want the
> file, or I can delete it immediately. Given that actually-removed
> files can usually be undeleted if their disk space hasn't been
> reclaimed, I don't think trash cans really help much.

If you're relying on the ability to undelete, you're already in trouble, you
just don't know it. As I understand it, you can't undelete from SSDs. Or
maybe it's the opposite, you can't securely delete from SSDs? I forget. In
any case, undelete is *unreliable* -- if you're relying on it for "Oh, I
changed my mind", then you're in trouble.

The primary purpose of trash cans is to protect users who are, um, let's
say, less than comfortable with computers, or careless, or both. But even
experienced users can make mistakes, or change my mind, and I find myself
recovering files from the trash, oh, once or twice a year.

I think every person who deletes files from the command line has
accidentally deleted the wrong file *more than once*. (And again, I hold my
hand up sheepishly here too: been there, done that.) If we were sensible,
we'd give ourselves a safety net just like the desktop guys, but there's a
machismo associated with command line use than makes us tough it out. No
matter how many times we delete the wrong file and have to recover it from
backups (or recreate it from scratch because the backups aren't working),
we tell ourselves it will be the very last time and it will never happen
again.

At least many Linux distros alias mv to mv -i by default, which is a start,
and use rm -i for root.

Anyway, this isn't meant to be a critical look at sys admin psychology :-)


> Plus, I think it's a UI insanity when a console program dumps
> something in the trash, which to me has always felt like a GUI file
> manager tool.

I'm not sure why you should feel that way. Every other file manager function
is more or less unified between GUI and CLIs. There's only one file
manager, and I think you would be annoyed if (say) renaming a file via a
CLI only showed the new name at the CLI, not from the GUI.

If I was writing a file manager tool, it shouldn't matter whether it used a
GUI or CLI, or a rich Curses text-based interface. I'd want to offer both
delete immediately and move to trash.


> That said, though, I think a standardized API for trashing files is
> not a bad thing; and it's not something that will need huge API
> changes, so the costs of feature-freeze would be low.

Indeed.



-- 
Steven



From ian.g.kelly at gmail.com  Thu May 14 22:49:36 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 14 May 2015 20:49:36 -0600
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <CAPTjJmokNo3e1=LEBBnC_6zQmtP=Smh7bXm-uCsGEew4sKZ52Q@mail.gmail.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <mj2g70$eij$1@reader1.panix.com>
 <55554ddb$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmokNo3e1=LEBBnC_6zQmtP=Smh7bXm-uCsGEew4sKZ52Q@mail.gmail.com>
Message-ID: <CALwzidkTXnbwcGPa5tK_fV_Rg1B7pX2rVK+0JPVdwLXu+CD7NQ@mail.gmail.com>

On May 14, 2015 7:55 PM, "Chris Angelico" <rosuav at gmail.com> wrote:
> (Though when
> it comes to the bikeshedding phase, I'm sure there'll be some who say
> "if it can't be trashed, just hard delete it", and others who say "if
> it can't be trashed, raise an exception". And neither is truly wrong.)

The answer is "raise an exception". Moving to trash and deleting are
different operations, and one shouldn't be substituted for the other any
more than a failed attempt to create a hard link should create a soft link
instead.

If the user wants, they can catch the exception and delete the file
instead. Recovering from an accidental deletion would be more difficult.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150514/dd5142fb/attachment.html>

From rosuav at gmail.com  Thu May 14 22:56:17 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 15 May 2015 12:56:17 +1000
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <CALwzidkTXnbwcGPa5tK_fV_Rg1B7pX2rVK+0JPVdwLXu+CD7NQ@mail.gmail.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <mj2g70$eij$1@reader1.panix.com>
 <55554ddb$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmokNo3e1=LEBBnC_6zQmtP=Smh7bXm-uCsGEew4sKZ52Q@mail.gmail.com>
 <CALwzidkTXnbwcGPa5tK_fV_Rg1B7pX2rVK+0JPVdwLXu+CD7NQ@mail.gmail.com>
Message-ID: <CAPTjJmqLjhbTYvHxwgTXWSNe-SKVpmKA9uh0PkeNCYZQ2osrXw@mail.gmail.com>

On Fri, May 15, 2015 at 12:49 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On May 14, 2015 7:55 PM, "Chris Angelico" <rosuav at gmail.com> wrote:
>> (Though when
>> it comes to the bikeshedding phase, I'm sure there'll be some who say
>> "if it can't be trashed, just hard delete it", and others who say "if
>> it can't be trashed, raise an exception". And neither is truly wrong.)
>
> The answer is "raise an exception". Moving to trash and deleting are
> different operations, and one shouldn't be substituted for the other any
> more than a failed attempt to create a hard link should create a soft link
> instead.
>
> If the user wants, they can catch the exception and delete the file instead.
> Recovering from an accidental deletion would be more difficult.

Yes, but sometimes it's at the file system's discretion - particularly
when you're working with network mounts. The application may not even
know that the file got hard deleted.

ChrisA


From rustompmody at gmail.com  Thu May 14 23:14:54 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Thu, 14 May 2015 20:14:54 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
Message-ID: <12b5b082-c04f-405f-baa1-ff5a53122b53@googlegroups.com>

On Wednesday, May 13, 2015 at 8:14:39 PM UTC+5:30, zipher wrote:
> On Tuesday, May 12, 2015 at 10:35:29 PM UTC-5, Rustom Mody wrote:
> > On Wednesday, May 13, 2015 at 8:00:50 AM UTC+5:30, Steven D'Aprano wrote:
> > > Why can't a language be designed with a *practical and concrete* need in
> > > mind? As far as I know, only one language designed from theoretical first
> > > principles has had any measure of mainstream success, Lisp, and that was
> > > probably because there weren't that many decent alternatives at the time.
> > 
> > How history U-turns!!
> > Lisp actually got every major/fundamental thing wrong
> > - variables scopes were dynamic by mistake
> > - lambdas were non-first class because the locution 'first-class' was still 8 
> > years in the future
> 
> I think you're confused.  LISP doesn't have variables.  It's a lambda calculus with an entirely different model computation than other programming languages which use variables all the time.  To the extent that it DOES have variables, it's to accomidate those coming over from iterative programming.
> 
> And the idea of lambdas were already encoded by the use of special expressions, set-off by parenthesis.  So they practically *defined* the concept of lambdas.  

See
https://groups.google.com/forum/#!msg/haskell-cafe/gDwF__-HMXE/oCKCbco2bS8J

| I asked McCarthy was the use of the LAMBDA notation in Lisp because the 
| language was functional, or was it just a convenient notation for anonymous 
| functions?  His answer was short and very definitive: he said it was a
| convenient notation --- *he didn't consider Lisp to be a functional language.*


From xiongziqi84 at gmail.com  Thu May 14 23:31:34 2015
From: xiongziqi84 at gmail.com (Ziqi Xiong)
Date: Fri, 15 May 2015 03:31:34 +0000
Subject: Looking for direction
In-Reply-To: <20150514121759.73f98b76@bigbox.christie.dr>
References: <mailman.465.1431559626.12865.python-list@python.org>
 <5553f8fe$0$13012$c3e8da3$5496439d@news.astraweb.com>
 <5554D40C.9090505@pacbell.net>
 <20150514121759.73f98b76@bigbox.christie.dr>
Message-ID: <CAFiuz6HL5HwP1HWdpvS_TMX0i4nKdV3ZG11Uxyuq7NpRAcUhDQ@mail.gmail.com>

maybe we can change this list to dict, using item[0] and item[1] as keys,
the whole item as value . then you can update by the same key i think
Tim Chase <python.list at tim.thechases.com>?2015?5?15? ??01:17???

> On 2015-05-14 09:57, 20/20 Lab wrote:
> > On 05/13/2015 06:23 PM, Steven D'Aprano wrote:
> >>> I have a LARGE csv file that I need to process.  110+ columns,
> >>> 72k rows.  I managed to write enough to reduce it to a few
> >>> hundred rows, and the five columns I'm interested in.
> > I actually stumbled across the csv module after coding enough to
> > make a list of lists.  So that is more the reason I approached the
> > list; Nothing like spending hours (or days) coding something that
> > already exists and just dont know about.
> >>> Now is were I have my problem:
> >>>
> >>> myList = [ [123, "XXX", "Item", "Qty", "Noise"],
> >>>              [72976, "YYY", "Item", "Qty", "Noise"],
> >>>              [123, "XXX" "ItemTypo", "Qty", "Noise"]    ]
> >>>
> >>> Basically, I need to check for rows with duplicate accounts
> >>> row[0] and staff (row[1]), and if so, remove that row, and add
> >>> it's Qty to the original row. I really dont have a clue how to
> >>> go about this.
> >>
> >> processed = {}  # hold the processed data in a dict
> >>
> >> for row in myList:
> >>      account, staff = row[0:2]
> >>      key = (account, staff)  # Put them in a tuple.
> >>      if key in processed:
> >>          # We've already seen this combination.
> >>          processed[key][3] += row[3]  # Add the quantities.
> >>      else:
> >>          # Never seen this combination before.
> >>          processed[key] = row
> >>
> >> newlist = list(processed.values())
> >>
> > It does, immensely.  I'll make this work.  Thank you again for the
> > link from yesterday and apologies for hitting the wrong reply
> > button.  I'll have to study more on the usage and implementations
> > of dictionaries and tuples.
>
> In processing the initial CSV file, I suspect that using a
> csv.DictReader would make the code a bit cleaner.  Additionally,
> as you're processing through the initial file, unless you need
> the intermediate data, you should be able to do it in one pass.
> Something like
>
>   HEADER_ACCOUNT = "account"
>   HEADER_STAFF = "staff"
>   HEADER_QTY = "Qty"
>
>   processed = {}
>   with open("data.csv") as f:
>     reader = csv.DictReader(f)
>     for row in reader:
>       if should_process_row(row):
>         account = row[HEADER_ACCOUNT]
>         staff = row[HEADER_STAFF]
>         qty = row[HEADER_QTY]
>         try:
>           row[HEADER_QTY] = qty = int(qty)
>         except Exception:
>           # not a numeric quantity?
>           continue
>         # from Steven's code
>         key = (account, staff)
>         if key in processed:
>           processed[key][HEADER_QTY] += qty
>         else:
>           processed[key][HEADER_QTY] = row
>   so_something_with(processed.values())
>
> I find that using names is a lot clearer than using arbitrary
> indexing.  Barring that, using indexes-as-constants still would
> add further clarity.
>
> -tkc
>
>
>
>
> .
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150515/aa6e677f/attachment.html>

From skybuck2000 at hotmail.com  Thu May 14 23:58:40 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Fri, 15 May 2015 05:58:40 +0200
Subject: Feature Request: Reposition Execution
In-Reply-To: <5553145b$0$11119$c3e8da3@news.astraweb.com>
References: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>
 <mailman.354.1431345441.12865.python-list@python.org>
 <4568f$5552602d$5419aafe$61337@news.ziggo.nl> <miuual$nek$1@dont-email.me>
 <5553145b$0$11119$c3e8da3@news.astraweb.com>
Message-ID: <79f34$55556ef0$5419aafe$39323@news.ziggo.nl>



"Steven D'Aprano"  wrote in message 
news:5553145b$0$11119$c3e8da3 at news.astraweb.com...

On Wednesday 13 May 2015 17:27, Christian Gollwitzer wrote:

> A clean way to exit your script could be to raise an exception. It
> should propagate to the toplevel and halt your script. However it is not
> possible to back and resume the execution.

"
while True:
    try:
        run_script()  # May raise TryAgain
        break
    except TryAgain:
        pass


If you prefer to only retry a finite number of times:


for i in range(10):
    try:
        run_script()  # May raise TryAgain
        break
    except TryAgain:
        pass
else:
    # break skips past the for...else block
    raise GiveUpError('too many failures')
"

Hi,

Thanks for the ideas, I haven't tried them yet.

I wonder if they will work in a multi-threaded fashion.

I doubt it.

The run_script runs in it's own thread.

The exception would have to be raise from another thread. (The other thread 
check for errors and needs to abort the run script).

I doubt thread B can interrupt thread A via exceptions/exception handling 
?!?

Bye,
  Skybuck. 



From zhangxiangsunny at gmail.com  Thu May 14 23:58:48 2015
From: zhangxiangsunny at gmail.com (Xiang Zhang)
Date: Thu, 14 May 2015 20:58:48 -0700 (PDT)
Subject: Python: How to find out values of all feasible x under constraints.
Message-ID: <12da2c70-fd4e-430b-83d2-fad4ac1e3f8f@googlegroups.com>

Dear all,

I am writing a code using Python now.

I want to know how to find out values of all feasible x under constraints.

x = [x_1, x_2, x_3,..., x_10]


constraints:
x_i = 0,1,2,3 or 4,          where i=1,2,....10
x_1 + x_2 + x_3 +...+x_10 <= 15

How to find out all the feasible solutions x (domain of x) using python, like [0,0,0,0,0,0,0,0,0,0], [1,1,1,1,1,1,1,1,1,1] etc ? What should be the code?

Any hint or help would be highly appreciated!

Sincerely,

Xiang Zhang


From steve+comp.lang.python at pearwood.info  Fri May 15 00:09:39 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 15 May 2015 14:09:39 +1000
Subject: Survey -- Move To Trash function in Python?
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <mj2g70$eij$1@reader1.panix.com>
 <55554ddb$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmokNo3e1=LEBBnC_6zQmtP=Smh7bXm-uCsGEew4sKZ52Q@mail.gmail.com>
 <CALwzidkTXnbwcGPa5tK_fV_Rg1B7pX2rVK+0JPVdwLXu+CD7NQ@mail.gmail.com>
 <mailman.23.1431658586.17265.python-list@python.org>
Message-ID: <55557185$0$12987$c3e8da3$5496439d@news.astraweb.com>

On Fri, 15 May 2015 12:56 pm, Chris Angelico wrote:

> On Fri, May 15, 2015 at 12:49 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> On May 14, 2015 7:55 PM, "Chris Angelico" <rosuav at gmail.com> wrote:
>>> (Though when
>>> it comes to the bikeshedding phase, I'm sure there'll be some who say
>>> "if it can't be trashed, just hard delete it", and others who say "if
>>> it can't be trashed, raise an exception". And neither is truly wrong.)
>>
>> The answer is "raise an exception". Moving to trash and deleting are
>> different operations, and one shouldn't be substituted for the other any
>> more than a failed attempt to create a hard link should create a soft
>> link instead.
>>
>> If the user wants, they can catch the exception and delete the file
>> instead. Recovering from an accidental deletion would be more difficult.
> 
> Yes, but sometimes it's at the file system's discretion - particularly
> when you're working with network mounts. The application may not even
> know that the file got hard deleted.

Citation needed.

"Move to trash" is a move operation, not an unlink. By definition, it is
reversible. Which platforms/file systems implements it as an unlink?

(I think the answer is, "broken ones".)

I've never heard of any file system implement a move to trash operation as a
primitive, it's always implemented at the application layer (desktop
management being an application for these purposes). As far as the file
system is concerned, it is just a file rename, or very occasionally if the
implementation is lousy, a copy followed by delete.



-- 
Steven



From rosuav at gmail.com  Fri May 15 00:25:45 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 15 May 2015 14:25:45 +1000
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <55557185$0$12987$c3e8da3$5496439d@news.astraweb.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <mj2g70$eij$1@reader1.panix.com>
 <55554ddb$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmokNo3e1=LEBBnC_6zQmtP=Smh7bXm-uCsGEew4sKZ52Q@mail.gmail.com>
 <CALwzidkTXnbwcGPa5tK_fV_Rg1B7pX2rVK+0JPVdwLXu+CD7NQ@mail.gmail.com>
 <mailman.23.1431658586.17265.python-list@python.org>
 <55557185$0$12987$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmpS2F1j1gWauSgN=bsF60X8jDZ1JQG8qpWMQyfmT3Ot2Q@mail.gmail.com>

On Fri, May 15, 2015 at 2:09 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
>> Yes, but sometimes it's at the file system's discretion - particularly
>> when you're working with network mounts. The application may not even
>> know that the file got hard deleted.
>
> Citation needed.
>
> "Move to trash" is a move operation, not an unlink. By definition, it is
> reversible. Which platforms/file systems implements it as an unlink?
>
> (I think the answer is, "broken ones".)

With individual files, you're right that it's broken systems; I do
recall having some setups with network mounts where the "trash"
request was sent over to a remote system and that system, for whatever
reason, chose to delete since it was unable to trash. However, it's
worth noting that a lot of trash cans have strict capacity
limitations, and trashing an entire directory may well result in most
of its content being hard-deleted (or, to be technically correct,
trashed and then purged before the user had a chance to do anything).
And to add insult to injury, some systems (which shall remain
nameless, despite being well known for using UTF-16 file names and
other such anomalies) have such appalling trash-management algorithms
that deleting an entire directory becomes a ridiculously slow
operation - I suspect there's some sort of O(n) linear search being
done for every trashed file, resulting in quadratic performance for
trashing a directory. And when the directory in question is a set of
files that can't usefully be used individually, I'd really rather tell
the application to just hard-delete the whole lot... but that
application didn't even give that option. Anyway.

The main thing is that trashing invites the system to delete the file
at its leisure, so it'll consume disk space and be undeletable for an
unspecified and highly uncertain period of time. If you're depending
on your trash can to protect you, you're just as vulnerable as someone
who depends on disk-level undeletion. There's no solution to that; it
comes down to one of the two hardest problems in computing (cache
management).

ChrisA


From greg.ewing at canterbury.ac.nz  Fri May 15 02:05:33 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Fri, 15 May 2015 18:05:33 +1200
Subject: Building CPython
In-Reply-To: <xpb5x.597872$X95.548374@fx10.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net> <zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4>
Message-ID: <crlglfFgf05U1@mid.individual.net>

BartC wrote:
> It appears to be those "<=" and "+" operations in the code above where 
> much of the time is spent. When I trace out the execution paths a bit 
> more, I'll have a better picture of how many lines of C code are 
> involved in each iteration.

The path from decoding a bytecode to the C code that
implements it can be rather convoluted, but there are
reasons for each of the complications -- mainly to
do with supporting the ability to override operators
with C and/or Python code.

If you removed those abilities, the implemention
would be simpler, and possibly faster. But then the
language wouldn't be Python any more.

-- 
Greg


From auriocus at gmx.de  Fri May 15 02:06:18 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Fri, 15 May 2015 08:06:18 +0200
Subject: Feature Request: Reposition Execution
In-Reply-To: <79f34$55556ef0$5419aafe$39323@news.ziggo.nl>
References: <2d482$5550967d$5419aafe$27910@news.ziggo.nl>
 <mailman.354.1431345441.12865.python-list@python.org>
 <4568f$5552602d$5419aafe$61337@news.ziggo.nl> <miuual$nek$1@dont-email.me>
 <5553145b$0$11119$c3e8da3@news.astraweb.com>
 <79f34$55556ef0$5419aafe$39323@news.ziggo.nl>
Message-ID: <mj42an$6v9$1@dont-email.me>

Am 15.05.15 um 05:58 schrieb Skybuck Flying:
> Thanks for the ideas, I haven't tried them yet.
>
> I wonder if they will work in a multi-threaded fashion.
>
> I doubt it.
>
> The run_script runs in it's own thread.

It would be of enormous help if you would create a minimal script just 
like the above for your situation. What does it mean, runs in a thread? 
How do you achieve that from Jython?

> The exception would have to be raise from another thread. (The other
> thread check for errors and needs to abort the run script).

Well usually you can pass messages between the threads, wait for 
termination etc. I've got no experience with threading in Jython; 
however this site
http://www.jython.org/jythonbook/en/1.0/Concurrency.html
suggests that there is a large number of primitives available to do that.

Concerning "Resume", what do you expect: A) starting afresh from the 
beginning or B) continue the execution at the point where the exception 
was thrown? Because A) is/should be  easy, B) could be impossible

	Christian



From lac at openend.se  Fri May 15 03:03:13 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 15 May 2015 09:03:13 +0200
Subject: new python.org mailing list, elections-wg
Message-ID: <201505150703.t4F73DQk015513@fido.openend.se>

It is for discussing voting software (currently Helios and Evote)
with the end result that the new PSF election Commissioner Ian
Cordasco will pick one and use it for the next PSF e?lection.
We're hoping to turn it into a real PSF workgroup.  It looks like
it is going to be a fairly nerdy place.  All are welcome, this
is a public list.  Interested parties please join at:
https://mail.python.org/mailman/listinfo/elections-wg

Thank you.
Laura


From __peter__ at web.de  Fri May 15 03:47:28 2015
From: __peter__ at web.de (Peter Otten)
Date: Fri, 15 May 2015 09:47:28 +0200
Subject: Python: How to find out values of all feasible x under
 constraints.
References: <12da2c70-fd4e-430b-83d2-fad4ac1e3f8f@googlegroups.com>
Message-ID: <mj48ah$nn0$1@ger.gmane.org>

Xiang Zhang wrote:

> I want to know how to find out values of all feasible x under constraints.
> 
> x = [x_1, x_2, x_3,..., x_10]
> 
> 
> constraints:
> x_i = 0,1,2,3 or 4,          where i=1,2,....10
> x_1 + x_2 + x_3 +...+x_10 <= 15

That are 5**10 == 9765625 candidates. That's still feasible to check using 
brute force.

> How to find out all the feasible solutions x (domain of x) using python,
> like [0,0,0,0,0,0,0,0,0,0], [1,1,1,1,1,1,1,1,1,1] etc ? What should be the
> code?
> 
> Any hint or help would be highly appreciated!

>>> solutions = [x for x in itertools.product(range(5), repeat=10) if sum(x) 
<= 15]
>>> len(solutions)
1556215

The first ten solutions:

>>> solutions[:10]
[(0, 0, 0, 0, 0, 0, 0, 0, 0, 0), (0, 0, 0, 0, 0, 0, 0, 0, 0, 1), (0, 0, 0, 
0, 0, 0, 0, 0, 0, 2), (0, 0, 0, 0, 0, 0, 0, 0, 0, 3), (0, 0, 0, 0, 0, 0, 0, 
0, 0, 4), (0, 0, 0, 0, 0, 0, 0, 0, 1, 0), (0, 0, 0, 0, 0, 0, 0, 0, 1, 1), 
(0, 0, 0, 0, 0, 0, 0, 0, 1, 2), (0, 0, 0, 0, 0, 0, 0, 0, 1, 3), (0, 0, 0, 0, 
0, 0, 0, 0, 1, 4)]




From marko at pacujo.net  Fri May 15 04:59:30 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 15 May 2015 11:59:30 +0300
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net>
Message-ID: <87bnhmgqrx.fsf@elektro.pacujo.net>

Gregory Ewing <greg.ewing at canterbury.ac.nz>:

> BartC wrote:
>> It appears to be those "<=" and "+" operations in the code above
>> where much of the time is spent. When I trace out the execution paths
>> a bit more, I'll have a better picture of how many lines of C code
>> are involved in each iteration.
>
> The path from decoding a bytecode to the C code that implements it can
> be rather convoluted, but there are reasons for each of the
> complications -- mainly to do with supporting the ability to override
> operators with C and/or Python code.
>
> If you removed those abilities, the implemention would be simpler, and
> possibly faster. But then the language wouldn't be Python any more.

I agree that Python's raison-d'?tre is its dynamism and expressive
power. It definitely shouldn't be sacrificed for performance.

However, in some respects, Python might be going overboard with its
dynamism; are all those dunder methods really needed? Must "false" be
defined so broadly? Must a method lookup necessarily involve object
creation?


Marko


From marko at pacujo.net  Fri May 15 05:20:14 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 15 May 2015 12:20:14 +0300
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <c8bcda07-6c80-4cae-b1b0-2e434135fa42@googlegroups.com>
Message-ID: <877fsagptd.fsf@elektro.pacujo.net>

wxjmfauth at gmail.com:

> Implement unicode correctly.

Did they reject your patch?


Marko


From bc at freeuk.com  Fri May 15 05:32:39 2015
From: bc at freeuk.com (BartC)
Date: Fri, 15 May 2015 10:32:39 +0100
Subject: Building CPython
In-Reply-To: <crlglfFgf05U1@mid.individual.net>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net> <zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4> <crlglfFgf05U1@mid.individual.net>
Message-ID: <Z2j5x.640864$ms.43141@fx30.am4>

On 15/05/2015 07:05, Gregory Ewing wrote:
> BartC wrote:
>> It appears to be those "<=" and "+" operations in the code above where
>> much of the time is spent. When I trace out the execution paths a bit
>> more, I'll have a better picture of how many lines of C code are
>> involved in each iteration.
>
> The path from decoding a bytecode to the C code that
> implements it can be rather convoluted, but there are
> reasons for each of the complications -- mainly to
> do with supporting the ability to override operators
> with C and/or Python code.
>
> If you removed those abilities, the implemention
> would be simpler, and possibly faster. But then the
> language wouldn't be Python any more.

That's the challenge; programs must still work as they did before. (But 
I suppose it can be exasperating for CPython developers if 99% of 
programs could be made faster but can't be because of the 1% that rely 
on certain features).

Still, I'm just seeing what there is in CPython that limits the 
performance, and whether anything can be done /easily/ without going to 
solutions such as PyPy which are unsatisfactory IMO (by being even more 
fantastically complicated, but they don't always give a speed-up either).

For example, there is a /specific/ byte-code called BINARY_ADD, which 
then proceeds to call a /generic/ binary-op handler! This throws away 
the advantage of knowing at byte-code generation time exactly which 
operation is needed.

(Unless I'm just looking at a bunch of macros or maybe there is some 
inlining going on with the compiler reducing all those table-indexing 
operations for 'Add', with direct accesses, but it didn't look like it. 
I'm just glad it doesn't use C++ which would have made it truly 
impossible to figure out what's going on.)

(BTW since I'm having to use Linux to compile this anyway, is there a 
tool available that will tell me whether something in the C sources is a 
function or macro? (And preferably where the definition might be located.))

-- 
Bartc


From rosuav at gmail.com  Fri May 15 05:43:57 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 15 May 2015 19:43:57 +1000
Subject: Building CPython
In-Reply-To: <87bnhmgqrx.fsf@elektro.pacujo.net>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net>
 <87bnhmgqrx.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmrkoQY60HfyMRkf-wL6GKmCjW7QRm9AyT68ouUOFP5mEg@mail.gmail.com>

On Fri, May 15, 2015 at 6:59 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> However, in some respects, Python might be going overboard with its
> dynamism; are all those dunder methods really needed?

Yes - at least, most of them. As regards operators, there are three
options: either you have magic methods for all of them (Python style),
or none of them (Java style, no operator overloading), or you
hybridize and permit just a handful of them (and then you have to
decide which). There's really no reason not to have them. The other
dunder methods are a mixed bag; some are to allow you to customize
object creation itself (a class's __new__ method could be considered
equivalent to an instance-specific __call__ method on the type
object), some let you pretend to be different types of number
(__int__, __index__, __complex__), which allows you to duck-type
integerness rather than having to subclass int and rely on magic; and
others let you customize the behaviour of well-known functions, such
as __len__ for len() and __repr__ for repr(). Without dunder methods
for all of these, it would be difficult to make an object "play
nicely" with the overall Python ecosystem. There are others, though,
which are less crucial (__getstate__ and so on for pickle), but you
can ignore them if you're not using those features.

> Must "false" be defined so broadly?

Different languages define true and false differently. REXX says that
"1" is true and "0" is false, and anything else is an error. Pike says
that the integer 0 is false and anything else is true; the philosophy
is that a "thing" is true and the absence of any thing is false.
Python says that an empty "thing" is false and a non-empty "thing" is
true; if next(iter(x)) raises StopIteration, x is probably false, and
vice versa. All three have their merits, all three have their
consequences. One consequence of Python's model is that a __bool__
method is needed on any object that might be empty (unless it defines
__len__, in which case that makes a fine fall-back); it's normal in
Python code to distinguish between "if x:" and "if x is not None:",
where the former sees if x has anything in it, but the latter sees if
there x even exists. (More or less.)

> Must a method lookup necessarily involve object creation?

Actually, no. Conceptually, this method call:

foo.bar(1, 2, 3)

involves looking up 'foo' in the current namespace, looking up
attribute 'bar' on it, treating the result as a function, and calling
it with three integer objects as its arguments. And the language
definition demands that this work even if the "foo.bar" part is broken
out:

def return_function():
    return foo.bar

def call_function():
    return_function()(1, 2, 3)

But a particular Python implementation is most welcome to notice the
extremely common situation of method calls and optimize it. I'm not
sure if PyPy does this, but I do remember reading about at least one
Python that does; CPython has an optimization for the actual memory
allocations involved, though I think it does actually construct some
sort of object for each one; as long as the resulting behaviour is
within spec, objects needn't be created just to be destroyed.

Dynamism doesn't have to be implemented naively, just as long as the
slow path is there if anyone needs it.

ChrisA


From steve+comp.lang.python at pearwood.info  Fri May 15 06:14:31 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 15 May 2015 20:14:31 +1000
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
Message-ID: <5555c708$0$13009$c3e8da3$5496439d@news.astraweb.com>

On Fri, 15 May 2015 06:59 pm, Marko Rauhamaa wrote:

> However, in some respects, Python might be going overboard with its
> dynamism; are all those dunder methods really needed? Must "false" be
> defined so broadly? Must a method lookup necessarily involve object
> creation?

Yes, what do you mean, and no.

(1) Yes, the dunder methods are necessary to support operator overloading
and various other protocols. The existence of dunder methods doesn't have
any runtime costs except for that related to memory usage. Fortunately, the
dunder methods only exist on the class itself, not each and every instance.

(2) What do you mean by false being defined so broadly?

The singleton instance False is not defined broadly at all. It is a built-in
constant, and there's only one of it. In Python 3, "False" is even a
keyword, so you cannot redefine the name. (It's not a keyword in Python 2
because of historical reasons.)

Perhaps you are talking about falsey (false-like) instances, e.g. 

    if []: ...
    else: ...

will run the else block. That's quite broad, in a sense, but it has no real
runtime cost over and above a more Pascal-ish language would force you to
have:

    if len([]) == 0: ...
    else: ...

Because True and False are singletons, the overhead of testing something in
a boolean context is no greater than the cost of testing the same condition
by hand. That is, it makes no difference whether you manually compare the
list's length to zero, or let its __nonzero__ or __bool__ method do the
same. (If anything, using an implicit boolean test will be faster than an
explicit manual test, because it doesn't have to call the len() global.)

(3) Method lookups don't *necessarily* have to involve object creation. The
only semantics which Python requires (so far as I understand it) are:

- Taking a reference to an unbound method:

    ref = str.upper

  may return the function object itself, which clearly already exists.

- Taking a reference to a bound method:

    ref = "some string".upper

  must return a method object, but it doesn't have to be recreated from
  scratch each and every time. It could cache it once, then always return
  that. That, I believe, is an implementation detail.

- Calling a method may bypass creating a method, and just use the 
  function object directly, provided Python knows that the descriptor
  has no side-effects.

For example, since FunctionType.__get__ has no side-effects, a Python
interpreter could special-case function objects and avoid calling the
descriptor protocol when it knows that the method is just going to be
called immediately.


But... having said all that... how do you know that these issues are
bottlenecks that eliminating them would speed up the code by any
significant amount?




-- 
Steven



From rosuav at gmail.com  Fri May 15 06:25:22 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 15 May 2015 20:25:22 +1000
Subject: Building CPython
In-Reply-To: <5555c708$0$13009$c3e8da3$5496439d@news.astraweb.com>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net>
 <87bnhmgqrx.fsf@elektro.pacujo.net>
 <5555c708$0$13009$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmp8PGqPh7pL550niXv3izgQdyYztszN-0K_6+Cx5s_-MA@mail.gmail.com>

On Fri, May 15, 2015 at 8:14 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> (If anything, using an implicit boolean test will be faster than an
> explicit manual test, because it doesn't have to call the len() global.)

Even more so: Some objects may be capable of determining their own
lengths, but can ascertain their own emptiness more quickly. So len(x)
might have to chug chug chug to figure out exactly how many results
there are (imagine a database query or something), where bool(x)
merely has to see whether or not a single one exists (imagine a
database query with "LIMIT 1" tacked on).

ChrisA


From Cecil at decebal.nl  Fri May 15 06:29:35 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 15 May 2015 12:29:35 +0200
Subject: Trying to build Python
Message-ID: <87a8x65e28.fsf@Equus.decebal.nl>

I am trying to build Python 3 on a Debian system. It is successful,
but a few things where missing. Partly I solved it, but a few things
keep unresolved:
    _bz2
    _sqlite3
    readline
    _dbm
    _ssl
    _gdbm

What do I need to do to get those resolved also?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From marko at pacujo.net  Fri May 15 06:50:29 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 15 May 2015 13:50:29 +0300
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <mailman.31.1431683047.17265.python-list@python.org>
Message-ID: <87382yglmy.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> On Fri, May 15, 2015 at 6:59 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
>> Must a method lookup necessarily involve object creation?
>
> Actually, no.
> [...]
> a particular Python implementation is most welcome to notice the
> extremely common situation of method calls and optimize it.

I'm not sure that is feasible given the way it has been specified. You'd
have to prove the class attribute lookup produces the same outcome in
consecutive method references.

Also:

   >>> class X:
   ...   def f(self): pass
   ... 
   >>> x = X()
   >>> f = x.f
   >>> ff = x.f
   >>> f is ff
   False

Would a compliant Python implementation be allowed to respond "True"?

Maybe. At least method objects seem immutable:

   >>> f.__name__
   'f'
   >>> f.__name__ = 'g'
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   AttributeError: 'method' object has no attribute '__name__'
   >>> f.__eq__ = None
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   AttributeError: 'method' object attribute '__eq__' is read-only
   >>> f.xyz = 'xyz'
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   AttributeError: 'method' object has no attribute 'xyz'


Marko


From auriocus at gmx.de  Fri May 15 06:51:16 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Fri, 15 May 2015 12:51:16 +0200
Subject: Building CPython
In-Reply-To: <mailman.14.1431629476.17265.python-list@python.org>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <mailman.4.1431619772.17265.python-list@python.org>
 <oy45x.458028$zE7.268573@fx40.am4>
 <CAPTjJmq7amGAYn63C_w2uHp0AGNPxzQHVngQ5bnXrJrVxg5nXg@mail.gmail.com>
 <mailman.14.1431629476.17265.python-list@python.org>
Message-ID: <mj4j11$rc5$1@dont-email.me>

Am 14.05.15 um 20:50 schrieb Terry Reedy:
> On 5/14/2015 1:11 PM, Chris Angelico wrote:
>
>> 2) make test - run the entire test suite. Takes just as long every
>> time, but most of it won't have changed.
>
> The test runner has an option, -jn, to run tests in n processes instead
> of just 1.  On my 6 core pentium, -j5 cuts time to almost exactly 1/5th
> of otherwise.  -j10 seems faster but have not times it.  I suspect that
> 'make test' does not use -j option.
>

Just to clarify, -j is an option of GNU make to run the Makefile in 
parallel. Unless the Makefile is buggy, this should result in the same 
output. You can also set an environment variable to enable this 
permanently (until you log out) like

export MAKEFLAGS=-j5

Put this into your .bashrc or .profile, and it'll become permanent.

	Christian


From marko at pacujo.net  Fri May 15 06:52:27 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 15 May 2015 13:52:27 +0300
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <c8bcda07-6c80-4cae-b1b0-2e434135fa42@googlegroups.com>
 <877fsagptd.fsf@elektro.pacujo.net>
 <5f588ae2-4f89-477a-a9ad-ac7913cf7459@googlegroups.com>
Message-ID: <87y4kqf6z8.fsf@elektro.pacujo.net>

wxjmfauth at gmail.com:

> Le vendredi 15 mai 2015 11:20:25 UTC+2, Marko Rauhamaa a ?crit?:
>> wxjmfauth at gmail.com:
>> 
>> > Implement unicode correctly.
>> Did they reject your patch?
>
> You can not patch something that is wrong by design.

Are you saying the Python language spec is unfixable or that the CPython
implementation is unfixable?

If CPython is unfixable, you can develop a better Python implementation.

If Python itself is unfixable, what brings you here?


Marko


From rustompmody at gmail.com  Fri May 15 07:02:30 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Fri, 15 May 2015 04:02:30 -0700 (PDT)
Subject: Trying to build Python
In-Reply-To: <87a8x65e28.fsf@Equus.decebal.nl>
References: <87a8x65e28.fsf@Equus.decebal.nl>
Message-ID: <a8f18c31-4da2-473b-85a1-309b0ac60ea0@googlegroups.com>

On Friday, May 15, 2015 at 4:28:13 PM UTC+5:30, Cecil Westerhof wrote:
> I am trying to build Python 3 on a Debian system. It is successful,
> but a few things where missing. Partly I solved it, but a few things
> keep unresolved:
>     _bz2
>     _sqlite3
>     readline
>     _dbm
>     _ssl
>     _gdbm
> 
> What do I need to do to get those resolved also?
> 

$ aptitude build-dep python3

should fetch the requirements of python3


From rosuav at gmail.com  Fri May 15 07:16:45 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 15 May 2015 21:16:45 +1000
Subject: Trying to build Python
In-Reply-To: <a8f18c31-4da2-473b-85a1-309b0ac60ea0@googlegroups.com>
References: <87a8x65e28.fsf@Equus.decebal.nl>
 <a8f18c31-4da2-473b-85a1-309b0ac60ea0@googlegroups.com>
Message-ID: <CAPTjJmrkOtVgewxZV+RszcJQVkBR9mxSkpf6_0Hx7=sFoNWYmw@mail.gmail.com>

On Fri, May 15, 2015 at 9:02 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> On Friday, May 15, 2015 at 4:28:13 PM UTC+5:30, Cecil Westerhof wrote:
>> I am trying to build Python 3 on a Debian system. It is successful,
>> but a few things where missing. Partly I solved it, but a few things
>> keep unresolved:
>>     _bz2
>>     _sqlite3
>>     readline
>>     _dbm
>>     _ssl
>>     _gdbm
>>
>> What do I need to do to get those resolved also?
>>
>
> $ aptitude build-dep python3
>
> should fetch the requirements of python3

And if not, try grabbing some development libraries:

$ sudo apt-get install libbz2-dev libsqlite3-dev libreadline-dev
libssl-dev libgdbm-dev

I'm not sure about the _dbm module, not sure what it needs.

ChrisA


From steve+comp.lang.python at pearwood.info  Fri May 15 08:10:15 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 15 May 2015 22:10:15 +1000
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <c8bcda07-6c80-4cae-b1b0-2e434135fa42@googlegroups.com>
 <877fsagptd.fsf@elektro.pacujo.net>
 <5f588ae2-4f89-477a-a9ad-ac7913cf7459@googlegroups.com>
 <87y4kqf6z8.fsf@elektro.pacujo.net>
Message-ID: <5555e228$0$13002$c3e8da3$5496439d@news.astraweb.com>

On Fri, 15 May 2015 08:52 pm, Marko Rauhamaa wrote:

> wxjmfauth at gmail.com:
> 
>> Le vendredi 15 mai 2015 11:20:25 UTC+2, Marko Rauhamaa a ?crit?:
>>> wxjmfauth at gmail.com:
>>> 
>>> > Implement unicode correctly.
>>> Did they reject your patch?
>>
>> You can not patch something that is wrong by design.
> 
> Are you saying the Python language spec is unfixable or that the CPython
> implementation is unfixable?

JMF is obsessed with a trivial and artificial performance regression in the
handling of Unicode strings since Python 3.3, which introduced a
significant memory optimization for Unicode strings. Each individual string
uses a code unit no larger than necessary, thus if a string contains
nothing but ASCII or Latin 1 characters, it will use one byte per
character; if it fits into the Basic Multilingual Plane, two bytes per
character; and only use four bytes per character if there are "astral"
characters in the string.

(That is, Python strings select from a Latin-1, UCS-2 and UTF-32 encoded
form at creation time, according to the largest code point in the string.)

The benefit of this is that most strings will use 1/2 or 1/4 of the memory
that they otherwise would need, which gives an impressive memory saving.
That leads to demonstrable speed-ups in real-world code, however it is
possible to find artificial benchmarks that experience a slowdown compared
to Python 3.2.

JMF found one such artificial benchmark, involving creating and throwing
away many strings as fast as possible without doing any work with them, and
from this has built this fantasy in his head that Python is not compliant
with the Unicode spec and is logically, mathematically broken.


-- 
Steven



From rosuav at gmail.com  Fri May 15 08:34:36 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 15 May 2015 22:34:36 +1000
Subject: Building CPython
In-Reply-To: <5555e228$0$13002$c3e8da3$5496439d@news.astraweb.com>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net>
 <87bnhmgqrx.fsf@elektro.pacujo.net>
 <c8bcda07-6c80-4cae-b1b0-2e434135fa42@googlegroups.com>
 <877fsagptd.fsf@elektro.pacujo.net>
 <5f588ae2-4f89-477a-a9ad-ac7913cf7459@googlegroups.com>
 <87y4kqf6z8.fsf@elektro.pacujo.net>
 <5555e228$0$13002$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmr=eroNMgTy9ByeEbnfWhzT+S_4jAhZ5rUmG95HPvOEcg@mail.gmail.com>

On Fri, May 15, 2015 at 10:10 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> The benefit of this is that most strings will use 1/2 or 1/4 of the memory
> that they otherwise would need, which gives an impressive memory saving.
> That leads to demonstrable speed-ups in real-world code, however it is
> possible to find artificial benchmarks that experience a slowdown compared
> to Python 3.2.

It's also possible to find a number of situations in which a narrow
build of 3.2 was faster than 3.3, due to the buggy handling of
surrogates. I've no idea whether jmf is still complaining against that
basis or not, as I don't see his posts.

ChrisA


From breamoreboy at yahoo.co.uk  Fri May 15 08:35:18 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 15 May 2015 13:35:18 +0100
Subject: Python: How to find out values of all feasible x under
 constraints.
In-Reply-To: <12da2c70-fd4e-430b-83d2-fad4ac1e3f8f@googlegroups.com>
References: <12da2c70-fd4e-430b-83d2-fad4ac1e3f8f@googlegroups.com>
Message-ID: <mj4p69$je7$1@ger.gmane.org>

On 15/05/2015 04:58, Xiang Zhang wrote:
> Dear all,
>
> I am writing a code using Python now.
>
> I want to know how to find out values of all feasible x under constraints.
>
> x = [x_1, x_2, x_3,..., x_10]
>
>
> constraints:
> x_i = 0,1,2,3 or 4,          where i=1,2,....10
> x_1 + x_2 + x_3 +...+x_10 <= 15
>
> How to find out all the feasible solutions x (domain of x) using python, like [0,0,0,0,0,0,0,0,0,0], [1,1,1,1,1,1,1,1,1,1] etc ? What should be the code?
>
> Any hint or help would be highly appreciated!
>
> Sincerely,
>
> Xiang Zhang
>

There are several constraint libraries on pypi if you don't want to roll 
your own.  See for example 
https://pypi.python.org/pypi/python-constraint/1.2

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Fri May 15 08:38:47 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 15 May 2015 13:38:47 +0100
Subject: Building CPython
In-Reply-To: <877fsagptd.fsf@elektro.pacujo.net>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <c8bcda07-6c80-4cae-b1b0-2e434135fa42@googlegroups.com>
 <877fsagptd.fsf@elektro.pacujo.net>
Message-ID: <mj4pcq$nfk$1@ger.gmane.org>

On 15/05/2015 10:20, Marko Rauhamaa wrote:
> wxjmfauth at gmail.com:
>
>> Implement unicode correctly.
>
> Did they reject your patch?
>
> Marko
>

Please don't feed him, it's been obvious for years that he hasn't the 
faintest idea what he's talking about.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Fri May 15 08:41:12 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 15 May 2015 13:41:12 +0100
Subject: Building CPython
In-Reply-To: <87y4kqf6z8.fsf@elektro.pacujo.net>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <c8bcda07-6c80-4cae-b1b0-2e434135fa42@googlegroups.com>
 <877fsagptd.fsf@elektro.pacujo.net>
 <5f588ae2-4f89-477a-a9ad-ac7913cf7459@googlegroups.com>
 <87y4kqf6z8.fsf@elektro.pacujo.net>
Message-ID: <mj4phb$nfk$2@ger.gmane.org>

On 15/05/2015 11:52, Marko Rauhamaa wrote:
> wxjmfauth at gmail.com:
>
>> Le vendredi 15 mai 2015 11:20:25 UTC+2, Marko Rauhamaa a ?crit :
>>> wxjmfauth at gmail.com:
>>>
>>>> Implement unicode correctly.
>>> Did they reject your patch?
>>
>> You can not patch something that is wrong by design.
>
> Are you saying the Python language spec is unfixable or that the CPython
> implementation is unfixable?
>
> If CPython is unfixable, you can develop a better Python implementation.
>
> If Python itself is unfixable, what brings you here?
>
> Marko
>

I forgot to mention earlier that I report all his rubbish as abuse on 
google groups.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From steve+comp.lang.python at pearwood.info  Fri May 15 08:43:09 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 15 May 2015 22:43:09 +1000
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <mailman.31.1431683047.17265.python-list@python.org>
 <87382yglmy.fsf@elektro.pacujo.net>
Message-ID: <5555e9de$0$13007$c3e8da3$5496439d@news.astraweb.com>

On Fri, 15 May 2015 08:50 pm, Marko Rauhamaa wrote:

> Chris Angelico <rosuav at gmail.com>:
> 
>> On Fri, May 15, 2015 at 6:59 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
>>> Must a method lookup necessarily involve object creation?
>>
>> Actually, no.
>> [...]
>> a particular Python implementation is most welcome to notice the
>> extremely common situation of method calls and optimize it.
> 
> I'm not sure that is feasible given the way it has been specified. You'd
> have to prove the class attribute lookup produces the same outcome in
> consecutive method references.

Sure. But some implementations may have a more, um, flexible approach to
correctness, and offer more aggressive optimizations which break the letter
of Python's semantics but work for 90% of cases. Just because CPython
doesn't do so, doesn't mean that some new implementation might not offer a
series of aggressive optimizations which the caller (or maybe the module?)
can turn on as needed, e.g.:

- assume methods never change;
- assume classes are static;
- assume built-in names always refer to the known built-in;

etc. Such an optimized Python, when running with those optimizations turned
on, is not *strictly* Python, but "buyer beware" applies here. If the
optimizations break your code or make testing hard, don't use it.


> Also:
> 
>    >>> class X:
>    ...   def f(self): pass
>    ...
>    >>> x = X()
>    >>> f = x.f
>    >>> ff = x.f
>    >>> f is ff
>    False
> 
> Would a compliant Python implementation be allowed to respond "True"?

Certainly.

When you retrieve x.f, Python applies the usual "attribute lookup" code,
which simplified looks like this:

if 'f' in x.__dict__:
    attr = x.__dict__['f']
else:
    for K in type(x).__mro__:
        # Walk the parent classes of x in the method resolution order
        if 'f' in K.__dict__:
            attr = K.__dict__['f']
            break
    else:  # no break
        raise AttributeError
# if we get here, we know x.f exists and is bound to attr
# now apply the descriptor protocol (simplified)
if hasattr(attr, '__get__'):
    attr = attr.__get__(x, type(x))
# Finally we can call x.f()
return attr(x, *args)

Functions have a __get__ method which returns the method object! Imagine
they look something like this:

class FunctionType:
    def __call__(self, *args, **kwargs):
        # Actually call the code that does stuff

    def __get__(self, instance, cls):
        if cls is None:
            # Unbound instance
            return self
        return MethodType(self, instance)  # self is the function


This implementation creates a new method object every time you look it up.
But functions *could* do this:

    def __get__(self, instance, cls):
        if cls is None:
            # Unbound instance
            return self
        if self._method is None:
            self._method = MethodType(self, instance)  # Cache it.
        return self._method



What's more, a compliant implementation could reach the "if we get here"
point in the lookup procedure above, and do this:

# if we get here, we know attr exists
if type(attr) is FunctionType:  # Fast pointer comparison.
    return attr(x, *args)
else:
    # do the descriptor protocol thing, and then call attr


It can only do this if it knows that x.f is a real function, not some sort
of callable or function subclass, because in that case who knows what
side-effects the __get__ method might have.

How much time would it save? Probably very little. After all, unless the
method call itself did bugger-all work, the time to create the method
object is probably insignificant. But it's a possible optimization.



-- 
Steven



From breamoreboy at yahoo.co.uk  Fri May 15 09:04:19 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 15 May 2015 14:04:19 +0100
Subject: pip grabs tar.gz file instead of whl?
Message-ID: <mj4qsn$k5m$1@ger.gmane.org>

I was fed up with trying to install from pypi to Windows.  Setup.py more 
often than not wouldn't be able to find the VS compiler. So I thought 
I'd try the direct route to the excellent Christoph Gohlke site at 
http://www.lfd.uci.edu/~gohlke/pythonlibs/ which is all whl files these 
days.  However as you can see below despite my best efforts I'm still 
processing the tar.gz file, so what am I doing wrong?

C:\Users\Mark\Documents\MyPython>pip install --no-cache-dir 
--trusted-host http://www.lfd.uci.edu/ -U -f 
http://www.lfd.uci.edu/~gohlke/pythonlibs/ numba
Collecting numba
   This repository located at www.lfd.uci.edu is not a trusted host, if 
this repository is available via HTTPS it is recommend to use HTTPS 
instead, otherwise you may silence this warning with '--trusted-host 
www.lfd.uci.edu'.
   DEPRECATION: Implicitly allowing locations which are not hosted at a 
secure origin is deprecated and will require the use of --trusted-host 
in the future.
   Downloading numba-0.18.2.tar.gz (507kB)
     100% |################################| 507kB 373kB/s

I'm aware of the work around, download the file, then install the local 
copy, but that's a whole extra step :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From Cecil at decebal.nl  Fri May 15 10:25:22 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 15 May 2015 16:25:22 +0200
Subject: Trying to build Python
References: <87a8x65e28.fsf@Equus.decebal.nl>
 <a8f18c31-4da2-473b-85a1-309b0ac60ea0@googlegroups.com>
 <mailman.33.1431688614.17265.python-list@python.org>
Message-ID: <87617t6hpp.fsf@Equus.decebal.nl>

Op Friday 15 May 2015 13:16 CEST schreef Chris Angelico:

> On Fri, May 15, 2015 at 9:02 PM, Rustom Mody <rustompmody at gmail.com> wrote:
>> On Friday, May 15, 2015 at 4:28:13 PM UTC+5:30, Cecil Westerhof wrote:
>>> I am trying to build Python 3 on a Debian system. It is
>>> successful, but a few things where missing. Partly I solved it,
>>> but a few things keep unresolved: _bz2 _sqlite3 readline _dbm _ssl
>>> _gdbm
>>>
>>> What do I need to do to get those resolved also?
>>>
>>
>> $ aptitude build-dep python3
>>
>> should fetch the requirements of python3

It should, but it did not. I had to install tcl-dev and tk-dev and
then still got the above missing dependencies.


> And if not, try grabbing some development libraries:
>
> $ sudo apt-get install libbz2-dev libsqlite3-dev libreadline-dev
> libssl-dev libgdbm-dev

That did the trick. :-D


> I'm not sure about the _dbm module, not sure what it needs.

I get no complaints anymore, so: problem solved.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From ian.g.kelly at gmail.com  Fri May 15 11:00:46 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 15 May 2015 09:00:46 -0600
Subject: Building CPython
In-Reply-To: <5555e9de$0$13007$c3e8da3$5496439d@news.astraweb.com>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <mailman.31.1431683047.17265.python-list@python.org>
 <87382yglmy.fsf@elektro.pacujo.net>
 <5555e9de$0$13007$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CALwzidntduTbzto0WF0qsuUjEAGymzc6dufV3EXVhZ6atF9MQA@mail.gmail.com>

On Fri, May 15, 2015 at 6:43 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> How much time would it save? Probably very little. After all, unless the
> method call itself did bugger-all work, the time to create the method
> object is probably insignificant. But it's a possible optimization.

An interesting alternative (if it's not already being done) might be
to maintain a limited free-list of method objects, removing the need
to allocate memory for one before filling it in with data.


From ian.g.kelly at gmail.com  Fri May 15 11:04:46 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 15 May 2015 09:04:46 -0600
Subject: Building CPython
In-Reply-To: <CALwzidntduTbzto0WF0qsuUjEAGymzc6dufV3EXVhZ6atF9MQA@mail.gmail.com>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <mailman.31.1431683047.17265.python-list@python.org>
 <87382yglmy.fsf@elektro.pacujo.net>
 <5555e9de$0$13007$c3e8da3$5496439d@news.astraweb.com>
 <CALwzidntduTbzto0WF0qsuUjEAGymzc6dufV3EXVhZ6atF9MQA@mail.gmail.com>
Message-ID: <CALwzidm66a=RahLOiPTJGtTfDxwxu44ge8CqyDpxWa16euXyvQ@mail.gmail.com>

On Fri, May 15, 2015 at 9:00 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Fri, May 15, 2015 at 6:43 AM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>> How much time would it save? Probably very little. After all, unless the
>> method call itself did bugger-all work, the time to create the method
>> object is probably insignificant. But it's a possible optimization.
>
> An interesting alternative (if it's not already being done) might be
> to maintain a limited free-list of method objects, removing the need
> to allocate memory for one before filling it in with data.

Looks like it is already being done:
https://hg.python.org/cpython/file/e7c7431f91b2/Objects/methodobject.c#l7


From rosuav at gmail.com  Fri May 15 11:06:58 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 16 May 2015 01:06:58 +1000
Subject: Building CPython
In-Reply-To: <CALwzidntduTbzto0WF0qsuUjEAGymzc6dufV3EXVhZ6atF9MQA@mail.gmail.com>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net>
 <87bnhmgqrx.fsf@elektro.pacujo.net>
 <mailman.31.1431683047.17265.python-list@python.org>
 <87382yglmy.fsf@elektro.pacujo.net>
 <5555e9de$0$13007$c3e8da3$5496439d@news.astraweb.com>
 <CALwzidntduTbzto0WF0qsuUjEAGymzc6dufV3EXVhZ6atF9MQA@mail.gmail.com>
Message-ID: <CAPTjJmryXd30MiFOG0U9X8xotLOQij9TB3su+Npa1LbpbOkEQg@mail.gmail.com>

On Sat, May 16, 2015 at 1:00 AM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Fri, May 15, 2015 at 6:43 AM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>> How much time would it save? Probably very little. After all, unless the
>> method call itself did bugger-all work, the time to create the method
>> object is probably insignificant. But it's a possible optimization.
>
> An interesting alternative (if it's not already being done) might be
> to maintain a limited free-list of method objects, removing the need
> to allocate memory for one before filling it in with data.

It is already done. Some stats were posted recently to python-dev, and
(if I read them correctly) method objects are among the free-list
types. So the actual memory (de)allocations are optimized, and all
that's left is setting a couple of pointers to select an object and a
function.

ChrisA


From ned at nedbatchelder.com  Fri May 15 12:37:34 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Fri, 15 May 2015 09:37:34 -0700 (PDT)
Subject: Trying to build Python
In-Reply-To: <87617t6hpp.fsf@Equus.decebal.nl>
References: <87a8x65e28.fsf@Equus.decebal.nl>
 <a8f18c31-4da2-473b-85a1-309b0ac60ea0@googlegroups.com>
 <mailman.33.1431688614.17265.python-list@python.org>
 <87617t6hpp.fsf@Equus.decebal.nl>
Message-ID: <df6d75b5-5172-41be-8561-d90b814f31f5@googlegroups.com>

On Friday, May 15, 2015 at 10:40:14 AM UTC-4, Cecil Westerhof wrote:
> Op Friday 15 May 2015 13:16 CEST schreef Chris Angelico:
> 
> > On Fri, May 15, 2015 at 9:02 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> >> On Friday, May 15, 2015 at 4:28:13 PM UTC+5:30, Cecil Westerhof wrote:
> >>> I am trying to build Python 3 on a Debian system. It is
> >>> successful, but a few things where missing. Partly I solved it,
> >>> but a few things keep unresolved: _bz2 _sqlite3 readline _dbm _ssl
> >>> _gdbm
> >>>
> >>> What do I need to do to get those resolved also?
> >>>
> >>
> >> $ aptitude build-dep python3
> >>
> >> should fetch the requirements of python3
> 
> It should, but it did not. I had to install tcl-dev and tk-dev and
> then still got the above missing dependencies.

Just to be clear: "build-dep python3" will install the minimum
requirements to build Python.  This worked for you: as you
described, your first build succeeded: it produced an
executable "python".

A few of the more exotic modules that need OS-level support
are built only if the OS support is present.  This is the
message you got at the end of the build.

> 
> 
> > And if not, try grabbing some development libraries:
> >
> > $ sudo apt-get install libbz2-dev libsqlite3-dev libreadline-dev
> > libssl-dev libgdbm-dev
> 
> That did the trick. :-D

And this is how you install those other libraries to get
those modules built! :)

--Ned.

> 
> 
> > I'm not sure about the _dbm module, not sure what it needs.
> 
> I get no complaints anymore, so: problem solved.
> 
> -- 
> Cecil Westerhof
> Senior Software Engineer
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof



From Cecil at decebal.nl  Fri May 15 14:17:18 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 15 May 2015 20:17:18 +0200
Subject: Minimising stack trace
Message-ID: <871tih66z5.fsf@Equus.decebal.nl>

While playing with recursion I get:
    RuntimeError: maximum recursion depth exceeded in comparison

But then I get a very long stack trace. Is there a way to make this a
lot shorter. Now I ?lose? interesting information because of the
length of the stack trace.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From random832 at fastmail.us  Fri May 15 14:27:05 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 15 May 2015 14:27:05 -0400
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <CAPTjJmpS2F1j1gWauSgN=bsF60X8jDZ1JQG8qpWMQyfmT3Ot2Q@mail.gmail.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <mj2g70$eij$1@reader1.panix.com>
 <55554ddb$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmokNo3e1=LEBBnC_6zQmtP=Smh7bXm-uCsGEew4sKZ52Q@mail.gmail.com>
 <CALwzidkTXnbwcGPa5tK_fV_Rg1B7pX2rVK+0JPVdwLXu+CD7NQ@mail.gmail.com>
 <mailman.23.1431658586.17265.python-list@python.org>
 <55557185$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmpS2F1j1gWauSgN=bsF60X8jDZ1JQG8qpWMQyfmT3Ot2Q@mail.gmail.com>
Message-ID: <1431714425.3284050.269876137.5380A46A@webmail.messagingengine.com>

On Fri, May 15, 2015, at 00:25, Chris Angelico wrote:
> The main thing is that trashing invites the system to delete the file
> at its leisure,

I've never seen a system whose trash can emptied itself without user
intervention.


From rosuav at gmail.com  Fri May 15 14:33:16 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 16 May 2015 04:33:16 +1000
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <1431714425.3284050.269876137.5380A46A@webmail.messagingengine.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <mj2g70$eij$1@reader1.panix.com>
 <55554ddb$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmokNo3e1=LEBBnC_6zQmtP=Smh7bXm-uCsGEew4sKZ52Q@mail.gmail.com>
 <CALwzidkTXnbwcGPa5tK_fV_Rg1B7pX2rVK+0JPVdwLXu+CD7NQ@mail.gmail.com>
 <mailman.23.1431658586.17265.python-list@python.org>
 <55557185$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmpS2F1j1gWauSgN=bsF60X8jDZ1JQG8qpWMQyfmT3Ot2Q@mail.gmail.com>
 <1431714425.3284050.269876137.5380A46A@webmail.messagingengine.com>
Message-ID: <CAPTjJmqfcv4_6+b8Xtkto1nWoQLccOCgYDQaAXbdyQq2-2uyBg@mail.gmail.com>

On Sat, May 16, 2015 at 4:27 AM,  <random832 at fastmail.us> wrote:
> On Fri, May 15, 2015, at 00:25, Chris Angelico wrote:
>> The main thing is that trashing invites the system to delete the file
>> at its leisure,
>
> I've never seen a system whose trash can emptied itself without user
> intervention.

They always grow to infinity on your systems? Huh. That's probably the
better option, but I've had to manage Windowses back as far as XP
recently (back as far as W95 but not for many years, fortunately) and
the system would very kindly remove from the trash... I'm fairly sure
the algorithm was either random or outright malicious, but it did
remove stuff.

ChrisA


From ethan at stoneleaf.us  Fri May 15 14:45:14 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 15 May 2015 11:45:14 -0700
Subject: pip grabs tar.gz file instead of whl?
In-Reply-To: <mj4qsn$k5m$1@ger.gmane.org>
References: <mj4qsn$k5m$1@ger.gmane.org>
Message-ID: <55563EBA.8080606@stoneleaf.us>

On 05/15/2015 06:04 AM, Mark Lawrence wrote:

> C:\Users\Mark\Documents\MyPython>pip install --no-cache-dir
> --trusted-host http://www.lfd.uci.edu/ -U -f
> http://www.lfd.uci.edu/~gohlke/pythonlibs/ numba
 >
> Collecting numba
>    This repository located at www.lfd.uci.edu is not a trusted host, if
> this repository is available via HTTPS it is recommend to use HTTPS
> instead, otherwise you may silence this warning with '--trusted-host
> www.lfd.uci.edu'.
>    DEPRECATION: Implicitly allowing locations which are not hosted at a
> secure origin is deprecated and will require the use of --trusted-host
> in the future.
>    Downloading numba-0.18.2.tar.gz (507kB)
>      100% |################################| 507kB 373kB/s

I particularly like how, when you specify --trusted-host, you still get 
all the warnings to use --trusted-host... not.

--
~Ethan~


From rosuav at gmail.com  Fri May 15 14:55:07 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 16 May 2015 04:55:07 +1000
Subject: pip grabs tar.gz file instead of whl?
In-Reply-To: <55563EBA.8080606@stoneleaf.us>
References: <mj4qsn$k5m$1@ger.gmane.org>
	<55563EBA.8080606@stoneleaf.us>
Message-ID: <CAPTjJmpku_aPPnNq4auYhOy=Y-hJSZUErQPcjizLMpwCvmf0oQ@mail.gmail.com>

On Sat, May 16, 2015 at 4:45 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 05/15/2015 06:04 AM, Mark Lawrence wrote:
>
>> C:\Users\Mark\Documents\MyPython>pip install --no-cache-dir
>> --trusted-host http://www.lfd.uci.edu/ -U -f
>> http://www.lfd.uci.edu/~gohlke/pythonlibs/ numba
>
>>
>>
>> Collecting numba
>>    This repository located at www.lfd.uci.edu is not a trusted host, if
>> this repository is available via HTTPS it is recommend to use HTTPS
>> instead, otherwise you may silence this warning with '--trusted-host
>> www.lfd.uci.edu'.
>>    DEPRECATION: Implicitly allowing locations which are not hosted at a
>> secure origin is deprecated and will require the use of --trusted-host
>> in the future.
>>    Downloading numba-0.18.2.tar.gz (507kB)
>>      100% |################################| 507kB 373kB/s
>
>
> I particularly like how, when you specify --trusted-host, you still get all
> the warnings to use --trusted-host... not.

Actually, I suspect the note is telling you to use "--trusted-host
www.lfd.uci.edu" rather than "--trusted-host http://www.lfd.uci.edu/"
- so it's as if the option wasn't specified at all. But pip can be a
little hard to understand sometimes (just ask Prince Edward, who had
no end of trouble understanding the poor chipmonk!), so I may very
well be wrong here.

ChrisA


From ned at nedbatchelder.com  Fri May 15 15:04:27 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Fri, 15 May 2015 12:04:27 -0700 (PDT)
Subject: Minimising stack trace
In-Reply-To: <871tih66z5.fsf@Equus.decebal.nl>
References: <871tih66z5.fsf@Equus.decebal.nl>
Message-ID: <2c2b0f1d-ce16-41b4-a12d-1cface2bf154@googlegroups.com>

On Friday, May 15, 2015 at 2:50:12 PM UTC-4, Cecil Westerhof wrote:
> While playing with recursion I get:
>     RuntimeError: maximum recursion depth exceeded in comparison
> 
> But then I get a very long stack trace. Is there a way to make this a
> lot shorter. Now I 'lose' interesting information because of the
> length of the stack trace.

There isn't a way to shorten the stack trace.  If you are losing
information at the top because of your terminal window, you can
likely increase the number of lines it will keep for you.

Another option is to reduce the maximum stack depth, so that it
is exceeded sooner, which produces shorter stack traces:

    import sys;
    sys.setrecursionlimit(50)


--Ned.


From skip.montanaro at gmail.com  Fri May 15 15:14:07 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Fri, 15 May 2015 14:14:07 -0500
Subject: Minimising stack trace
In-Reply-To: <2c2b0f1d-ce16-41b4-a12d-1cface2bf154@googlegroups.com>
References: <871tih66z5.fsf@Equus.decebal.nl>
 <2c2b0f1d-ce16-41b4-a12d-1cface2bf154@googlegroups.com>
Message-ID: <CANc-5UyMteqNdyetufYX0kaUObpGQcoN5YqCoPMRKOYpSg02VQ@mail.gmail.com>

A third alternative is to take a look at asyncore.compact_traceback.
That will only help modestly in Cecil's example, but I found it
helpful.

Skip

On Fri, May 15, 2015 at 2:04 PM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> On Friday, May 15, 2015 at 2:50:12 PM UTC-4, Cecil Westerhof wrote:
>> While playing with recursion I get:
>>     RuntimeError: maximum recursion depth exceeded in comparison
>>
>> But then I get a very long stack trace. Is there a way to make this a
>> lot shorter. Now I 'lose' interesting information because of the
>> length of the stack trace.
>
> There isn't a way to shorten the stack trace.  If you are losing
> information at the top because of your terminal window, you can
> likely increase the number of lines it will keep for you.
>
> Another option is to reduce the maximum stack depth, so that it
> is exceeded sooner, which produces shorter stack traces:
>
>     import sys;
>     sys.setrecursionlimit(50)
>
>
> --Ned.
> --
> https://mail.python.org/mailman/listinfo/python-list


From PointedEars at web.de  Fri May 15 15:41:17 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Fri, 15 May 2015 21:41:17 +0200
Subject: Minimising stack trace
References: <871tih66z5.fsf@Equus.decebal.nl>
Message-ID: <1585804.vCLx6kbzfH@PointedEars.de>

Cecil Westerhof wrote:

> While playing with recursion I get:
>     RuntimeError: maximum recursion depth exceeded in comparison
> 
> But then I get a very long stack trace. Is there a way to make this a
> lot shorter. Now I ?lose? interesting information because of the length of 
> the stack trace.

<http://www.catb.org/~esr/faqs/smart-questions.html>

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From breamoreboy at yahoo.co.uk  Fri May 15 17:00:38 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 15 May 2015 22:00:38 +0100
Subject: pip grabs tar.gz file instead of whl?
In-Reply-To: <CAPTjJmpku_aPPnNq4auYhOy=Y-hJSZUErQPcjizLMpwCvmf0oQ@mail.gmail.com>
References: <mj4qsn$k5m$1@ger.gmane.org>	<55563EBA.8080606@stoneleaf.us>
 <CAPTjJmpku_aPPnNq4auYhOy=Y-hJSZUErQPcjizLMpwCvmf0oQ@mail.gmail.com>
Message-ID: <mj5mpp$t1g$1@ger.gmane.org>

On 15/05/2015 19:55, Chris Angelico wrote:
> On Sat, May 16, 2015 at 4:45 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
>> On 05/15/2015 06:04 AM, Mark Lawrence wrote:
>>
>>> C:\Users\Mark\Documents\MyPython>pip install --no-cache-dir
>>> --trusted-host http://www.lfd.uci.edu/ -U -f
>>> http://www.lfd.uci.edu/~gohlke/pythonlibs/ numba
>>
>>>
>>>
>>> Collecting numba
>>>     This repository located at www.lfd.uci.edu is not a trusted host, if
>>> this repository is available via HTTPS it is recommend to use HTTPS
>>> instead, otherwise you may silence this warning with '--trusted-host
>>> www.lfd.uci.edu'.
>>>     DEPRECATION: Implicitly allowing locations which are not hosted at a
>>> secure origin is deprecated and will require the use of --trusted-host
>>> in the future.
>>>     Downloading numba-0.18.2.tar.gz (507kB)
>>>       100% |################################| 507kB 373kB/s
>>
>>
>> I particularly like how, when you specify --trusted-host, you still get all
>> the warnings to use --trusted-host... not.
>
> Actually, I suspect the note is telling you to use "--trusted-host
> www.lfd.uci.edu" rather than "--trusted-host http://www.lfd.uci.edu/"
> - so it's as if the option wasn't specified at all. But pip can be a
> little hard to understand sometimes (just ask Prince Edward, who had
> no end of trouble understanding the poor chipmonk!), so I may very
> well be wrong here.
>
> ChrisA
>

I'm putting this here FTR as I'm too knackered to follow up right now.

After a bit more digging I ended up with this.

C:\Users\Mark\Documents\MyPython>pip install --trusted-host 
www.lfd.uci.edu/ -U 
http://www.lfd.uci.edu/~gohlke/pythonlibs/numba-0.18.2-cp34-none-win_amd64.whl
Collecting numba==0.18.2 from 
http://www.lfd.uci.edu/~gohlke/pythonlibs/numba-0.18.2-cp34-none-win_amd64.whl
   Downloading 
http://www.lfd.uci.edu/~gohlke/pythonlibs/numba-0.18.2-cp34-none-win_amd64.whl
   Exception:
   Traceback (most recent call last):
     File "C:\Python34\lib\site-packages\pip\basecommand.py", line 246, 
in main
       status = self.run(options, args)
     File "C:\Python34\lib\site-packages\pip\commands\install.py", line 
342, in run
       requirement_set.prepare_files(finder)
     File "C:\Python34\lib\site-packages\pip\req\req_set.py", line 345, 
in prepare_files
       functools.partial(self._prepare_file, finder))
     File "C:\Python34\lib\site-packages\pip\req\req_set.py", line 290, 
in _walk_req_to_install
       more_reqs = handler(req_to_install)
     File "C:\Python34\lib\site-packages\pip\req\req_set.py", line 487, 
in _prepare_file
       download_dir, do_download, session=self.session,
     File "C:\Python34\lib\site-packages\pip\download.py", line 827, in 
unpack_url
       session,
     File "C:\Python34\lib\site-packages\pip\download.py", line 677, in 
unpack_http_url
       unpack_file(from_path, location, content_type, link)
     File "C:\Python34\lib\site-packages\pip\utils\__init__.py", line 
630, in unpack_file
       flatten=not filename.endswith('.whl')
     File "C:\Python34\lib\site-packages\pip\utils\__init__.py", line 
509, in unzip_file
       zip = zipfile.ZipFile(zipfp, allowZip64=True)
     File "C:\Python34\lib\zipfile.py", line 937, in __init__
       self._RealGetContents()
     File "C:\Python34\lib\zipfile.py", line 978, in _RealGetContents
       raise BadZipFile("File is not a zip file")
   zipfile.BadZipFile: File is not a zip file

The same file installed perfectly when downloaded and installed in two 
steps.  Whether this is simply a known bug with zipfile handling, pip 
itself, a combination of both or what I've no idea.

I find it slightly irritating that a tool recommended by the Python 
Packaging Authority behaves in such a way, but then I didn't have to dip 
into my pocket to pay for it and certainly won't lose any sleep over it 
as there's such a simple work around.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From tjreedy at udel.edu  Fri May 15 17:05:16 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 15 May 2015 17:05:16 -0400
Subject: Building CPython
In-Reply-To: <87bnhmgqrx.fsf@elektro.pacujo.net>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
Message-ID: <mj5n2m$1g2$1@ger.gmane.org>

On 5/15/2015 4:59 AM, Marko Rauhamaa wrote:
> Must a method lookup necessarily involve object creation?

Where is matters, inside loops, method lookup can be avoided after doing 
it once.

for i in range(1000000): ob.meth(i)

versus

meth = ob.meth
for i in range(1000000): meth(i)

For working with a single stack:

a = []
apush = stack.append
apop = stack.pop
# Now apush(x), apop(x), and test with 'if a:' as desired.
Being able to do this was part of my rational for adding list.pop, about 
15 years ago.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Fri May 15 17:19:06 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 15 May 2015 17:19:06 -0400
Subject: Building CPython
In-Reply-To: <mj4j11$rc5$1@dont-email.me>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <mailman.4.1431619772.17265.python-list@python.org>
 <oy45x.458028$zE7.268573@fx40.am4>
 <CAPTjJmq7amGAYn63C_w2uHp0AGNPxzQHVngQ5bnXrJrVxg5nXg@mail.gmail.com>
 <mailman.14.1431629476.17265.python-list@python.org>
 <mj4j11$rc5$1@dont-email.me>
Message-ID: <mj5nsk$ejp$1@ger.gmane.org>

On 5/15/2015 6:51 AM, Christian Gollwitzer wrote:
> Am 14.05.15 um 20:50 schrieb Terry Reedy:
>> On 5/14/2015 1:11 PM, Chris Angelico wrote:
>>
>>> 2) make test - run the entire test suite. Takes just as long every
>>> time, but most of it won't have changed.
>>
>> The test runner has an option, -jn, to run tests in n processes instead
>> of just 1.  On my 6 core pentium, -j5 cuts time to almost exactly 1/5th
>> of otherwise.  -j10 seems faster but have not times it.  I suspect that
>> 'make test' does not use -j option.
>>
>
> Just to clarify, -j is an option of GNU make to run the Makefile in
> parallel. Unless the Makefile is buggy, this should result in the same
> output. You can also set an environment variable to enable this
> permanently (until you log out) like
>
> export MAKEFLAGS=-j5
>
> Put this into your .bashrc or .profile, and it'll become permanent.

This is not applicable when running on Windows.  I was specifically 
referring, for instance, to entering at command prompt

current_dir> python -m test -j10

AFAIK, test.__main__ does not look at environmental variables.

ps. I have been wondering why 'j' rather than, say 'c' or 'p' was used 
for 'run in parallel on multiple cores'.  Your comment suggests that 'j' 
follows precedent.

-- 
Terry Jan Reedy



From bc at freeuk.com  Fri May 15 17:54:40 2015
From: bc at freeuk.com (BartC)
Date: Fri, 15 May 2015 22:54:40 +0100
Subject: Building CPython
In-Reply-To: <87bnhmgqrx.fsf@elektro.pacujo.net>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net>	<zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4>	<crlglfFgf05U1@mid.individual.net>
 <87bnhmgqrx.fsf@elektro.pacujo.net>
Message-ID: <DWt5x.439666$qW.359562@fx20.am4>

On 15/05/2015 09:59, Marko Rauhamaa wrote:

>> The path from decoding a bytecode to the C code that implements it can
>> be rather convoluted, but there are reasons for each of the
>> complications -- mainly to do with supporting the ability to override
>> operators with C and/or Python code.
>>
>> If you removed those abilities, the implemention would be simpler, and
>> possibly faster. But then the language wouldn't be Python any more.
>
> I agree that Python's raison-d'?tre is its dynamism and expressive
> power. It definitely shouldn't be sacrificed for performance.
>
> However, in some respects, Python might be going overboard with its
> dynamism; are all those dunder methods really needed? Must "false" be
> defined so broadly? Must a method lookup necessarily involve object
> creation?

What /is/ a method lookup? Is it when you have this:

  A.B()

and need to find whether the expression A (or its class or type) has a 
name B associated with it? (And it then needs to check whether B is 
something that can be called.)

If so, does that have to be done using Python's Dict mechanism? (Ie. 
searching for a key 'B' by name and seeing if the object associated with 
it is a method. That does not sound efficient.)

(And I guess Python's classes come into play so if B is not part of A's 
class then it might be part of some base-class. I can see that it can 
get complicated, but I don't use OO so can't speculate further.)

(In the language whose implementation I'm comparing with CPython's, it 
doesn't have classes. A.B() can still appear, but .B will need to be an 
attribute that the (bytecode) compiler already knows from a prior 
definition (usually, some struct or record if A is an expression).

If there is only one .B it knows, then a simple check that A is the 
correct type is all that is needed. Otherwise a runtime search through 
all .Bs (and a compact table will have been set up for this) is needed 
to find the .B that matches A's type. But this is all still pretty quick.

In this context, A.B will need to be some function pointer, and A.B() 
will call it.)

-- 
Bartc


From hossamalagmy at gmail.com  Fri May 15 18:34:04 2015
From: hossamalagmy at gmail.com (=?windows-1256?B?1O3TyNM=?=)
Date: Fri, 15 May 2015 15:34:04 -0700 (PDT)
Subject: =?windows-1256?B?5MrtzMkgx+HU5cfPySDH4cfIys/Hxu3JIDIwMTUg?=
 =?windows-1256?B?x+HK0eMgx+HLx+Ts?=
Message-ID: <4bb2e349-a5c8-497d-b444-c1cd4cfebb7b@googlegroups.com>

????? ??????? ?????????? 2015 ????? ??????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


????? ??????? ?????????? 2015 ???? ?????? ???? ?????????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

????? ??????? ?????????? ?????? ??????? 2015 ???? ??????
https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

????? ??????? ?????????? ?????? ?????????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
?????? ?? ????? ???? ???????? ?????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
????? ??????? ?????????? ?????? ????? ?????? 2015 ????? ?????? ???? ??????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
????? ??????? ?????????? ?????? ?????? 2015 ????? ?????? ???? ??????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
????? ??????? ?????????? ?????? ??? ????? ?????? 2015 ???? ??????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
????? ??????? ?????????? ?????? ??? ????? ????? ?????? 2015 ???? ??????
https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

????? ?????? ??? ??? ?????? ????? ?? ?? ???????? ???????? ?????? 2015
https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

????? ???? ?????? ????????? ????? ?????? 2015 ???? ??????
https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

??????? ????????? ??????? ????? ??????? ?????? ?????? ???? ?????? ????????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
??????? ????????? ??????? ????? ??????? ?????? ????????? ??????? ?????????? ????? ?????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
??? ?????? ???? ???????? ?? ???????? ?????????? ??????? ????????? 2015 ????? ??????


https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
?????? ??? ??????? ???????? ??????? ????? ??????? ??????? ????????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

????? ??????? ?????????? ?????? ??????? 2015 ????? ?????? ???? ??????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

????? ??????? ?????????? ?????? ???? ????? 2015 ????? ?????? ???? ??????
https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


????? ??????? ?????????? ?????? ?????? ?????? 2015 ????? ?????? ???? ??????
https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


????? ??????? ?????????? ?????? ??? ???? 2015 ????? ?????? ???? ??????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

????? ??????? ?????????? ?????? ????? 2015 ????? ?????? ???? ??????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

????? ??????? ?????????? ?????? ??????? 2015 ????? ?????? ???? ??????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
????? ??????? ?????????? ?????? ???????? 2015 ????? ?????? ???? ??????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
????? ??????? ?????????? ?????? ??????? 2015 ????? ?????? ???? ??????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
???? ?????? ???????? ?????????? ??????? ?????????? ?????? ?????????? ????? ?????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


???? ?????? ????? ??????? ???? ?????? ???????? ?????? ?????? ????? ?????? 2015


https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

???? ?????? ???????? ?????????? ???? ?????? ????????? ?????? ??????? ????? ?????? 2015
https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


???? ?????? ????? ???????? ???? ?????? ???????? ???? ?????? ???????? ????? ?????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


???? ?????? ????? ???? ????? ???????? ?????? ???????? ???? ????? ?????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


????? ?????? ?? ?????? ????? ??????? ??????? ????????? ????? ?????? 2015



https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
???? ????????? ???????? ?????? ???????? ?????????? ??????? ????????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

?????? ???????? ?????????? ???? ????? ???????? ?????? ?????????? ????? ?????? 2015


https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl
?????? ????? ?????????? ???? ????? ??????? ?????? ?????????? ????? ?????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

?????? ????? ?????????? ???? ?????? ???????? ?????? ????? ?????? ????? ?????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


?????? ????? ?????????? TIME FOR ENGLISH ??????? ?????????? ?????? ??????? ????? ?????? 2015
https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


??????? ????????? ??????? ????? ?????????? ??????? ?????????? ?????? ?????????? ????? ?????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


???? ?????? ????? ?????????? ??????? ????????? ?????? ??????????? 2015 ????? ??????


https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

?????? ????? ?????????? TIME FOR ENGLISH ??????? ?????????? ????? ?????? 2015
https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl



?????? ????? ?????????? ???? ?????? ???????? ?????? ?????????? ????? ?????? 2015


https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

???? ?????? ????? ?????????? ???? ????? ???????? ????? ?????? 2015


https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

???? ?????? ???????? ?????????? ??????? ?????????? ????? ?????? 2015 ??????? ?? ??????

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


?????? ?????? ????? ???? ????? ???????? ?????? ?????????? ????? ?????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl

???? ?????? ????? ???? ????? ???????? ?????? ?????????? ????? ?????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl


???? ?????? ?????? ???? ????? ???????? ?????? ?????????? ????? ?????? 2015

https://plus.google.com/u/0/106736449589858805776


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?ref=hl



From marko at pacujo.net  Fri May 15 18:44:41 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 16 May 2015 01:44:41 +0300
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4>
Message-ID: <87twvdsbom.fsf@elektro.pacujo.net>

BartC <bc at freeuk.com>:

> What /is/ a method lookup? Is it when you have this:
>
>  A.B()
>
> and need to find whether the expression A (or its class or type) has a
> name B associated with it? (And it then needs to check whether B is
> something that can be called.)
>
> If so, does that have to be done using Python's Dict mechanism? (Ie.
> searching for a key 'B' by name and seeing if the object associated
> with it is a method. That does not sound efficient.)

That is a general feature among high-level programming languages. In
Python, it is even more complicated:

 * first the object's dict is looked up for the method name

 * if the method is not found (it usually isn't), the dict of the
   object's class is consulted

 * if the method is found (it usually is), a function object is
   instantiated that delegates to the class's method and embeds a "self"
   reference to the object to the call

IOW, two dict lookups plus an object construction for each method call.


Marko


From breamoreboy at yahoo.co.uk  Fri May 15 19:27:58 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 16 May 2015 00:27:58 +0100
Subject: Building CPython
In-Reply-To: <87twvdsbom.fsf@elektro.pacujo.net>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4> <87twvdsbom.fsf@elektro.pacujo.net>
Message-ID: <mj5ve1$180$1@ger.gmane.org>

On 15/05/2015 23:44, Marko Rauhamaa wrote:
> BartC <bc at freeuk.com>:
>
>> What /is/ a method lookup? Is it when you have this:
>>
>>   A.B()
>>
>> and need to find whether the expression A (or its class or type) has a
>> name B associated with it? (And it then needs to check whether B is
>> something that can be called.)
>>
>> If so, does that have to be done using Python's Dict mechanism? (Ie.
>> searching for a key 'B' by name and seeing if the object associated
>> with it is a method. That does not sound efficient.)
>
> That is a general feature among high-level programming languages. In
> Python, it is even more complicated:
>
>   * first the object's dict is looked up for the method name
>
>   * if the method is not found (it usually isn't), the dict of the
>     object's class is consulted
>
>   * if the method is found (it usually is), a function object is
>     instantiated that delegates to the class's method and embeds a "self"
>     reference to the object to the call
>
> IOW, two dict lookups plus an object construction for each method call.
>
>
> Marko
>

As a picture paints a thousand words is anybody aware of a site or sites 
that show this diagramatically, as I think I and possibly others would 
find it far easier to grasp.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From tjreedy at udel.edu  Fri May 15 19:54:31 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 15 May 2015 19:54:31 -0400
Subject: Building CPython
In-Reply-To: <DWt5x.439666$qW.359562@fx20.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net>	<zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4>	<crlglfFgf05U1@mid.individual.net>
 <87bnhmgqrx.fsf@elektro.pacujo.net> <DWt5x.439666$qW.359562@fx20.am4>
Message-ID: <mj6101$nki$1@ger.gmane.org>

On 5/15/2015 5:54 PM, BartC wrote:

> What /is/ a method lookup? Is it when you have this:
>
>   A.B()

This is parsed as (A.B)()

> and need to find whether the expression A (or its class or type) has a
> name B associated with it?

Yes.  Dotted names imply an attribute lookup.

> (And it then needs to check whether B is something that can be called.)

The object resulting from the attribute lookup, A.B (not B exactly), is 
called in a separate operation (with a separate bytecode). It depends on 
the object having a .__call__ method.  The .__call__ method is 
*executed* (rather than *called*, which would lead to infinite regress).

-- 
Terry Jan Reedy



From sohcahtoa82 at gmail.com  Fri May 15 20:04:49 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Fri, 15 May 2015 17:04:49 -0700 (PDT)
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <mailman.42.1431714427.17265.python-list@python.org>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <mj2g70$eij$1@reader1.panix.com>
 <55554ddb$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmokNo3e1=LEBBnC_6zQmtP=Smh7bXm-uCsGEew4sKZ52Q@mail.gmail.com>
 <CALwzidkTXnbwcGPa5tK_fV_Rg1B7pX2rVK+0JPVdwLXu+CD7NQ@mail.gmail.com>
 <mailman.23.1431658586.17265.python-list@python.org>
 <55557185$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <CAPTjJmpS2F1j1gWauSgN=bsF60X8jDZ1JQG8qpWMQyfmT3Ot2Q@mail.gmail.com>
 <mailman.42.1431714427.17265.python-list@python.org>
Message-ID: <1a90a69d-2376-49bc-afeb-db230d5ba24d@googlegroups.com>

On Friday, May 15, 2015 at 11:27:18 AM UTC-7, rand... at fastmail.us wrote:
> On Fri, May 15, 2015, at 00:25, Chris Angelico wrote:
> > The main thing is that trashing invites the system to delete the file
> > at its leisure,
> 
> I've never seen a system whose trash can emptied itself without user
> intervention.

Windows won't empty the entire trash without user intervention, but is IS configured by default to only use up to a specific amount of hard drive space.  Once the trash reaches its capacity, it'll start removing files from it to make room for more.


From breamoreboy at yahoo.co.uk  Fri May 15 20:16:04 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 16 May 2015 01:16:04 +0100
Subject: Python: How to find out values of all feasible x under
 constraints.
In-Reply-To: <mj4p69$je7$1@ger.gmane.org>
References: <12da2c70-fd4e-430b-83d2-fad4ac1e3f8f@googlegroups.com>
 <mj4p69$je7$1@ger.gmane.org>
Message-ID: <mj6287$9k0$1@ger.gmane.org>

On 15/05/2015 13:35, Mark Lawrence wrote:
> On 15/05/2015 04:58, Xiang Zhang wrote:
>> Dear all,
>>
>> I am writing a code using Python now.
>>
>> I want to know how to find out values of all feasible x under
>> constraints.
>>
>> x = [x_1, x_2, x_3,..., x_10]
>>
>>
>> constraints:
>> x_i = 0,1,2,3 or 4,          where i=1,2,....10
>> x_1 + x_2 + x_3 +...+x_10 <= 15
>>
>> How to find out all the feasible solutions x (domain of x) using
>> python, like [0,0,0,0,0,0,0,0,0,0], [1,1,1,1,1,1,1,1,1,1] etc ? What
>> should be the code?
>>
>> Any hint or help would be highly appreciated!
>>
>> Sincerely,
>>
>> Xiang Zhang
>>
>
> There are several constraint libraries on pypi if you don't want to roll
> your own.  See for example
> https://pypi.python.org/pypi/python-constraint/1.2
>

Forgot to mention this http://numberjack.ucc.ie/

Maybe of general interest see also http://www.csplib.org/ 
http://www.hakank.org/constraint_programming_blog/



-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From bc at freeuk.com  Fri May 15 20:43:48 2015
From: bc at freeuk.com (BartC)
Date: Sat, 16 May 2015 01:43:48 +0100
Subject: Building CPython
In-Reply-To: <87twvdsbom.fsf@elektro.pacujo.net>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net>	<zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4>	<crlglfFgf05U1@mid.individual.net>
 <87bnhmgqrx.fsf@elektro.pacujo.net>	<DWt5x.439666$qW.359562@fx20.am4>
 <87twvdsbom.fsf@elektro.pacujo.net>
Message-ID: <bpw5x.561870$Zf4.150532@fx22.am4>

On 15/05/2015 23:44, Marko Rauhamaa wrote:
> BartC <bc at freeuk.com>:
>
>> What /is/ a method lookup? Is it when you have this:
>>
>>   A.B()
>>
>> and need to find whether the expression A (or its class or type) has a
>> name B associated with it? (And it then needs to check whether B is
>> something that can be called.)
>>
>> If so, does that have to be done using Python's Dict mechanism? (Ie.
>> searching for a key 'B' by name and seeing if the object associated
>> with it is a method. That does not sound efficient.)
>
> That is a general feature among high-level programming languages. In
> Python, it is even more complicated:
>
>   * first the object's dict is looked up for the method name
>
>   * if the method is not found (it usually isn't), the dict of the
>     object's class is consulted
>
>   * if the method is found (it usually is), a function object is
>     instantiated that delegates to the class's method and embeds a "self"
>     reference to the object to the call
>
> IOW, two dict lookups plus an object construction for each method call.

OK, I didn't know that objects have their own set of attributes that are 
distinct from the class they belong to. I really ought to learn more 
Python!.

(Yet, I have this crazy urge now to create my own bytecode interpreter 
for, if not exactly Python itself, then an equivalent language. Just to 
see if I can do any better than CPython, given the same language 
restraints.

Although I'm hampered a little by not knowing Python well enough. Nor 
OOP, but those are minor details... Anyway it sounds more fun than 
trying to decipher the layers of macros and conditional code that appear 
to be the CPython sources.)

 > IOW, two dict lookups plus an object construction for each method call.

I suppose in many cases an object will have no attributes of its own, 
and so it can rapidly bypass the first lookup. I don't understand the 
need for an object creation (to represent A.B so that it can call it?) 
but perhaps such an object can already exist, prepared ready for use.

-- 
Bartc


From greg.ewing at canterbury.ac.nz  Fri May 15 20:55:51 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Sat, 16 May 2015 12:55:51 +1200
Subject: Building CPython
In-Reply-To: <Z2j5x.640864$ms.43141@fx30.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net> <zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4> <crlglfFgf05U1@mid.individual.net>
 <Z2j5x.640864$ms.43141@fx30.am4>
Message-ID: <crnisnF2efoU1@mid.individual.net>

BartC wrote:
> For example, there is a /specific/ byte-code called BINARY_ADD, which 
> then proceeds to call a /generic/ binary-op handler! This throws away 
> the advantage of knowing at byte-code generation time exactly which 
> operation is needed.

While inlining the binary-op handling might give you a
slightly shorter code path, it wouldn't necessarily speed
anything up. It's possible, for example, that the shared
binary-op handler fits in the instruction cache, but the
various inlined copies of it don't, leading to a slowdown.

The only way to be sure about things like that is to try
them and measure. The days when you could predict the speed
of a program just by counting the number of instructions
executed are long gone.

-- 
Greg


From steve+comp.lang.python at pearwood.info  Fri May 15 20:57:42 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sat, 16 May 2015 10:57:42 +1000
Subject: Minimising stack trace
References: <871tih66z5.fsf@Equus.decebal.nl>
 <1585804.vCLx6kbzfH@PointedEars.de>
Message-ID: <55569607$0$13008$c3e8da3$5496439d@news.astraweb.com>

On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote:

> Cecil Westerhof wrote:
> 
>> While playing with recursion I get:
>>     RuntimeError: maximum recursion depth exceeded in comparison
>> 
>> But then I get a very long stack trace. Is there a way to make this a
>> lot shorter. Now I ?lose? interesting information because of the length
>> of the stack trace.
> 
> <http://www.catb.org/~esr/faqs/smart-questions.html>

There ought to be a website that explains how to give smart answers.


-- 
Steven



From python at mrabarnett.plus.com  Fri May 15 21:16:52 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 16 May 2015 02:16:52 +0100
Subject: Building CPython
In-Reply-To: <bpw5x.561870$Zf4.150532@fx22.am4>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net>	<zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4>	<crlglfFgf05U1@mid.individual.net>
 <87bnhmgqrx.fsf@elektro.pacujo.net>	<DWt5x.439666$qW.359562@fx20.am4>
 <87twvdsbom.fsf@elektro.pacujo.net> <bpw5x.561870$Zf4.150532@fx22.am4>
Message-ID: <55569A84.4040307@mrabarnett.plus.com>

On 2015-05-16 01:43, BartC wrote:
> On 15/05/2015 23:44, Marko Rauhamaa wrote:
>> BartC <bc at freeuk.com>:
>>
>>> What /is/ a method lookup? Is it when you have this:
>>>
>>>   A.B()
>>>
>>> and need to find whether the expression A (or its class or type) has a
>>> name B associated with it? (And it then needs to check whether B is
>>> something that can be called.)
>>>
>>> If so, does that have to be done using Python's Dict mechanism? (Ie.
>>> searching for a key 'B' by name and seeing if the object associated
>>> with it is a method. That does not sound efficient.)
>>
>> That is a general feature among high-level programming languages. In
>> Python, it is even more complicated:
>>
>>   * first the object's dict is looked up for the method name
>>
>>   * if the method is not found (it usually isn't), the dict of the
>>     object's class is consulted
>>
>>   * if the method is found (it usually is), a function object is
>>     instantiated that delegates to the class's method and embeds a "self"
>>     reference to the object to the call
>>
>> IOW, two dict lookups plus an object construction for each method call.
>
> OK, I didn't know that objects have their own set of attributes that are
> distinct from the class they belong to. I really ought to learn more
> Python!.
>
> (Yet, I have this crazy urge now to create my own bytecode interpreter
> for, if not exactly Python itself, then an equivalent language. Just to
> see if I can do any better than CPython, given the same language
> restraints.
>
> Although I'm hampered a little by not knowing Python well enough. Nor
> OOP, but those are minor details... Anyway it sounds more fun than
> trying to decipher the layers of macros and conditional code that appear
> to be the CPython sources.)
>
>   > IOW, two dict lookups plus an object construction for each method call.
>
> I suppose in many cases an object will have no attributes of its own,
> and so it can rapidly bypass the first lookup. I don't understand the
> need for an object creation (to represent A.B so that it can call it?)
> but perhaps such an object can already exist, prepared ready for use.
>
It's possible to do:

     f = A.B
     ...
     f()

so it's necessary to have an object for A.B.

The question is how much you would gain from optimising A.B() as a
special case (increase in speed vs increase in complexity).



From ned at nedbatchelder.com  Fri May 15 21:24:11 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Fri, 15 May 2015 18:24:11 -0700 (PDT)
Subject: Minimising stack trace
In-Reply-To: <55569607$0$13008$c3e8da3$5496439d@news.astraweb.com>
References: <871tih66z5.fsf@Equus.decebal.nl>
 <1585804.vCLx6kbzfH@PointedEars.de>
 <55569607$0$13008$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <9cbf3b8e-e830-46f5-a636-27c0d07f460a@googlegroups.com>

On Friday, May 15, 2015 at 8:57:53 PM UTC-4, Steven D'Aprano wrote:
> On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote:
> 
> > Cecil Westerhof wrote:
> > 
> >> While playing with recursion I get:
> >>     RuntimeError: maximum recursion depth exceeded in comparison
> >> 
> >> But then I get a very long stack trace. Is there a way to make this a
> >> lot shorter. Now I 'lose' interesting information because of the length
> >> of the stack trace.
> > 
> > <http://www.catb.org/~esr/faqs/smart-questions.html>
> 
> There ought to be a website that explains how to give smart answers.

Or one that explains how to be kind to those that don't know
the ropes yet.  Though pointing impatient experts at a
website probably won't get them to change any more than
brusquely pointing newbs at a website will.

--Ned.


From rosuav at gmail.com  Fri May 15 21:26:12 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 16 May 2015 11:26:12 +1000
Subject: pip grabs tar.gz file instead of whl?
In-Reply-To: <mj5mpp$t1g$1@ger.gmane.org>
References: <mj4qsn$k5m$1@ger.gmane.org> <55563EBA.8080606@stoneleaf.us>
 <CAPTjJmpku_aPPnNq4auYhOy=Y-hJSZUErQPcjizLMpwCvmf0oQ@mail.gmail.com>
 <mj5mpp$t1g$1@ger.gmane.org>
Message-ID: <CAPTjJmpndhpUnJZ_HxjiVFG_A8W4nC2RJYUmrEC4q0C9RAPZdQ@mail.gmail.com>

On Sat, May 16, 2015 at 7:00 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> The same file installed perfectly when downloaded and installed in two
> steps.  Whether this is simply a known bug with zipfile handling, pip
> itself, a combination of both or what I've no idea.
>
> I find it slightly irritating that a tool recommended by the Python
> Packaging Authority behaves in such a way, but then I didn't have to dip
> into my pocket to pay for it and certainly won't lose any sleep over it as
> there's such a simple work around.

The way I see it, pip is great for handling the most common case where
you just want to name a package and say "go fetch", but if you want to
override its decisions, you should use the lower-level facilities eg
manual downloading and setup.py. It's like with Debian packages: I can
type "sudo apt-get install blah" and it'll run off and grab it, check
its signatures, make sure everything's right, and then install it; but
if I want to install something from a different location, the best way
is usually to download it manually, do my own checking, and then "sudo
dpkg -i blah.deb" to actually install it - no apt-get involvement at
all. This shouldn't normally be a problem; you don't *have* to use pip
here, you just want to end up with the package properly installed.

ChrisA


From rosuav at gmail.com  Fri May 15 21:33:10 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 16 May 2015 11:33:10 +1000
Subject: Minimising stack trace
In-Reply-To: <9cbf3b8e-e830-46f5-a636-27c0d07f460a@googlegroups.com>
References: <871tih66z5.fsf@Equus.decebal.nl>
 <1585804.vCLx6kbzfH@PointedEars.de>
 <55569607$0$13008$c3e8da3$5496439d@news.astraweb.com>
 <9cbf3b8e-e830-46f5-a636-27c0d07f460a@googlegroups.com>
Message-ID: <CAPTjJmrvvZ7Ck9fnv8iASn_DdWFKszuo+0f3jWCo_bnCiMGJtg@mail.gmail.com>

On Sat, May 16, 2015 at 11:24 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> On Friday, May 15, 2015 at 8:57:53 PM UTC-4, Steven D'Aprano wrote:
>> On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote:
>>
>> > Cecil Westerhof wrote:
>> >
>> >> While playing with recursion I get:
>> >>     RuntimeError: maximum recursion depth exceeded in comparison
>> >>
>> >> But then I get a very long stack trace. Is there a way to make this a
>> >> lot shorter. Now I 'lose' interesting information because of the length
>> >> of the stack trace.
>> >
>> > <http://www.catb.org/~esr/faqs/smart-questions.html>
>>
>> There ought to be a website that explains how to give smart answers.
>
> Or one that explains how to be kind to those that don't know
> the ropes yet.  Though pointing impatient experts at a
> website probably won't get them to change any more than
> brusquely pointing newbs at a website will.

You mean like this?

http://www.catb.org/~esr/faqs/smart-questions.html#idp64834912

(Not sure whether you knew and were hinting, or didn't know that
that's explicitly mentioned.)

ChrisA


From ned at nedbatchelder.com  Fri May 15 21:37:05 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Fri, 15 May 2015 18:37:05 -0700 (PDT)
Subject: Minimising stack trace
In-Reply-To: <mailman.55.1431739994.17265.python-list@python.org>
References: <871tih66z5.fsf@Equus.decebal.nl>
 <1585804.vCLx6kbzfH@PointedEars.de>
 <55569607$0$13008$c3e8da3$5496439d@news.astraweb.com>
 <9cbf3b8e-e830-46f5-a636-27c0d07f460a@googlegroups.com>
 <mailman.55.1431739994.17265.python-list@python.org>
Message-ID: <b644d3ca-58cc-4dd6-adaa-c9cc73ec39dd@googlegroups.com>

On Friday, May 15, 2015 at 9:33:54 PM UTC-4, Chris Angelico wrote:
> On Sat, May 16, 2015 at 11:24 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> > On Friday, May 15, 2015 at 8:57:53 PM UTC-4, Steven D'Aprano wrote:
> >> On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote:
> >>
> >> > Cecil Westerhof wrote:
> >> >
> >> >> While playing with recursion I get:
> >> >>     RuntimeError: maximum recursion depth exceeded in comparison
> >> >>
> >> >> But then I get a very long stack trace. Is there a way to make this a
> >> >> lot shorter. Now I 'lose' interesting information because of the length
> >> >> of the stack trace.
> >> >
> >> > <http://www.catb.org/~esr/faqs/smart-questions.html>
> >>
> >> There ought to be a website that explains how to give smart answers.
> >
> > Or one that explains how to be kind to those that don't know
> > the ropes yet.  Though pointing impatient experts at a
> > website probably won't get them to change any more than
> > brusquely pointing newbs at a website will.
> 
> You mean like this?
> 
> http://www.catb.org/~esr/faqs/smart-questions.html#idp64834912
> 
> (Not sure whether you knew and were hinting, or didn't know that
> that's explicitly mentioned.)

I didn't realize that was in there. :) It's a good demonstration
that we can all improve in various ways.  Being right isn't the
highest ideal, there are other things to aspire to as well.

--Ned.


From steve+comp.lang.python at pearwood.info  Fri May 15 21:55:49 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sat, 16 May 2015 11:55:49 +1000
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4> <87twvdsbom.fsf@elektro.pacujo.net>
 <mailman.50.1431732526.17265.python-list@python.org>
Message-ID: <5556a3a5$0$12989$c3e8da3$5496439d@news.astraweb.com>

On Sat, 16 May 2015 09:27 am, Mark Lawrence wrote:

> On 15/05/2015 23:44, Marko Rauhamaa wrote:
>> BartC <bc at freeuk.com>:
>>
>>> What /is/ a method lookup? Is it when you have this:
>>>
>>>   A.B()
>>>
>>> and need to find whether the expression A (or its class or type) has a
>>> name B associated with it? (And it then needs to check whether B is
>>> something that can be called.)
>>>
>>> If so, does that have to be done using Python's Dict mechanism? (Ie.
>>> searching for a key 'B' by name and seeing if the object associated
>>> with it is a method. That does not sound efficient.)

It's not as inefficient as you may think. Dicts are hash tables, and hash
tables are a standard computer science data structure for performing very
fast searches at constant (or near constant) time.

The dict is basically an array of pointers to (key, value). To look a name
up in the dict, you hash the string which gives you an index into the
array, then look at that position. If it is blank, you know there is no
match. If it points to a string, you compare that string to your string. If
they are equal, then you have a match. If they aren't equal, you have a
collision, and you have to look elsewhere (details differ) but typically
you don't end up looking in more than one or two positions. So all pretty
fast, and close enough to constant time.

To speed things up even further, I think that the hash value is cached with
the string, so it only needs to be calculated the first time.



>> That is a general feature among high-level programming languages. In
>> Python, it is even more complicated:
>>
>>   * first the object's dict is looked up for the method name
>>
>>   * if the method is not found (it usually isn't), the dict of the
>>     object's class is consulted
>>
>>   * if the method is found (it usually is), a function object is
>>     instantiated that delegates to the class's method and embeds a "self"
>>     reference to the object to the call

It's the other way around. The function object already exists: you created
it by writing `def method(self, *args): ... ` inside the class body. def
always makes a function. It's the *method* object which is created on the
fly, delegating to the function.



>> IOW, two dict lookups plus an object construction for each method call.
>>
>>
>> Marko
>>
> 
> As a picture paints a thousand words is anybody aware of a site or sites
> that show this diagramatically, as I think I and possibly others would
> find it far easier to grasp.

No I'm not aware of any such site, but I can try to make it more obvious
with an example.

Suppose we have a hierarchy of classes, starting from the root of the
hierarchy (object) to a specific instance:

class Animal(object): 
    pass

class Mammal(Animal):
    pass

class Dog(Mammal):
    def bark(self): ...

laddy = Dog()


We then look up a method:

laddy.bark()

In a non-dynamic language like Java, the compiler knows exactly where bark
is defined ("in the Dog class") and can call it directly. In dynamic
languages like Python, the compiler can't be sure that bark hasn't been
shadowed or overridden at runtime, so it has to search for the first match
found. Simplified:

* Does laddy.__dict__ contain the key "bark"? If so, we have a match.

* For each class in the MRO (Method Resolution Order), namely 
  [Dog, Mammal, Animal, object], does the class __dict__ contain the
  key "bark"? If so, we have a match.

* Do any of those classes in the MRO have a __getattr__ method? If
  so, then try calling __getattr__("bark") and see if it returns 
  a match.

* Give up and raise AttributeError.

[Aside: some of the things I haven't covered: __slots__, __getattribute__,
how metaclasses can affect this.]

In the case of laddy.bark, the matching attribute is found as
Dog.__dict__['bark']:

    temp = Dog.__bark__['bark']  # laddy.bark, is a function

At this point, the descriptor protocol applies. You can ignore this part if
you like, and just pretend that laddy.bark returns a method instead of a
function, but if you want to know what actually happens in all it's gory
details, it is something like this (again, simplified):

* Does the attribute have a __get__ method? If not, then we just 
  use the object as-is, with no changes.

* But if it does have a __get__ method, then it is a descriptor and
  we call the __get__ method to get the object we actually use.


Since functions are descriptors, we get this:

    temp = temp.__get__(laddy, Dog)  # returns a method object

and finally we can call the method:

    temp()  # laddy.bark()


None of these individual operations are particularly expensive, nor are
there a lot of them. For a typical instance, the MRO usually contains only
two or three classes, and __dict__ lookups are fast. Nevertheless, even
though each method lookup is individually *almost* as fast as the sort of
pre-compiled all-but-instantaneous access which Java can do, it all adds
up. So in Java, a long chain of dots:

    foo.bar.baz.foobar.spam.eggs.cheese

can be resolved at compile-time, and takes no more time than

    foo.cheese

but in Python's case it has to be resolved at run-time, so if you care about
speed, you should try to avoid long chains of dots in performance critical
loops. E.g. instead of:

    for x in sequence:
        foo.bar.baz.foobar.spam.eggs.cheese(x)

you can write:

    cheese = foo.bar.baz.foobar.spam.eggs.cheese
    for x in sequence:
        cheese(x)



-- 
Steven



From breamoreboy at yahoo.co.uk  Fri May 15 21:58:29 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 16 May 2015 02:58:29 +0100
Subject: pip grabs tar.gz file instead of whl?
In-Reply-To: <CAPTjJmpndhpUnJZ_HxjiVFG_A8W4nC2RJYUmrEC4q0C9RAPZdQ@mail.gmail.com>
References: <mj4qsn$k5m$1@ger.gmane.org> <55563EBA.8080606@stoneleaf.us>
 <CAPTjJmpku_aPPnNq4auYhOy=Y-hJSZUErQPcjizLMpwCvmf0oQ@mail.gmail.com>
 <mj5mpp$t1g$1@ger.gmane.org>
 <CAPTjJmpndhpUnJZ_HxjiVFG_A8W4nC2RJYUmrEC4q0C9RAPZdQ@mail.gmail.com>
Message-ID: <mj6889$s8r$1@ger.gmane.org>

On 16/05/2015 02:26, Chris Angelico wrote:
> On Sat, May 16, 2015 at 7:00 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>> The same file installed perfectly when downloaded and installed in two
>> steps.  Whether this is simply a known bug with zipfile handling, pip
>> itself, a combination of both or what I've no idea.
>>
>> I find it slightly irritating that a tool recommended by the Python
>> Packaging Authority behaves in such a way, but then I didn't have to dip
>> into my pocket to pay for it and certainly won't lose any sleep over it as
>> there's such a simple work around.
>
> The way I see it, pip is great for handling the most common case where
> you just want to name a package and say "go fetch", but if you want to
> override its decisions, you should use the lower-level facilities eg
> manual downloading and setup.py. It's like with Debian packages: I can
> type "sudo apt-get install blah" and it'll run off and grab it, check
> its signatures, make sure everything's right, and then install it; but
> if I want to install something from a different location, the best way
> is usually to download it manually, do my own checking, and then "sudo
> dpkg -i blah.deb" to actually install it - no apt-get involvement at
> all. This shouldn't normally be a problem; you don't *have* to use pip
> here, you just want to end up with the package properly installed.
>
> ChrisA
>

Being on Windows, as I said at the beginning of the thread, the biggest 
problem is that setup.py can't find VS if there is no whl file to 
install.  Hence it is far easier to get the binaries from elsewhere. 
Hopefully this problem will disappear in the future as the whl standard 
becomes prevelant.

As for sudo I always thought that was a type of Japanese wrestling :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Fri May 15 22:15:44 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 16 May 2015 12:15:44 +1000
Subject: Building CPython
In-Reply-To: <5556a3a5$0$12989$c3e8da3$5496439d@news.astraweb.com>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net>
 <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4>
 <87twvdsbom.fsf@elektro.pacujo.net>
 <mailman.50.1431732526.17265.python-list@python.org>
 <5556a3a5$0$12989$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmp358KLTP-cdqNayL3B4+HybgzniyJFAgKesjvpg17_hQ@mail.gmail.com>

On Sat, May 16, 2015 at 11:55 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> but in Python's case it has to be resolved at run-time, so if you care about
> speed, you should try to avoid long chains of dots in performance critical
> loops. E.g. instead of:
>
>     for x in sequence:
>         foo.bar.baz.foobar.spam.eggs.cheese(x)
>
> you can write:
>
>     cheese = foo.bar.baz.foobar.spam.eggs.cheese
>     for x in sequence:
>         cheese(x)

Like all advice, of course, this should not be taken on its own. Some
code tries to avoid long chains of dots by adding wrapper methods:

for x in sequence:
    foo.cheese(x)

where foo.cheese() delegates to self.bar.cheese() and so on down the
line. That, of course, will be far FAR slower, in pretty much any
language (unless the compiler can in-line the code completely, in
which case it's effectively being optimized down to the original
anyway); the dots aren't as bad as you might think :)

Just always remember the one cardinal rule of optimization: MEASURE
FIRST. You have no idea how slow something is until you measure it.

(I'm not addressing my comments to Steven here, who I'm fairly sure is
aware of all of this(!), but it's his post that gave the example that
I'm quoting.)

ChrisA


From breamoreboy at yahoo.co.uk  Fri May 15 22:17:27 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 16 May 2015 03:17:27 +0100
Subject: Building CPython
In-Reply-To: <5556a3a5$0$12989$c3e8da3$5496439d@news.astraweb.com>
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4> <87twvdsbom.fsf@elektro.pacujo.net>
 <mailman.50.1431732526.17265.python-list@python.org>
 <5556a3a5$0$12989$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mj69bq$cud$2@ger.gmane.org>

On 16/05/2015 02:55, Steven D'Aprano wrote:
> On Sat, 16 May 2015 09:27 am, Mark Lawrence wrote:
>
>> On 15/05/2015 23:44, Marko Rauhamaa wrote:
>>> BartC <bc at freeuk.com>:
>>>
>>>> What /is/ a method lookup? Is it when you have this:
>>>>
>>>>    A.B()
>>>>
>>>> and need to find whether the expression A (or its class or type) has a
>>>> name B associated with it? (And it then needs to check whether B is
>>>> something that can be called.)
>>>>
>>>> If so, does that have to be done using Python's Dict mechanism? (Ie.
>>>> searching for a key 'B' by name and seeing if the object associated
>>>> with it is a method. That does not sound efficient.)
>
> It's not as inefficient as you may think. Dicts are hash tables, and hash
> tables are a standard computer science data structure for performing very
> fast searches at constant (or near constant) time.
>
> The dict is basically an array of pointers to (key, value). To look a name
> up in the dict, you hash the string which gives you an index into the
> array, then look at that position. If it is blank, you know there is no
> match. If it points to a string, you compare that string to your string. If
> they are equal, then you have a match. If they aren't equal, you have a
> collision, and you have to look elsewhere (details differ) but typically
> you don't end up looking in more than one or two positions. So all pretty
> fast, and close enough to constant time.
>
> To speed things up even further, I think that the hash value is cached with
> the string, so it only needs to be calculated the first time.
>
>
>
>>> That is a general feature among high-level programming languages. In
>>> Python, it is even more complicated:
>>>
>>>    * first the object's dict is looked up for the method name
>>>
>>>    * if the method is not found (it usually isn't), the dict of the
>>>      object's class is consulted
>>>
>>>    * if the method is found (it usually is), a function object is
>>>      instantiated that delegates to the class's method and embeds a "self"
>>>      reference to the object to the call
>
> It's the other way around. The function object already exists: you created
> it by writing `def method(self, *args): ... ` inside the class body. def
> always makes a function. It's the *method* object which is created on the
> fly, delegating to the function.
>
>
>
>>> IOW, two dict lookups plus an object construction for each method call.
>>>
>>>
>>> Marko
>>>
>>
>> As a picture paints a thousand words is anybody aware of a site or sites
>> that show this diagramatically, as I think I and possibly others would
>> find it far easier to grasp.
>
> No I'm not aware of any such site, but I can try to make it more obvious
> with an example.
>
> Suppose we have a hierarchy of classes, starting from the root of the
> hierarchy (object) to a specific instance:
>
> class Animal(object):
>      pass
>
> class Mammal(Animal):
>      pass
>
> class Dog(Mammal):
>      def bark(self): ...
>
> laddy = Dog()
>
>
> We then look up a method:
>
> laddy.bark()
>
> In a non-dynamic language like Java, the compiler knows exactly where bark
> is defined ("in the Dog class") and can call it directly. In dynamic
> languages like Python, the compiler can't be sure that bark hasn't been
> shadowed or overridden at runtime, so it has to search for the first match
> found. Simplified:
>
> * Does laddy.__dict__ contain the key "bark"? If so, we have a match.
>
> * For each class in the MRO (Method Resolution Order), namely
>    [Dog, Mammal, Animal, object], does the class __dict__ contain the
>    key "bark"? If so, we have a match.
>
> * Do any of those classes in the MRO have a __getattr__ method? If
>    so, then try calling __getattr__("bark") and see if it returns
>    a match.
>
> * Give up and raise AttributeError.
>
> [Aside: some of the things I haven't covered: __slots__, __getattribute__,
> how metaclasses can affect this.]
>
> In the case of laddy.bark, the matching attribute is found as
> Dog.__dict__['bark']:
>
>      temp = Dog.__bark__['bark']  # laddy.bark, is a function
>
> At this point, the descriptor protocol applies. You can ignore this part if
> you like, and just pretend that laddy.bark returns a method instead of a
> function, but if you want to know what actually happens in all it's gory
> details, it is something like this (again, simplified):
>
> * Does the attribute have a __get__ method? If not, then we just
>    use the object as-is, with no changes.
>
> * But if it does have a __get__ method, then it is a descriptor and
>    we call the __get__ method to get the object we actually use.
>
>
> Since functions are descriptors, we get this:
>
>      temp = temp.__get__(laddy, Dog)  # returns a method object
>
> and finally we can call the method:
>
>      temp()  # laddy.bark()
>
>
> None of these individual operations are particularly expensive, nor are
> there a lot of them. For a typical instance, the MRO usually contains only
> two or three classes, and __dict__ lookups are fast. Nevertheless, even
> though each method lookup is individually *almost* as fast as the sort of
> pre-compiled all-but-instantaneous access which Java can do, it all adds
> up. So in Java, a long chain of dots:
>
>      foo.bar.baz.foobar.spam.eggs.cheese
>
> can be resolved at compile-time, and takes no more time than
>
>      foo.cheese
>
> but in Python's case it has to be resolved at run-time, so if you care about
> speed, you should try to avoid long chains of dots in performance critical
> loops. E.g. instead of:
>
>      for x in sequence:
>          foo.bar.baz.foobar.spam.eggs.cheese(x)
>
> you can write:
>
>      cheese = foo.bar.baz.foobar.spam.eggs.cheese
>      for x in sequence:
>          cheese(x)
>

Thanks for taking the time over this, I'll study it later today after 
hopefully a good night's sleep :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Fri May 15 22:17:50 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 16 May 2015 12:17:50 +1000
Subject: pip grabs tar.gz file instead of whl?
In-Reply-To: <mj6889$s8r$1@ger.gmane.org>
References: <mj4qsn$k5m$1@ger.gmane.org> <55563EBA.8080606@stoneleaf.us>
 <CAPTjJmpku_aPPnNq4auYhOy=Y-hJSZUErQPcjizLMpwCvmf0oQ@mail.gmail.com>
 <mj5mpp$t1g$1@ger.gmane.org>
 <CAPTjJmpndhpUnJZ_HxjiVFG_A8W4nC2RJYUmrEC4q0C9RAPZdQ@mail.gmail.com>
 <mj6889$s8r$1@ger.gmane.org>
Message-ID: <CAPTjJmojOqYqagyR4E8siwUsBf7nZcZpwNgxUckP1toXKgmtBA@mail.gmail.com>

On Sat, May 16, 2015 at 11:58 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>> The way I see it, pip is great for handling the most common case where
>> you just want to name a package and say "go fetch", but if you want to
>> override its decisions, you should use the lower-level facilities eg
>> manual downloading and setup.py. It's like with Debian packages: I can
>> type "sudo apt-get install blah" and it'll run off and grab it, check
>> its signatures, make sure everything's right, and then install it; but
>> if I want to install something from a different location, the best way
>> is usually to download it manually, do my own checking, and then "sudo
>> dpkg -i blah.deb" to actually install it - no apt-get involvement at
>> all. This shouldn't normally be a problem; you don't *have* to use pip
>> here, you just want to end up with the package properly installed.
>>
>> ChrisA
>>
>
> Being on Windows, as I said at the beginning of the thread, the biggest
> problem is that setup.py can't find VS if there is no whl file to install.
> Hence it is far easier to get the binaries from elsewhere. Hopefully this
> problem will disappear in the future as the whl standard becomes prevelant.
>
> As for sudo I always thought that was a type of Japanese wrestling :)

That'll be sumo :) sudo is "su do this", and it's like using "su",
then doing something, and then dropping out again. (su gives you a new
prompt as superuser, or as some other user.)

I don't know what the exact installation steps are for a whl, which is
why I mentioned setup.py. Whatever those lower-level facilities are,
those are what you'd use once you decide to skip pip and do your own
downloading.

ChrisA


From rustompmody at gmail.com  Fri May 15 22:51:51 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Fri, 15 May 2015 19:51:51 -0700 (PDT)
Subject: Minimising stack trace
In-Reply-To: <9cbf3b8e-e830-46f5-a636-27c0d07f460a@googlegroups.com>
References: <871tih66z5.fsf@Equus.decebal.nl>
 <1585804.vCLx6kbzfH@PointedEars.de>
 <55569607$0$13008$c3e8da3$5496439d@news.astraweb.com>
 <9cbf3b8e-e830-46f5-a636-27c0d07f460a@googlegroups.com>
Message-ID: <5581b381-8c45-4aa1-b037-32aee59f4463@googlegroups.com>

On Saturday, May 16, 2015 at 6:54:23 AM UTC+5:30, Ned Batchelder wrote:
> On Friday, May 15, 2015 at 8:57:53 PM UTC-4, Steven D'Aprano wrote:
> > On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote:
> > 
> > > Cecil Westerhof wrote:
> > > 
> > >> While playing with recursion I get:
> > >>     RuntimeError: maximum recursion depth exceeded in comparison
> > >> 
> > >> But then I get a very long stack trace. Is there a way to make this a
> > >> lot shorter. Now I 'lose' interesting information because of the length
> > >> of the stack trace.
> > > 
> > > <http://www.catb.org/~esr/faqs/smart-questions.html>
> > 
> > There ought to be a website that explains how to give smart answers.
> 
> Or one that explains how to be kind to those that don't know
> the ropes yet.  Though pointing impatient experts at a
> website probably won't get them to change any more than
> brusquely pointing newbs at a website will.

How about a 'answerer-quota' ? ?

? Once for every 20 useful answers you are allowed to vent on any subject of 
  your choice...

  - Stupid questions (and point to catb's smart questions)
  - Google groups
  - Top posting
  - Html mail

?


From rustompmody at gmail.com  Fri May 15 22:59:36 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Fri, 15 May 2015 19:59:36 -0700 (PDT)
Subject: Minimising stack trace
In-Reply-To: <5581b381-8c45-4aa1-b037-32aee59f4463@googlegroups.com>
References: <871tih66z5.fsf@Equus.decebal.nl>
 <1585804.vCLx6kbzfH@PointedEars.de>
 <55569607$0$13008$c3e8da3$5496439d@news.astraweb.com>
 <9cbf3b8e-e830-46f5-a636-27c0d07f460a@googlegroups.com>
 <5581b381-8c45-4aa1-b037-32aee59f4463@googlegroups.com>
Message-ID: <51084a44-3ac1-4c9e-a83d-d75b03625828@googlegroups.com>

On Saturday, May 16, 2015 at 8:22:05 AM UTC+5:30, Rustom Mody wrote:
> On Saturday, May 16, 2015 at 6:54:23 AM UTC+5:30, Ned Batchelder wrote:
> > On Friday, May 15, 2015 at 8:57:53 PM UTC-4, Steven D'Aprano wrote:
> > > On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote:
> > > 
> > > > Cecil Westerhof wrote:
> > > > 
> > > >> While playing with recursion I get:
> > > >>     RuntimeError: maximum recursion depth exceeded in comparison
> > > >> 
> > > >> But then I get a very long stack trace. Is there a way to make this a
> > > >> lot shorter. Now I 'lose' interesting information because of the length
> > > >> of the stack trace.
> > > > 
> > > > <http://www.catb.org/~esr/faqs/smart-questions.html>
> > > 
> > > There ought to be a website that explains how to give smart answers.
> > 
> > Or one that explains how to be kind to those that don't know
> > the ropes yet.  Though pointing impatient experts at a
> > website probably won't get them to change any more than
> > brusquely pointing newbs at a website will.
> 
> How about a 'answerer-quota' ? ?
> 
> ? Once for every 20 useful answers you are allowed to vent on any subject of 
>   your choice...
> 
>   - Stupid questions (and point to catb's smart questions)
>   - Google groups
>   - Top posting
>   - Html mail
> 
> ?

Forgot the most important:   Using print's ?


From dreamingforward at gmail.com  Fri May 15 23:25:50 2015
From: dreamingforward at gmail.com (zipher)
Date: Fri, 15 May 2015 20:25:50 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <mailman.474.1431570976.12865.python-list@python.org>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
 <mailman.449.1431530832.12865.python-list@python.org>
 <71ee2dbb-27a3-45ad-b903-bffca6d8923f@googlegroups.com>
 <5553fd0e$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <mailman.474.1431570976.12865.python-list@python.org>
Message-ID: <4e580ae1-e62d-4619-bd06-07543e1067e0@googlegroups.com>

On Wednesday, May 13, 2015 at 9:36:27 PM UTC-5, Mark Lawrence wrote:
> On 14/05/2015 02:40, Steven D'Aprano wrote:
> > On Thu, 14 May 2015 04:07 am, zipher wrote:
> >
> >>
> >> No, you haven't understood, padawan.  Lambda *is* the function, not it's
> >> definition.  Perhaps you will understand what I mean by that, perhaps you
> >> won't.  It's subtle.
> >
> > Subtle like a kick to the head.
> 
>  From the very first drivel that he posted on python ideas just over two 
> years ago, he's shown that he's incapable of any logical thought 
> relating to computing, in just the same way that the RUE has never 
> posted anything logical about the FSR.  Please can we stop feeding him.

The truth is quite the opposite.  There is something very strange going on around the nets.  People are very confused.  Javascript has confused them.   Tell me where is the lexer in Javascript?  or the virtual machine?

The point I'm making about LISP is that in LISP you don't define a function, it IS the function.  Instead of f(x,y) = x+y, which is how you code a traditional program, you only show/program f itself.  In the lambda calculus, it is expressed f = /(x,y).x+y where / is the lambda character, not f(x) or f OF x.  Do you see why the difference is subtle? 

Java has a virtual machine.  Python has a virtual machine.  Do you think Chrome and these browsers include virtual machines to run on 50 different architectures?  I'm actually curious if anyone has a non-magical answer to it.  We all assume that computers can only act logically and have always acted such.  But they way some people act and the way somethings are being done is making questions.  Perhaps I'm the only one.

Mark


From rustompmody at gmail.com  Fri May 15 23:36:29 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Fri, 15 May 2015 20:36:29 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <4e580ae1-e62d-4619-bd06-07543e1067e0@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
 <mailman.449.1431530832.12865.python-list@python.org>
 <71ee2dbb-27a3-45ad-b903-bffca6d8923f@googlegroups.com>
 <5553fd0e$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <mailman.474.1431570976.12865.python-list@python.org>
 <4e580ae1-e62d-4619-bd06-07543e1067e0@googlegroups.com>
Message-ID: <adcfd61e-9c84-4120-89cf-78dff449f6a3@googlegroups.com>

On Saturday, May 16, 2015 at 8:56:00 AM UTC+5:30, zipher wrote:
> On Wednesday, May 13, 2015 at 9:36:27 PM UTC-5, Mark Lawrence wrote:
> > On 14/05/2015 02:40, Steven D'Aprano wrote:
> > > On Thu, 14 May 2015 04:07 am, zipher wrote:
> > >
> > >>
> > >> No, you haven't understood, padawan.  Lambda *is* the function, not it's
> > >> definition.  Perhaps you will understand what I mean by that, perhaps you
> > >> won't.  It's subtle.
> > >
> > > Subtle like a kick to the head.
> > 
> >  From the very first drivel that he posted on python ideas just over two 
> > years ago, he's shown that he's incapable of any logical thought 
> > relating to computing, in just the same way that the RUE has never 
> > posted anything logical about the FSR.  Please can we stop feeding him.
> 
> The truth is quite the opposite.  There is something very strange going on around the nets.  People are very confused.  Javascript has confused them.   Tell me where is the lexer in Javascript?  or the virtual machine?
> 
> The point I'm making about LISP is that in LISP you don't define a function, it IS the function.  Instead of f(x,y) = x+y, which is how you code a traditional program, you only show/program f itself.  In the lambda calculus, it is expressed f = /(x,y).x+y where / is the lambda character, not f(x) or f OF x.  Do you see why the difference is subtle? 
> 
> Java has a virtual machine.  Python has a virtual machine.  Do you think Chrome and these browsers include virtual machines to run on 50 different architectures?  I'm actually curious if anyone has a non-magical answer to it.  We all assume that computers can only act logically and have always acted such.  But they way some people act and the way somethings are being done is making questions.  Perhaps I'm the only one.
> 

As for chrome and its specific VM
http://en.wikipedia.org/wiki/V8_%28JavaScript_engine%29

As for the rest...
We are as confused by your confusions as you are

[Your continual assertions about everyone everywhere being totally confused
reminds me of my first trip to N America where I was unnerved to find everyone
driving on the wrong side of the road]


From breamoreboy at yahoo.co.uk  Fri May 15 23:49:23 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 16 May 2015 04:49:23 +0100
Subject: Minimising stack trace
In-Reply-To: <5581b381-8c45-4aa1-b037-32aee59f4463@googlegroups.com>
References: <871tih66z5.fsf@Equus.decebal.nl>
 <1585804.vCLx6kbzfH@PointedEars.de>
 <55569607$0$13008$c3e8da3$5496439d@news.astraweb.com>
 <9cbf3b8e-e830-46f5-a636-27c0d07f460a@googlegroups.com>
 <5581b381-8c45-4aa1-b037-32aee59f4463@googlegroups.com>
Message-ID: <mj6eo7$ndr$1@ger.gmane.org>

On 16/05/2015 03:51, Rustom Mody wrote:
> On Saturday, May 16, 2015 at 6:54:23 AM UTC+5:30, Ned Batchelder wrote:
>> On Friday, May 15, 2015 at 8:57:53 PM UTC-4, Steven D'Aprano wrote:
>>> On Sat, 16 May 2015 05:41 am, Thomas 'PointedEars' Lahn wrote:
>>>
>>>> Cecil Westerhof wrote:
>>>>
>>>>> While playing with recursion I get:
>>>>>      RuntimeError: maximum recursion depth exceeded in comparison
>>>>>
>>>>> But then I get a very long stack trace. Is there a way to make this a
>>>>> lot shorter. Now I 'lose' interesting information because of the length
>>>>> of the stack trace.
>>>>
>>>> <http://www.catb.org/~esr/faqs/smart-questions.html>
>>>
>>> There ought to be a website that explains how to give smart answers.
>>
>> Or one that explains how to be kind to those that don't know
>> the ropes yet.  Though pointing impatient experts at a
>> website probably won't get them to change any more than
>> brusquely pointing newbs at a website will.
>
> How about a 'answerer-quota' ? ?
>
> ? Once for every 20 useful answers you are allowed to vent on any subject of
>    your choice...
>
>    - Stupid questions (and point to catb's smart questions)
>    - Google groups
>    - Top posting
>    - Html mail
>

Blatant discrimination against people who live in Mudeford, which of 
course straddles the mighty River Mude.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Sat May 16 00:12:20 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 16 May 2015 05:12:20 +0100
Subject: pip grabs tar.gz file instead of whl?
In-Reply-To: <CAPTjJmojOqYqagyR4E8siwUsBf7nZcZpwNgxUckP1toXKgmtBA@mail.gmail.com>
References: <mj4qsn$k5m$1@ger.gmane.org> <55563EBA.8080606@stoneleaf.us>
 <CAPTjJmpku_aPPnNq4auYhOy=Y-hJSZUErQPcjizLMpwCvmf0oQ@mail.gmail.com>
 <mj5mpp$t1g$1@ger.gmane.org>
 <CAPTjJmpndhpUnJZ_HxjiVFG_A8W4nC2RJYUmrEC4q0C9RAPZdQ@mail.gmail.com>
 <mj6889$s8r$1@ger.gmane.org>
 <CAPTjJmojOqYqagyR4E8siwUsBf7nZcZpwNgxUckP1toXKgmtBA@mail.gmail.com>
Message-ID: <mj6g38$b89$1@ger.gmane.org>

On 16/05/2015 03:17, Chris Angelico wrote:
> On Sat, May 16, 2015 at 11:58 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>>> The way I see it, pip is great for handling the most common case where
>>> you just want to name a package and say "go fetch", but if you want to
>>> override its decisions, you should use the lower-level facilities eg
>>> manual downloading and setup.py. It's like with Debian packages: I can
>>> type "sudo apt-get install blah" and it'll run off and grab it, check
>>> its signatures, make sure everything's right, and then install it; but
>>> if I want to install something from a different location, the best way
>>> is usually to download it manually, do my own checking, and then "sudo
>>> dpkg -i blah.deb" to actually install it - no apt-get involvement at
>>> all. This shouldn't normally be a problem; you don't *have* to use pip
>>> here, you just want to end up with the package properly installed.
>>>
>>> ChrisA
>>>
>>
>> Being on Windows, as I said at the beginning of the thread, the biggest
>> problem is that setup.py can't find VS if there is no whl file to install.
>> Hence it is far easier to get the binaries from elsewhere. Hopefully this
>> problem will disappear in the future as the whl standard becomes prevelant.
>>
>
> I don't know what the exact installation steps are for a whl, which is
> why I mentioned setup.py. Whatever those lower-level facilities are,
> those are what you'd use once you decide to skip pip and do your own
> downloading.
>
> ChrisA
>

The whole point is that setup.py never works because it can't find VS 
despite the fact that I know I've got the correct version installed.  If 
I download a whl file, pip installs that version perfectly. If I try to 
get pip to download and install the very same file it gave the zipfile 
error I referred to earlier.  Hopefully all the problems with pip will 
get ironed out, or where do we go, distutils3? :(

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From dreamingforward at gmail.com  Sat May 16 00:24:49 2015
From: dreamingforward at gmail.com (zipher)
Date: Fri, 15 May 2015 21:24:49 -0700 (PDT)
Subject: Instead of deciding between Python or Lisp for a programming
 intro course...What about an intro course that uses *BOTH*? Good idea?
In-Reply-To: <adcfd61e-9c84-4120-89cf-78dff449f6a3@googlegroups.com>
References: <02dba7aa-8466-4937-a8d8-82ffd03e5568@googlegroups.com>
 <87wq0gyvyr.fsf@elektro.pacujo.net>
 <mailman.333.1431308306.12865.python-list@python.org>
 <ed1b5d2c-7d29-4102-b310-ee544cf033a3@googlegroups.com>
 <55515f9d$0$12987$c3e8da3$5496439d@news.astraweb.com>
 <569169cf-d232-48c0-bd49-91090e9c0ddb@googlegroups.com>
 <mitbph$2sq$3@dont-email.me>
 <complexity-20150512191339@ram.dialup.fu-berlin.de>
 <d67fa8aa-6ea8-4d22-b8bf-69d6a23392ed@googlegroups.com>
 <1fdbee0a-b425-44ff-b706-b6b407358bf1@googlegroups.com>
 <5552b74e$0$12985$c3e8da3$5496439d@news.astraweb.com>
 <a5a1345d-04e3-478e-abbd-36138ef200fd@googlegroups.com>
 <5a982567-8d89-42b0-9f47-956e61a21274@googlegroups.com>
 <mailman.449.1431530832.12865.python-list@python.org>
 <71ee2dbb-27a3-45ad-b903-bffca6d8923f@googlegroups.com>
 <5553fd0e$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <mailman.474.1431570976.12865.python-list@python.org>
 <4e580ae1-e62d-4619-bd06-07543e1067e0@googlegroups.com>
 <adcfd61e-9c84-4120-89cf-78dff449f6a3@googlegroups.com>
Message-ID: <16619215-1a32-49ae-962d-d5846d31702d@googlegroups.com>

On Friday, May 15, 2015 at 10:36:45 PM UTC-5, Rustom Mody wrote:
> On Saturday, May 16, 2015 at 8:56:00 AM UTC+5:30, zipher wrote:
> > On Wednesday, May 13, 2015 at 9:36:27 PM UTC-5, Mark Lawrence wrote:
> > > On 14/05/2015 02:40, Steven D'Aprano wrote:
> > > > On Thu, 14 May 2015 04:07 am, zipher wrote:
> > > >
> > > >>
> > > >> No, you haven't understood, padawan.  Lambda *is* the function, not it's
> > > >> definition.  Perhaps you will understand what I mean by that, perhaps you
> > > >> won't.  It's subtle.
> > > >
> > > > Subtle like a kick to the head.
> > > 
> > >  From the very first drivel that he posted on python ideas just over two 
> > > years ago, he's shown that he's incapable of any logical thought 
> > > relating to computing, in just the same way that the RUE has never 
> > > posted anything logical about the FSR.  Please can we stop feeding him.
> > 
> > The truth is quite the opposite.  There is something very strange going on around the nets.  People are very confused.  Javascript has confused them.   Tell me where is the lexer in Javascript?  or the virtual machine?
> > 
> > The point I'm making about LISP is that in LISP you don't define a function, it IS the function.  Instead of f(x,y) = x+y, which is how you code a traditional program, you only show/program f itself.  In the lambda calculus, it is expressed f = /(x,y).x+y where / is the lambda character, not f(x) or f OF x.  Do you see why the difference is subtle? 
> > 
> > Java has a virtual machine.  Python has a virtual machine.  Do you think Chrome and these browsers include virtual machines to run on 50 different architectures?  I'm actually curious if anyone has a non-magical answer to it.  We all assume that computers can only act logically and have always acted such.  But they way some people act and the way somethings are being done is making questions.  Perhaps I'm the only one.
> > 
> 
> As for chrome and its specific VM
> http://en.wikipedia.org/wiki/V8_%28JavaScript_engine%29
> 
> As for the rest...
> We are as confused by your confusions as you are
> 
> [Your continual assertions about everyone everywhere being totally confused
> reminds me of my first trip to N America where I was unnerved to find everyone
> driving on the wrong side of the road]

Thanks for that note, actually, it's quite apropos.  I'm a space cowboy who's trying to tell people what I've seen from "outside the fishbowl".  To those in the fishbowl, they don't think that they're in water, because they've never been outside of it--the concept doesn't exist.  But I left the orthodoxy (without throwing it away) and saw from the outside.  Zen is one of the nodal points of that outside.  And from that view, I can tell you that Python has abandoned much of it`s Zen to the point that is now only held by the programmers who still believe in it like some lost ancient tablet that they keep in their pocket.

It can come back easily, though.  One conversation in person with a whiteboard would do it.  I'm starting to see why I'm seen as such an iconoclast:  I've been outside of it for a long time and have taken it for granted -- I couldn't see my own "water".  Oh well.  Maybe the relationship will be mended with the community, or maybe not.  My psychological states go from one extreme to another, so that's my "thing" in case anyone was wondering what my issue is--it's happened ever since getting tazered.  It sucked.

Mark


From tjreedy at udel.edu  Sat May 16 00:28:48 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 16 May 2015 00:28:48 -0400
Subject: pip grabs tar.gz file instead of whl?
In-Reply-To: <mj6g38$b89$1@ger.gmane.org>
References: <mj4qsn$k5m$1@ger.gmane.org> <55563EBA.8080606@stoneleaf.us>
 <CAPTjJmpku_aPPnNq4auYhOy=Y-hJSZUErQPcjizLMpwCvmf0oQ@mail.gmail.com>
 <mj5mpp$t1g$1@ger.gmane.org>
 <CAPTjJmpndhpUnJZ_HxjiVFG_A8W4nC2RJYUmrEC4q0C9RAPZdQ@mail.gmail.com>
 <mj6889$s8r$1@ger.gmane.org>
 <CAPTjJmojOqYqagyR4E8siwUsBf7nZcZpwNgxUckP1toXKgmtBA@mail.gmail.com>
 <mj6g38$b89$1@ger.gmane.org>
Message-ID: <mj6h2a$g1m$1@ger.gmane.org>

On 5/16/2015 12:12 AM, Mark Lawrence wrote:

> The whole point is that setup.py never works because it can't find VS
> despite the fact that I know I've got the correct version installed.  If
> I download a whl file, pip installs that version perfectly. If I try to
> get pip to download and install the very same file it gave the zipfile
> error I referred to earlier.  Hopefully all the problems with pip will
> get ironed out, or where do we go, distutils3? :(

Unlike distutils, pip is being actively maintained and improved.  It 
must have a comm channel for bug reports and suggestions.

-- 
Terry Jan Reedy



From zljubisicmob at gmail.com  Sat May 16 00:38:10 2015
From: zljubisicmob at gmail.com (zljubisicmob at gmail.com)
Date: Fri, 15 May 2015 21:38:10 -0700 (PDT)
Subject: How to deploy a custom common module?
Message-ID: <05d4eec1-1590-41ae-9315-7bb3cd2b22ee@googlegroups.com>

While working on one python script (test.py), I developed some functions that I will probably need in my future projects, so I decided to put such functions in another python file (cmn_funcs.py).

So in my test.py there is import cmn_funcs in order to use common functions and everything works well.

When I deploy test.py on another computer, I put (rsync) both test.py and cmn_funcs.py in the same remote directory.

If I create another python project (test2.py) in new directory, that needs common functions, what should I do with cmn_funcs.py?

Cmn_funcs.py should be in some shared directory accessible by my every current and future python project.

Furthermore, I would like to preserve simplicity of using rsync for upgrading these three python files on remote computers. 

How to deploy custom a custom common module?
Where to put it?
How to use it?
How to upgrade it?


From breamoreboy at yahoo.co.uk  Sat May 16 00:56:43 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 16 May 2015 05:56:43 +0100
Subject: pip grabs tar.gz file instead of whl?
In-Reply-To: <mj6h2a$g1m$1@ger.gmane.org>
References: <mj4qsn$k5m$1@ger.gmane.org> <55563EBA.8080606@stoneleaf.us>
 <CAPTjJmpku_aPPnNq4auYhOy=Y-hJSZUErQPcjizLMpwCvmf0oQ@mail.gmail.com>
 <mj5mpp$t1g$1@ger.gmane.org>
 <CAPTjJmpndhpUnJZ_HxjiVFG_A8W4nC2RJYUmrEC4q0C9RAPZdQ@mail.gmail.com>
 <mj6889$s8r$1@ger.gmane.org>
 <CAPTjJmojOqYqagyR4E8siwUsBf7nZcZpwNgxUckP1toXKgmtBA@mail.gmail.com>
 <mj6g38$b89$1@ger.gmane.org> <mj6h2a$g1m$1@ger.gmane.org>
Message-ID: <mj6ime$fg5$1@ger.gmane.org>

On 16/05/2015 05:28, Terry Reedy wrote:
> On 5/16/2015 12:12 AM, Mark Lawrence wrote:
>
>> The whole point is that setup.py never works because it can't find VS
>> despite the fact that I know I've got the correct version installed.  If
>> I download a whl file, pip installs that version perfectly. If I try to
>> get pip to download and install the very same file it gave the zipfile
>> error I referred to earlier.  Hopefully all the problems with pip will
>> get ironed out, or where do we go, distutils3? :(
>
> Unlike distutils, pip is being actively maintained and improved.  It
> must have a comm channel for bug reports and suggestions.
>

https://groups.google.com/forum/#!forum/python-virtualenv

or

gmane.comp.python.virtualenv

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From Cecil at decebal.nl  Sat May 16 03:22:41 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 16 May 2015 09:22:41 +0200
Subject: Minimising stack trace
References: <871tih66z5.fsf@Equus.decebal.nl>
 <2c2b0f1d-ce16-41b4-a12d-1cface2bf154@googlegroups.com>
Message-ID: <878ucpf0la.fsf@Equus.decebal.nl>

Op Friday 15 May 2015 21:04 CEST schreef Ned Batchelder:

> On Friday, May 15, 2015 at 2:50:12 PM UTC-4, Cecil Westerhof wrote:
>> While playing with recursion I get:
>> RuntimeError: maximum recursion depth exceeded in comparison
>>
>> But then I get a very long stack trace. Is there a way to make this
>> a lot shorter. Now I 'lose' interesting information because of the
>> length of the stack trace.
>
> There isn't a way to shorten the stack trace.  If you are losing
> information at the top because of your terminal window, you can
> likely increase the number of lines it will keep for you.

Well, I am not really losing information, but it happens in a script
with some output. One of the things I output is information about how
deep I am going. If I then get a stack trace of a 1000 lines that is
not very helpful. Especially because except the first and last every
message is the same. What would be a lot more helpful would be
something like:
    RuntimeError                              Traceback (most recent call last)
    /home/cecil/Python/mathDecebal.py in <module>()
        355         for i in range(start, end + 1):
        356             factorial_iter      = factorial_iterative(i)
    --> 357             factorial_recur     = factorial_recursive(i)
        358             factorial_recur_old = factorial_recursive_old(i)
        359             factorial_tail      = factorial_tail_recursion(i)

    /home/cecil/Python/mathDecebal.py in factorial_recursive(x, y, z)
         51     if x < 2:
         52         return y
    ---> 53     return y if z > x else factorial_recursive(x, z * y, z + 1)
         54 
         55 def factorial_recursive_old(x, y = 1):

    Last call repeated 153 times

    /home/cecil/Python/mathDecebal.py in factorial_recursive(x, y, z)
         48 
         49 def factorial_recursive(x, y = 1, z = 1):
    ---> 50     assert x >= 0
         51     if x < 2:
         52         return y

    RuntimeError: maximum recursion depth exceeded in comparison

I would find that a lot clearer and I do not think you are losing
anything useful.


> Another option is to reduce the maximum stack depth, so that it
> is exceeded sooner, which produces shorter stack traces:
>
> import sys;
> sys.setrecursionlimit(50)

Well that would break my code. I just got the above problem with my
math functions. I had it tweaked for testing. But when running the
test in ipython3 it goes wrong. It looks like ipython3 has a smaller
stack, or puts more on the stack as ipython, python3 and python2.

Yes, that is correct. When running:
    python3 mathDecebal.py
there is no problem, but when running:
    ipython3 mathDecebal.py
I get a stack overflow.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From marko at pacujo.net  Sat May 16 04:08:25 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 16 May 2015 11:08:25 +0300
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4> <87twvdsbom.fsf@elektro.pacujo.net>
 <bpw5x.561870$Zf4.150532@fx22.am4>
Message-ID: <87egmhrll2.fsf@elektro.pacujo.net>

BartC <bc at freeuk.com>:

> I suppose in many cases an object will have no attributes of its own,
> and so it can rapidly bypass the first lookup.

Almost all objects have quite many instance attributes. That's what
tells objects apart.

> I don't understand the need for an object creation (to represent A.B
> so that it can call it?) but perhaps such an object can already exist,
> prepared ready for use.

Note that almost identical semantics could be achieved without a class.
Thus, these two constructs are almost identical:

    class C:
        def __init__(self, x):
            self.x = x

        def square(self):
            return self.x * self.x

        def cube(self):
            return self.x * self.square()

    ##

    class O: pass

    def C(x):
        o = O()

        def square():
            return x * x

        def cube():
            return x * square()

        o.square = square
        o.cube = cube
        return o


IOW, the class is a virtually superfluous concept in Python. Python has
gotten it probably without much thought (other languages at the time had
it). I comes with advantages and disadvantages:

 + improves readability

 + makes objects slightly smaller

 + makes object instantiation slightly faster

 - goes against the grain of ducktyping

 - makes method calls slower

 - makes method call semantics a bit tricky


Marko


From steve+comp.lang.python at pearwood.info  Sat May 16 05:40:55 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sat, 16 May 2015 19:40:55 +1000
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4> <87twvdsbom.fsf@elektro.pacujo.net>
 <bpw5x.561870$Zf4.150532@fx22.am4> <87egmhrll2.fsf@elektro.pacujo.net>
Message-ID: <555710a8$0$13001$c3e8da3$5496439d@news.astraweb.com>

On Sat, 16 May 2015 06:08 pm, Marko Rauhamaa wrote:

> Note that almost identical semantics could be achieved without a class.
> Thus, these two constructs are almost identical:
[...]

> IOW, the class is a virtually superfluous concept in Python. Python has
> gotten it probably without much thought (other languages at the time had
> it). I comes with advantages and disadvantages:

Your example is effectively just a way of using closures instead of a class
instance.

Almost anything you can do with classes, you can do with closures. The big
advantage of classes over closures is that you have an interface to access
arbitrary class attributes, while you would need a separate closure for
each and every attribute you want access to.

For example, here is sketch:

class K:
    def method(self, arg):
        return self.spam + arg


k = K()
the_method = k.method  # bound method


as a closure becomes:


def make_closure(instance):  # instance is equivalent to self above
    def method(arg):
        return instance.spam + arg
    return method

the_method = make_closure(obj)  # Some object with a spam field.


The big advantage of a closure is that you have much more strict
encapsulation. The big disadvantage of a closure is that you have much more
strict encapsulation.


>  + improves readability

I wouldn't say that.


>  + makes objects slightly smaller
> 
>  + makes object instantiation slightly faster

Are you sure? Have you actually benchmarked this?

 
>  - goes against the grain of ducktyping
> 
>  - makes method calls slower
> 
>  - makes method call semantics a bit tricky


A couple more negatives:


- no such thing as inheritance;

- "is-a" relationship tests don't work;

- an unfamiliar idiom for most people;



Also, at least with Python's implementation, a couple of mixed blessings:

? closures are closed against modification;

? internals of the closure are strictly private;




-- 
Steven



From bruceg113355 at gmail.com  Sat May 16 09:28:19 2015
From: bruceg113355 at gmail.com (bruceg113355 at gmail.com)
Date: Sat, 16 May 2015 06:28:19 -0700 (PDT)
Subject: Fastest way to remove the first x characters from a very long string
Message-ID: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>

I have a string that contains 10 million characters.

The string is formatted as:

"0000001 : some hexadecimal text ... \n
0000002 : some hexadecimal text ... \n
0000003 : some hexadecimal text ... \n
...
0100000 : some hexadecimal text ... \n
0100001 : some hexadecimal text ... \n"

and I need the string to look like:

"some hexadecimal text ... \n
some hexadecimal text ... \n
some hexadecimal text ... \n
...
some hexadecimal text ... \n
some hexadecimal text ... \n"

I can split the string at the ":" then iterate through the list removing the first 8 characters then convert back to a string. This method works, but it takes too long to execute.

Any tricks to remove the first n characters of each line in a string faster?

Thanks,
Bruce


From joel.goldstick at gmail.com  Sat May 16 09:43:35 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Sat, 16 May 2015 09:43:35 -0400
Subject: Fastest way to remove the first x characters from a very long
 string
In-Reply-To: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
Message-ID: <CAPM-O+wxZ-GMhC-_vgs4Bg2BT1MZef5kQwpc3pAHGhts-M6LHg@mail.gmail.com>

On Sat, May 16, 2015 at 9:28 AM,  <bruceg113355 at gmail.com> wrote:
> I have a string that contains 10 million characters.
>
> The string is formatted as:
>
> "0000001 : some hexadecimal text ... \n
> 0000002 : some hexadecimal text ... \n
> 0000003 : some hexadecimal text ... \n
> ...
> 0100000 : some hexadecimal text ... \n
> 0100001 : some hexadecimal text ... \n"
>
> and I need the string to look like:
>
> "some hexadecimal text ... \n
> some hexadecimal text ... \n
> some hexadecimal text ... \n
> ...
> some hexadecimal text ... \n
> some hexadecimal text ... \n"
>
> I can split the string at the ":" then iterate through the list removing the first 8 characters then convert back to a string. This method works, but it takes too long to execute.
>
> Any tricks to remove the first n characters of each line in a string faster?
>
slicing might be faster than searching for :

Do you need to do this all at once?  If not, use a generator

> Thanks,
> Bruce
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Joel Goldstick
http://joelgoldstick.com


From rosuav at gmail.com  Sat May 16 09:45:57 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 16 May 2015 23:45:57 +1000
Subject: Fastest way to remove the first x characters from a very long
 string
In-Reply-To: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
Message-ID: <CAPTjJmoB94Ontj0sR3FZLUX7e-O0kYRig6pcSC8=1R-CPT1J_A@mail.gmail.com>

On Sat, May 16, 2015 at 11:28 PM,  <bruceg113355 at gmail.com> wrote:
> I have a string that contains 10 million characters.
>
> The string is formatted as:
>
> "0000001 : some hexadecimal text ... \n
> 0000002 : some hexadecimal text ... \n
> 0000003 : some hexadecimal text ... \n
> ...
> 0100000 : some hexadecimal text ... \n
> 0100001 : some hexadecimal text ... \n"
>
> and I need the string to look like:
>
> "some hexadecimal text ... \n
> some hexadecimal text ... \n
> some hexadecimal text ... \n
> ...
> some hexadecimal text ... \n
> some hexadecimal text ... \n"
>
> I can split the string at the ":" then iterate through the list removing the first 8 characters then convert back to a string. This method works, but it takes too long to execute.
>
> Any tricks to remove the first n characters of each line in a string faster?

Given that your definition is "each line", what I'd advise is first
splitting the string into lines, then changing each line, and then
rejoining them into a single string.

lines = original_text.split("\n")
new_text = "\n".join(line[8:] for line in lines)

Would that work?

ChrisA


From Cecil at decebal.nl  Sat May 16 09:49:42 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 16 May 2015 15:49:42 +0200
Subject: Minimising stack trace
References: <871tih66z5.fsf@Equus.decebal.nl>
Message-ID: <87382wfx8p.fsf@Equus.decebal.nl>

Op Friday 15 May 2015 20:17 CEST schreef Cecil Westerhof:

> While playing with recursion I get:
> RuntimeError: maximum recursion depth exceeded in comparison
>
> But then I get a very long stack trace. Is there a way to make this
> a lot shorter. Now I ?lose? interesting information because of the
> length of the stack trace.

I found something. I could use:
    import sys

    sys.tracebacklimit = 10

The output is different, but I find it useful enough.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From marko at pacujo.net  Sat May 16 09:59:26 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 16 May 2015 16:59:26 +0300
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4> <87twvdsbom.fsf@elektro.pacujo.net>
 <bpw5x.561870$Zf4.150532@fx22.am4> <87egmhrll2.fsf@elektro.pacujo.net>
 <555710a8$0$13001$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <878ucosjwh.fsf@elektro.pacujo.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info>:

> A couple more negatives:
>
> - no such thing as inheritance;

Untrue. My simple Scheme object system (125 lines incl. documentation)
supports multiple inheritance without classes. Maybe I should port that
to Python...

> - "is-a" relationship tests don't work;

>From the ducktyping point of view, that is an advantage. The whole
Linnaean categorization of objects is unnecessary ontological chaff. How
many times have people here had to advise newcomers not to inspect type
and instance relations of objects and just call the method?

> - an unfamiliar idiom for most people;

That's impossible to ascertain objectively. Java and JavaScript
programmers (of all people!) routinely deal with closures.


Marko


From bruceg113355 at gmail.com  Sat May 16 10:02:25 2015
From: bruceg113355 at gmail.com (bruceg113355 at gmail.com)
Date: Sat, 16 May 2015 07:02:25 -0700 (PDT)
Subject: Fastest way to remove the first x characters from a very long
 string
In-Reply-To: <mailman.72.1431783967.17265.python-list@python.org>
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
 <mailman.72.1431783967.17265.python-list@python.org>
Message-ID: <4272d9d9-3d5b-4b8b-9875-6b66634b490c@googlegroups.com>

On Saturday, May 16, 2015 at 9:46:17 AM UTC-4, Chris Angelico wrote:
> On Sat, May 16, 2015 at 11:28 PM,  <bruceg113355 at gmail.com> wrote:
> > I have a string that contains 10 million characters.
> >
> > The string is formatted as:
> >
> > "0000001 : some hexadecimal text ... \n
> > 0000002 : some hexadecimal text ... \n
> > 0000003 : some hexadecimal text ... \n
> > ...
> > 0100000 : some hexadecimal text ... \n
> > 0100001 : some hexadecimal text ... \n"
> >
> > and I need the string to look like:
> >
> > "some hexadecimal text ... \n
> > some hexadecimal text ... \n
> > some hexadecimal text ... \n
> > ...
> > some hexadecimal text ... \n
> > some hexadecimal text ... \n"
> >
> > I can split the string at the ":" then iterate through the list removing the first 8 characters then convert back to a string. This method works, but it takes too long to execute.
> >
> > Any tricks to remove the first n characters of each line in a string faster?
> 
> Given that your definition is "each line", what I'd advise is first
> splitting the string into lines, then changing each line, and then
> rejoining them into a single string.
> 
> lines = original_text.split("\n")
> new_text = "\n".join(line[8:] for line in lines)
> 
> Would that work?
> 
> ChrisA


Hi Chris,

I meant to say I can split the string at the \n.

Your approach using .join is what I was looking for.
Thank you,

Bruce


From mal at europython.eu  Sat May 16 10:36:35 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Sat, 16 May 2015 16:36:35 +0200
Subject: EuroPython 2015: Come with your partners
Message-ID: <555755F3.3090204@europython.eu>

We are happy to announce the official EuroPython Partner Program for
EuroPython 2015 in Bilbao:

              *** EuroPython 2015 Partner Program ***

      https://ep2015.europython.eu/en/events/partner-program/

There is plenty to see in and around Bilbao. We have worked out a
set of interesting tours, together with a local tour company to choose
from, for partners and EuroPython attendees alike:

 * Panoramic tour of Bilbao
 * Excursion to San Sebastian
 * Guggenheim Museum
 * Excursion to La Rioja
 * Boat tour of Bilbao

The tours include travel, lunch and tickets (where needed).

We?d like to encourage early sign-up, since seats are limited.

                Registration deadline is June 20th.

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From invalid at invalid.invalid  Sat May 16 10:59:53 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Sat, 16 May 2015 14:59:53 +0000 (UTC)
Subject: Fastest way to remove the first x characters from a very long
 string
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
Message-ID: <mj7m19$qn$1@reader1.panix.com>

On 2015-05-16, bruceg113355 at gmail.com <bruceg113355 at gmail.com> wrote:

> I have a string that contains 10 million characters.
>
> The string is formatted as:
>
> "0000001 : some hexadecimal text ... \n
> 0000002 : some hexadecimal text ... \n
> 0000003 : some hexadecimal text ... \n
> ...
> 0100000 : some hexadecimal text ... \n
> 0100001 : some hexadecimal text ... \n"
>
> and I need the string to look like:
>
> "some hexadecimal text ... \n
> some hexadecimal text ... \n
> some hexadecimal text ... \n
> ...
> some hexadecimal text ... \n
> some hexadecimal text ... \n"
>
> I can split the string at the ":" then iterate through the list
> removing the first 8 characters then convert back to a string. This
> method works, but it takes too long to execute.
>
> Any tricks to remove the first n characters of each line in a string faster?

Well, if the strings are all in a file, I'd probably just use sed:

$ sed 's/^........//g' file1.txt >file2.txt

or

$ sed 's/^.*://g' file1.txt >file2.txt


From rustompmody at gmail.com  Sat May 16 11:13:31 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sat, 16 May 2015 08:13:31 -0700 (PDT)
Subject: Fastest way to remove the first x characters from a very long
 string
In-Reply-To: <mj7m19$qn$1@reader1.panix.com>
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
 <mj7m19$qn$1@reader1.panix.com>
Message-ID: <3faf260f-7777-4a60-8212-981340e478b3@googlegroups.com>

On Saturday, May 16, 2015 at 8:30:02 PM UTC+5:30, Grant Edwards wrote:
> On 2015-05-16, bruceg113355 wrote:
> 
> > I have a string that contains 10 million characters.
> >
> > The string is formatted as:
> >
> > "0000001 : some hexadecimal text ... \n
> > 0000002 : some hexadecimal text ... \n
> > 0000003 : some hexadecimal text ... \n
> > ...
> > 0100000 : some hexadecimal text ... \n
> > 0100001 : some hexadecimal text ... \n"
> >
> > and I need the string to look like:
> >
> > "some hexadecimal text ... \n
> > some hexadecimal text ... \n
> > some hexadecimal text ... \n
> > ...
> > some hexadecimal text ... \n
> > some hexadecimal text ... \n"
> >
> > I can split the string at the ":" then iterate through the list
> > removing the first 8 characters then convert back to a string. This
> > method works, but it takes too long to execute.
> >
> > Any tricks to remove the first n characters of each line in a string faster?
> 
> Well, if the strings are all in a file, I'd probably just use sed:
> 
> $ sed 's/^........//g' file1.txt >file2.txt
> 
> or
> 
> $ sed 's/^.*://g' file1.txt >file2.txt


And if they are not in a file you could start by putting them (it) there :-)

Seriously... How does your 'string' come into existence?
How/when do you get hold of it?


From Seymore4Head at Hotmail.invalid  Sat May 16 11:26:04 2015
From: Seymore4Head at Hotmail.invalid (Seymore4Head)
Date: Sat, 16 May 2015 11:26:04 -0400
Subject: Secret code in Ex Machina
Message-ID: <lboela1bmhoiel6a8pnn90m2knagaksi16@4ax.com>

http://www.reddit.com/r/movies/comments/365f9b/secret_code_in_ex_machina/


From bruceg113355 at gmail.com  Sat May 16 12:22:02 2015
From: bruceg113355 at gmail.com (bruceg113355 at gmail.com)
Date: Sat, 16 May 2015 09:22:02 -0700 (PDT)
Subject: Fastest way to remove the first x characters from a very long
 string
In-Reply-To: <measurements-20150516160609@ram.dialup.fu-berlin.de>
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
 <mailman.72.1431783967.17265.python-list@python.org>
 <4272d9d9-3d5b-4b8b-9875-6b66634b490c@googlegroups.com>
 <measurements-20150516160609@ram.dialup.fu-berlin.de>
Message-ID: <f3836ca1-3f87-4c5c-9839-4d8a35aa77e4@googlegroups.com>

On Saturday, May 16, 2015 at 10:06:31 AM UTC-4, Stefan Ram wrote:
> bruceg113355 at gmail.com writes:
> >Your approach using .join is what I was looking for.
> 
>   I'd appreciate a report of your measurements.

# Original Approach
# -----------------
ss = ss.split("\n")
ss1 = ""
for sdata in ss:
    ss1 = ss1 + (sdata[OFFSET:] + "\n")


# Chris's Approach
# ----------------
lines = ss.split("\n")
new_text = "\n".join(line[8:] for line in lines)  


Test #1, Number of Characters: 165110
Original Approach: 18ms
Chris's Approach:   1ms

Test #2, Number of Characters: 470763
Original Approach: 593ms
Chris's Approach:   16ms

Test #3, Number of Characters: 944702
Original Approach: 2.824s
Chris's Approach:    47ms

Test #4, Number of Characters: 5557394
Original Approach: 122s
Chris's Approach:   394ms


From bruceg113355 at gmail.com  Sat May 16 12:24:23 2015
From: bruceg113355 at gmail.com (bruceg113355 at gmail.com)
Date: Sat, 16 May 2015 09:24:23 -0700 (PDT)
Subject: Fastest way to remove the first x characters from a very long
 string
In-Reply-To: <3faf260f-7777-4a60-8212-981340e478b3@googlegroups.com>
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
 <mj7m19$qn$1@reader1.panix.com>
 <3faf260f-7777-4a60-8212-981340e478b3@googlegroups.com>
Message-ID: <5b815f6c-d639-4193-a644-ea2e1a1759a6@googlegroups.com>

On Saturday, May 16, 2015 at 11:13:45 AM UTC-4, Rustom Mody wrote:
> On Saturday, May 16, 2015 at 8:30:02 PM UTC+5:30, Grant Edwards wrote:
> > On 2015-05-16, bruceg113355 wrote:
> > 
> > > I have a string that contains 10 million characters.
> > >
> > > The string is formatted as:
> > >
> > > "0000001 : some hexadecimal text ... \n
> > > 0000002 : some hexadecimal text ... \n
> > > 0000003 : some hexadecimal text ... \n
> > > ...
> > > 0100000 : some hexadecimal text ... \n
> > > 0100001 : some hexadecimal text ... \n"
> > >
> > > and I need the string to look like:
> > >
> > > "some hexadecimal text ... \n
> > > some hexadecimal text ... \n
> > > some hexadecimal text ... \n
> > > ...
> > > some hexadecimal text ... \n
> > > some hexadecimal text ... \n"
> > >
> > > I can split the string at the ":" then iterate through the list
> > > removing the first 8 characters then convert back to a string. This
> > > method works, but it takes too long to execute.
> > >
> > > Any tricks to remove the first n characters of each line in a string faster?
> > 
> > Well, if the strings are all in a file, I'd probably just use sed:
> > 
> > $ sed 's/^........//g' file1.txt >file2.txt
> > 
> > or
> > 
> > $ sed 's/^.*://g' file1.txt >file2.txt
> 
> 
> And if they are not in a file you could start by putting them (it) there :-)
> 
> Seriously... How does your 'string' come into existence?
> How/when do you get hold of it?

Data is coming from a wxPython TextCtrl widget.
The widget is displaying data received on a serial port for a user to analyze.


From irmen.NOSPAM at xs4all.nl  Sat May 16 12:55:34 2015
From: irmen.NOSPAM at xs4all.nl (Irmen de Jong)
Date: Sat, 16 May 2015 18:55:34 +0200
Subject: Fastest way to remove the first x characters from a very long
 string
In-Reply-To: <5b815f6c-d639-4193-a644-ea2e1a1759a6@googlegroups.com>
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
 <mj7m19$qn$1@reader1.panix.com>
 <3faf260f-7777-4a60-8212-981340e478b3@googlegroups.com>
 <5b815f6c-d639-4193-a644-ea2e1a1759a6@googlegroups.com>
Message-ID: <55577688$0$2821$e4fe514c@news.xs4all.nl>

On 16-5-2015 18:24, bruceg113355 at gmail.com wrote:
> Data is coming from a wxPython TextCtrl widget.

Hm, there should be a better source of the data before it ends up in the textctrl widget.

> The widget is displaying data received on a serial port for a user to analyze.

If this is read from a serial port, can't you process the data directly when it arrives?
This may give you the chance to simply operate on the line as soon as it arrives from
the port, before pasting it all in the textctrl



Irmen



From ian.g.kelly at gmail.com  Sat May 16 12:57:23 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 16 May 2015 10:57:23 -0600
Subject: Fastest way to remove the first x characters from a very long
 string
In-Reply-To: <f3836ca1-3f87-4c5c-9839-4d8a35aa77e4@googlegroups.com>
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
 <mailman.72.1431783967.17265.python-list@python.org>
 <4272d9d9-3d5b-4b8b-9875-6b66634b490c@googlegroups.com>
 <measurements-20150516160609@ram.dialup.fu-berlin.de>
 <f3836ca1-3f87-4c5c-9839-4d8a35aa77e4@googlegroups.com>
Message-ID: <CALwzid=Bv7uJ_smvYmQxynObhY_g9gVzh7ZH9QHocHFANjrHsg@mail.gmail.com>

On Sat, May 16, 2015 at 10:22 AM,  <bruceg113355 at gmail.com> wrote:
> # Chris's Approach
> # ----------------
> lines = ss.split("\n")
> new_text = "\n".join(line[8:] for line in lines)

Looks like the approach you have may be fast enough already, but I'd
wager the generator expression could be replaced with:

    map(operator.itemgetter(slice(8, None)), lines)

for a modest speed-up. On the downside, this is less readable.
Substitute itertools.imap for map if using Python 2.x.


From rosuav at gmail.com  Sat May 16 12:59:06 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 17 May 2015 02:59:06 +1000
Subject: Fastest way to remove the first x characters from a very long
 string
In-Reply-To: <f3836ca1-3f87-4c5c-9839-4d8a35aa77e4@googlegroups.com>
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
 <mailman.72.1431783967.17265.python-list@python.org>
 <4272d9d9-3d5b-4b8b-9875-6b66634b490c@googlegroups.com>
 <measurements-20150516160609@ram.dialup.fu-berlin.de>
 <f3836ca1-3f87-4c5c-9839-4d8a35aa77e4@googlegroups.com>
Message-ID: <CAPTjJmqpUZyvYYieAnyLm4=pDz_q_h3rXNObzdLWxbyz855qJQ@mail.gmail.com>

On Sun, May 17, 2015 at 2:22 AM,  <bruceg113355 at gmail.com> wrote:
> # Original Approach
> # -----------------
> ss = ss.split("\n")
> ss1 = ""
> for sdata in ss:
>     ss1 = ss1 + (sdata[OFFSET:] + "\n")
>
>
> # Chris's Approach
> # ----------------
> lines = ss.split("\n")
> new_text = "\n".join(line[8:] for line in lines)

Ah, yep. This is exactly what str.join() exists for :) Though do make
sure the results are the same for each - there are two noteworthy
differences between these two. Your version has a customizable OFFSET,
where mine is hard-coded; I'm sure you know how to change that part.
The subtler one is that "\n".join(...) won't put a \n after the final
string - your version ends up adding one more newline. If that's
important to you, you'll have to add one explicitly. (I suspect
probably not, though; ss.split("\n") won't expect a final newline, so
you'll get a blank entry in the list if there is one, and then you'll
end up reinstating the newline when that blank gets joined in.) Just
remember to check correctness before performance, and you should be
safe.

ChrisA


From bruceg113355 at gmail.com  Sat May 16 13:35:53 2015
From: bruceg113355 at gmail.com (bruceg113355 at gmail.com)
Date: Sat, 16 May 2015 10:35:53 -0700 (PDT)
Subject: Fastest way to remove the first x characters from a very long
 string
In-Reply-To: <mailman.78.1431795548.17265.python-list@python.org>
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
 <mailman.72.1431783967.17265.python-list@python.org>
 <4272d9d9-3d5b-4b8b-9875-6b66634b490c@googlegroups.com>
 <measurements-20150516160609@ram.dialup.fu-berlin.de>
 <f3836ca1-3f87-4c5c-9839-4d8a35aa77e4@googlegroups.com>
 <mailman.78.1431795548.17265.python-list@python.org>
Message-ID: <8354e4ba-0a80-48a1-b2e9-6edb4b67ff36@googlegroups.com>

On Saturday, May 16, 2015 at 12:59:19 PM UTC-4, Chris Angelico wrote:
> On Sun, May 17, 2015 at 2:22 AM,  <bruceg113355 at gmail.com> wrote:
> > # Original Approach
> > # -----------------
> > ss = ss.split("\n")
> > ss1 = ""
> > for sdata in ss:
> >     ss1 = ss1 + (sdata[OFFSET:] + "\n")
> >
> >
> > # Chris's Approach
> > # ----------------
> > lines = ss.split("\n")
> > new_text = "\n".join(line[8:] for line in lines)
> 
> Ah, yep. This is exactly what str.join() exists for :) Though do make
> sure the results are the same for each - there are two noteworthy
> differences between these two. Your version has a customizable OFFSET,
> where mine is hard-coded; I'm sure you know how to change that part.
> The subtler one is that "\n".join(...) won't put a \n after the final
> string - your version ends up adding one more newline. If that's
> important to you, you'll have to add one explicitly. (I suspect
> probably not, though; ss.split("\n") won't expect a final newline, so
> you'll get a blank entry in the list if there is one, and then you'll
> end up reinstating the newline when that blank gets joined in.) Just
> remember to check correctness before performance, and you should be
> safe.
> 
> ChrisA

Hi Chris,

Your approach more than meets my requirements.
Data is formatted correctly and performance is simply amazing. 
OFFSET and \n are small details.

Thank you again,
Bruce



From DLipman~NoSpam~ at Verizon.Net  Sat May 16 14:09:11 2015
From: DLipman~NoSpam~ at Verizon.Net (David H. Lipman)
Date: Sat, 16 May 2015 14:09:11 -0400
Subject: Secret code in Ex Machina
In-Reply-To: <lboela1bmhoiel6a8pnn90m2knagaksi16@4ax.com>
References: <lboela1bmhoiel6a8pnn90m2knagaksi16@4ax.com>
Message-ID: <C5-dnbTSKvlTGsrInZ2dnUU7-amdnZ2d@giganews.com>

From: "Seymore4Head" <Seymore4Head at Hotmail.invalid>

> http://www.reddit.com/r/movies/comments/365f9b/secret_code_in_ex_machina/

LOL - It is like an Easter Egg in a movie.

C O O L  !

-- 
Dave
Multi-AV Scanning Tool - http://multi-av.thespykiller.co.uk
http://www.pctipp.ch/downloads/dl/35905.asp


From steve+comp.lang.python at pearwood.info  Sat May 16 14:18:41 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 17 May 2015 04:18:41 +1000
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4> <87twvdsbom.fsf@elektro.pacujo.net>
 <bpw5x.561870$Zf4.150532@fx22.am4> <87egmhrll2.fsf@elektro.pacujo.net>
 <555710a8$0$13001$c3e8da3$5496439d@news.astraweb.com>
 <878ucosjwh.fsf@elektro.pacujo.net>
Message-ID: <55578a02$0$13003$c3e8da3$5496439d@news.astraweb.com>

On Sat, 16 May 2015 11:59 pm, Marko Rauhamaa wrote:

> Steven D'Aprano <steve+comp.lang.python at pearwood.info>:
> 
>> A couple more negatives:
>>
>> - no such thing as inheritance;
> 
> Untrue. My simple Scheme object system (125 lines incl. documentation)

Ah yes, I've seen Javascript code like that too. Each line is thirty
thousand characters long...

*wink*


> supports multiple inheritance without classes. Maybe I should port that
> to Python...

I'd like to see it, but somehow I don't think that your "Scheme object
system" is another name for "closures". We were talking about closures,
weren't we?

It sounds like you have implemented a form of prototype-based object
programming. The question of whether prototype-OOP has inheritance is an
interesting one. Clearly prototypes implement something *like* inheritance,
but it is based in delegation or copying. Delegation-based cloning is quite
close to class-based inheritance, but copying-based cloning is not.

My sense is that I prefer to say that prototypes don't have inheritance in
the same sense as classes, but they have something that plays the same role
as inheritance. But if you want to call it inheritance, I can't really
argue.



>> - "is-a" relationship tests don't work;
> 
> From the ducktyping point of view, that is an advantage. The whole
> Linnaean categorization of objects is unnecessary ontological chaff. How
> many times have people here had to advise newcomers not to inspect type
> and instance relations of objects and just call the method?

Um, is that a trick question? I don't remember the last time.



>> - an unfamiliar idiom for most people;
> 
> That's impossible to ascertain objectively. Java and JavaScript
> programmers (of all people!) routinely deal with closures.

I don't think so. I can't say for Javascript, but for Java, there's a lot of
confusion around closures, they've been described as "evil" with the
recommendation not to use them, and people cannot even agree when they were
introduced!

http://java.dzone.com/articles/whats-wrong-java-8-currying-vs
www.javaworld.com/javaworld/jw-06-2008/jw-06-closures.html


I mean, people had to *debate* the introduction of closures? There were
three competing proposals for them, plus an argument for "don't add them".
Some people say closures were added in Java 7, others say closures have
been there since the beginning, and James Gosling himself says that Java
used inner classes instead of closures but the result was painful...

It seems to me that in the Java community, there's a lot of confusion over
closures, which are seen as an advanced (and rather scary) feature. Hardly
routine.


-- 
Steven



From marko at pacujo.net  Sat May 16 14:55:55 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 16 May 2015 21:55:55 +0300
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4> <87twvdsbom.fsf@elektro.pacujo.net>
 <bpw5x.561870$Zf4.150532@fx22.am4> <87egmhrll2.fsf@elektro.pacujo.net>
 <555710a8$0$13001$c3e8da3$5496439d@news.astraweb.com>
 <878ucosjwh.fsf@elektro.pacujo.net>
 <55578a02$0$13003$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87zj54qrlw.fsf@elektro.pacujo.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info>:

> On Sat, 16 May 2015 11:59 pm, Marko Rauhamaa wrote:
>> supports multiple inheritance without classes. Maybe I should port that
>> to Python...
>
> I'd like to see it, but somehow I don't think that your "Scheme object
> system" is another name for "closures". We were talking about closures,
> weren't we?

Ok, here's a quick port that have barely tried out:

========================================================================

### Simple OO Framework

class _O: pass

def make_object(*procedures, base=None, bases=None):
    o = _O()
    methods = {}
    setattr(o, '%methods', methods)
    if base is not None:
        inherit_single(o, base)
    elif bases is not None:
        inherit_multi(o, bases)
    for procedure in procedures:
        methods[procedure.__name__] = procedure
        setattr(o, procedure.__name__, procedure)
    return o

def inherit_single(o, base):
    methods = getattr(o, '%methods')
    for name, method in getattr(base, '%methods').items():
        methods[name] = method
        setattr(o, name, method)

def inherit_multi(o, bases):
    for base in bases:
        inherit_single(o, base)

### Used as follows

def TCPClient():
    def connect(socket_address):
        ...
    return make_object(connect)

def SMTPClient():
    tcp_client = TCPClient()
    def connect(host):
        tcp_client.connect((host, 25))
    def send_message(message):
        ...
    return make_object(send_message, base=tcp_client)

client = SMTPClient()
client.connect('mail.example.com')
========================================================================

> I mean, people had to *debate* the introduction of closures? There were
> three competing proposals for them, plus an argument for "don't add them".
> Some people say closures were added in Java 7, others say closures have
> been there since the beginning, and James Gosling himself says that Java
> used inner classes instead of closures but the result was painful...

I'm with those who say anonymous and named inner classes have been there
forever and serve the purpose of closures. Yes, Java's boilerplate
requirements are painful, but if you don't like that, use Python.

> It seems to me that in the Java community, there's a lot of confusion over
> closures, which are seen as an advanced (and rather scary) feature. Hardly
> routine.

Importantly, anonymous inner classes have been in active use by Java
newbs from day one.


Marko


From chris at cdreimer.com  Sat May 16 15:20:38 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sat, 16 May 2015 12:20:38 -0700
Subject: Rule of order for dot operators?
Message-ID: <55579886.3010001@cdreimer.com>

Greetings,

Noobie question regarding a single line of code that transforms a URL 
slug ("this-is-a-slug") into a title ("This Is A Slug").

title = slug.replace('-',' ').title()

This line also works if I switched the dot operators around.

title = slug.title().replace('-',' ')

I'm reading the first example as character replacement first and title 
capitalization second, and the second example as title capitalization 
first and character replacement second.

Does python perform the dot operators from left to right or according to 
a rule of order (i.e., multiplication/division before add/subtract)?

Thank you,

Chris Reimer


From gherron at digipen.edu  Sat May 16 15:32:45 2015
From: gherron at digipen.edu (Gary Herron)
Date: Sat, 16 May 2015 12:32:45 -0700
Subject: Rule of order for dot operators?
In-Reply-To: <55579886.3010001@cdreimer.com>
References: <55579886.3010001@cdreimer.com>
Message-ID: <55579B5D.3000704@digipen.edu>

On 05/16/2015 12:20 PM, C.D. Reimer wrote:
> Greetings,
>
> Noobie question regarding a single line of code that transforms a URL 
> slug ("this-is-a-slug") into a title ("This Is A Slug").
>
> title = slug.replace('-',' ').title()
>
> This line also works if I switched the dot operators around.
>
> title = slug.title().replace('-',' ')
>
> I'm reading the first example as character replacement first and title 
> capitalization second, and the second example as title capitalization 
> first and character replacement second.
>
> Does python perform the dot operators from left to right or according 
> to a rule of order (i.e., multiplication/division before add/subtract)?

Yes, that's correct.

Gary Herron



>
> Thank you,
>
> Chris Reimer


-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418



From joel.goldstick at gmail.com  Sat May 16 15:33:42 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Sat, 16 May 2015 15:33:42 -0400
Subject: Rule of order for dot operators?
In-Reply-To: <55579886.3010001@cdreimer.com>
References: <55579886.3010001@cdreimer.com>
Message-ID: <CAPM-O+yhGNXcq77bb9DopQOn2oBd=oPu7fUCG4-WK93+BJFLZA@mail.gmail.com>

On Sat, May 16, 2015 at 3:20 PM, C.D. Reimer <chris at cdreimer.com> wrote:
> Greetings,
>
> Noobie question regarding a single line of code that transforms a URL slug
> ("this-is-a-slug") into a title ("This Is A Slug").
>
> title = slug.replace('-',' ').title()
>
> This line also works if I switched the dot operators around.
>
> title = slug.title().replace('-',' ')
>
> I'm reading the first example as character replacement first and title
> capitalization second, and the second example as title capitalization first
> and character replacement second.
>
> Does python perform the dot operators from left to right or according to a
> rule of order (i.e., multiplication/division before add/subtract)?
>
> Thank you,
>
> Chris Reimer
> --
> https://mail.python.org/mailman/listinfo/python-list

>From left to right.  So in your first example it replaces - with
space, then capitalizes each word.
In your second example, it does the caps first, then gets rid of the dashes

-- 
Joel Goldstick
http://joelgoldstick.com


From __peter__ at web.de  Sat May 16 15:34:20 2015
From: __peter__ at web.de (Peter Otten)
Date: Sat, 16 May 2015 21:34:20 +0200
Subject: Rule of order for dot operators?
References: <55579886.3010001@cdreimer.com>
Message-ID: <mj8640$dat$1@ger.gmane.org>

C.D. Reimer wrote:

> Greetings,
> 
> Noobie question regarding a single line of code that transforms a URL
> slug ("this-is-a-slug") into a title ("This Is A Slug").
> 
> title = slug.replace('-',' ').title()
> 
> This line also works if I switched the dot operators around.
> 
> title = slug.title().replace('-',' ')
> 
> I'm reading the first example as character replacement first and title
> capitalization second, and the second example as title capitalization
> first and character replacement second.
> 
> Does python perform the dot operators from left to right or according to
> a rule of order (i.e., multiplication/division before add/subtract)?

You can find out yourself by using operations where the order does matter:

"Test".upper().lower()




From python.list at tim.thechases.com  Sat May 16 15:38:42 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Sat, 16 May 2015 14:38:42 -0500
Subject: Rule of order for dot operators?
In-Reply-To: <55579886.3010001@cdreimer.com>
References: <55579886.3010001@cdreimer.com>
Message-ID: <20150516143842.7ff45b9f@bigbox.christie.dr>

On 2015-05-16 12:20, C.D. Reimer wrote:
> Does python perform the dot operators from left to right or
> according to a rule of order (i.e., multiplication/division before
> add/subtract)?

Yes, Python evaluates dot-operators from left to right.

-tkc




From PointedEars at web.de  Sat May 16 15:40:08 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sat, 16 May 2015 21:40:08 +0200
Subject: Rule of order for dot operators?
References: <mailman.80.1431804386.17265.python-list@python.org>
Message-ID: <3341326.d8VUBGAoep@PointedEars.de>

C.D. Reimer wrote:
^^^^
Who?

> Noobie

What?

> question regarding a single line of code that transforms a URL
> slug ("this-is-a-slug") into a title ("This Is A Slug").
> 
> title = slug.replace('-',' ').title()
> 
> This line also works if I switched the dot operators around.
> 
> title = slug.title().replace('-',' ')
> 
> I'm reading the first example as character replacement first and title
> capitalization second, and the second example as title capitalization
> first and character replacement second.

You are reading correctly.

> Does python perform the dot operators from left to right or according to
> a rule of order (i.e., multiplication/division before add/subtract)?

Yes.  If you debug the code, which you should have done before posting [1] , 
you will see that

  'this-is-a-slug'.title() == 'This-Is-A-Slug'

It follows that in this special case it does not matter if you call .title() 
before or after .replace().

However, for greater efficiency, in general you should call .replace() in 
such a way that the length of the string it operates on is minimized.  For 
example, if feasible, always slice *before* .replace().

[1] <http://www.catb.org/~esr/faqs/smart-questions.html>
 
-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From chris at cdreimer.com  Sat May 16 15:55:21 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sat, 16 May 2015 12:55:21 -0700
Subject: Rule of order for dot operators?
In-Reply-To: <3341326.d8VUBGAoep@PointedEars.de>
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
Message-ID: <5557A0A9.9000705@cdreimer.com>

On 5/16/2015 12:40 PM, Thomas 'PointedEars' Lahn wrote:
> However, for greater efficiency, in general you should call .replace() 
> in such a way that the length of the string it operates on is 
> minimized. For example, if feasible, always slice *before* .replace().

Slice was how I got the slug from the URL in the first place. :)

Thank you,

Chris Reimer



From chris at cdreimer.com  Sat May 16 16:01:54 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sat, 16 May 2015 13:01:54 -0700
Subject: Rule of order for dot operators?
In-Reply-To: <mj8640$dat$1@ger.gmane.org>
References: <55579886.3010001@cdreimer.com> <mj8640$dat$1@ger.gmane.org>
Message-ID: <5557A232.9080408@cdreimer.com>

On 5/16/2015 12:34 PM, Peter Otten wrote:
> You can find out yourself by using operations where the order does 
> matter: "Test".upper().lower() 

I was wondering about that and couldn't think of an example off the top 
of my head.

Thank you,

Chris Reimer


From cs at zip.com.au  Sat May 16 18:41:14 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Sun, 17 May 2015 08:41:14 +1000
Subject: Fastest way to remove the first x characters from a very long
 string
In-Reply-To: <8354e4ba-0a80-48a1-b2e9-6edb4b67ff36@googlegroups.com>
References: <8354e4ba-0a80-48a1-b2e9-6edb4b67ff36@googlegroups.com>
Message-ID: <20150516224114.GA30814@cskk.homeip.net>

On 16May2015 10:35, bruceg113355 at gmail.com <bruceg113355 at gmail.com> wrote:
>On Saturday, May 16, 2015 at 12:59:19 PM UTC-4, Chris Angelico wrote:
>> On Sun, May 17, 2015 at 2:22 AM,  <bruceg113355 at gmail.com> wrote:
>> > # Original Approach
>> > # -----------------
>> > ss = ss.split("\n")
>> > ss1 = ""
>> > for sdata in ss:
>> >     ss1 = ss1 + (sdata[OFFSET:] + "\n")
>> >
>> > # Chris's Approach
>> > # ----------------
>> > lines = ss.split("\n")
>> > new_text = "\n".join(line[8:] for line in lines)
[...]
>
>Your approach more than meets my requirements.
>Data is formatted correctly and performance is simply amazing.
>OFFSET and \n are small details.

The only comment I'd make at this point is to consider if you really need a 
single string at the end. Keeping it as a list of lines may be more flexible.  
(It will consume more memory.) If you're doing more stuff with the string as 
lines then you'd need to re-split it, and so forth.

Cheers,
Cameron Simpson <cs at zip.com.au>


From marko at pacujo.net  Sat May 16 18:51:08 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sun, 17 May 2015 01:51:08 +0300
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4> <87twvdsbom.fsf@elektro.pacujo.net>
 <bpw5x.561870$Zf4.150532@fx22.am4> <87egmhrll2.fsf@elektro.pacujo.net>
 <555710a8$0$13001$c3e8da3$5496439d@news.astraweb.com>
 <878ucosjwh.fsf@elektro.pacujo.net>
 <55578a02$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <87zj54qrlw.fsf@elektro.pacujo.net>
Message-ID: <87lhgoqgpv.fsf@elektro.pacujo.net>

Marko Rauhamaa <marko at pacujo.net>:

> Ok, here's a quick port that have barely tried out:

And here's a more complete port (with some possible dunder abuse):

========================================================================
### Simple OO Framework

class _O: pass

def make_object(*procedures, base=None, bases=None):
    o = _O()
    methods = {}
    o.__methods__ = methods
    o.__derived__ = None
    if base is not None:
        _inherit_single(o, base)
    elif bases is not None:
        _inherit_multi(o, bases)
    for procedure in procedures:
        methods[procedure.__name__] = procedure
        def method(*args, __procedure__=procedure, __dispatch__=True, **kwargs):
            if not __dispatch__ or o.__derived__ is None:
                return __procedure__(*args, **kwargs)
            derived = o
            while derived.__derived__ is not None:
                derived = derived.__derived__
            return getattr(derived, __procedure__.__name__)(*args, **kwargs)
        setattr(o, procedure.__name__, method)
    return o

def _inherit_single(o, base):
    methods = o.__methods__
    for name, method in base.__methods__.items():
        methods[name] = method
        setattr(o, name, method)

def _inherit_multi(o, bases):
    for base in bases:
        _inherit_single(o, base)

def delegate(method, *args, **kwargs):
    return method(*args, __dispatch__=False, **kwargs)

### Used as follows

def TCPClient():
    def connect(address):
        pass
    def shut_down():
        pass
    return make_object(connect, shut_down)

def SMTPClient():
    tcp_client = TCPClient()
    def connect(address):
        delegate(tcp_client.connect, address)
        do_stuff()
    def send_message(message):
        pass
    return make_object(connect, send_message, base=tcp_client)

client = SMTPClient()
client.connect(None)
========================================================================


Marko


From denismfmcmahon at gmail.com  Sat May 16 19:24:04 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sat, 16 May 2015 23:24:04 +0000 (UTC)
Subject: Fastest way to remove the first x characters from a very long
 string
References: <6a383ce2-5975-4225-b4f2-f744c9d7a516@googlegroups.com>
Message-ID: <mj8jik$f1c$5@dont-email.me>

On Sat, 16 May 2015 06:28:19 -0700, bruceg113355 wrote:

> I have a string that contains 10 million characters.
> 
> The string is formatted as:
> 
> "0000001 : some hexadecimal text ... \n 0000002 : some hexadecimal text
> ... \n 0000003 : some hexadecimal text ... \n ...
> 0100000 : some hexadecimal text ... \n 0100001 : some hexadecimal text
> ... \n"
> 
> and I need the string to look like:
> 
> "some hexadecimal text ... \n some hexadecimal text ... \n some
> hexadecimal text ... \n ...
> some hexadecimal text ... \n some hexadecimal text ... \n"

Looks to me as if you have a 10 Mbyte encoded file with line numbers as 
ascii text and you're trying to strip the line numbers before decoding 
the file.

Are you looking for a one-off solution, or do you have a lot of these 
files?

If you have a lot of files to process, you could try using something like 
sed.

sed -i.old 's/^\d+ : //' *.ext

-- 
Denis McMahon, denismfmcmahon at gmail.com


From steve+comp.lang.python at pearwood.info  Sat May 16 21:31:52 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 17 May 2015 11:31:52 +1000
Subject: Rule of order for dot operators?
References: <mailman.80.1431804386.17265.python-list@python.org>
Message-ID: <5557ef89$0$12982$c3e8da3$5496439d@news.astraweb.com>

On Sun, 17 May 2015 05:20 am, C.D. Reimer wrote:

> Greetings,
> 
> Noobie question regarding a single line of code that transforms a URL
> slug ("this-is-a-slug") into a title ("This Is A Slug").
> 
> title = slug.replace('-',' ').title()
> 
> This line also works if I switched the dot operators around.

Technically, dot is not an operator, but even if it was, swapping the *dots*
around makes no difference:

slug.replace('-',' ').title()  => slug.replace('-',' ').title()

Because a dot is a dot, right? What you mean is that you're swapping the
*methods* around, not just the dots:

slug.replace('-',' ').title()  => slug.title().replace('-',' ')


> title = slug.title().replace('-',' ')
> 
> I'm reading the first example as character replacement first and title
> capitalization second, and the second example as title capitalization
> first and character replacement second.
> 
> Does python perform the dot operators from left to right or according to
> a rule of order (i.e., multiplication/division before add/subtract)?

Left to right.




-- 
Steven



From steve+comp.lang.python at pearwood.info  Sat May 16 21:45:02 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Sun, 17 May 2015 11:45:02 +1000
Subject: Rule of order for dot operators?
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
Message-ID: <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>

On Sun, 17 May 2015 05:40 am, Thomas 'PointedEars' Lahn wrote:

> C.D. Reimer wrote:
> ^^^^
> Who?

Don't be a dick, Thomas. Lots of people use their initials. You use your
nickname as part of your sender address, why are you questioning somebody
for using their initials?


>> Noobie
> 
> What?

Where? When? Why? How?

I'm pretty sure you've been on the Internet for long enough to know
what "noobie", "n00b", "newbie" etc mean.

But just in case you need help:

http://lmgtfy.com/?q=noobie



-- 
Steven



From jsf80238 at gmail.com  Sat May 16 22:06:49 2015
From: jsf80238 at gmail.com (Jason Friedman)
Date: Sat, 16 May 2015 20:06:49 -0600
Subject: How to deploy a custom common module?
In-Reply-To: <05d4eec1-1590-41ae-9315-7bb3cd2b22ee@googlegroups.com>
References: <05d4eec1-1590-41ae-9315-7bb3cd2b22ee@googlegroups.com>
Message-ID: <CANy1k1imsEwvzHeYex7s7mFRRyfyZ0tfRu2AFoju-+GHfyJPxQ@mail.gmail.com>

> When I deploy test.py on another computer, I put (rsync) both test.py and cmn_funcs.py in the same remote directory.
>
> If I create another python project (test2.py) in new directory, that needs common functions, what should I do with cmn_funcs.py?

I put my shared code in a separate folder, named something like
/path/to/module_dir.

I then add to /etc/profile.d/something.sh:

export PYTHONPATH=$PYTHONPATH:/path/to/module_dir


From irmen.NOSPAM at xs4all.nl  Sat May 16 22:32:04 2015
From: irmen.NOSPAM at xs4all.nl (Irmen de Jong)
Date: Sun, 17 May 2015 04:32:04 +0200
Subject: How to deploy a custom common module?
In-Reply-To: <mailman.88.1431828418.17265.python-list@python.org>
References: <05d4eec1-1590-41ae-9315-7bb3cd2b22ee@googlegroups.com>
 <mailman.88.1431828418.17265.python-list@python.org>
Message-ID: <5557fda6$0$2961$e4fe514c@news.xs4all.nl>

On 17-5-2015 4:06, Jason Friedman wrote:
>> When I deploy test.py on another computer, I put (rsync) both test.py and cmn_funcs.py in the same remote directory.
>>
>> If I create another python project (test2.py) in new directory, that needs common functions, what should I do with cmn_funcs.py?
> 
> I put my shared code in a separate folder, named something like
> /path/to/module_dir.
> 
> I then add to /etc/profile.d/something.sh:
> 
> export PYTHONPATH=$PYTHONPATH:/path/to/module_dir
> 


I think you could use the user site packages directory as well for this... then you
should not have to change the PYTHONPATH.

https://docs.python.org/3.5/library/site.html#site.USER_SITE

(if you don't want to or cannot install into to the system's site-packages directory)


Irmen



From rustompmody at gmail.com  Sun May 17 01:06:38 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sat, 16 May 2015 22:06:38 -0700 (PDT)
Subject: Rule of order for dot operators?
In-Reply-To: <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <67e40749-f6a2-45e4-90ee-d189084fb2f7@googlegroups.com>

On Sunday, May 17, 2015 at 7:15:13 AM UTC+5:30, Steven D'Aprano wrote:
> On Sun, 17 May 2015 05:40 am, Thomas 'PointedEars' Lahn wrote:
> 
> > C.D. Reimer wrote:
> > ^^^^
> > Who?
> 
> Don't be a dick, Thomas. Lots of people use their initials. You use your
> nickname as part of your sender address, why are you questioning somebody
> for using their initials?

Not only is it a ridiculous objection, I dont even understand what the objection
is.

Among the eminent this is quite common
Dijkstra -- ewd
Stallman -- rms
Eric Raymond -- esr

And even for non-eminent like your truly, most of my working life most people
called me 'rpm' than Rusi/Mody or any such.


From baillargeon at visi.com  Sun May 17 04:58:48 2015
From: baillargeon at visi.com (Bernard Baillargeon)
Date: Sun, 17 May 2015 03:58:48 -0500
Subject: install 3.4.3 or 3.5.0a4 -immediate fail
Message-ID: <001001d0907f$b092c740$11b855c0$@visi.com>

Win 8.1pro 64b

             Both/either of the 64b versions fail on "idle", "python" with
'no encodings' module error.

             It is there.  install had set the path to include install dir &
the \scripts subdir.

Just uninstalled, will have to try later.  Been working this -googling the
many instances of this failure in windows environment -for 5 hours and
cannot waste more.

 

Bernie 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150517/3e804037/attachment.html>

From jonas at wielicki.name  Sun May 17 08:25:29 2015
From: jonas at wielicki.name (Jonas Wielicki)
Date: Sun, 17 May 2015 14:25:29 +0200
Subject: Building CPython
In-Reply-To: <crnisnF2efoU1@mid.individual.net>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net> <zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4> <crlglfFgf05U1@mid.individual.net>
 <Z2j5x.640864$ms.43141@fx30.am4> <crnisnF2efoU1@mid.individual.net>
Message-ID: <555888B9.5090209@wielicki.name>

On 16.05.2015 02:55, Gregory Ewing wrote:
> BartC wrote:
>> For example, there is a /specific/ byte-code called BINARY_ADD, which
>> then proceeds to call a /generic/ binary-op handler! This throws away
>> the advantage of knowing at byte-code generation time exactly which
>> operation is needed.
> 
> While inlining the binary-op handling might give you a
> slightly shorter code path, it wouldn't necessarily speed
> anything up. It's possible, for example, that the shared
> binary-op handler fits in the instruction cache, but the
> various inlined copies of it don't, leading to a slowdown.
> 
> The only way to be sure about things like that is to try
> them and measure. The days when you could predict the speed
> of a program just by counting the number of instructions
> executed are long gone.

That, and also, the days where you could guess the number of
instructions executed from looking at the code are also gone. Compilers,
and especially C or C++ compilers, are huge beasts with an insane number
of different optimizations which yield pretty impressive results. Not to
mention that they may know the architecture you?re targeting and can
optimize each build for a different architecture; which is not really
possible if you do optimizations which e.g. rely on cache
characteristics or instruction timings or interactions by hand.

I changed my habits to just trust my compiler a few years ago and have
more readable code in exchange for that. The compiler does a fairly
great job, although gcc still outruns clang for *my* usecases. YMMV.

regards,
jwi


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20150517/e1e08da1/attachment.sig>

From bc at freeuk.com  Sun May 17 09:41:10 2015
From: bc at freeuk.com (BartC)
Date: Sun, 17 May 2015 14:41:10 +0100
Subject: Building CPython
In-Reply-To: <mailman.90.1431866130.17265.python-list@python.org>
References: <7JN4x.37133$Q41.15375@fx25.am4>	<mailman.464.1431556504.12865.python-list@python.org>	<6w35x.645690$I97.19867@fx31.am4>
 <874mnfunpn.fsf@elektro.pacujo.net> <zR85x.441429$Zf4.246168@fx22.am4>
 <xpb5x.597872$X95.548374@fx10.am4> <crlglfFgf05U1@mid.individual.net>
 <Z2j5x.640864$ms.43141@fx30.am4> <crnisnF2efoU1@mid.individual.net>
 <mailman.90.1431866130.17265.python-list@python.org>
Message-ID: <YT06x.482260$_r1.466432@fx39.am4>

On 17/05/2015 13:25, Jonas Wielicki wrote:
> On 16.05.2015 02:55, Gregory Ewing wrote:
>> BartC wrote:
>>> For example, there is a /specific/ byte-code called BINARY_ADD, which
>>> then proceeds to call a /generic/ binary-op handler! This throws away
>>> the advantage of knowing at byte-code generation time exactly which
>>> operation is needed.
>>
>> While inlining the binary-op handling might give you a
>> slightly shorter code path, it wouldn't necessarily speed
>> anything up. It's possible, for example, that the shared
>> binary-op handler fits in the instruction cache, but the
>> various inlined copies of it don't, leading to a slowdown.
>>
>> The only way to be sure about things like that is to try
>> them and measure. The days when you could predict the speed
>> of a program just by counting the number of instructions
>> executed are long gone.
>
> That, and also, the days where you could guess the number of
> instructions executed from looking at the code are also gone. Compilers,
> and especially C or C++ compilers, are huge beasts with an insane number
> of different optimizations which yield pretty impressive results. Not to
> mention that they may know the architecture you?re targeting and can
> optimize each build for a different architecture; which is not really
> possible if you do optimizations which e.g. rely on cache
> characteristics or instruction timings or interactions by hand.
>
> I changed my habits to just trust my compiler a few years ago and have
> more readable code in exchange for that. The compiler does a fairly
> great job, although gcc still outruns clang for *my* usecases.

> YMMV.

It does. For my interpreter projects, gcc -O3 does a pretty good job.

For running a suite of standard benchmarks ('spectral', 'fannkuch', 
'binary-tree', all that lot) in the bytecode language under test, then 
gcc is 30% faster than my own language/compiler. (And 25% faster than 
clang.)

(In that project, gcc can do a lot of inlining, which doesn't seem to be 
practical in CPython as functions are all over the place.)

However, when I plug in an ASM dispatcher to my version (which tries to 
deal with simple bytecodes or some common object types before passing 
control to the HLL to deal with), then I can get /twice as fast/ as gcc 
-O3. (For real programs the difference is narrower, but usually still 
faster than gcc.)

(This approach I don't think will work with CPython, because there don't 
appear to be any simple cases for ASM to deal with! The ASM dispatcher 
keeps essential globals such as the stack pointer and program counter in 
registers, and uses chained 'threaded' code rather than function calls. 
A proportion of byte-codes need to be handled in this environment, 
otherwise it could actually slow things down, as the switch to/from HLL 
code is expensive.)

-- 
Bartc


From denismfmcmahon at gmail.com  Sun May 17 11:43:36 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sun, 17 May 2015 15:43:36 +0000 (UTC)
Subject: Rule of order for dot operators?
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mjacv8$ods$6@dont-email.me>

On Sun, 17 May 2015 11:45:02 +1000, Steven D'Aprano wrote:

> On Sun, 17 May 2015 05:40 am, Thomas 'PointedEars' Lahn wrote:
> 
>> C.D. Reimer wrote:
>> ^^^^
>> Who?
> 
> Don't be a dick, Thomas.

Thomas is a professional dick, he can't help it, he's been a professional 
dick for years in php and javascript groups, and now he's obviously 
spreading himself further afield. He usually confines his wisdom to 
pointing out faults in other's posts, rather than offering any 
constructive input himself.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From pegah.alizadeh at gmail.com  Sun May 17 12:18:41 2015
From: pegah.alizadeh at gmail.com (pegah Aliz)
Date: Sun, 17 May 2015 09:18:41 -0700 (PDT)
Subject: I do not have access to the right _hierarchy.py source file
Message-ID: <4b48babb-e34e-4d48-a4c7-f40990b619ea@googlegroups.com>

Hello Everybody,

This question seems simple, but I can't find the solution:

I use scipy.cluster.hierarchy to do a hierarchical clustering on a set of points using "cosine" similarity metric. As an example, I have:


import scipy.cluster.hierarchy as hac
import matplotlib.pyplot as plt

Points = 
  np.array([[ 0.         , 0.23508573],
 [ 0.00754775 , 0.26717266],
 [ 0.00595464 , 0.27775905],
 [ 0.01220563 , 0.23622067],
 [ 0.00542628 , 0.14185873],
 [ 0.03078922 , 0.11273108],
 [ 0.06707743 ,-0.1061131 ],
 [ 0.04411757 ,-0.10775407],
 [ 0.01349434 , 0.00112159],
 [ 0.04066034 , 0.11639591],
 [ 0.         , 0.29046682],
 [ 0.07338036 , 0.00609912],
 [ 0.01864988 , 0.0316196 ],
 [ 0.         , 0.07270636],
 [ 0.         ,  0.        ]]) 


z = hac.linkage(Points, metric='cosine', method='complete')
labels = hac.fcluster(z, 0.1, criterion="distance")


plt.scatter(Points[:, 0], Points[:, 1], c=labels.astype(np.float))
plt.show()


Since I use cosine metric, in some cases the dot product of two vectors can be negative or norm of some vectors can be zero. It means z output will have some negative or infinite elements which is not valid for fcluster (as below): 

z = 
[[  0.00000000e+00   1.00000000e+01   0.00000000e+00   2.00000000e+00]
 [  1.30000000e+01   1.50000000e+01   0.00000000e+00   3.00000000e+00]
 [  8.00000000e+00   1.10000000e+01   4.26658708e-13   2.00000000e+00]
 [  1.00000000e+00   2.00000000e+00   2.31748880e-05   2.00000000e+00]
 [  3.00000000e+00   4.00000000e+00   8.96700489e-05   2.00000000e+00]
 [  1.60000000e+01   1.80000000e+01   3.98805492e-04   5.00000000e+00]
 [  1.90000000e+01   2.00000000e+01   1.33225099e-03   7.00000000e+00]
 [  5.00000000e+00   9.00000000e+00   2.41120340e-03   2.00000000e+00]
 [  6.00000000e+00   7.00000000e+00   1.52914684e-02   2.00000000e+00]
 [  1.20000000e+01   2.20000000e+01   3.52441432e-02   3.00000000e+00]
 [  2.10000000e+01   2.40000000e+01   1.38662986e-01   1.00000000e+01]
 [  1.70000000e+01   2.30000000e+01   6.99056531e-01   4.00000000e+00]
 [  2.50000000e+01   2.60000000e+01   1.92543748e+00   1.40000000e+01]
 [ -1.00000000e+00   2.70000000e+01              inf   1.50000000e+01]]

To solve this problem, I checked linkage() function and inside it I needed to check _hierarchy.linkage() method. I use pycharm text editor and when I asked for "linkage" source code, it opened up a python file namely "_hierarchy.py" inside the directory like the following:

.PyCharm40/system/python_stubs/-1247972723/scipy/cluster/_hierarchy.py
 
This python file doesn't have any definition for all included functions.  
I am wondering what is the correct source of this function to revise it and solve my problem.
I would be appreciated if someone helps me to explore the correct source.

Thanks and Regards
Pegah


From zachary.ware+pylist at gmail.com  Sun May 17 12:42:23 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Sun, 17 May 2015 11:42:23 -0500
Subject: I do not have access to the right _hierarchy.py source file
In-Reply-To: <4b48babb-e34e-4d48-a4c7-f40990b619ea@googlegroups.com>
References: <4b48babb-e34e-4d48-a4c7-f40990b619ea@googlegroups.com>
Message-ID: <CAKJDb-P1cVT9PZ37+X4drF9Fe=QguV2wbuyZT_8FOVmBJZ7HSw@mail.gmail.com>

On May 17, 2015 11:20 AM, "pegah Aliz" <pegah.alizadeh at gmail.com> wrote:
>
> Hello Everybody,
>
> This question seems simple, but I can't find the solution:

<snip>

> To solve this problem, I checked linkage() function and inside it I
needed to check _hierarchy.linkage() method. I use pycharm text editor and
when I asked for "linkage" source code, it opened up a python file namely
"_hierarchy.py" inside the directory like the following:
>
> .PyCharm40/system/python_stubs/-1247972723/scipy/cluster/_hierarchy.py
>
> This python file doesn't have any definition for all included functions.
> I am wondering what is the correct source of this function to revise it
and solve my problem.
> I would be appreciated if someone helps me to explore the correct source.

What you're seeing is basically an implementation detail of PyCharm: they
include "stub" files for many standard library and third party packages
that have C source in order to provide parameter completion and other
features. Chances are very good that you actually need '_hierarchy.c' from
somewhere in SciPy's source; unfortunately I can't help you beyond telling
you that.

Hope this helps,
--
Zach
(On a phone)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150517/8692fac4/attachment.html>

From gherron at digipen.edu  Sun May 17 12:44:11 2015
From: gherron at digipen.edu (Gary Herron)
Date: Sun, 17 May 2015 09:44:11 -0700
Subject: I do not have access to the right _hierarchy.py source file
In-Reply-To: <4b48babb-e34e-4d48-a4c7-f40990b619ea@googlegroups.com>
References: <4b48babb-e34e-4d48-a4c7-f40990b619ea@googlegroups.com>
Message-ID: <5558C55B.50804@digipen.edu>

On 05/17/2015 09:18 AM, pegah Aliz wrote:

...
> To solve this problem, I checked linkage() function and inside it I needed to check _hierarchy.linkage() method. I use pycharm text editor and when I asked for "linkage" source code, it opened up a python file namely "_hierarchy.py" inside the directory like the following:
>
> .PyCharm40/system/python_stubs/-1247972723/scipy/cluster/_hierarchy.py
>   
> This python file doesn't have any definition for all included functions.
> I am wondering what is the correct source of this function to revise it and solve my problem.
> I would be appreciated if someone helps me to explore the correct source.
>
> Thanks and Regards
> Pegah

Please tell us:

  * What platform you are on;  Linux, Windows, ...
  * How you installed PyCharm
  * What contents that file currently has
  * Why you think it's incorrect.

I think it's far more likely that that file is correct, and you are 
somehow misinterpreting its contents, but we can't even begin to guess 
until you show us its current content.

Gary

-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150517/992488cc/attachment.html>

From __peter__ at web.de  Sun May 17 12:47:43 2015
From: __peter__ at web.de (Peter Otten)
Date: Sun, 17 May 2015 18:47:43 +0200
Subject: I do not have access to the right _hierarchy.py source file
References: <4b48babb-e34e-4d48-a4c7-f40990b619ea@googlegroups.com>
Message-ID: <mjagnh$p2s$1@ger.gmane.org>

pegah Aliz wrote:

> To solve this problem, I checked linkage() function and inside it I needed
> to check _hierarchy.linkage() method. I use pycharm text editor and when I
> asked for "linkage" source code, it opened up a python file namely
> "_hierarchy.py" inside the directory like the following:
> 
> .PyCharm40/system/python_stubs/-1247972723/scipy/cluster/_hierarchy.py
> 
> This python file doesn't have any definition for all included functions.
> I am wondering what is the correct source of this function to revise it
> and solve my problem. I would be appreciated if someone helps me to
> explore the correct source.

The actual _hierarchy module is probably written in C. Before you turn to 
hacking that you should ask for advice on a scipy news group or mailing 
list. Perhaps the experts can point you to a better way to approach the 
underlying problem.




From pegah.alizadeh at gmail.com  Sun May 17 13:00:22 2015
From: pegah.alizadeh at gmail.com (pegah Aliz)
Date: Sun, 17 May 2015 10:00:22 -0700 (PDT)
Subject: I do not have access to the right _hierarchy.py source file
In-Reply-To: <4b48babb-e34e-4d48-a4c7-f40990b619ea@googlegroups.com>
References: <4b48babb-e34e-4d48-a4c7-f40990b619ea@googlegroups.com>
Message-ID: <f65ed128-ed04-4227-93a0-cc5709f79908@googlegroups.com>

On Sunday, May 17, 2015 at 6:18:51 PM UTC+2, pegah Aliz wrote:
> Hello Everybody,
> 
> This question seems simple, but I can't find the solution:
> 
> I use scipy.cluster.hierarchy to do a hierarchical clustering on a set of points using "cosine" similarity metric. As an example, I have:
> 
> 
> import scipy.cluster.hierarchy as hac
> import matplotlib.pyplot as plt
> 
> Points = 
>   np.array([[ 0.         , 0.23508573],
>  [ 0.00754775 , 0.26717266],
>  [ 0.00595464 , 0.27775905],
>  [ 0.01220563 , 0.23622067],
>  [ 0.00542628 , 0.14185873],
>  [ 0.03078922 , 0.11273108],
>  [ 0.06707743 ,-0.1061131 ],
>  [ 0.04411757 ,-0.10775407],
>  [ 0.01349434 , 0.00112159],
>  [ 0.04066034 , 0.11639591],
>  [ 0.         , 0.29046682],
>  [ 0.07338036 , 0.00609912],
>  [ 0.01864988 , 0.0316196 ],
>  [ 0.         , 0.07270636],
>  [ 0.         ,  0.        ]]) 
> 
> 
> z = hac.linkage(Points, metric='cosine', method='complete')
> labels = hac.fcluster(z, 0.1, criterion="distance")
> 
> 
> plt.scatter(Points[:, 0], Points[:, 1], c=labels.astype(np.float))
> plt.show()
> 
> 
> Since I use cosine metric, in some cases the dot product of two vectors can be negative or norm of some vectors can be zero. It means z output will have some negative or infinite elements which is not valid for fcluster (as below): 
> 
> z = 
> [[  0.00000000e+00   1.00000000e+01   0.00000000e+00   2.00000000e+00]
>  [  1.30000000e+01   1.50000000e+01   0.00000000e+00   3.00000000e+00]
>  [  8.00000000e+00   1.10000000e+01   4.26658708e-13   2.00000000e+00]
>  [  1.00000000e+00   2.00000000e+00   2.31748880e-05   2.00000000e+00]
>  [  3.00000000e+00   4.00000000e+00   8.96700489e-05   2.00000000e+00]
>  [  1.60000000e+01   1.80000000e+01   3.98805492e-04   5.00000000e+00]
>  [  1.90000000e+01   2.00000000e+01   1.33225099e-03   7.00000000e+00]
>  [  5.00000000e+00   9.00000000e+00   2.41120340e-03   2.00000000e+00]
>  [  6.00000000e+00   7.00000000e+00   1.52914684e-02   2.00000000e+00]
>  [  1.20000000e+01   2.20000000e+01   3.52441432e-02   3.00000000e+00]
>  [  2.10000000e+01   2.40000000e+01   1.38662986e-01   1.00000000e+01]
>  [  1.70000000e+01   2.30000000e+01   6.99056531e-01   4.00000000e+00]
>  [  2.50000000e+01   2.60000000e+01   1.92543748e+00   1.40000000e+01]
>  [ -1.00000000e+00   2.70000000e+01              inf   1.50000000e+01]]
> 
> To solve this problem, I checked linkage() function and inside it I needed to check _hierarchy.linkage() method. I use pycharm text editor and when I asked for "linkage" source code, it opened up a python file namely "_hierarchy.py" inside the directory like the following:
> 
> .PyCharm40/system/python_stubs/-1247972723/scipy/cluster/_hierarchy.py
>  
> This python file doesn't have any definition for all included functions.  
> I am wondering what is the correct source of this function to revise it and solve my problem.
> I would be appreciated if someone helps me to explore the correct source.
> 
> Thanks and Regards
> Pegah



1 - The platform is Linux
2 - After downloading .tar file, making file and configuring, I use pycharm.sh
3 - these are contents of _hierarchy.py : 

# encoding: utf-8
# module scipy.cluster._hierarchy
# from /users/alizadeh/.local/lib/python2.7/site-packages/scipy/cluster/_hierarchy.so
# by generator 1.136
# no doc

# imports
import __builtin__ as __builtins__ # <module '__builtin__' (built-in)>
import numpy as np # /usr/lib/pymodules/python2.7/numpy/__init__.pyc

# functions

def calculate_cluster_sizes(*args, **kwargs): # real signature unknown
    """
    Calculate the size of each cluster. The result is the fourth column of
        the linkage matrix.
    
        Parameters
        ----------
        Z : ndarray
            The linkage matrix. The fourth column can be empty.
        cs : ndarray
            The array to store the sizes.
        n : ndarray
            The number of observations.
    """
    pass

def cluster_dist(*args, **kwargs): # real signature unknown
    """
    Form flat clusters by distance criterion.
    
        Parameters
        ----------
        Z : ndarray
            The linkage matrix.
        T : ndarray
            The array to store the cluster numbers. The i'th observation belongs to
            cluster `T[i]`.
        cutoff : double
            Clusters are formed when distances are less than or equal to `cutoff`.
        n : int
            The number of observations.
    """
    pass

def cluster_in(*args, **kwargs): # real signature unknown
    """
    Form flat clusters by inconsistent criterion.
    
        Parameters
        ----------
        Z : ndarray
            The linkage matrix.
        R : ndarray
            The inconsistent matrix.
        T : ndarray
            The array to store the cluster numbers. The i'th observation belongs to
            cluster `T[i]`.
        cutoff : double
            Clusters are formed when the inconsistent values are less than or
            or equal to `cutoff`.
        n : int
            The number of observations.
    """
    pass

def cluster_maxclust_dist(*args, **kwargs): # real signature unknown
    """
    Form flat clusters by maxclust criterion.
    
        Parameters
        ----------
        Z : ndarray
            The linkage matrix.
        T : ndarray
            The array to store the cluster numbers. The i'th observation belongs to
            cluster `T[i]`.
        n : int
            The number of observations.
        mc : int
            The maximum number of clusters.
    """
    pass

def cluster_maxclust_monocrit(*args, **kwargs): # real signature unknown
    """
    Form flat clusters by maxclust_monocrit criterion.
    
        Parameters
        ----------
        Z : ndarray
            The linkage matrix.
        MC : ndarray
            The monotonic criterion array.
        T : ndarray
            The array to store the cluster numbers. The i'th observation belongs to
            cluster `T[i]`.
        n : int
            The number of observations.
        max_nc : int
            The maximum number of clusters.
    """
    pass

def cluster_monocrit(*args, **kwargs): # real signature unknown
    """
    Form flat clusters by monocrit criterion.
    
        Parameters
        ----------
        Z : ndarray
            The linkage matrix.
        MC : ndarray
            The monotonic criterion array.
        T : ndarray
            The array to store the cluster numbers. The i'th observation belongs to
            cluster `T[i]`.
        cutoff : double
            Clusters are formed when the MC values are less than or equal to
            `cutoff`.
        n : int
            The number of observations.
    """
    pass

def cophenetic_distances(*args, **kwargs): # real signature unknown
    """
    Calculate the cophenetic distances between each observation
    
        Parameters
        ----------
        Z : ndarray
            The linkage matrix.
        d : ndarray
            The condensed matrix to store the cophenetic distances.
        n : int
            The number of observations.
    """
    pass

def get_max_dist_for_each_cluster(*args, **kwargs): # real signature unknown
    """
    Get the maximum inconsistency coefficient for each non-singleton cluster.
    
        Parameters
        ----------
        Z : ndarray
            The linkage matrix.
        MD : ndarray
            The array to store the result.
        n : int
            The number of observations.
    """
    pass

4 - because in hierarchy.py I have a line like this:
 
      _hierarchy.linkage(dm, Z, n,
                         int(_cpy_non_euclid_methods[method]))
which Z value is different before and after it.


From pegah.alizadeh at gmail.com  Sun May 17 13:02:57 2015
From: pegah.alizadeh at gmail.com (pegah Aliz)
Date: Sun, 17 May 2015 10:02:57 -0700 (PDT)
Subject: I do not have access to the right _hierarchy.py source file
In-Reply-To: <4b48babb-e34e-4d48-a4c7-f40990b619ea@googlegroups.com>
References: <4b48babb-e34e-4d48-a4c7-f40990b619ea@googlegroups.com>
Message-ID: <6101fa82-c9d3-4c0d-8531-77bf4f05497e@googlegroups.com>

On Sunday, May 17, 2015 at 6:18:51 PM UTC+2, pegah Aliz wrote:
> Hello Everybody,
> 
> This question seems simple, but I can't find the solution:
> 
> I use scipy.cluster.hierarchy to do a hierarchical clustering on a set of points using "cosine" similarity metric. As an example, I have:
> 
> 
> import scipy.cluster.hierarchy as hac
> import matplotlib.pyplot as plt
> 
> Points = 
>   np.array([[ 0.         , 0.23508573],
>  [ 0.00754775 , 0.26717266],
>  [ 0.00595464 , 0.27775905],
>  [ 0.01220563 , 0.23622067],
>  [ 0.00542628 , 0.14185873],
>  [ 0.03078922 , 0.11273108],
>  [ 0.06707743 ,-0.1061131 ],
>  [ 0.04411757 ,-0.10775407],
>  [ 0.01349434 , 0.00112159],
>  [ 0.04066034 , 0.11639591],
>  [ 0.         , 0.29046682],
>  [ 0.07338036 , 0.00609912],
>  [ 0.01864988 , 0.0316196 ],
>  [ 0.         , 0.07270636],
>  [ 0.         ,  0.        ]]) 
> 
> 
> z = hac.linkage(Points, metric='cosine', method='complete')
> labels = hac.fcluster(z, 0.1, criterion="distance")
> 
> 
> plt.scatter(Points[:, 0], Points[:, 1], c=labels.astype(np.float))
> plt.show()
> 
> 
> Since I use cosine metric, in some cases the dot product of two vectors can be negative or norm of some vectors can be zero. It means z output will have some negative or infinite elements which is not valid for fcluster (as below): 
> 
> z = 
> [[  0.00000000e+00   1.00000000e+01   0.00000000e+00   2.00000000e+00]
>  [  1.30000000e+01   1.50000000e+01   0.00000000e+00   3.00000000e+00]
>  [  8.00000000e+00   1.10000000e+01   4.26658708e-13   2.00000000e+00]
>  [  1.00000000e+00   2.00000000e+00   2.31748880e-05   2.00000000e+00]
>  [  3.00000000e+00   4.00000000e+00   8.96700489e-05   2.00000000e+00]
>  [  1.60000000e+01   1.80000000e+01   3.98805492e-04   5.00000000e+00]
>  [  1.90000000e+01   2.00000000e+01   1.33225099e-03   7.00000000e+00]
>  [  5.00000000e+00   9.00000000e+00   2.41120340e-03   2.00000000e+00]
>  [  6.00000000e+00   7.00000000e+00   1.52914684e-02   2.00000000e+00]
>  [  1.20000000e+01   2.20000000e+01   3.52441432e-02   3.00000000e+00]
>  [  2.10000000e+01   2.40000000e+01   1.38662986e-01   1.00000000e+01]
>  [  1.70000000e+01   2.30000000e+01   6.99056531e-01   4.00000000e+00]
>  [  2.50000000e+01   2.60000000e+01   1.92543748e+00   1.40000000e+01]
>  [ -1.00000000e+00   2.70000000e+01              inf   1.50000000e+01]]
> 
> To solve this problem, I checked linkage() function and inside it I needed to check _hierarchy.linkage() method. I use pycharm text editor and when I asked for "linkage" source code, it opened up a python file namely "_hierarchy.py" inside the directory like the following:
> 
> .PyCharm40/system/python_stubs/-1247972723/scipy/cluster/_hierarchy.py
>  
> This python file doesn't have any definition for all included functions.  
> I am wondering what is the correct source of this function to revise it and solve my problem.
> I would be appreciated if someone helps me to explore the correct source.
> 
> Thanks and Regards
> Pegah

@Peter Thank you. I will do that. 


From fred at evoluware.eu  Sun May 17 13:15:57 2015
From: fred at evoluware.eu (Fred Spiessens)
Date: Sun, 17 May 2015 10:15:57 -0700 (PDT)
Subject: My attempts in playing with tail-recursion in python
In-Reply-To: <521df4f7$0$2551$426a74cc@news.free.fr>
References: <521df4f7$0$2551$426a74cc@news.free.fr>
Message-ID: <ed8a0e14-2b7e-41a2-a663-7db275b5ae05@googlegroups.com>

Hi Thomas, I like what you've been doing.
I think it would also be great if the "leave the loop" detector would be the actual stop condition in the recursion, applied to the arguments of the call.
That would of course force you to split the recursive function in two functions: one to detect the stop condition, and another one that makes the next call, but in my opinion, that would make perfect sense.


From PointedEars at web.de  Sun May 17 13:17:06 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 17 May 2015 19:17:06 +0200
Subject: Rule of order for dot operators?
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <mailman.84.1431806129.17265.python-list@python.org>
Message-ID: <4369306.J7Y2hDxjVg@PointedEars.de>

C.D. Reimer wrote:

> On 5/16/2015 12:40 PM, Thomas 'PointedEars' Lahn wrote:
>> However, for greater efficiency, in general you should call .replace()
>> in such a way that the length of the string it operates on is
>> minimized. For example, if feasible, always slice *before* .replace().
> 
> Slice was how I got the slug from the URL in the first place. :)

Consider using a regular expression or the urllib object instead.  See 
RFC 3986, Appendix B, and <https://docs.python.org/3/library/urllib.html>, 
respectively.

> Thank you,

You are welcome.

> Chris Reimer

Please use this or something to that effect in your ?From? header field 
value.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From PointedEars at web.de  Sun May 17 13:18:41 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sun, 17 May 2015 19:18:41 +0200
Subject: Rule of order for dot operators?
References: <55579886.3010001@cdreimer.com>
 <mailman.86.1431807388.17265.python-list@python.org>
Message-ID: <22376984.KGljeZNGsi@PointedEars.de>

Tim Chase wrote:

> On 2015-05-16 12:20, C.D. Reimer wrote:
>> Does python perform the dot operators from left to right or
>> according to a rule of order (i.e., multiplication/division before
>> add/subtract)?
> 
> Yes, Python evaluates dot-operators from left to right.

?.? is _not_ an operator in Python:

<https://docs.python.org/3/reference/lexical_analysis.html#operators>

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From ned at nedbatchelder.com  Sun May 17 13:30:50 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sun, 17 May 2015 10:30:50 -0700 (PDT)
Subject: Rule of order for dot operators?
In-Reply-To: <4369306.J7Y2hDxjVg@PointedEars.de>
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <mailman.84.1431806129.17265.python-list@python.org>
 <4369306.J7Y2hDxjVg@PointedEars.de>
Message-ID: <d061de7f-a039-491a-964d-ec81b5ff8f5b@googlegroups.com>

On Sunday, May 17, 2015 at 1:19:59 PM UTC-4, Thomas 'PointedEars' Lahn wrote:
> C.D. Reimer wrote:
> > Chris Reimer
> 
> Please use this or something to that effect in your "From" header field 
> value.

Please stop making strange requests of others. "C.D. Reimer" is
a perfectly reasonable name to use.

--Ned.

> 
> -- 
> PointedEars


From chris at cdreimer.com  Sun May 17 13:50:53 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sun, 17 May 2015 10:50:53 -0700
Subject: Rule of order for dot operators?
In-Reply-To: <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5558D4FD.1090907@cdreimer.com>

On 5/16/2015 6:45 PM, Steven D'Aprano wrote:
> On Sun, 17 May 2015 05:40 am, Thomas 'PointedEars' Lahn wrote:
>
>> C.D. Reimer wrote:
>> ^^^^
>> Who?
> Don't be a dick, Thomas. Lots of people use their initials. You use your
> nickname as part of your sender address, why are you questioning somebody
> for using their initials?

I used my initials to hide my online presence from the Real World(tm). If a hiring manager looks up my legal name on the Internet, he or she will find a bunch of Usenet postings when I was a SuSE Linux noob in the 1990's. The only online accounts I have under my legal name is a Yahoo email address and a LinkedIn profile. After working at one employer that allowed anything found on the Internet as ammo in the office politics, a blank online slate provides better protection from such nonsense.

Besides, I got called by my initials in school when the compact discs (CD) became popular. :)

As for my question, my 2007 Core Python Programming book (based on python 2.5) indexed the dot for search operations. Some code examples show a single call (i.e., object.method()) but not multiple calls (i.e., object.method().method()). Since I wasn't sure what I was looking for, an Internet search turned up nothing useful. Hence, IMHO, a noobie question.

Maybe I need a newer python book?

Thank you,

Chris Reimer



From chris at cdreimer.com  Sun May 17 14:35:20 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sun, 17 May 2015 11:35:20 -0700
Subject: Rule of order for dot operators?
In-Reply-To: <4369306.J7Y2hDxjVg@PointedEars.de>
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <mailman.84.1431806129.17265.python-list@python.org>
 <4369306.J7Y2hDxjVg@PointedEars.de>
Message-ID: <5558DF68.3050609@cdreimer.com>

On 5/17/2015 10:17 AM, Thomas 'PointedEars' Lahn wrote:
> C.D. Reimer wrote:
>
> Consider using a regular expression or the urllib object instead. See 
> RFC 3986, Appendix B, and 
> <https://docs.python.org/3/library/urllib.html>, respectively.

That wouldn't work for me. I'm in the process of converting a WordPress 
website into a static website.

I wrote a script that pulled the HTML content from the SQL file to save 
each post in a text file with the URL as the file name (i.e., 
"2015-01-01-this-is-a-slug.html"). That created 275 files in the source 
folder.

Since I'm using Grav CMS (http://getgrav.org/) for my static website, I 
wrote a script to get the file names from the source folder, slice each 
file name into their respective component (i.e., year, month, day, slug, 
and title from the slug), convert the HTML into Markdown, and copy the 
content into a file called item.md inside a new folder (i.e., 
20150101.this-is-a-slug) in the destination folder.

After I get done cleaning up 275 item.md files in a Markdown editor, 
I'll write another script to create an .htaccess file to forward old url 
(i.e., /2015/01/01/this-is-a-slug) to the new URL (i.e., 
/blog/this-is-a-slug).

Gotta love string manipulations. ;)

Thank you,

Chris Reimer


From rustompmody at gmail.com  Sun May 17 15:13:16 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sun, 17 May 2015 12:13:16 -0700 (PDT)
Subject: Rule of order for dot operators?
In-Reply-To: <d061de7f-a039-491a-964d-ec81b5ff8f5b@googlegroups.com>
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <mailman.84.1431806129.17265.python-list@python.org>
 <4369306.J7Y2hDxjVg@PointedEars.de>
 <d061de7f-a039-491a-964d-ec81b5ff8f5b@googlegroups.com>
Message-ID: <9012c2cd-4a3e-4915-bfb8-79a13143e0e7@googlegroups.com>

On Sunday, May 17, 2015 at 11:01:01 PM UTC+5:30, Ned Batchelder wrote:
> On Sunday, May 17, 2015 at 1:19:59 PM UTC-4, Thomas 'PointedEars' Lahn wrote:
> > C.D. Reimer wrote:
> > > Chris Reimer
> > 
> > Please use this or something to that effect in your "From" header field 
> > value.
> 
> Please stop making strange requests of others. "C.D. Reimer" is
> a perfectly reasonable name to use.
> 
> --Ned.

And what about 'PointedEars'?
If that can be accepted so can 'C.D. Reimer'.





From dfnsonfsduifb at gmx.de  Sun May 17 15:39:58 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sun, 17 May 2015 21:39:58 +0200
Subject: textwrap.wrap() breaks non-breaking spaces
Message-ID: <mjaqqd$t0m$1@news.albasani.net>

Hey there,

so that textwrap.wrap() breks non-breaking spaces, is this a bug or
intended behavior? For example:

Python 3.4.0 (default, Apr 11 2014, 13:05:11)
[GCC 4.8.2] on linux

>>> import textwrap
>>> for line in textwrap.wrap("foo dont\xa0break " * 20): print(line)
...
foo dont break foo dont break foo dont break foo dont break foo dont
break foo dont break foo dont break foo dont break foo dont break foo
dont break foo dont break foo dont break foo dont break foo dont break
foo dont break foo dont break foo dont break foo dont break foo dont
break foo dont break

Apparently it does recognize that \xa0 is a kind of space, but it thinks
it can break any space. The point of \xa0 being exactly to avoid this
kind of thing.

Any remedy or ideas?

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From python.list at tim.thechases.com  Sun May 17 16:12:36 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Sun, 17 May 2015 15:12:36 -0500
Subject: textwrap.wrap() breaks non-breaking spaces
In-Reply-To: <mjaqqd$t0m$1@news.albasani.net>
References: <mjaqqd$t0m$1@news.albasani.net>
Message-ID: <20150517151236.1b7a9c2e@bigbox.christie.dr>

On 2015-05-17 21:39, Johannes Bauer wrote:
> Hey there,
> 
> so that textwrap.wrap() breks non-breaking spaces, is this a bug or
> intended behavior? For example:
> 
> Python 3.4.0 (default, Apr 11 2014, 13:05:11)
> [GCC 4.8.2] on linux
> 
> >>> import textwrap
> >>> for line in textwrap.wrap("foo dont\xa0break " * 20):
> >>> print(line)
> ...
> foo dont break foo dont break foo dont break foo dont break foo dont
> break foo dont break foo dont break foo dont break foo dont break
> foo dont break foo dont break foo dont break foo dont break foo
> dont break foo dont break foo dont break foo dont break foo dont
> break foo dont break foo dont break
> 
> Apparently it does recognize that \xa0 is a kind of space, but it
> thinks it can break any space. The point of \xa0 being exactly to
> avoid this kind of thing.
> 
> Any remedy or ideas?

Since it uses a TextWrapper class, you can subclass that and
then assert that the spaces found for splitting aren't
non-breaking spaces.  Note that, to use the "\u00a0"
notation, the particular string has to be a non-raw string.
You can compare the two regular expressions with those in
the original source file in your $STDLIB/textwrap.py

  import textwrap
  import re
  
  class MyWrapper(textwrap.TextWrapper):
    wordsep_re = re.compile(
      '((?!\u00a0)\\s+|'                      # any whitespace
      r'[^\s\w]*\w+[^0-9\W]-(?=\w+[^0-9\W])|' # hyphenated words
      r'(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w))') # em-dash
  
    # This less funky little regex just split on recognized spaces. E.g.
    #   "Hello there -- you goof-ball, use the -b option!"
    # splits into
    #   Hello/ /there/ /--/ /you/ /goof-ball,/ /use/ /the/ /-b/ /option!/
    wordsep_simple_re = re.compile('((?!\u00a0)\\s+)')
  
  s = 'foo dont\u00a0break ' * 20
  
  wrapper = MyWrapper()
  for line in wrapper.wrap(s):
    print(line)

Based on my tests, it gives the results you were looking
for.

-tkc







From ned at nedbatchelder.com  Sun May 17 16:24:24 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sun, 17 May 2015 13:24:24 -0700 (PDT)
Subject: textwrap.wrap() breaks non-breaking spaces
In-Reply-To: <mjaqqd$t0m$1@news.albasani.net>
References: <mjaqqd$t0m$1@news.albasani.net>
Message-ID: <322460a5-ac25-467e-ae90-9a610e3bbb5f@googlegroups.com>

On Sunday, May 17, 2015 at 3:40:07 PM UTC-4, Johannes Bauer wrote:
> Hey there,
> 
> so that textwrap.wrap() breks non-breaking spaces, is this a bug or
> intended behavior? For example:
> 
> Python 3.4.0 (default, Apr 11 2014, 13:05:11)
> [GCC 4.8.2] on linux
> 
> >>> import textwrap
> >>> for line in textwrap.wrap("foo dont\xa0break " * 20): print(line)
> ...
> foo dont break foo dont break foo dont break foo dont break foo dont
> break foo dont break foo dont break foo dont break foo dont break foo
> dont break foo dont break foo dont break foo dont break foo dont break
> foo dont break foo dont break foo dont break foo dont break foo dont
> break foo dont break
> 
> Apparently it does recognize that \xa0 is a kind of space, but it thinks
> it can break any space. The point of \xa0 being exactly to avoid this
> kind of thing.

There's a Python bug about this: http://bugs.python.org/issue20491

--Ned.

> 
> Any remedy or ideas?
> 
> Cheers,
> Johannes


From marko at pacujo.net  Sun May 17 16:49:15 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sun, 17 May 2015 23:49:15 +0300
Subject: Building CPython
References: <7JN4x.37133$Q41.15375@fx25.am4>
 <mailman.464.1431556504.12865.python-list@python.org>
 <6w35x.645690$I97.19867@fx31.am4> <874mnfunpn.fsf@elektro.pacujo.net>
 <zR85x.441429$Zf4.246168@fx22.am4> <xpb5x.597872$X95.548374@fx10.am4>
 <crlglfFgf05U1@mid.individual.net> <87bnhmgqrx.fsf@elektro.pacujo.net>
 <DWt5x.439666$qW.359562@fx20.am4> <87twvdsbom.fsf@elektro.pacujo.net>
 <bpw5x.561870$Zf4.150532@fx22.am4> <87egmhrll2.fsf@elektro.pacujo.net>
 <555710a8$0$13001$c3e8da3$5496439d@news.astraweb.com>
 <878ucosjwh.fsf@elektro.pacujo.net>
 <55578a02$0$13003$c3e8da3$5496439d@news.astraweb.com>
 <87zj54qrlw.fsf@elektro.pacujo.net>
 <87lhgoqgpv.fsf@elektro.pacujo.net>
Message-ID: <87mw13orp0.fsf@elektro.pacujo.net>


Iterating. A line still missing:

> ========================================================================
> ### Simple OO Framework
>
> class _O: pass
>
> def make_object(*procedures, base=None, bases=None):
>     o = _O()
>     methods = {}
>     o.__methods__ = methods
>     o.__derived__ = None
>     if base is not None:
>         _inherit_single(o, base)
>     elif bases is not None:
>         _inherit_multi(o, bases)
>     for procedure in procedures:
>         methods[procedure.__name__] = procedure
>         def method(*args, __procedure__=procedure, __dispatch__=True, **kwargs):
>             if not __dispatch__ or o.__derived__ is None:
>                 return __procedure__(*args, **kwargs)
>             derived = o
>             while derived.__derived__ is not None:
>                 derived = derived.__derived__
>             return getattr(derived, __procedure__.__name__)(*args, **kwargs)
>         setattr(o, procedure.__name__, method)
>     return o
>
> def _inherit_single(o, base):
>     methods = o.__methods__
>     for name, method in base.__methods__.items():
>         methods[name] = method
>         setattr(o, name, method)
      base.__derived__ = o
>
> def _inherit_multi(o, bases):
>     for base in bases:
>         _inherit_single(o, base)
>
> def delegate(method, *args, **kwargs):
>     return method(*args, __dispatch__=False, **kwargs)
>
> ### Used as follows
>
> def TCPClient():
>     def connect(address):
>         pass
>     def shut_down():
>         pass
>     return make_object(connect, shut_down)
>
> def SMTPClient():
>     tcp_client = TCPClient()
>     def connect(address):
>         delegate(tcp_client.connect, address)
>         do_stuff()
>     def send_message(message):
>         pass
>     return make_object(connect, send_message, base=tcp_client)
>
> client = SMTPClient()
> client.connect(None)
> ========================================================================


From steve+comp.lang.python at pearwood.info  Mon May 18 01:21:05 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 18 May 2015 15:21:05 +1000
Subject: Python 3: yum is dead, long live dnf!
Message-ID: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>

As part of Red Hat's move to Python 3, yum is officially deprecated and 
replaced by dnf:

http://dnf.baseurl.org/2015/05/11/yum-is-dead-long-live-dnf/

Quote:

    Yum would not survive the ?Python 3 as default? Fedora 
    initiative meanwhile DNF is able to run on Python 2 and 
    Python 3.



-- 
Steve



From rosuav at gmail.com  Mon May 18 03:40:50 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 18 May 2015 17:40:50 +1000
Subject: Rule of order for dot operators?
In-Reply-To: <5558D4FD.1090907@cdreimer.com>
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <5558D4FD.1090907@cdreimer.com>
Message-ID: <CAPTjJmoGGYAa0cNugk=545c8KXWG3jZiqp=SE6DSPdmrJcdyrg@mail.gmail.com>

On Mon, May 18, 2015 at 3:50 AM, C.D. Reimer <chris at cdreimer.com> wrote:
> On 5/16/2015 6:45 PM, Steven D'Aprano wrote:
>>
>> On Sun, 17 May 2015 05:40 am, Thomas 'PointedEars' Lahn wrote:
>>
>>> C.D. Reimer wrote:
>>> ^^^^
>>> Who?
>>
>> Don't be a dick, Thomas. Lots of people use their initials. You use your
>> nickname as part of your sender address, why are you questioning somebody
>> for using their initials?
>
>
> I used my initials to hide my online presence from the Real World(tm). If a
> hiring manager looks up my legal name on the Internet, he or she will find a
> bunch of Usenet postings when I was a SuSE Linux noob in the 1990's. The
> only online accounts I have under my legal name is a Yahoo email address and
> a LinkedIn profile. After working at one employer that allowed anything
> found on the Internet as ammo in the office politics, a blank online slate
> provides better protection from such nonsense.

Personally, I'd rather establish a strong presence under my actual
name than try to hide; otherwise, there'll still _be_ some sort of
presence, but it's less under your control. But that's a matter of
opinion.

> Besides, I got called by my initials in school when the compact discs (CD)
> became popular. :)

Could be worse. You could have been called Carrier Detect, Current
Directory (or Change Directory), candela (if written "cd"), Cadmium
(if written "Cd"), or any number of other things. I've heard tell you
can learn a lot about someone by what they first think of as an
expansion of "CD". :)

> As for my question, my 2007 Core Python Programming book (based on python
> 2.5) indexed the dot for search operations. Some code examples show a single
> call (i.e., object.method()) but not multiple calls (i.e.,
> object.method().method()). Since I wasn't sure what I was looking for, an
> Internet search turned up nothing useful. Hence, IMHO, a noobie question.
>
> Maybe I need a newer python book?

Based on 2.5? Probably; get yourself a book that focuses on 3.x,
ideally. But not because stuff is fundamentally different - more
because you'll be missing out on some improvements. From 2.5 to 2.7
there were a number of neat feature additions, many of them enabling
2.x/3.x compatibility, plus some tidyups and such; what you'd normally
expect of a couple of versions' worth of development. Most of what the
book says is probably still valid.

ChrisA


From alister.nospam.ware at ntlworld.com  Mon May 18 06:18:49 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Mon, 18 May 2015 10:18:49 +0000 (UTC)
Subject: Python 3: yum is dead, long live dnf!
References: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>
Message-ID: <mjcea8$7ca$1@speranza.aioe.org>

On Mon, 18 May 2015 15:21:05 +1000, Steven D'Aprano wrote:

> As part of Red Hat's move to Python 3, yum is officially deprecated and
> replaced by dnf:
> 
> http://dnf.baseurl.org/2015/05/11/yum-is-dead-long-live-dnf/
> 
> Quote:
> 
>     Yum would not survive the ?Python 3 as default? Fedora initiative
>     meanwhile DNF is able to run on Python 2 and Python 3.


Poor choice of name DNF= Did Not Finish 


-- 
In war, truth is the first casualty.
		-- U Thant


From alister.nospam.ware at ntlworld.com  Mon May 18 06:28:49 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Mon, 18 May 2015 10:28:49 +0000 (UTC)
Subject: Python 3: yum is dead, long live dnf!
References: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>
 <mjcea8$7ca$1@speranza.aioe.org>
Message-ID: <mjcet1$8p0$1@speranza.aioe.org>

On Mon, 18 May 2015 10:18:49 +0000, alister wrote:

> On Mon, 18 May 2015 15:21:05 +1000, Steven D'Aprano wrote:
> 
>> As part of Red Hat's move to Python 3, yum is officially deprecated and
>> replaced by dnf:
>> 
>> http://dnf.baseurl.org/2015/05/11/yum-is-dead-long-live-dnf/
>> 
>> Quote:
>> 
>>     Yum would not survive the ?Python 3 as default? Fedora initiative
>>     meanwhile DNF is able to run on Python 2 and Python 3.
> 
> 
> Poor choice of name DNF= Did Not Finish


Which may be fitting
it just waisted 10 min downloading everything before discovering I did 
not have permission (forgot to sudo)


-- 
Civilization is the limitless multiplication of unnecessary necessities.
		-- Mark Twain


From breamoreboy at yahoo.co.uk  Mon May 18 06:30:57 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 18 May 2015 11:30:57 +0100
Subject: Python 3: yum is dead, long live dnf!
In-Reply-To: <mjcea8$7ca$1@speranza.aioe.org>
References: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>
 <mjcea8$7ca$1@speranza.aioe.org>
Message-ID: <mjcf17$tvu$1@ger.gmane.org>

On 18/05/2015 11:18, alister wrote:
> On Mon, 18 May 2015 15:21:05 +1000, Steven D'Aprano wrote:
>
>> As part of Red Hat's move to Python 3, yum is officially deprecated and
>> replaced by dnf:
>>
>> http://dnf.baseurl.org/2015/05/11/yum-is-dead-long-live-dnf/
>>
>> Quote:
>>
>>      Yum would not survive the ?Python 3 as default? Fedora initiative
>>      meanwhile DNF is able to run on Python 2 and Python 3.
>
>
> Poor choice of name DNF= Did Not Finish
>

Does Not Fail?

As an aside, from the signature of your message, I thought "In war, 
truth is the first casualty." was first said by Winston Churchill, not U 
Thant?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From redstone-cold at 163.com  Mon May 18 06:31:36 2015
From: redstone-cold at 163.com (iMath)
Date: Mon, 18 May 2015 03:31:36 -0700 (PDT)
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <4d32dbce-a69f-4fd5-b31a-137393204f0a@googlegroups.com>

? 2015?5?14???? UTC+8??11:45:38?Steven D'Aprano???
> I'd like to do a little survey, and get a quick show of hands.
> 
> How many people have written GUI or text-based applications or scripts where
> a "Move file to trash" function would be useful?
> 
> Would you like to see that in the standard library, even if it meant that
> the library had feature-freeze and could gain no more functionality?
> 
> 
> -- 
> Steven

Send2Trash is a small package that sends files to the Trash
https://pypi.python.org/pypi/Send2Trash

I am using PyQt, I would like to see that in the standard library


From rosuav at gmail.com  Mon May 18 06:38:23 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 18 May 2015 20:38:23 +1000
Subject: Python 3: yum is dead, long live dnf!
In-Reply-To: <mjcf17$tvu$1@ger.gmane.org>
References: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>
 <mjcea8$7ca$1@speranza.aioe.org> <mjcf17$tvu$1@ger.gmane.org>
Message-ID: <CAPTjJmpQb7r8kfYYGeEnu4NnwT9cC7gJoFR=vXvtJg0mPp4mRw@mail.gmail.com>

On Mon, May 18, 2015 at 8:30 PM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> As an aside, from the signature of your message, I thought "In war, truth is
> the first casualty." was first said by Winston Churchill, not U Thant?

I prefer the White King's line. Alice commented that she should
perhaps be grateful for the truth, even though a lie might well have
been more persuasive; the King replies, "In war, truth first! There's
always time for lies later."

ChrisA


From alister.nospam.ware at ntlworld.com  Mon May 18 06:58:18 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Mon, 18 May 2015 10:58:18 +0000 (UTC)
Subject: Python 3: yum is dead, long live dnf!
References: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>
 <mjcea8$7ca$1@speranza.aioe.org>
 <mailman.101.1431945074.17265.python-list@python.org>
Message-ID: <mjcgka$ctq$1@speranza.aioe.org>

On Mon, 18 May 2015 11:30:57 +0100, Mark Lawrence wrote:

> On 18/05/2015 11:18, alister wrote:
>> On Mon, 18 May 2015 15:21:05 +1000, Steven D'Aprano wrote:
>>
>>> As part of Red Hat's move to Python 3, yum is officially deprecated
>>> and replaced by dnf:
>>>
>>> http://dnf.baseurl.org/2015/05/11/yum-is-dead-long-live-dnf/
>>>
>>> Quote:
>>>
>>>      Yum would not survive the ?Python 3 as default? Fedora initiative
>>>      meanwhile DNF is able to run on Python 2 and Python 3.
>>
>>
>> Poor choice of name DNF= Did Not Finish
>>
>>
> Does Not Fail?
> 
> As an aside, from the signature of your message, I thought "In war,
> truth is the first casualty." was first said by Winston Churchill, not U
> Thant?

Dont ask me, it is generated by fortune running on my Linux box


-- 
The aim of a joke is not to degrade the human being but to remind him that
he is already degraded.
		-- George Orwell


From steve+comp.lang.python at pearwood.info  Mon May 18 07:49:08 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Mon, 18 May 2015 21:49:08 +1000
Subject: Python 3: yum is dead, long live dnf!
References: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>
 <mjcea8$7ca$1@speranza.aioe.org>
Message-ID: <5559d1b5$0$12990$c3e8da3$5496439d@news.astraweb.com>

On Mon, 18 May 2015 08:18 pm, alister wrote:

> On Mon, 18 May 2015 15:21:05 +1000, Steven D'Aprano wrote:
> 
>> As part of Red Hat's move to Python 3, yum is officially deprecated and
>> replaced by dnf:
>> 
>> http://dnf.baseurl.org/2015/05/11/yum-is-dead-long-live-dnf/
>> 
>> Quote:
>> 
>>     Yum would not survive the ?Python 3 as default? Fedora initiative
>>     meanwhile DNF is able to run on Python 2 and Python 3.
> 
> 
> Poor choice of name DNF= Did Not Finish

:-)

Computer programs, like art, are never finished, merely abandoned.

Except for TeX, of course, which upon Donald Knuth's death will be released
as version ? and all remaining bugs declared to be features.




-- 
Steven



From nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de  Mon May 18 08:00:12 2015
From: nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de (Thomas Rachel)
Date: Mon, 18 May 2015 14:00:12 +0200
Subject: Rule of order for dot operators?
In-Reply-To: <mailman.80.1431804386.17265.python-list@python.org>
References: <mailman.80.1431804386.17265.python-list@python.org>
Message-ID: <mjck8h$o18$1@r01.glglgl.de>

Am 16.05.2015 um 21:20 schrieb C.D. Reimer:

> Does python perform the dot operators from left to right or according to
> a rule of order (i.e., multiplication/division before add/subtract)?

In this case, it does the only thing it can do:

title = slug.replace('-',' ').title()

is performed as

* take slug
* get its replace method
* call it and take the result
* get this result's title method
* call it and store its result into title.

OTOH,

title = slug.title().replace('-',' ')

is performed as

* take slug
* get its title method
* call it and take the result
* get this result's replace method
* call it and store its result into title.

Any other order would be unintuitive and just wrong.

The reason why the result is the same is because .title() title-cases 
letters after a space as well as after a '-'.

In other cases, it wouldn't do so, so you have to take care what you do.


Thomas


From alister.nospam.ware at ntlworld.com  Mon May 18 08:08:05 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Mon, 18 May 2015 12:08:05 +0000 (UTC)
Subject: Python 3: yum is dead, long live dnf!
References: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>
 <mjcea8$7ca$1@speranza.aioe.org> <mjcet1$8p0$1@speranza.aioe.org>
 <mailman.105.1431950679.17265.python-list@python.org>
Message-ID: <mjckn5$ngs$1@speranza.aioe.org>

On Mon, 18 May 2015 15:08:07 +0300, Mihamina Rakotomandimby wrote:

> On 05/18/2015 01:28 PM, alister wrote:
>> Which may be fitting it just waisted 10 min downloading everything
>> before discovering I did not have permission (forgot to sudo)
> 
> I think if you resume the transaction, downloaded things are locally
> cached: aren't they?

No, it downloaded them all again



-- 
Don't wake me up too soon...
Gonna take a ride across the moon...
You and me.


From mihamina.rakotomandimby at rktmb.org  Mon May 18 08:08:07 2015
From: mihamina.rakotomandimby at rktmb.org (Mihamina Rakotomandimby)
Date: Mon, 18 May 2015 15:08:07 +0300
Subject: Python 3: yum is dead, long live dnf!
In-Reply-To: <mjcet1$8p0$1@speranza.aioe.org>
References: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>
 <mjcea8$7ca$1@speranza.aioe.org> <mjcet1$8p0$1@speranza.aioe.org>
Message-ID: <5559D627.7010401@rktmb.org>

On 05/18/2015 01:28 PM, alister wrote:
> Which may be fitting
> it just waisted 10 min downloading everything before discovering I did
> not have permission (forgot to sudo)

I think if you resume the transaction, downloaded things are locally 
cached: aren't they?


From mihamina.rakotomandimby at rktmb.org  Mon May 18 08:15:31 2015
From: mihamina.rakotomandimby at rktmb.org (Mihamina Rakotomandimby)
Date: Mon, 18 May 2015 15:15:31 +0300
Subject: Python 3: yum is dead, long live dnf!
In-Reply-To: <mjckn5$ngs$1@speranza.aioe.org>
References: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>
 <mjcea8$7ca$1@speranza.aioe.org> <mjcet1$8p0$1@speranza.aioe.org>
 <mailman.105.1431950679.17265.python-list@python.org>
 <mjckn5$ngs$1@speranza.aioe.org>
Message-ID: <5559D7E3.7020309@rktmb.org>

On 05/18/2015 03:08 PM, alister wrote:
> On Mon, 18 May 2015 15:08:07 +0300, Mihamina Rakotomandimby wrote:
>
>> On 05/18/2015 01:28 PM, alister wrote:
>>> Which may be fitting it just waisted 10 min downloading everything
>>> before discovering I did not have permission (forgot to sudo)
>> I think if you resume the transaction, downloaded things are locally
>> cached: aren't they?
> No, it downloaded them all again
>
>

That should be reprted as a usefull missing feature...


From breamoreboy at yahoo.co.uk  Mon May 18 09:46:35 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 18 May 2015 14:46:35 +0100
Subject: Survey -- Move To Trash function in Python?
In-Reply-To: <4d32dbce-a69f-4fd5-b31a-137393204f0a@googlegroups.com>
References: <5554c318$0$12999$c3e8da3$5496439d@news.astraweb.com>
 <4d32dbce-a69f-4fd5-b31a-137393204f0a@googlegroups.com>
Message-ID: <mjcqg0$2c6$1@ger.gmane.org>

On 18/05/2015 11:31, iMath wrote:
> ? 2015?5?14???? UTC+8??11:45:38?Steven D'Aprano???
>> I'd like to do a little survey, and get a quick show of hands.
>>
>> How many people have written GUI or text-based applications or scripts where
>> a "Move file to trash" function would be useful?
>>
>> Would you like to see that in the standard library, even if it meant that
>> the library had feature-freeze and could gain no more functionality?
>>
>>
>> --
>> Steven
>
> Send2Trash is a small package that sends files to the Trash
> https://pypi.python.org/pypi/Send2Trash
>
> I am using PyQt, I would like to see that in the standard library
>

There are lots of people who would like to see "anything that you'd care 
to name" in the stdlib.  My preference is to see it remain very stable, 
so that "Python in a Nutshell" still fits into a pocket, and not a 44 
ton articulated lorry.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Mon May 18 09:50:37 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 18 May 2015 14:50:37 +0100
Subject: Python 3: yum is dead, long live dnf!
In-Reply-To: <5559d1b5$0$12990$c3e8da3$5496439d@news.astraweb.com>
References: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>
 <mjcea8$7ca$1@speranza.aioe.org>
 <5559d1b5$0$12990$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mjcqni$6ie$1@ger.gmane.org>

On 18/05/2015 12:49, Steven D'Aprano wrote:
> On Mon, 18 May 2015 08:18 pm, alister wrote:
>
>> On Mon, 18 May 2015 15:21:05 +1000, Steven D'Aprano wrote:
>>
>>> As part of Red Hat's move to Python 3, yum is officially deprecated and
>>> replaced by dnf:
>>>
>>> http://dnf.baseurl.org/2015/05/11/yum-is-dead-long-live-dnf/
>>>
>>> Quote:
>>>
>>>      Yum would not survive the ?Python 3 as default? Fedora initiative
>>>      meanwhile DNF is able to run on Python 2 and Python 3.
>>
>>
>> Poor choice of name DNF= Did Not Finish
>
> :-)
>
> Computer programs, like art, are never finished, merely abandoned.
>
> Except for TeX, of course, which upon Donald Knuth's death will be released
> as version ? and all remaining bugs declared to be features.
>

I didn't realise that there is an emoticon depicting Stonehenge, thanks 
for that :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From subhabrata.banerji at gmail.com  Mon May 18 11:05:38 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Mon, 18 May 2015 08:05:38 -0700 (PDT)
Subject: Flask Post returning error
Message-ID: <304362fc-8c0c-49d6-8ef5-a60f05179c42@googlegroups.com>

Dear Group,

I am trying to practice Flask and trying to correspond with it with through requests. 

I am being able to DELETE, GET. But as I am doing POST it is not posting the data rather returning null. 

I tried to search Flask and requests tutorials but did not get much. 
I am newly practising Flask, and on Python2.7+ with Windows 7 Professional. 

>>> var1=requests.get('http://127.0.0.1:5000/todos')
>>> var2=var2.text
>>> print var3
{
    "todo1": {
        "task": "build an API"
    }, 
    "todo2": {
        "task": "How are you?"
    }, 
    "todo3": {
        "task": "profit!"
    }
}
>>> import json
>>> payload = {'key1': 'value1'}
>>> var4=requests.post('http://127.0.0.1:5000/todos', data=json.dumps(payload))
>>> var5=requests.get('http://127.0.0.1:5000/todos')
>>> var6=var5.text
>>> print var6
{
    "todo1": {
        "task": "build an API"
    }, 
    "todo3": {
        "task": null
    }
}

If anyone may kindly suggest what is the error I am doing. 

Regards,
Subhabrata Banerjee. 


From breamoreboy at yahoo.co.uk  Mon May 18 13:41:47 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 18 May 2015 18:41:47 +0100
Subject: Green Tree Snakes - the missing Python AST docs
Message-ID: <mjd891$kd1$1@ger.gmane.org>

I wondered what on earth "Green Tree Snakes" was referring to until I 
caught on to the AST part.  So for those of you who like diving under 
the bonnet and don't care about possibly, or even probably, very dirty 
hands, here's a link.

https://greentreesnakes.readthedocs.org/en/latest/nodes.html

Giga kudos to Thomas Kluyver methinks.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From marfig at gmail.com  Mon May 18 15:23:37 2015
From: marfig at gmail.com (Mario Figueiredo)
Date: Mon, 18 May 2015 20:23:37 +0100
Subject: Slices time complexity
Message-ID: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>

I'd like to understand what I'm being told about slices in
https://wiki.python.org/moin/TimeComplexity

Particularly, what's a 'del slice' and a 'set slice' and whether this
information pertains to both CPython 2.7 and 3.4.

>From the above link it seems slices work in linear time on all cases.
And this really has a big impact on certain operations. For instance,
the code below may surprise some people when they realize it doesn't
run in linear time on 3.4:

    def minimum(values):
        if len(values) == 1:
            return values[0]
        else:
            m = minimum(values[1:])
            return m if m < values[0] else values[0]

Other languages implement slices. I'm currently being faced with a Go
snippet that mirrors the exact code above and it does run in linear
time.

Is there any reason why Python 3.4 implementation of slices cannot be
a near constant operation?


From rosuav at gmail.com  Mon May 18 15:36:44 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 19 May 2015 05:36:44 +1000
Subject: Slices time complexity
In-Reply-To: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
Message-ID: <CAPTjJmoYvxrQBfzR+8cWE1y=RJsL6oiKV_wdCz2KiOkPr6piRQ@mail.gmail.com>

On Tue, May 19, 2015 at 5:23 AM, Mario Figueiredo <marfig at gmail.com> wrote:
> From the above link it seems slices work in linear time on all cases.
> And this really has a big impact on certain operations. For instance,
> the code below may surprise some people when they realize it doesn't
> run in linear time on 3.4:
>
>     def minimum(values):
>         if len(values) == 1:
>             return values[0]
>         else:
>             m = minimum(values[1:])
>             return m if m < values[0] else values[0]

https://xkcd.com/1270/

Is there really a reason to code this in such a bizarrely inefficient
way? Linear or not, it's bound to be less efficient than the more
obvious form:

def minimum(values):
    values = iter(values)
    min = next(values)
    for value in values:
        if value < min: min = value
    return min

And if you know your value domain (maybe you're working with floats,
or positive integers, or something) and can put a hard-coded base
value in, it becomes even simpler:

def minimum(values):
    min = 0 # or float("-inf") etc
    for value in values:
        if value < min: min = value
    return min

It's obvious that this code will complete in linear time. It's also
pretty obvious how it's working: it steps through the collection,
comparing each value against the current smallest. With your recursive
version, it effectively steps backward through the list, comparing
each value against the current smallest, all while unwinding the
stack. It can't even be tail-call-optimized in its current state.

What's the point of optimizing slicing to allow you to use a poor
algorithm, instead of fixing your algorithm?

ChrisA


From toddrjen at gmail.com  Mon May 18 15:42:11 2015
From: toddrjen at gmail.com (Todd)
Date: Mon, 18 May 2015 21:42:11 +0200
Subject: Slices time complexity
In-Reply-To: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
Message-ID: <CAFpSVpKiTKFCahdgn0rDN0XYtdcZDR=P-4YrXeUFv9tVFWFYAg@mail.gmail.com>

On May 18, 2015 9:26 PM, "Mario Figueiredo" <marfig at gmail.com> wrote:
>
> I'd like to understand what I'm being told about slices in
> https://wiki.python.org/moin/TimeComplexity
>
> Particularly, what's a 'del slice' and a 'set slice' and whether this
> information pertains to both CPython 2.7 and 3.4.
>
> From the above link it seems slices work in linear time on all cases.
> And this really has a big impact on certain operations. For instance,
> the code below may surprise some people when they realize it doesn't
> run in linear time on 3.4:
>
>     def minimum(values):
>         if len(values) == 1:
>             return values[0]
>         else:
>             m = minimum(values[1:])
>             return m if m < values[0] else values[0]
>
> Other languages implement slices. I'm currently being faced with a Go
> snippet that mirrors the exact code above and it does run in linear
> time.
>
> Is there any reason why Python 3.4 implementation of slices cannot be
> a near constant operation?

In this case you are copying the items (or rather pointers to the items)
from the list to a new list. This is inherently a O(k) operation.

There are other ways to implement it. I recall the was talk of a way to get
views of sequences, although this would involve keeping the original list
in memory and any changes to the new list would be passed to the original
(this is how numpy works) . It is also possible to do copy-on-write, which
avoids altering the original list but has the same memory problems while
still involving a O(k) copy operation if you want to change the list, and
it is more complex to implement (this is how MATLAB works) .

But to have a new list that can be edited independently requires coping
something, and that will be a O(k) operation, unless you use a radically
different data structure with its own limitations.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150518/6d30fa63/attachment.html>

From marfig at gmail.com  Mon May 18 15:49:10 2015
From: marfig at gmail.com (Mario Figueiredo)
Date: Mon, 18 May 2015 20:49:10 +0100
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.112.1431977807.17265.python-list@python.org>
Message-ID: <icgklalal1m13g6bch4v5mhf7bv80hfspt@4ax.com>

On Tue, 19 May 2015 05:36:44 +1000, Chris Angelico <rosuav at gmail.com>
wrote:

>What's the point of optimizing slicing to allow you to use a poor
>algorithm, instead of fixing your algorithm?
>

Chris, thank you for your input. But the code isn't really the
question, is it?

It's just an example. It was being used earlier to demonstrate Time
Complexity calculations in another forum. It's not real live code.
Never will be. Besides we already have a min() function in Python.


From ian.g.kelly at gmail.com  Mon May 18 15:49:45 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 18 May 2015 13:49:45 -0600
Subject: Slices time complexity
In-Reply-To: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
Message-ID: <CALwzid=v+1FMy1tTAc=Ps7EOChETxMddNj1UA+Jxt9mfrr7obg@mail.gmail.com>

On Mon, May 18, 2015 at 1:23 PM, Mario Figueiredo <marfig at gmail.com> wrote:
> I'd like to understand what I'm being told about slices in
> https://wiki.python.org/moin/TimeComplexity
>
> Particularly, what's a 'del slice' and a 'set slice' and whether this
> information pertains to both CPython 2.7 and 3.4.

"Del Slice" is the operation where a slice of a list is deleted, and
"Set Slice" is the operation where a slice is replaced. E.g.:

>>> x = list(range(100))
>>> del x[2:98]
>>> x
[0, 1, 98, 99]
>>> x[1:3] = [7, 6, 5, 4, 3]
>>> x
[0, 7, 6, 5, 4, 3, 99]


> Other languages implement slices. I'm currently being faced with a Go
> snippet that mirrors the exact code above and it does run in linear
> time.
>
> Is there any reason why Python 3.4 implementation of slices cannot be
> a near constant operation?

The semantics are different. IIRC, a slice in Go is just a view of
some underlying array; if you change the array (or some other slice of
it), the change will be reflected in the slice. A slice of a list in
Python, OTOH, constructs a completely independent list.

It may be possible that lists in CPython could be made to share their
internal arrays with other lists on a copy-on-write basis, which could
allow slicing to be O(1) as long as neither list is modified while the
array is being shared. I expect this would be a substantial piece of
work, and I don't know if it's something that anybody has looked into.


From fabien.maussion at gmail.com  Mon May 18 15:54:20 2015
From: fabien.maussion at gmail.com (Fabien)
Date: Mon, 18 May 2015 21:54:20 +0200
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
Message-ID: <mjdg1a$2ig$1@speranza.aioe.org>

On 05/18/2015 09:49 PM, Ian Kelly wrote:
> It may be possible that lists in CPython could be made to share their
> internal arrays with other lists on a copy-on-write basis, which could
> allow slicing to be O(1) as long as neither list is modified while the
> array is being shared. I expect this would be a substantial piece of
> work, and I don't know if it's something that anybody has looked into.

Isn't Numpy doing this (not sure, not a python nor a numpy expert):

 >>> import numpy as np
 >>> a = np.array([1,2,3,4])
 >>> b = a[1:]
 >>> b[0] = 9
 >>> a
array([1, 9, 3, 4])


Fabien


From rosuav at gmail.com  Mon May 18 16:04:07 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 19 May 2015 06:04:07 +1000
Subject: Slices time complexity
In-Reply-To: <icgklalal1m13g6bch4v5mhf7bv80hfspt@4ax.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.112.1431977807.17265.python-list@python.org>
 <icgklalal1m13g6bch4v5mhf7bv80hfspt@4ax.com>
Message-ID: <CAPTjJmprn92cM-ueLSupQreW3K+NRxdtjEyrD5GyA+YrAtq9-w@mail.gmail.com>

On Tue, May 19, 2015 at 5:49 AM, Mario Figueiredo <marfig at gmail.com> wrote:
> On Tue, 19 May 2015 05:36:44 +1000, Chris Angelico <rosuav at gmail.com>
> wrote:
>
>>What's the point of optimizing slicing to allow you to use a poor
>>algorithm, instead of fixing your algorithm?
>>
>
> Chris, thank you for your input. But the code isn't really the
> question, is it?
>
> It's just an example. It was being used earlier to demonstrate Time
> Complexity calculations in another forum. It's not real live code.
> Never will be. Besides we already have a min() function in Python.

General rule of optimization: Do the simplest thing first, and make it
more complicated only if the speed benefit is worth it. In this case,
slicing a list is done in the obvious and simple way: construct a new
list of the appropriate length, and assign all its elements. (The
details may be optimized some at the C level, but it still constructs
a new list.) You're asking why Python doesn't have a much more
complicated system (Todd suggests views and COW; another way is to do
a LISP-style linked list, which has similar consequences to views, but
is more efficient if you do this specific thing of processing the
first element and recursing for the rest), and the answer is: There's
always a better way to write your algorithm.

So if your code is intended to demonstrate how a poor algorithm can
turn a linear task into a quadratic one, congrats! You've succeeded.
If you're trying to showcase how terrible Python is, well, I'm sure
you could do that in more effective ways, but they'll still come down
to trying to write <insert other language name here> code using the
Python interpreter. If you write idiomatic Python code, using
iterators and loops rather than recursion, you'll find your code is
cleaner and faster than it would be if you fight against the language.

ChrisA


From toddrjen at gmail.com  Mon May 18 16:23:41 2015
From: toddrjen at gmail.com (Todd)
Date: Mon, 18 May 2015 22:23:41 +0200
Subject: Slices time complexity
In-Reply-To: <mjdg1a$2ig$1@speranza.aioe.org>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <mjdg1a$2ig$1@speranza.aioe.org>
Message-ID: <CAFpSVpKui2FHRp5pU9L759sLBftLv-bKmxaeBdr2knJw8nbLTQ@mail.gmail.com>

On May 18, 2015 9:56 PM, "Fabien" <fabien.maussion at gmail.com> wrote:
>
> On 05/18/2015 09:49 PM, Ian Kelly wrote:
>>
>> It may be possible that lists in CPython could be made to share their
>> internal arrays with other lists on a copy-on-write basis, which could
>> allow slicing to be O(1) as long as neither list is modified while the
>> array is being shared. I expect this would be a substantial piece of
>> work, and I don't know if it's something that anybody has looked into.
>
>
> Isn't Numpy doing this (not sure, not a python nor a numpy expert):
>
> >>> import numpy as np
> >>> a = np.array([1,2,3,4])
> >>> b = a[1:]
> >>> b[0] = 9
> >>> a
> array([1, 9, 3, 4])
>
>
> Fabien

Numpy arrays use views. Matlab arrays use copy-on-write.  But as discussed
in the recent thread about string views, these approaches have a memory
penalty since they require keeping the original array in memory. And the
copy-on-write approach still has a O(k) complexity if you try to make any
changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150518/010fd6cb/attachment.html>

From cybervigilante at gmail.com  Mon May 18 16:59:55 2015
From: cybervigilante at gmail.com (Jim Mooney)
Date: Mon, 18 May 2015 13:59:55 -0700 (PDT)
Subject: -- redacted --
In-Reply-To: <68c91385-4595-43e5-b704-adced93d0930@googlegroups.com>
References: <1baf4641-757b-4e30-8be1-f95b53ebc12c@googlegroups.com>
 <68c91385-4595-43e5-b704-adced93d0930@googlegroups.com>
Message-ID: <12040758-da54-4e73-8ac3-3ee41e87f1c9@googlegroups.com>

-- redacted --


From marfig at gmail.com  Mon May 18 17:04:43 2015
From: marfig at gmail.com (Mario Figueiredo)
Date: Mon, 18 May 2015 22:04:43 +0100
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
Message-ID: <19kkla9krk0auuivanhh39lfc8i1csf2ja@4ax.com>

On Mon, 18 May 2015 13:49:45 -0600, Ian Kelly <ian.g.kelly at gmail.com>
wrote:

>> Other languages implement slices. I'm currently being faced with a Go
>> snippet that mirrors the exact code above and it does run in linear
>> time.
>>
>> Is there any reason why Python 3.4 implementation of slices cannot be
>> a near constant operation?
>
>The semantics are different. IIRC, a slice in Go is just a view of
>some underlying array; if you change the array (or some other slice of
>it), the change will be reflected in the slice. A slice of a list in
>Python, OTOH, constructs a completely independent list.
>
>It may be possible that lists in CPython could be made to share their
>internal arrays with other lists on a copy-on-write basis, which could
>allow slicing to be O(1) as long as neither list is modified while the
>array is being shared. I expect this would be a substantial piece of
>work, and I don't know if it's something that anybody has looked into.

This is what I was after. Thank you Ian.

So we basically don't have a view of a list. Makes sense now, since
slice does create a new list. I should have seen that. Thank you once
again.

It would a good addition though. Even if only on specialized
implementations like pypy. Inspecting and not changing lists is a good
chunk of our code. But that's besides the point. I was more interested
in understanding the behavior. Thank you once again.


From breamoreboy at yahoo.co.uk  Mon May 18 17:38:26 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 18 May 2015 22:38:26 +0100
Subject: Slices time complexity
In-Reply-To: <19kkla9krk0auuivanhh39lfc8i1csf2ja@4ax.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <19kkla9krk0auuivanhh39lfc8i1csf2ja@4ax.com>
Message-ID: <mjdm4o$9fe$1@ger.gmane.org>

On 18/05/2015 22:04, Mario Figueiredo wrote:
> On Mon, 18 May 2015 13:49:45 -0600, Ian Kelly <ian.g.kelly at gmail.com>
> wrote:
>
>>> Other languages implement slices. I'm currently being faced with a Go
>>> snippet that mirrors the exact code above and it does run in linear
>>> time.
>>>
>>> Is there any reason why Python 3.4 implementation of slices cannot be
>>> a near constant operation?
>>
>> The semantics are different. IIRC, a slice in Go is just a view of
>> some underlying array; if you change the array (or some other slice of
>> it), the change will be reflected in the slice. A slice of a list in
>> Python, OTOH, constructs a completely independent list.
>>
>> It may be possible that lists in CPython could be made to share their
>> internal arrays with other lists on a copy-on-write basis, which could
>> allow slicing to be O(1) as long as neither list is modified while the
>> array is being shared. I expect this would be a substantial piece of
>> work, and I don't know if it's something that anybody has looked into.
>
> This is what I was after. Thank you Ian.
>
> So we basically don't have a view of a list. Makes sense now, since
> slice does create a new list. I should have seen that. Thank you once
> again.
>
> It would a good addition though. Even if only on specialized
> implementations like pypy. Inspecting and not changing lists is a good
> chunk of our code. But that's besides the point. I was more interested
> in understanding the behavior. Thank you once again.
>

Not directly affecting slices but the idea of views has come into Python 
3, please see:-

https://www.python.org/dev/peps/pep-3118/

https://docs.python.org/3/whatsnew/3.3.html#pep-3118-new-memoryview-implementation-and-buffer-protocol-documentation

https://docs.python.org/3/library/stdtypes.html#typememoryview

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From cs at zip.com.au  Mon May 18 18:29:50 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 19 May 2015 08:29:50 +1000
Subject: Rule of order for dot operators?
In-Reply-To: <55579886.3010001@cdreimer.com>
References: <55579886.3010001@cdreimer.com>
Message-ID: <20150518222950.GA49144@cskk.homeip.net>

On 16May2015 12:20, C.D. Reimer <chris at cdreimer.com> wrote:
>title = slug.replace('-',' ').title()
>This line also works if I switched the dot operators around.
>title = slug.title().replace('-',' ')
>
>I'm reading the first example as character replacement first and title 
>capitalization second, and the second example as title capitalization 
>first and character replacement second.
>
>Does python perform the dot operators from left to right or according
>to a rule of order (i.e., multiplication/division before add/subtract)?

I've been thinking about the mindset that asks this question.

I think the reason you needed to ask it was that you were thinking in terms of 
each .foo() operation acting on the original "slug". They do not.

"slug" is an expression returning, of course, itself.

"slug.title()" is an expression returning a new string which is a titlecased 
version of "slug".

"slug.title().replace(...)" is an expression which _operates on_ that new 
string, _not_ on "slug".

In particular, each .foo() need not return a string - it might return anything, 
and the following .bah() will work on that anything.

Cheers,
Cameron Simpson <cs at zip.com.au>


From tjreedy at udel.edu  Mon May 18 19:04:32 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 18 May 2015 19:04:32 -0400
Subject: Slices time complexity
In-Reply-To: <19kkla9krk0auuivanhh39lfc8i1csf2ja@4ax.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <19kkla9krk0auuivanhh39lfc8i1csf2ja@4ax.com>
Message-ID: <mjdr63$uh0$1@ger.gmane.org>

On 5/18/2015 5:04 PM, Mario Figueiredo wrote:

>>> Other languages implement slices. I'm currently being faced with a Go
>>> snippet that mirrors the exact code above and it does run in linear
>>> time.

>>> Is there any reason why Python 3.4 implementation of slices cannot be
>>> a near constant operation?
>>
>> The semantics are different. IIRC, a slice in Go is just a view of
>> some underlying array; if you change the array (or some other slice of
>> it), the change will be reflected in the slice. A slice of a list in
>> Python, OTOH, constructs a completely independent list.
>>
>> It may be possible that lists in CPython could be made to share their
>> internal arrays with other lists on a copy-on-write basis, which could
>> allow slicing to be O(1) as long as neither list is modified while the
>> array is being shared. I expect this would be a substantial piece of
>> work, and I don't know if it's something that anybody has looked into.
>
> This is what I was after. Thank you Ian.
>
> So we basically don't have a view of a list.

Actually we do if you think about things the right way.  An index can be 
viewed as representing the slice of a list from the indexed item to the 
end.  In this view, "for i in range(len(seq)):" works with progressively 
shrinking slices, the same as with the recursive version of the algorithm.

The analogy is better with iterators.  iter(seq) returns a seq_iterator 
that initially represent a tail slice consisting of the entire sequence. 
  Each next(seq_iter) call return the head of the sequence and mutates 
seq_iter to represent a reduced tail-slice.  The effect is the same as 
repeatedly stripping the head from a linked list. For statements 
automate the next calls and StopIteration checks.

-- 
Terry Jan Reedy



From greg.ewing at canterbury.ac.nz  Mon May 18 19:59:24 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Tue, 19 May 2015 11:59:24 +1200
Subject: Python 3: yum is dead, long live dnf!
In-Reply-To: <mailman.101.1431945074.17265.python-list@python.org>
References: <555976c1$0$2804$c3e8da3$76491128@news.astraweb.com>
 <mjcea8$7ca$1@speranza.aioe.org>
 <mailman.101.1431945074.17265.python-list@python.org>
Message-ID: <crvcmsF1un0U1@mid.individual.net>

Mark Lawrence wrote:
> As an aside, from the signature of your message, I thought "In war, 
> truth is the first casualty." was first said by Winston Churchill,

But... did he say it during wartime?

-- 
Greg


From rustompmody at gmail.com  Mon May 18 21:32:11 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 18 May 2015 18:32:11 -0700 (PDT)
Subject: Rule of order for dot operators?
In-Reply-To: <mailman.118.1431989304.17265.python-list@python.org>
References: <55579886.3010001@cdreimer.com>
 <mailman.118.1431989304.17265.python-list@python.org>
Message-ID: <fabab250-d7a5-49f4-a9ef-359bd003a607@googlegroups.com>

On Tuesday, May 19, 2015 at 4:18:36 AM UTC+5:30, Cameron Simpson wrote:
> On 16May2015 12:20, C.D. Reimer  wrote:
> >title = slug.replace('-',' ').title()
> >This line also works if I switched the dot operators around.
> >title = slug.title().replace('-',' ')
> >
> >I'm reading the first example as character replacement first and title 
> >capitalization second, and the second example as title capitalization 
> >first and character replacement second.
> >
> >Does python perform the dot operators from left to right or according
> >to a rule of order (i.e., multiplication/division before add/subtract)?
> 
> I've been thinking about the mindset that asks this question.
> 
> I think the reason you needed to ask it was that you were thinking in terms of 
> each .foo() operation acting on the original "slug". They do not.
> 
> "slug" is an expression returning, of course, itself.
> 
> "slug.title()" is an expression returning a new string which is a titlecased 
> version of "slug".
> 
> "slug.title().replace(...)" is an expression which _operates on_ that new 
> string, _not_ on "slug".
> 
> In particular, each .foo() need not return a string - it might return anything, 
> and the following .bah() will work on that anything.

For an arbitrary binary operator ?
x ? y ? z
can group as
(x?y)?z
or
x?(y?z)

One could (conceivably) apply the same rule to x.y.z
Except that x.(y.z) is a bit hard to give a meaning to!!


From rustompmody at gmail.com  Mon May 18 22:20:39 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 18 May 2015 19:20:39 -0700 (PDT)
Subject: Slices time complexity
In-Reply-To: <mailman.114.1431978628.17265.python-list@python.org>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
Message-ID: <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>

On Tuesday, May 19, 2015 at 1:20:55 AM UTC+5:30, Ian wrote:
> It may be possible that lists in CPython could be made to share their
> internal arrays with other lists on a copy-on-write basis, which could
> allow slicing to be O(1) as long as neither list is modified while the
> array is being shared. I expect this would be a substantial piece of
> work, and I don't know if it's something that anybody has looked into.

One fundamental difference/choice in language semantics is copy vs reference
C#/.Net call it value-type and reference-type. [The names may be a bit odd...]

Functional languages OTOH tend to the philosophy:
- maximise sharing internally
- mutability taboo
which has its own costs -- eg 'obvious' data structures like arrays become hard/impossible

Its quite likely that your proposal - slices as COW-views -- will have 
significant penalties in other usage scenarios.  Imagine plastering more and 
more complex slice-views on an array inter-mixed with updates.

I must say I am impressed by C#/.Net for making the value/object distinction
first-class all the way from language to VM.


From ron3200 at gmail.com  Mon May 18 22:43:49 2015
From: ron3200 at gmail.com (Ron Adam)
Date: Mon, 18 May 2015 22:43:49 -0400
Subject: Rule of order for dot operators?
In-Reply-To: <fabab250-d7a5-49f4-a9ef-359bd003a607@googlegroups.com>
References: <55579886.3010001@cdreimer.com>
 <mailman.118.1431989304.17265.python-list@python.org>
 <fabab250-d7a5-49f4-a9ef-359bd003a607@googlegroups.com>
Message-ID: <mje816$2kd$1@ger.gmane.org>



On 05/18/2015 09:32 PM, Rustom Mody wrote:
>> >In particular, each .foo() need not return a string - it might return anything,
>> >and the following .bah() will work on that anything.
> For an arbitrary binary operator ?
> x ? y ? z
> can group as
> (x?y)?z
> or
> x?(y?z)
>
> One could (conceivably) apply the same rule to x.y.z
> Except that x.(y.z) is a bit hard to give a meaning to!!

Yes.

Having just implementing something similar for nested scopes, it turns out 
it can't be operators because if it was, then the names y and z would be 
resolved in the wrong scope.

          y = "m"
          z = "n"
          a = x . y . z

Which of course wouldn't do what we want.

          a = x . "m" . "n"

And most likely this would give an error.


The name-part after the dot is evaluated in the next inner scope.  y is 
resolved in x's scope, and z is resolved in y's scope.

Which is why you can implement objects with closures, but you need to delay 
name resolution to do that.   Which is what the "." does.


Pythons attribute lookup is a bit more complex than this of course.

      https://docs.python.org/3.4/howto/descriptor.html


Cheers,
    Ron







From miki.tebeka at gmail.com  Tue May 19 01:04:25 2015
From: miki.tebeka at gmail.com (Miki Tebeka)
Date: Mon, 18 May 2015 22:04:25 -0700 (PDT)
Subject: Flask Post returning error
In-Reply-To: <304362fc-8c0c-49d6-8ef5-a60f05179c42@googlegroups.com>
References: <304362fc-8c0c-49d6-8ef5-a60f05179c42@googlegroups.com>
Message-ID: <5fda31ec-b4ba-4be1-a21a-01db5c428058@googlegroups.com>

> If anyone may kindly suggest what is the error I am doing. 
It's close to impossible to know without seeing the server side code.


From steve+comp.lang.python at pearwood.info  Tue May 19 01:13:58 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 19 May 2015 15:13:58 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
Message-ID: <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 19 May 2015 12:20, Rustom Mody wrote:

> I must say I am impressed by C#/.Net for making the value/object
> distinction first-class all the way from language to VM.

I'm not sure what you mean by that. Are you referring to something similar 
to Java's distinction between native/unboxed types versus objects and boxed 
values?

Apart from the possible efficiency gains, what benefit do you see from 
distinguishing between "values which are objects" versus "values which are 
not objects"?


-- 
Steve


From rustompmody at gmail.com  Tue May 19 01:33:29 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 18 May 2015 22:33:29 -0700 (PDT)
Subject: Slices time complexity
In-Reply-To: <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>

On Tuesday, May 19, 2015 at 10:44:10 AM UTC+5:30, Steven D'Aprano wrote:
> On Tuesday 19 May 2015 12:20, Rustom Mody wrote:
> 
> > I must say I am impressed by C#/.Net for making the value/object
> > distinction first-class all the way from language to VM.
> 
> I'm not sure what you mean by that. 

Neither am I <wink> 
[Dont know too much about that environment]

> Are you referring to something similar 
> to Java's distinction between native/unboxed types versus objects and boxed 
> values?

Yes except that it seems to be more core to C# than to Java (or python):
https://msdn.microsoft.com/en-us/library/s1ax56ch.aspx
And then this distinction goes all the way down to the CLR
[in ways that I am not very clear about]
eg http://www.informit.com/articles/article.aspx?p=30608&seqNum=3

> Apart from the possible efficiency gains, what benefit do you see from 
> distinguishing between "values which are objects" versus "values which are 
> not objects"?

As I said, in the context of a low level language its probably a bit of a misnomer
However conceptually/pedagogically making a fundamenal distinction of
timeless | time
value | object
immutable | mutable
expression | statement
function | procedure

is key to getting programming [and is something that Pascal got better than most
of its successors].

The FPers want to squeeze the whole world into column 1
The OOPers want to do the opposite and are embarrassed by the existence of
column-1 [status of int in java etc]
Unless one is committed to some philosophical extreme position --
Only One True Way -- I believe accepting two fundamentals is the most sane choice 
 


From rosuav at gmail.com  Tue May 19 02:25:35 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 19 May 2015 16:25:35 +1000
Subject: Rule of order for dot operators?
In-Reply-To: <mje816$2kd$1@ger.gmane.org>
References: <55579886.3010001@cdreimer.com>
 <mailman.118.1431989304.17265.python-list@python.org>
 <fabab250-d7a5-49f4-a9ef-359bd003a607@googlegroups.com>
 <mje816$2kd$1@ger.gmane.org>
Message-ID: <CAPTjJmqf4n4nzxs_TLBc8QAZsLkXRNxY_z8Gb0ZMmkf1kW5rpQ@mail.gmail.com>

On Tue, May 19, 2015 at 12:43 PM, Ron Adam <ron3200 at gmail.com> wrote:
> Having just implementing something similar for nested scopes, it turns out
> it can't be operators because if it was, then the names y and z would be
> resolved in the wrong scope.
>
>          y = "m"
>          z = "n"
>          a = x . y . z
>
> Which of course wouldn't do what we want.
>
>          a = x . "m" . "n"
>
> And most likely this would give an error.

If you want to implement the dot as an operator, you could do it by
having a special syntactic element called an "atom", which is used for
these kinds of identifier-like tokens. The dot operator could then
take an object and an atom, and effectively return getattr(obj,
stringify(atom)). I'm fairly sure this would result in the same syntax
as Python uses.

ChrisA


From marko at pacujo.net  Tue May 19 03:12:38 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 19 May 2015 10:12:38 +0300
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
Message-ID: <878uclf3bt.fsf@elektro.pacujo.net>

Rustom Mody <rustompmody at gmail.com>:

> However conceptually/pedagogically making a fundamenal distinction of
> timeless | time
> value | object
> immutable | mutable
> expression | statement
> function | procedure
>
> is key to getting programming [and is something that Pascal got better
> than most of its successors].
>
> The FPers want to squeeze the whole world into column 1
> The OOPers want to do the opposite and are embarrassed by the existence of
> column-1 [status of int in java etc]
> Unless one is committed to some philosophical extreme position --
> Only One True Way -- I believe accepting two fundamentals is the most
> sane choice

I sympathize. Can you get Python without getting a language like C
first? Can a baby be born without an umbilical cord? Can you skip Newton
and go straight to quantum mechanics and relativity? I have noticed some
experienced Java programmers are a bit lost in the woods because they
don't have an idea of what is going on under the hood.

Scheme almost gets away with the dilemma but must face it with pairs:

    A pair (sometimes called a dotted pair) is a record structure with
    two fields called the car and cdr fields (for historical reasons).
    Pairs are created by the procedure cons. The car and cdr fields are
    accessed by the procedures car and cdr. The car and cdr fields are
    assigned by the procedures set-car! and set-cdr!.

    Pairs are used primarily to represent lists.

    <URL: http://www.schemers.org/Documents/Standards/R5RS/HTM
    L/r5rs-Z-H-9.html#%_sec_6.3.2>

What is a "record structure?" What is a "field?" What is "creation?"

Still, I don't like the dichotomy of boxed/unboxed values. Is a Python
integer an "object" or a "value?" The Faithful have a surefire answer,
but I think the key is: who cares? What matters is the effect of a
program. If two metaphysical formulations of language semantics produce
the same outcome, neither is objectively better than the other.

Pedagogically, I think you could introduce topics with half-truths and
simplifications, and revisit them later with corrections when the
students are better equipped to handle the abstractions.


Marko


From steve+comp.lang.python at pearwood.info  Tue May 19 04:16:48 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 19 May 2015 18:16:48 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
Message-ID: <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 19 May 2015 17:12, Marko Rauhamaa wrote:

> Can you get Python without getting a language like C first? 

Yes.

There are a few senses in which C is very important to Python. Python was 
designed to be a glue language for interoperability with C libraries, so 
that's a very important sense, but in another way it is a trivial sense. You 
could replace C by any other language, or no language at all, and Python 
would still be more or less the same.

Another sense is that Python has borrowed some syntax and concepts from C, 
e.g. zero-based indexing, "continue" and "break" keywords, etc. But these 
are incidentals, the language would be mostly still the same if "continue" 
was spelled "next" in the Pascal tradition.

Python has been influenced by, and borrowed concepts from, a whole slew of 
languages, including Pascal, Algol, Modula 3, Icon, CLU, Lisp, Haskell, TCL, 
Smalltalk, ML, and especially ABC.

[...]
> Still, I don't like the dichotomy of boxed/unboxed values.

Distinguishing the two is important for mere reasons of implementation 
efficiency, but it makes the language semantics messy.


> Is a Python integer an "object" or a "value?" 

Why should that be a dichotomy?

Ints are values. Floats are values. Lists are values. Strings are values. 
Structs and records and union types and even functions are values. Why 
should objects not also be values?


> The Faithful have a surefire answer,
> but I think the key is: who cares? What matters is the effect of a
> program. If two metaphysical formulations of language semantics produce
> the same outcome, neither is objectively better than the other.

By outcome, do you mean the program's output? But the metaphysical 
formulation of the language is irrelevant to the program output. You can get 
the same output in Forth, Haskell, Prolog, C, Applescript or a Turing 
Machine, despite having radically different paradigms.

The outcome which matters is *usability*. Some paradigms are better suited 
to human understanding than others. Complexity and inconsistency is hard. A 
language where the rules for dealing with lists is different from those for 
floats will be harder to use than a language where they are both treated in 
the same way.

I'm not talking about the functionality available to each type -- obviously 
lists and floats use different syntax and different functionality. But if 
you wrote `$spam = alist` to assign a list, and `spam := $afloat` to assign 
a float, that would be bad. A consistent language paradigm is better than an 
inconsistent one. Think of Perl with its complicated rules that you have to 
memorize before you know whether to prefix a variable with a $ or a @ or 
whatever the sigils are.

-- 
Steve



From steve+comp.lang.python at pearwood.info  Tue May 19 04:19:20 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 19 May 2015 18:19:20 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
Message-ID: <555af209$0$12995$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 19 May 2015 15:33, Rustom Mody wrote:

> However conceptually/pedagogically making a fundamenal distinction of
> timeless | time
> value | object
> immutable | mutable
> expression | statement
> function | procedure
> 
> is key to getting programming [and is something that Pascal got better
> than most of its successors].

Hmmm. Well, I don't quite know what distinction you are making between 
timeless and time, that's ambiguous, and I strongly disagree with the 
value/object distinction, but the rest seems reasonable.


-- 
Steve



From marko at pacujo.net  Tue May 19 04:39:35 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 19 May 2015 11:39:35 +0300
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <874mn9ezaw.fsf@elektro.pacujo.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info>:

>> What matters is the effect of a program. If two metaphysical
>> formulations of language semantics produce the same outcome, neither
>> is objectively better than the other.
>
> By outcome, do you mean the program's output?

Yes, the program's output, but also the interactions between program
parts.

> But the metaphysical formulation of the language is irrelevant to the
> program output. You can get the same output in Forth, Haskell, Prolog,
> C, Applescript or a Turing Machine, despite having radically different
> paradigms.

You misunderstood my point. A valid Python program would not produce the
same output when given to a Prolog interpreter (it would likely produce
a syntax error).

What I'm saying is that it doesn't matter what semantic description you
give Python constructs as long as the observed behavior is correct.

For example, you could explain Python's object references as pointers
(memory addresses) if you considered that helpful. (That's how Lisp
textbooks often explain cons cells.)


Marko


From rosuav at gmail.com  Tue May 19 04:50:29 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 19 May 2015 18:50:29 +1000
Subject: Slices time complexity
In-Reply-To: <874mn9ezaw.fsf@elektro.pacujo.net>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmoY8F_=NiSsjsaiB1PN_Tcxs9C9k6T0PqYHWWR912o7KQ@mail.gmail.com>

On Tue, May 19, 2015 at 6:39 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> For example, you could explain Python's object references as pointers
> (memory addresses) if you considered that helpful. (That's how Lisp
> textbooks often explain cons cells.)

Sorta-kinda-maybe, but if a C programmer's idea of pointers is invoked
to explain Python's object references, the differences will start to
be problematic:

1) Pointer arithmetic simply doesn't exist in Python. Arrays/lists are
not just pointers to their first elements, and subscripting is most
definitely NOT "add to pointer and dereference".
2) In fact, dereferencing as a whole isn't really a 'thing' either. At
best, it happens automatically.
3) References actually mean something. Copying a pointer doesn't.
Whether the Python you're using is refcounted (CPython) or
mark-and-sweep (uPy, I think) or some other model, an additional
reference to the same object will prevent it from being disposed of,
which isn't the case in C.
4) A pointer is itself a value. You can pass a
pointer-to-local-variable to another function and have that function
change a local variable.
5) Furthermore, since a pointer is a value, you can have pointers to
pointers, etc. Doesn't make any sense in Python.

A closer comparison is the C++ "reference", which works kinda like
pass-by-reference, kinda like a pointer, and kinda not like anything
at all, really. But that's still not the same thing as HLL object
semantics (the same semantics used by Python, Pike, ECMAScript, and a
bunch of other languages). As long as you are aware that analogies are
always limited, you can certainly use them as part of an explanation;
but you do have to take care.

ChrisA


From as at sci.fi  Tue May 19 05:14:43 2015
From: as at sci.fi (Anssi Saari)
Date: Tue, 19 May 2015 12:14:43 +0300
Subject: Running cool and silent
References: <d5b29c73-b44d-4437-ad3b-6832164ed550@googlegroups.com>
Message-ID: <vg3twv90vzw.fsf@coffee.modeemi.fi>

Rustom Mody <rustompmody at gmail.com> writes:

> For some reason my Dell laptop runs hot and noisy in linux but cool and
> silent in Windows-8
>
> Running
>
> $ echo "min_power" | sudo tee /sys/class/scsi_host/host*/link_power_management_policy
>
> makes the fan slow/stop.
>
> But I am not sure what it does!!
> [I dont want my laptop fried and its rather HOT out here right now :-) ]

That just turns on the power management on your SATA links. Goggle SATA
ALPM if you like. It's perfectly safe.


From as at sci.fi  Tue May 19 05:16:51 2015
From: as at sci.fi (Anssi Saari)
Date: Tue, 19 May 2015 12:16:51 +0300
Subject: Running cool and silent
References: <d5b29c73-b44d-4437-ad3b-6832164ed550@googlegroups.com>
Message-ID: <vg3pp5x0vwc.fsf@coffee.modeemi.fi>

Rustom Mody <rustompmody at gmail.com> writes:

> Does someone know what it does? [Dont remember where I found it]

One more comment: run Intel's Powertop to see what else you can do to
improve power management on your laptop.


From marko at pacujo.net  Tue May 19 05:35:04 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 19 May 2015 12:35:04 +0300
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
Message-ID: <87zj50ewqf.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> Sorta-kinda-maybe, but if a C programmer's idea of pointers is invoked
> to explain Python's object references, the differences will start to
> be problematic:
>
> 1) Pointer arithmetic simply doesn't exist in Python. Arrays/lists are
> not just pointers to their first elements, and subscripting is most
> definitely NOT "add to pointer and dereference".

Barely an issue.

> 2) In fact, dereferencing as a whole isn't really a 'thing' either. At
> best, it happens automatically.

Yes, you could say to a C programmer: "Python's . corresponds to C's ->"
and be done with it.

> 3) References actually mean something. Copying a pointer doesn't.
> Whether the Python you're using is refcounted (CPython) or
> mark-and-sweep (uPy, I think) or some other model, an additional
> reference to the same object will prevent it from being disposed of,
> which isn't the case in C.

"Disposing of" shouldn't concern a beginning Python programmer. Note
that Scheme does not address the whole topic in its standard. The memory
model is infinite if you will.

> 4) A pointer is itself a value. You can pass a
> pointer-to-local-variable to another function and have that function
> change a local variable.

Correct, variables are not first-class objects in Python. In C, they
are. Functions are not first-class objects in C. In Python, they are.

Still, that doesn't make the pointer semantics any less applicable to
Python.

> 5) Furthermore, since a pointer is a value, you can have pointers to
> pointers, etc. Doesn't make any sense in Python.

What you are saying is that references in general are not first-class
objects in Python. In C, they are.

So in Python, we have these memory locations (variables, references)
that are accessible through special syntax only. Ok. Again, that in no
way invalidates pointer semantics. (And Guido could approve a perfectly
backwards-compatible PEP tomorrow that elevates references to the
first-class status.)

> As long as you are aware that analogies are always limited, you can
> certainly use them as part of an explanation; but you do have to take
> care.

I'm talking about a model, not a mere analogy.


Marko


From subhabrata.banerji at gmail.com  Tue May 19 05:36:31 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Tue, 19 May 2015 02:36:31 -0700 (PDT)
Subject: Flask Post returning error
In-Reply-To: <5fda31ec-b4ba-4be1-a21a-01db5c428058@googlegroups.com>
References: <304362fc-8c0c-49d6-8ef5-a60f05179c42@googlegroups.com>
 <5fda31ec-b4ba-4be1-a21a-01db5c428058@googlegroups.com>
Message-ID: <ce858845-4a05-4352-90d8-3b2c7f54a986@googlegroups.com>

On Tuesday, May 19, 2015 at 10:34:35 AM UTC+5:30, Miki Tebeka wrote:
> > If anyone may kindly suggest what is the error I am doing. 
> It's close to impossible to know without seeing the server side code.

Dear Sir,

I am trying to paste the code from:
from flask import Flask
from flask_restful import reqparse, abort, Api, Resource

app = Flask(__name__)
api = Api(app)

TODOS = {
    'todo1': {'task': 'build an API'},
    'todo2': {'task': 'How are you?'},
    'todo3': {'task': 'profit!'},
}


def abort_if_todo_doesnt_exist(todo_id):
    if todo_id not in TODOS:
        abort(404, message="Todo {} doesn't exist".format(todo_id))

parser = reqparse.RequestParser()
parser.add_argument('task', type=str)


# Todo
# shows a single todo item and lets you delete a todo item
class Todo(Resource):
    def get(self, todo_id):
        abort_if_todo_doesnt_exist(todo_id)
        return TODOS[todo_id]

    def delete(self, todo_id):
        abort_if_todo_doesnt_exist(todo_id)
        del TODOS[todo_id]
        return '', 204

    def put(self, todo_id):
        args = parser.parse_args()
        task = {'task': args['task']}
        TODOS[todo_id] = task
        return task, 201


# TodoList
# shows a list of all todos, and lets you POST to add new tasks
class TodoList(Resource):
    def get(self):
        return TODOS

    def post(self):
        args = parser.parse_args()
        todo_id = int(max(TODOS.keys()).lstrip('todo')) + 1
        todo_id = 'todo%i' % todo_id
        TODOS[todo_id] = {'task': args['task']}
        return TODOS[todo_id], 201

##
## Actually setup the Api resource routing here
##
api.add_resource(TodoList, '/todos')
api.add_resource(Todo, '/todos/<todo_id>')


if __name__ == '__main__':
    app.run(debug=True)

taken from:
http://flask.pocoo.org/docs/0.10/quickstart/

Regards,
Subhabrata Banerjee. 


From steve+comp.lang.python at pearwood.info  Tue May 19 05:45:02 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 19 May 2015 19:45:02 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
Message-ID: <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>

On Tuesday 19 May 2015 05:23, Mario Figueiredo wrote:

> I'd like to understand what I'm being told about slices in
> https://wiki.python.org/moin/TimeComplexity
> 
> Particularly, what's a 'del slice' and a 'set slice' and whether this
> information pertains to both CPython 2.7 and 3.4.


Get a slice: x = mylist[2:15]

Set a slice: mylist[2:15] = [2, 4, 6, 8]

Del a slice: del mylist[2:15]

 
> From the above link it seems slices work in linear time on all cases.

I wouldn't trust that is always the case, e.g. deleting a contiguous slice 
from the end of a list could be O(1).


> And this really has a big impact on certain operations. For instance,
> the code below may surprise some people when they realize it doesn't
> run in linear time on 3.4:

I'm sure that lots of things surprise people who assume that every 
language's performance characteristics are identical.

Python is not Lisp. Lisp is not Java. Java is not Forth. Forth is not Lua. 
Lua is not Fortran. Fortran is not PHP. Do you see a pattern yet? :-)


>     def minimum(values):
>         if len(values) == 1:
>             return values[0]
>         else:
>             m = minimum(values[1:])
>             return m if m < values[0] else values[0]
> 
> Other languages implement slices. I'm currently being faced with a Go
> snippet that mirrors the exact code above and it does run in linear
> time.

I spot a terminology error here. What Go calls a slice and what Python calls 
a slice are *not the same thing*. A Go slice is a view into an array. Python 
slicing is a *method call* to copy, assign, or delete part of a sequence.

Also, Python slices can accept three arguments, not just two, which may be 
arbitrary objects. Go slices cannot.


> Is there any reason why Python 3.4 implementation of slices cannot be
> a near constant operation?

Yes. (1) Backwards compatibility. (2) That would go against the design of 
Python slices that they are copies. (3) That would upset those who want and 
expect a slice to be a copy.

I'll tell you what -- Python slices have worked this way for over 20 years. 
You should propose to the Go designers that Go is confusing because it 
doesn't match Python's behaviour, and they should change the meaning of 
slices in Go to match Python. There's not as much Go code in the world as 
Python code, so that will be far less disruptive. Tell them that Go should 
be just like Python, otherwise it will confuse users who expect Go slices to 
behave like Python slices.

Do let us know what the Go designers say.



-- 
Steven



From storchaka at gmail.com  Tue May 19 06:15:07 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 19 May 2015 13:15:07 +0300
Subject: Slices time complexity
In-Reply-To: <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
Message-ID: <mjf2fc$r4o$1@ger.gmane.org>

On 19.05.15 12:45, Steven D'Aprano wrote:
> On Tuesday 19 May 2015 05:23, Mario Figueiredo wrote:
>>  From the above link it seems slices work in linear time on all cases.
>
> I wouldn't trust that is always the case, e.g. deleting a contiguous slice
> from the end of a list could be O(1).

It always has linear complexity. You need to decref removed elements.




From greg.ewing at canterbury.ac.nz  Tue May 19 07:00:28 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Tue, 19 May 2015 23:00:28 +1200
Subject: Slices time complexity
In-Reply-To: <mailman.124.1432025439.17265.python-list@python.org>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
Message-ID: <cs0jeeFb5egU1@mid.individual.net>

Chris Angelico wrote:
> 1) Pointer arithmetic simply doesn't exist in Python. Arrays/lists are
> not just pointers to their first elements, and subscripting is most
> definitely NOT "add to pointer and dereference".
> 2) In fact, dereferencing as a whole isn't really a 'thing' either. At
> best, it happens automatically.
> 3) References actually mean something. Copying a pointer doesn't.
> Whether the Python you're using is refcounted (CPython) or
> mark-and-sweep (uPy, I think) or some other model, an additional
> reference to the same object will prevent it from being disposed of,
> which isn't the case in C.
> 4) A pointer is itself a value. You can pass a
> pointer-to-local-variable to another function and have that function
> change a local variable.
> 5) Furthermore, since a pointer is a value, you can have pointers to
> pointers, etc. Doesn't make any sense in Python.

FWIW, most of those things are peculiar to C, and are not
necessarily shared even with other languages that have
explicit pointers. In Pascal, for example, there is no
pointer arithmetic, and (at least not without nonstandard
extension) you can't make a pointer point into the middle
of a stack frame or array or record, etc. (You *can* have
a pointer to a pointer, but only by heap-allocating a
memory block containing a single pointer, which is not
useful very often).

-- 
Greg


From oscar.j.benjamin at gmail.com  Tue May 19 07:09:48 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 19 May 2015 12:09:48 +0100
Subject: Slices time complexity
In-Reply-To: <mjf2fc$r4o$1@ger.gmane.org>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjf2fc$r4o$1@ger.gmane.org>
Message-ID: <CAHVvXxRDUNn90MraE4EUTkZK3E1k-_hPq3X-t9wxCfTMQirUDQ@mail.gmail.com>

On 19 May 2015 at 11:15, Serhiy Storchaka <storchaka at gmail.com> wrote:
> On 19.05.15 12:45, Steven D'Aprano wrote:
>>
>> On Tuesday 19 May 2015 05:23, Mario Figueiredo wrote:
>>>
>>>  From the above link it seems slices work in linear time on all cases.
>>
>>
>> I wouldn't trust that is always the case, e.g. deleting a contiguous slice
>> from the end of a list could be O(1).
>
> It always has linear complexity. You need to decref removed elements.

It has linear complexity in the length of the removed slice but not in
the length of the list. So deleting k elements from the end of a list
of length N is O(k) rather than O(N) which could be a big difference.
An algorithm that repeatedly deletes slices from the end until the
entire list was gone would still be O(N) whereas one that deletes from
the beginning would probably be O(N**2).

--
Oscar


From rustompmody at gmail.com  Tue May 19 07:41:19 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 19 May 2015 04:41:19 -0700 (PDT)
Subject: Slices time complexity
In-Reply-To: <555af209$0$12995$c3e8da3$5496439d@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <555af209$0$12995$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <b3fc2fba-f975-43ca-8f44-dc26ea5a1a27@googlegroups.com>

On Tuesday, May 19, 2015 at 1:49:31 PM UTC+5:30, Steven D'Aprano wrote:
> On Tuesday 19 May 2015 15:33, Rustom Mody wrote:
> 
> > However conceptually/pedagogically making a fundamenal distinction of
> > timeless | time
> > value | object
> > immutable | mutable
> > expression | statement
> > function | procedure
> > 
> > is key to getting programming [and is something that Pascal got better
> > than most of its successors].
> 
> Hmmm. Well, I don't quite know what distinction you are making between 
> timeless and time, that's ambiguous, and I strongly disagree with the 
> value/object distinction, but the rest seems reasonable.

Take Pascal and the statement:

y := x+1

A Pascal programmer (and more generally any imperative language programmer)
understands this in two frames:

The 'x+1' is understood mathematically; ie we dont need or wish or encourage
to think of it in terms of machine instructions.

OTOH the 'y := <rhs>' is to be understood procedurally or algorithmically or
in terms of some (maybe half-assed) machine abstraction.

It is only after learning to juggle these two framings that we can deal with
things like
x := x+1

C messes all this badly
1. Illegitimately coopting the '=' symbol for the assignment
2. By making expressions like ++ which are enough to confuse everyone
[ including the compiler eg  i = i++ --- does that increment or is it a no-op?]
3. Expressions are statement-ified by a ';' which discards the top-level value

Python is in some respects better than C -- at least assignment is a statement
In some its worse.  Think how frequent are questions out here on/around
- mixing up list.extend with +; sorted with sort
- comprehensions with side-effecting expressions etc

Think of the straigtforwardness of this error
>>> x = while
  File "<stdin>", line 1
    x = while
            ^
SyntaxError: invalid syntax



with clueless silence of this one:

>>> l = [1,2,3]
>>> l = l.append(4)
>>> l
>>> 

Most answers to issues like the second focus on the fact that None at
top-level vanishes.  A minor nuisance compared the to the real culprit, viz
the append allowable as an rhs.


From steve+comp.lang.python at pearwood.info  Tue May 19 07:43:44 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 19 May 2015 21:43:44 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
Message-ID: <555b21f2$0$13000$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 19 May 2015 18:39, Marko Rauhamaa wrote:

> What I'm saying is that it doesn't matter what semantic description you
> give Python constructs as long as the observed behavior is correct.

You really think that it doesn't matter which of the following two 
explanations I give for this behaviour?

x = 2
y = 3
print x+y  # prints 5

Explanation #1:

"Python adds 2 and 3 together and prints 5."

Explanation #2:

"Python takes the 2nd decimal place of pi (4), and the 3rd decimal place of 
e (8), multiplies them (32), then prints the sum of the digits (5)."

The first explanation has the advantage that it actually describes Python's 
semantic model for the + operator, and consequently it is far more likely to 
allow the user to predict the output of (say) 5+2. The second explanation 
explains the observed behaviour, but doesn't have any predictive power:

Explanation #2 continued:

"Of course Python only does that when the arguments are 2 and 3. With 
arguments 5 and 2, it takes the tenth decimal place of pi (5), the 9th 
decimal place of e**2 (8), and subtracts them from 20 (7)."

Explanations shouldn't just explain the observed behaviour. Any semi-
arbitrary collection of special cases can do that. To be useful, 
explanations ought to give the user *predictive power* -- if I calculate 
32767 + 1, what will Python do?

Explanation #1 predicts that Python will return 32768.
Explanation #2 makes no prediction at all.

Explanation #3, assuming that Python follows the same rules as C, might 
predict overflow to -32768, saturation (32767 + 1 = 32767), or "undefined 
behaviour" up to and including erasing your hard drive.

Only one of these models for Python allows you to make correct predictions, 
even though all three explain the observations given.

Having a good, consistent model for the programming language allows one to 
predict not just *what* it will do, but (in a rough and ready fashion) *how* 
it will do it. Having the *right* model allows you to predict *correctly*.


> For example, you could explain Python's object references as pointers
> (memory addresses) if you considered that helpful. (That's how Lisp
> textbooks often explain cons cells.)

Well, you could do that, but it would fail to explain the behaviour of 
Jython and IronPython.

Mixing descriptions of a specific implementation with descriptions of the 
high level semantics is sometimes useful but often misleading, and one needs 
to be careful how and when one does it.



-- 
Steve



From rustompmody at gmail.com  Tue May 19 07:46:17 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 19 May 2015 04:46:17 -0700 (PDT)
Subject: Slices time complexity
In-Reply-To: <878uclf3bt.fsf@elektro.pacujo.net>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
Message-ID: <acf12c3f-e42f-4dfd-808b-ab4b932ebc60@googlegroups.com>

On Tuesday, May 19, 2015 at 12:42:50 PM UTC+5:30, Marko Rauhamaa wrote:
> Rustom Mody :
> 
> > However conceptually/pedagogically making a fundamenal distinction of
> > timeless | time
> > value | object
> > immutable | mutable
> > expression | statement
> > function | procedure
> >
> > is key to getting programming [and is something that Pascal got better
> > than most of its successors].
> >
> > The FPers want to squeeze the whole world into column 1
> > The OOPers want to do the opposite and are embarrassed by the existence of
> > column-1 [status of int in java etc]
> > Unless one is committed to some philosophical extreme position --
> > Only One True Way -- I believe accepting two fundamentals is the most
> > sane choice
> 
> I sympathize. Can you get Python without getting a language like C
> first? Can a baby be born without an umbilical cord? Can you skip Newton
> and go straight to quantum mechanics and relativity? I have noticed some
> experienced Java programmers are a bit lost in the woods because they
> don't have an idea of what is going on under the hood.

And how would you classify C# in this scheme (pun unintended)?

Note that C# is in .Net what C is in Unix -- the primary building block language.

But C# also has claims to being higher level than C like java/python etc.


From rustompmody at gmail.com  Tue May 19 08:14:19 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 19 May 2015 05:14:19 -0700 (PDT)
Subject: Slices time complexity
In-Reply-To: <acf12c3f-e42f-4dfd-808b-ab4b932ebc60@googlegroups.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <acf12c3f-e42f-4dfd-808b-ab4b932ebc60@googlegroups.com>
Message-ID: <8c00b33a-b8c4-4469-a37e-191d9344bde9@googlegroups.com>

On Tuesday, May 19, 2015 at 5:16:29 PM UTC+5:30, Rustom Mody wrote:
> On Tuesday, May 19, 2015 at 12:42:50 PM UTC+5:30, Marko Rauhamaa wrote:
> > I sympathize. Can you get Python without getting a language like C
> > first? Can a baby be born without an umbilical cord? Can you skip Newton
> > and go straight to quantum mechanics and relativity? I have noticed some
> > experienced Java programmers are a bit lost in the woods because they
> > don't have an idea of what is going on under the hood.
> 
> And how would you classify C# in this scheme (pun unintended)?
> 
> Note that C# is in .Net what C is in Unix -- the primary building block language.
> 
> But C# also has claims to being higher level than C like java/python etc.

To be fair (and to make the opposite point of what I was making above)
I should describe a recent encounter with some C# folks.
They were doing something in what needed to be a fast loop.
Looking over the code I found some dictionary lookups.
I suggested digesting the dict-key into an int and using arrays instead of dicts.

They did it and got a 10x speedup.

So C# (just as your above cited typical Java programmers) certainly can produce
programmers who are clueless of what's 'under the hood'


From Cecil at decebal.nl  Tue May 19 08:24:45 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 19 May 2015 14:24:45 +0200
Subject: Why does the first loop go wrong with Python3
Message-ID: <878uckvjoy.fsf@Equus.decebal.nl>

I have the following code:
    from __future__     import division, print_function

    import subprocess

    p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
    for line in iter(p.stdout.readline, ''):
        print(line.rstrip().decode('utf-8'))

    p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
    for line in p.stdout.readlines():
        print(line.rstrip().decode('utf-8'))

This works in Python2. (Both give the same output.)

But when I execute this in Python3, then the first loop is stuck in a
loop where it continually prints a empty string. The second loop is
executed correctly in Python3.

In the current case it is not a problem for me, but when the output
becomes big, the second solution will need more memory. How can I get
the first version working in Python3?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From steve+comp.lang.python at pearwood.info  Tue May 19 08:42:16 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 19 May 2015 22:42:16 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
Message-ID: <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>

On Tue, 19 May 2015 06:50 pm, Chris Angelico wrote:

> On Tue, May 19, 2015 at 6:39 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
>> For example, you could explain Python's object references as pointers
>> (memory addresses) if you considered that helpful. (That's how Lisp
>> textbooks often explain cons cells.)
> 
> Sorta-kinda-maybe, but if a C programmer's idea of pointers is invoked
> to explain Python's object references, the differences will start to
> be problematic:
> 
> 1) Pointer arithmetic simply doesn't exist in Python. Arrays/lists are
> not just pointers to their first elements, and subscripting is most
> definitely NOT "add to pointer and dereference".
> 2) In fact, dereferencing as a whole isn't really a 'thing' either. At
> best, it happens automatically.
> 3) References actually mean something. Copying a pointer doesn't.
> Whether the Python you're using is refcounted (CPython) or
> mark-and-sweep (uPy, I think) or some other model, an additional
> reference to the same object will prevent it from being disposed of,
> which isn't the case in C.
> 4) A pointer is itself a value. You can pass a
> pointer-to-local-variable to another function and have that function
> change a local variable.
> 5) Furthermore, since a pointer is a value, you can have pointers to
> pointers, etc. Doesn't make any sense in Python.

Chris, that is one of the best explanations for why "references equals
pointers" is *not* a good explanation for Python's behaviour. I may have to
steal it :-)



-- 
Steven



From skip.montanaro at gmail.com  Tue May 19 08:59:00 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 19 May 2015 07:59:00 -0500
Subject: fork/exec & close file descriptors
Message-ID: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>

Due to presumed bugs in an underlying library over which I have no control,
I'm considering a restart in the wee hours of the morning. The basic
fork/exec dance is not a problem, but how do I discover all the open file
descriptors in the new child process to make sure they get closed? Do I
simply start at fd 3 and call os.close() on everything up to some largish
fd number? Some of these file descriptors will have been opened by stuff
well below the Python level, so I don't know them a priori.

Thx,

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150519/82d243c9/attachment.html>

From oscar.j.benjamin at gmail.com  Tue May 19 09:16:13 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 19 May 2015 14:16:13 +0100
Subject: Why does the first loop go wrong with Python3
In-Reply-To: <878uckvjoy.fsf@Equus.decebal.nl>
References: <878uckvjoy.fsf@Equus.decebal.nl>
Message-ID: <CAHVvXxQKBShmc9dh0SREFK79==aHYtDUfht6=fLivAuo6mVKdg@mail.gmail.com>

On 19 May 2015 at 13:24, Cecil Westerhof <Cecil at decebal.nl> wrote:
> I have the following code:
>     from __future__     import division, print_function
>
>     import subprocess
>
>     p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
>     for line in iter(p.stdout.readline, ''):
>         print(line.rstrip().decode('utf-8'))
>
>     p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
>     for line in p.stdout.readlines():
>         print(line.rstrip().decode('utf-8'))
>
> This works in Python2. (Both give the same output.)
>
> But when I execute this in Python3, then the first loop is stuck in a
> loop where it continually prints a empty string. The second loop is
> executed correctly in Python3.
>
> In the current case it is not a problem for me, but when the output
> becomes big, the second solution will need more memory. How can I get
> the first version working in Python3?

The problem is that Python 3 carefully distinguishes between the bytes
that come when reading from the stdout of a process and text which
must be decoded from the bytes. You're using iter(f, sentinel) and
checking for a sentinel value of ''. However in Python 3 the sentinel
returned will be b''.

Consider:
$ python3
Python 3.2.3 (default, Feb 27 2014, 21:31:18)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> '' == b''
False

If you change it from '' to b'' it will work.

However the normal way to do this is to iterate over stdout directly:

     p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
     for line in p.stdout:
         print(line.rstrip().decode('utf-8'))


--
Oscar


From rosuav at gmail.com  Tue May 19 09:24:47 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 19 May 2015 23:24:47 +1000
Subject: Slices time complexity
In-Reply-To: <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmrP9aOwcWS9mjyfJVHxnkP+yEuK09VCNhihvDGmqhCYQg@mail.gmail.com>

On Tue, May 19, 2015 at 10:42 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Tue, 19 May 2015 06:50 pm, Chris Angelico wrote:
>
>> On Tue, May 19, 2015 at 6:39 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
>>> For example, you could explain Python's object references as pointers
>>> (memory addresses) if you considered that helpful. (That's how Lisp
>>> textbooks often explain cons cells.)
>>
>> Sorta-kinda-maybe, but if a C programmer's idea of pointers is invoked
>> to explain Python's object references, the differences will start to
>> be problematic:
>>
>> 1) Pointer arithmetic simply doesn't exist in Python. Arrays/lists are
>> not just pointers to their first elements, and subscripting is most
>> definitely NOT "add to pointer and dereference".
>> 2) In fact, dereferencing as a whole isn't really a 'thing' either. At
>> best, it happens automatically.
>> 3) References actually mean something. Copying a pointer doesn't.
>> Whether the Python you're using is refcounted (CPython) or
>> mark-and-sweep (uPy, I think) or some other model, an additional
>> reference to the same object will prevent it from being disposed of,
>> which isn't the case in C.
>> 4) A pointer is itself a value. You can pass a
>> pointer-to-local-variable to another function and have that function
>> change a local variable.
>> 5) Furthermore, since a pointer is a value, you can have pointers to
>> pointers, etc. Doesn't make any sense in Python.
>
> Chris, that is one of the best explanations for why "references equals
> pointers" is *not* a good explanation for Python's behaviour. I may have to
> steal it :-)

:) They say that turnabout is fair play. I stole your "Python vs
MATLAB" python-tutor post about call-by-value vs call-by-reference vs
object semantics for years (until some other pages picked up the same
content). Go for it!

That said, though, I was specifically referring to C's pointer
semantics. As Gregory pointed out, Pascal has rather different pointer
semantics; but if you're talking about Python references and
considering using any concept of object addressing and pointers, it's
most likely going to be something similar to C/assembly, where a
pointer is a dereferenceable integer storing an offset into memory.

(But don't even get *started* on near vs far pointers, code vs data
segments and the thunks required to transform between them,
memory-mapped disk, virtual segments, overlapping segments, shared
memory, memory-mapped I/O, and all those other things that C
programmers sometimes contend with. Fun, but hardly anything to do
with Python objects. :) )

ChrisA


From rosuav at gmail.com  Tue May 19 09:33:33 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 19 May 2015 23:33:33 +1000
Subject: fork/exec & close file descriptors
In-Reply-To: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
Message-ID: <CAPTjJmoj4STV9+t2-9JWed3t6Hxk5ibtaxYSFfX31oGz=kKR1g@mail.gmail.com>

On Tue, May 19, 2015 at 10:59 PM, Skip Montanaro
<skip.montanaro at gmail.com> wrote:
> Due to presumed bugs in an underlying library over which I have no control,
> I'm considering a restart in the wee hours of the morning. The basic
> fork/exec dance is not a problem, but how do I discover all the open file
> descriptors in the new child process to make sure they get closed? Do I
> simply start at fd 3 and call os.close() on everything up to some largish fd
> number? Some of these file descriptors will have been opened by stuff well
> below the Python level, so I don't know them a priori.

What Python version are you targeting? Are you aware of PEP 446?

https://www.python.org/dev/peps/pep-0446/

tl;dr: As of Python 3.4, the default is to close file descriptors on
exec automatically - and atomically, where possible.

I'm not sure if there's a 2.7 backport available, though even if there
is, you'll need to manually set your files non-inheritable; AIUI the
default in 2.7 can't be changed for backward compatibility reasons
(the change in 3.4 *will* break code that was relying on automatic
inheritability of FDs - they now have to be explicitly tagged).

ChrisA


From massi_srb at msn.com  Tue May 19 09:42:56 2015
From: massi_srb at msn.com (massi_srb at msn.com)
Date: Tue, 19 May 2015 06:42:56 -0700 (PDT)
Subject: Help with Regular Expression
Message-ID: <960db3a7-54e8-40dc-83f3-4e7e8f675529@googlegroups.com>

Hi everyone,

I succesfully wrote a regex in python in order to substitute all the occurences in the form $"somechars" with another string. Here it is:

re.sub(ur"""(?u)(\$\"[^\"\\]*(?:\\.[^\"\\]*)*\")""", newstring, string)

Now I would need to exclude from the match all the string in the form $", ", can anyone help me to modufy it?
Thanks in advance!



From skip.montanaro at gmail.com  Tue May 19 09:44:07 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 19 May 2015 08:44:07 -0500
Subject: fork/exec & close file descriptors
In-Reply-To: <CAPTjJmoj4STV9+t2-9JWed3t6Hxk5ibtaxYSFfX31oGz=kKR1g@mail.gmail.com>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <CAPTjJmoj4STV9+t2-9JWed3t6Hxk5ibtaxYSFfX31oGz=kKR1g@mail.gmail.com>
Message-ID: <CANc-5UwUxYjg=rQQ0Z_nM3s7o4+ghVYzG-9ajQY=LERWAsPrKw@mail.gmail.com>

On Tue, May 19, 2015 at 8:33 AM, Chris Angelico <rosuav at gmail.com> wrote:

> What Python version are you targeting? Are you aware of PEP 446?


Yeah, I'm still on 2.7, and am aware of PEP 446. Note that many of the file
descriptors will not have been created by my Python code. They will have
been created by underlying C/C++ libraries, so I can't guarantee which
flags were set on file open.

I'm going to continue to pursue solutions which won't require a restart for
now, but would like to have a sane restart option in my back pocket should
it become necessary.

Thx,

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150519/b5b4dce1/attachment.html>

From steve+comp.lang.python at pearwood.info  Tue May 19 09:44:13 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 19 May 2015 23:44:13 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <87zj50ewqf.fsf@elektro.pacujo.net>
Message-ID: <555b3e2d$0$12993$c3e8da3$5496439d@news.astraweb.com>

On Tue, 19 May 2015 07:35 pm, Marko Rauhamaa wrote:

> Chris Angelico <rosuav at gmail.com>:
> 
>> Sorta-kinda-maybe, but if a C programmer's idea of pointers is invoked
>> to explain Python's object references, the differences will start to
>> be problematic:
>>
>> 1) Pointer arithmetic simply doesn't exist in Python. Arrays/lists are
>> not just pointers to their first elements, and subscripting is most
>> definitely NOT "add to pointer and dereference".
> 
> Barely an issue.
> 
>> 2) In fact, dereferencing as a whole isn't really a 'thing' either. At
>> best, it happens automatically.
> 
> Yes, you could say to a C programmer: "Python's . corresponds to C's ->"
> and be done with it.


You could say that, but you would be wrong. The amount of stuff that happens
with a dot in Python is far more than a pointer dereference. I can't think
of *any* part of Python's actual behaviour relating to attribute lookup
which it models correctly.

Among other issues, it fails to explain common behaviour like inheritance,
dynamic attribute access (__getattr__ and friends), and descriptors
(properties, methods, etc), never mind about *complicated* stuff like
metaclasses.


>> 3) References actually mean something. Copying a pointer doesn't.
>> Whether the Python you're using is refcounted (CPython) or
>> mark-and-sweep (uPy, I think) or some other model, an additional
>> reference to the same object will prevent it from being disposed of,
>> which isn't the case in C.
> 
> "Disposing of" shouldn't concern a beginning Python programmer. Note
> that Scheme does not address the whole topic in its standard. The memory
> model is infinite if you will.

Beginning Python programmers are not necessarily newbies to programming. If
Susan is a guru-level programmer with thirty years experience in C, C#,
Javascript, Lisp and Forth, but today is her first day using Python, are
you going to try to tell her that Python has infinite memory when she asks
what sort of memory management Python uses?

Even beginners to programming may be capable of intuiting for themselves
that disposal is an issue to be considered. "I know my computer has 4GB of
RAM, and I've just created a list of two billion strings. My computer is
running a bit slow. Maybe this has something to do with memory? How do I
dispose of those strings so they aren't using up memory?"


>> 4) A pointer is itself a value. You can pass a
>> pointer-to-local-variable to another function and have that function
>> change a local variable.
> 
> Correct, variables are not first-class objects in Python. In C, they
> are. Functions are not first-class objects in C. In Python, they are.

Variables are not first class values in C. (I assume you meant *values*
rather than "objects", on account of C not being an OOP language.) There is
no way, for example, to set x to *the variable y* in either C or Python. If
you could, that would imply something like this:

y = 42  # regular assignment
x ::= y  # set x to be the variable y, not the value of y
assert x == y  # naturally, since x and y are now two different 
               # names for the same variable
x = 23  # regular assignment
assert y == 23  # since y is just another name for x

You should note carefully that my example doesn't involve calling setter
functions, explicitly manipulating a namespace, pointer dereferences or any
other form of indirect assignment. It just uses name binding: a name on the
left, a value on the right.

You cannot do that in C or Python.

I'm not aware of any language which treats variables as first class values.


> Still, that doesn't make the pointer semantics any less applicable to
> Python.
> 
>> 5) Furthermore, since a pointer is a value, you can have pointers to
>> pointers, etc. Doesn't make any sense in Python.
> 
> What you are saying is that references in general are not first-class
> objects in Python. In C, they are.

References in Python are not values at all, so they are not first-class
values.

Pointers in C are values, and first-class values at that.

Arrays, on the other hand, are values but not first-class values in C: you
cannot do array assignment, pass an entire array by value, or return an
array from a function.



-- 
Steven



From rosuav at gmail.com  Tue May 19 09:54:37 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 19 May 2015 23:54:37 +1000
Subject: fork/exec & close file descriptors
In-Reply-To: <CANc-5UwUxYjg=rQQ0Z_nM3s7o4+ghVYzG-9ajQY=LERWAsPrKw@mail.gmail.com>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <CAPTjJmoj4STV9+t2-9JWed3t6Hxk5ibtaxYSFfX31oGz=kKR1g@mail.gmail.com>
 <CANc-5UwUxYjg=rQQ0Z_nM3s7o4+ghVYzG-9ajQY=LERWAsPrKw@mail.gmail.com>
Message-ID: <CAPTjJmrdPcW8EkLR9Kq+n9sYEWB8eJ3fg=_JFQ2x_4QhDeg90Q@mail.gmail.com>

On Tue, May 19, 2015 at 11:44 PM, Skip Montanaro
<skip.montanaro at gmail.com> wrote:
>
> On Tue, May 19, 2015 at 8:33 AM, Chris Angelico <rosuav at gmail.com> wrote:
>>
>> What Python version are you targeting? Are you aware of PEP 446?
>
>
> Yeah, I'm still on 2.7, and am aware of PEP 446. Note that many of the file
> descriptors will not have been created by my Python code. They will have
> been created by underlying C/C++ libraries, so I can't guarantee which flags
> were set on file open.
>
> I'm going to continue to pursue solutions which won't require a restart for
> now, but would like to have a sane restart option in my back pocket should
> it become necessary.

Fair enough. What you MAY be able to do is preempt it by going through
your FDs and setting them all CLOEXEC, but it won't make a lot of
difference compared to just going through them all and closing them
between fork and exec.

On Linux (and possibly some other Unixes), /proc/self/fd may be of
use. Enumerating files in that should tell you about your open files.
How useful that is I don't know, though.

ChrisA


From oscar.j.benjamin at gmail.com  Tue May 19 10:18:54 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 19 May 2015 15:18:54 +0100
Subject: Slices time complexity
In-Reply-To: <CAPTjJmoY8F_=NiSsjsaiB1PN_Tcxs9C9k6T0PqYHWWR912o7KQ@mail.gmail.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <CAPTjJmoY8F_=NiSsjsaiB1PN_Tcxs9C9k6T0PqYHWWR912o7KQ@mail.gmail.com>
Message-ID: <CAHVvXxSp=EcAO42W9CKKHMf2CxAKCfF2KNSP2d+FHq2PthDMwg@mail.gmail.com>

On 19 May 2015 at 09:50, Chris Angelico <rosuav at gmail.com> wrote:
> On Tue, May 19, 2015 at 6:39 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
>> For example, you could explain Python's object references as pointers
>> (memory addresses) if you considered that helpful. (That's how Lisp
>> textbooks often explain cons cells.)
>
> Sorta-kinda-maybe, but if a C programmer's idea of pointers is invoked
> to explain Python's object references, the differences will start to
> be problematic:
>
> 1) Pointer arithmetic simply doesn't exist in Python. Arrays/lists are
> not just pointers to their first elements, and subscripting is most
> definitely NOT "add to pointer and dereference".
> 2) In fact, dereferencing as a whole isn't really a 'thing' either. At
> best, it happens automatically.
> 3) References actually mean something. Copying a pointer doesn't.
> Whether the Python you're using is refcounted (CPython) or
> mark-and-sweep (uPy, I think) or some other model, an additional
> reference to the same object will prevent it from being disposed of,
> which isn't the case in C.
> 4) A pointer is itself a value. You can pass a
> pointer-to-local-variable to another function and have that function
> change a local variable.

I think this is actually one of those areas where the analogy does
make sense. In Python if I pass an object as an argument to another
function I can mutate the object but I can't rebind the name in the
parent frame. Similarly in C I can pass a pointer to anything into
another function and the other function will be able to mutate the
pointed-to variable but not change the value of the pointer in the
parent frame.

int b = 2;

void f(int* d)
{
    *d = 3; // Changes a which is pointed to by c and d.
    d = &b; // Doesn't affect c.
    *d = 4;  // Changes b
}

int main(int argc, char *argv[])
{
    int a = 1;
    int *c = &a;  // a and *c are the same object
    f(c);
    // c still points to a but a is now 3.
    printf("*c = %d\n", *c);
    return 0;
}

The effect is the same in Python except that I can't use an int for
the demo since they're immutable:

b = [2]

def g(d):
    d[0] = 3 # Changes a which is also c and d
    d = b     # Doesn't affect a or c
    d[0] = 4 # Changes b

def main():
    a = [1]
    c = a   # a and c are the same object
    f(c)
    # c is still a but a is now [3]
    print("c =", c)

main()


Having taught beginners to program with C as a first programming
language and with Python as a first programming language I would say
that in either case this is an important fundamental concept that
beginners need to get their heads round. If a C programmer finds it
easier to understand how Python handles it by relating it to pointers
then that's great.


> 5) Furthermore, since a pointer is a value, you can have pointers to
> pointers, etc. Doesn't make any sense in Python.

You can have a reference to a container object which references other
objects. The effect is more or less the same if you want to understand
how something like this works:

a = [[0] * 5] * 5


--
Oscar


From skip.montanaro at gmail.com  Tue May 19 10:31:14 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 19 May 2015 09:31:14 -0500
Subject: fork/exec & close file descriptors
In-Reply-To: <CAPTjJmrdPcW8EkLR9Kq+n9sYEWB8eJ3fg=_JFQ2x_4QhDeg90Q@mail.gmail.com>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <CAPTjJmoj4STV9+t2-9JWed3t6Hxk5ibtaxYSFfX31oGz=kKR1g@mail.gmail.com>
 <CANc-5UwUxYjg=rQQ0Z_nM3s7o4+ghVYzG-9ajQY=LERWAsPrKw@mail.gmail.com>
 <CAPTjJmrdPcW8EkLR9Kq+n9sYEWB8eJ3fg=_JFQ2x_4QhDeg90Q@mail.gmail.com>
Message-ID: <CANc-5UwWcJm_S7QMtP2gC1aW5U=iQRvrxVb=dPWjzE78-rL_9w@mail.gmail.com>

On Tue, May 19, 2015 at 8:54 AM, Chris Angelico <rosuav at gmail.com> wrote:

> On Linux (and possibly some other Unixes), /proc/self/fd may be of
> use.
>

Good point. Yes, /proc/PID/fd appears to contain all the entries for open
file descriptors (I am on Linux).

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150519/f0e77ce0/attachment.html>

From jon+usenet at unequivocal.co.uk  Tue May 19 10:32:33 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 19 May 2015 14:32:33 +0000 (UTC)
Subject: fork/exec & close file descriptors
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <CAPTjJmoj4STV9+t2-9JWed3t6Hxk5ibtaxYSFfX31oGz=kKR1g@mail.gmail.com>
 <mailman.136.1432043050.17265.python-list@python.org>
Message-ID: <slrnmlmie6.864.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-19, Skip Montanaro <skip.montanaro at gmail.com> wrote:
> Yeah, I'm still on 2.7, and am aware of PEP 446. Note that many of the file
> descriptors will not have been created by my Python code. They will have
> been created by underlying C/C++ libraries, so I can't guarantee which
> flags were set on file open.

There is no portable way to do this, the problem is Unix not Python.
The below code is a reasonable stab at it, but there is no 100%
guaranteed method. The code is untested but you get the idea.


  import errno
  import os


  def closeall(min=0, max=4096, keep=frozenset()):
      """Close all open file descriptors except for the given exceptions.

      Any file descriptors below or equal to `min`, or in the set `keep`
      will not be closed. Any file descriptors above `max` *might* not be
      closed.
      """
      # First try /proc/$$/pid
      try:
          for fd in os.listdir("/proc/%d/fd" % (os.getpid())):
	      try:
		  fd = int(fd)
              except ValueError:
	          continue
	      if fd >= min and fd not in keep:
	          os.close(int(fd))
          return
      except OSError as exc:
          if exc[0] != errno.ENOENT:
	      raise
      # If /proc was not available, fall back to closing a lot of descriptors.
      for fd in range(min, max):
          if fd not in keep:
	      try:
		  os.close(fd)
	      except OSError as exc:
		  if exc[0] != errno.EBADF:
		      raise


From nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de  Tue May 19 10:38:43 2015
From: nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de (Thomas Rachel)
Date: Tue, 19 May 2015 16:38:43 +0200
Subject: Why does the first loop go wrong with Python3
In-Reply-To: <mailman.133.1432041396.17265.python-list@python.org>
References: <878uckvjoy.fsf@Equus.decebal.nl>
 <mailman.133.1432041396.17265.python-list@python.org>
Message-ID: <mjfhto$8if$1@r01.glglgl.de>

Am 19.05.2015 um 15:16 schrieb Oscar Benjamin:

> However the normal way to do this is to iterate over stdout directly:

Depends. There may be differences when it comes to buffering etc...


Thomas


From Cecil at decebal.nl  Tue May 19 10:44:08 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 19 May 2015 16:44:08 +0200
Subject: Why does the first loop go wrong with Python3
References: <878uckvjoy.fsf@Equus.decebal.nl>
 <mailman.133.1432041396.17265.python-list@python.org>
Message-ID: <874mn8vd8n.fsf@Equus.decebal.nl>

Op Tuesday 19 May 2015 15:16 CEST schreef Oscar Benjamin:

> On 19 May 2015 at 13:24, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> I have the following code:
>> from __future__     import division, print_function
>>
>> import subprocess
>>
>> p = subprocess.Popen('ls -l', shell = True, stdout =
>> subprocess.PIPE) for line in iter(p.stdout.readline, ''):
>> print(line.rstrip().decode('utf-8'))
>>
>> p = subprocess.Popen('ls -l', shell = True, stdout =
>> subprocess.PIPE) for line in p.stdout.readlines():
>> print(line.rstrip().decode('utf-8'))
>>
>> This works in Python2. (Both give the same output.)
>>
>> But when I execute this in Python3, then the first loop is stuck in
>> a loop where it continually prints a empty string. The second loop
>> is executed correctly in Python3.
>>
>> In the current case it is not a problem for me, but when the output
>> becomes big, the second solution will need more memory. How can I
>> get the first version working in Python3?
>
> The problem is that Python 3 carefully distinguishes between the
> bytes that come when reading from the stdout of a process and text
> which must be decoded from the bytes. You're using iter(f, sentinel)
> and checking for a sentinel value of ''. However in Python 3 the
> sentinel returned will be b''.
>
> Consider: $ python3 Python 3.2.3 (default, Feb 27 2014, 21:31:18)
> [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or
> "license" for more information.
>>>> '' == b''
> False
>
> If you change it from '' to b'' it will work.
>
> However the normal way to do this is to iterate over stdout
> directly:
>
> p = subprocess.Popen('ls -l', shell = True, stdout =
> subprocess.PIPE) for line in p.stdout:
> print(line.rstrip().decode('utf-8'))

Works like a charm.

I looked at the documentation. Is it necessary to do a:
    p.wait()
afterwards?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From bartc at freeuk.com  Tue May 19 10:52:38 2015
From: bartc at freeuk.com (Bartc)
Date: Tue, 19 May 2015 15:52:38 +0100
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
Message-ID: <mjff5g$met$1@dont-email.me>

"Steven D'Aprano" <steve+comp.lang.python at pearwood.info> wrote in message
news:555b0621$0$2753$c3e8da3$76491128 at news.astraweb.com...
> On Tuesday 19 May 2015 05:23, Mario Figueiredo wrote:

>> And this really has a big impact on certain operations. For instance,
>> the code below may surprise some people when they realize it doesn't
>> run in linear time on 3.4:
>
> I'm sure that lots of things surprise people who assume that every
> language's performance characteristics are identical.
>
> Python is not Lisp. Lisp is not Java. Java is not Forth. Forth is not Lua.
> Lua is not Fortran. Fortran is not PHP. Do you see a pattern yet? :-)

But sometimes you want an algorithm that works perfectly well in one
language to work just as well in another. (Subject to understood factors,
eg, one language is implemented as native code, another byte-code.)

>> Other languages implement slices. I'm currently being faced with a Go
>> snippet that mirrors the exact code above and it does run in linear
>> time.
>
> I spot a terminology error here. What Go calls a slice and what Python
> calls
> a slice are *not the same thing*. A Go slice is a view into an array.
> Python
> slicing is a *method call* to copy, assign, or delete part of a sequence.
>
> Also, Python slices can accept three arguments, not just two, which may be
> arbitrary objects. Go slices cannot.

I'm surprised about the slice thing too; I was getting used to the idea of
Python only passing around references, even to small integer values, and now
here it goes and does a copy!

But apparently only a shallow (top-level) copy. Still, it sounds expensive
if you just want to pass part of a list to a function, not the whole thing,
a function which is not going to write into the list (like the OP's
example).

>> Is there any reason why Python 3.4 implementation of slices cannot be
>> a near constant operation?
>
> Yes. (1) Backwards compatibility. (2) That would go against the design of
> Python slices that they are copies. (3) That would upset those who want
> and
> expect a slice to be a copy.

Or you could just have a different kind of slice that is just a 'view'.
(Python is a big language to which almost everything else has been bolted
on, so why not?)

> I'll tell you what -- Python slices have worked this way for over 20
> years.
> You should propose to the Go designers that Go is confusing because it
> doesn't match Python's behaviour, and they should change the meaning of
> slices in Go to match Python. There's not as much Go code in the world as
> Python code, so that will be far less disruptive. Tell them that Go should
> be just like Python, otherwise it will confuse users who expect Go slices
> to
> behave like Python slices.

I agree with the OP, the way slices work in Python is confusing to those who
expect them to work like array assignments. So if B is a million-element
list, then A=B does not copy a million elements, neither shallow or deep.

But A=B[1:] copies 999999 shallow elements. (Pointers I expect, and might
need to change the reference counts of those 999999 elements. And when that
slice is discarded after perhaps a handful of accesses, it might need to
change them all back. if that is the case, then a 'view' kind of slice would
have been better, with an explicit copy operation if needed.)

-- 
Bartc





From rosuav at gmail.com  Tue May 19 11:13:49 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 20 May 2015 01:13:49 +1000
Subject: fork/exec & close file descriptors
In-Reply-To: <CANc-5UwWcJm_S7QMtP2gC1aW5U=iQRvrxVb=dPWjzE78-rL_9w@mail.gmail.com>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <CAPTjJmoj4STV9+t2-9JWed3t6Hxk5ibtaxYSFfX31oGz=kKR1g@mail.gmail.com>
 <CANc-5UwUxYjg=rQQ0Z_nM3s7o4+ghVYzG-9ajQY=LERWAsPrKw@mail.gmail.com>
 <CAPTjJmrdPcW8EkLR9Kq+n9sYEWB8eJ3fg=_JFQ2x_4QhDeg90Q@mail.gmail.com>
 <CANc-5UwWcJm_S7QMtP2gC1aW5U=iQRvrxVb=dPWjzE78-rL_9w@mail.gmail.com>
Message-ID: <CAPTjJmonuuAYNg+XiDmuuU2+kH-DY8pjtEH8cV+wKHwhh1zgpQ@mail.gmail.com>

On Wed, May 20, 2015 at 12:31 AM, Skip Montanaro
<skip.montanaro at gmail.com> wrote:
> On Tue, May 19, 2015 at 8:54 AM, Chris Angelico <rosuav at gmail.com> wrote:
>>
>> On Linux (and possibly some other Unixes), /proc/self/fd may be of
>> use.
>
>
> Good point. Yes, /proc/PID/fd appears to contain all the entries for open
> file descriptors (I am on Linux).

Yes, and /proc/self is usually a magic symlink to /proc/<your_pid> so
you can just look at /proc/self/fd instead of explicitly calling up
your own PID.

ChrisA


From ian.g.kelly at gmail.com  Tue May 19 11:15:48 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 19 May 2015 09:15:48 -0600
Subject: Slices time complexity
In-Reply-To: <mjf2fc$r4o$1@ger.gmane.org>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjf2fc$r4o$1@ger.gmane.org>
Message-ID: <CALwzidkfonf=vOUrMNH0NWEu54_EZpPDFWQqwYYdGaP-rT8Qfw@mail.gmail.com>

On May 19, 2015 4:16 AM, "Serhiy Storchaka" <storchaka at gmail.com> wrote:
>
> On 19.05.15 12:45, Steven D'Aprano wrote:
>>
>> On Tuesday 19 May 2015 05:23, Mario Figueiredo wrote:
>>>
>>>  From the above link it seems slices work in linear time on all cases.
>>
>>
>> I wouldn't trust that is always the case, e.g. deleting a contiguous
slice
>> from the end of a list could be O(1).
>
>
> It always has linear complexity. You need to decref removed elements.

Only in CPython. The operation might be O(1) in Pypy or Jython.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150519/ed128db9/attachment.html>

From ian.g.kelly at gmail.com  Tue May 19 11:49:24 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 19 May 2015 09:49:24 -0600
Subject: Why does the first loop go wrong with Python3
In-Reply-To: <874mn8vd8n.fsf@Equus.decebal.nl>
References: <878uckvjoy.fsf@Equus.decebal.nl>
 <mailman.133.1432041396.17265.python-list@python.org>
 <874mn8vd8n.fsf@Equus.decebal.nl>
Message-ID: <CALwzid=rPT+YOLm1QS1rELQ26x9BE61V78JZqzL9bGHMJPe5FQ@mail.gmail.com>

On Tue, May 19, 2015 at 8:44 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> I looked at the documentation. Is it necessary to do a:
>     p.wait()
> afterwards?

It's good practice to clean up zombie processes by waiting on them,
but they will also get cleaned up when your script exits.


From marko at pacujo.net  Tue May 19 11:59:21 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 19 May 2015 18:59:21 +0300
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <555b21f2$0$13000$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87lhgkeexy.fsf@elektro.pacujo.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info>:

> To be useful, explanations ought to give the user *predictive power*
> -- if I calculate 32767 + 1, what will Python do?
>
> Explanation #1 predicts that Python will return 32768.
> Explanation #2 makes no prediction at all.

Any semantic ruleset that correctly predicts the behavior of any given
Python program is equally valid.

> Only one of these models for Python allows you to make correct
> predictions, even though all three explain the observations given.

You're slaying straw men.

>> For example, you could explain Python's object references as pointers
>> (memory addresses) if you considered that helpful. (That's how Lisp
>> textbooks often explain cons cells.)
>
> Well, you could do that, but it would fail to explain the behaviour of 
> Jython and IronPython.

Jython, CPython and IronPython are supposed to be semantically
equivalent. So any valid CPython model is also a valid model for Jython
and IronPython.

Coincidentally:

   The reference values (often just references) are pointers to these
   objects, and a special null reference, which refers to no object.

   <URL: https://docs.oracle.com/javase/specs/jls/se8/html/jls-4.htm
   l#jls-4.3.1>

(without any explanation as to what a "pointer" might be).

> Mixing descriptions of a specific implementation with descriptions of
> the high level semantics is sometimes useful but often misleading, and
> one needs to be careful how and when one does it.

Nobody has proposed doing that.


Marko


From jkn_gg at nicorp.f9.co.uk  Tue May 19 12:29:01 2015
From: jkn_gg at nicorp.f9.co.uk (jkn)
Date: Tue, 19 May 2015 09:29:01 -0700 (PDT)
Subject: Slightly OT: Seeking (python-based) project diary tool, or framework
 to write one
Message-ID: <99067d97-cad4-42f8-8fd1-b1884bed769e@googlegroups.com>

Hi All
    as in the title, this is a little bit OT - but since ideally I'd like a
tool written in Python, and I know readers here have wide experience of
development/collaborative workflows etc ...

A few jobs ago the company I was with used a 'Project Diary' tool which I found
very useful. It was written in-house (in VB, I believe ;-o). I am looking for a
more widely available tool that gives similar features. 

It created and read a database which contained a series of projects. In this
context, a project was a discrete unit of development effort (eg. for a single
customer, say), and lasting from a couple of weeks to a year or more.

For each project you have a main view window which listed the 'items' in that
project. You could create a new item which was one of a large-ish number of
types. When you created a new item you were given the ability to edit a variety
of parameters, appropriate to the item type:

- info item:	freeform text/HTML
- contact item:	a form with email, telephone number fields
- datasheet item:	maybe a link to a PDF
- diary item	dated freeform
- task item	had title & fields like 'completion date', 'completion
                         percentage, etc.
- query item
- document refrence
- etc. etc.

(items could also be updated once created)

As a project develops, more and more items are added. There were sorting and
searchinf facilities to allow you to quickly find a particular item, and you
could generate custom reports based on seleted items, or date ranges,and so on.

I found this very useful and would like to find something similar, or create
something similar for my own purposes. I currently use an Excel spreadsheet
but this is a bit clunky. I am a moderately experienced Python programmer
(command line, Linux interfacing) but less experienced on the
Gui/framework/database side.

I would be interested in either pointers to something like this which might
already exists, or suggestions as to a framework that I might be able to use to
create something like this.

Other nice-to-haves for a possibly already-existing tool:

-	multiplatform
-	free, open source
-	written in Python
-	web-based probably OK.
-	extendible
-	etc. etc.

Thanks for your thoughts and any suggestions

	Jon N


From python.list at tim.thechases.com  Tue May 19 12:30:14 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Tue, 19 May 2015 11:30:14 -0500
Subject: Help with Regular Expression
In-Reply-To: <960db3a7-54e8-40dc-83f3-4e7e8f675529@googlegroups.com>
References: <960db3a7-54e8-40dc-83f3-4e7e8f675529@googlegroups.com>
Message-ID: <20150519113014.42bce8f7@bigbox.christie.dr>

On 2015-05-19 06:42, massi_srb at msn.com wrote:
> I succesfully wrote a regex in python in order to substitute all
> the occurences in the form $"somechars" with another string. Here
> it is:
> 
> re.sub(ur"""(?u)(\$\"[^\"\\]*(?:\\.[^\"\\]*)*\")""", newstring,
> string)

The expression is a little more precise than you describe it, but the
general idea is correct.

For the record, the "(?u)" happens to be unneeded here, and I find it
more clear if you pass the re.UNICODE flag to the function.

> Now I would need to exclude from the match all the string in the
> form $", ", can anyone help me to modufy it? Thanks in advance!

If you don't want commas or spaces, you should be able to just insert
them into your various negated character-classes:

  r"""(?u)(\$\"[^\"\\, ]*(?:\\.[^\"\\, ]*)*\")"""

Unless you want to allow commas and/or spaces, but disallow commas
followed by spaces.  That's a lot uglier.  If that's the case, it
would help to have a test-harness of your expected inputs and results:

  re_to_test = r"""(?u)(\$\"[^\"\\, ]*(?:\\.[^\"\\, ]*)*\")"""
  for test, expected in [
     ('Hello $"who"!', 'Hello world!'),
     ('Hello $"who.who"!', 'Hello world!'),
     ('Hello $"who.is.it"!', 'Hello world!'),
     ('Hello $"who, what"!', 'Hello world!'),
     ('Hello $"who,what,where"!', 'Hello world!'),
     ('Hello $"who, what, where"!', 'Hello $"who, what, where"!'),
     ('Hello $"who is it"!', 'Hello world!'),
     ]:
    result = re.sub(re_to_test, "world", test, re.UNICODE)
    if result == expected:
      report_passing(...)
    else:
      report_failure(...)

-tkc









From umasrinath88 at gmail.com  Tue May 19 12:30:15 2015
From: umasrinath88 at gmail.com (umasrinath88 at gmail.com)
Date: Tue, 19 May 2015 09:30:15 -0700 (PDT)
Subject: Convert c file to csv file(unix format) in python
Message-ID: <c9733ddb-7f97-4fcf-9dad-a4481d1d3e5d@googlegroups.com>

Hi All,

I have a python script file which converts .hex file  to c file.

Can anyone help me in converting .c file to csv file (unix format).


Regards,
Uma


From ethan at stoneleaf.us  Tue May 19 12:35:58 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 19 May 2015 09:35:58 -0700
Subject: fork/exec & close file descriptors
In-Reply-To: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
Message-ID: <555B666E.4050001@stoneleaf.us>

On 05/19/2015 05:59 AM, Skip Montanaro wrote:

> Due to presumed bugs in an underlying library over which I have no control, I'm considering a restart in the wee hours of the morning. The basic fork/exec dance is not a problem, but how do I discover
> all the open file descriptors in the new child process to make sure they get closed? Do I simply start at fd 3 and call os.close() on everything up to some largish fd number? Some of these file
> descriptors will have been opened by stuff well below the Python level, so I don't know them a priori.

Pandaemonium [1] (and I believe Ben Finney's daemon [2]) use something akin to the following:

def close_open_files(exclude):
     max_files = resource.getrlimit(resource.RLIMIT_NOFILE)[1]
     keep = set()
     for file in exclude:
         if isinstance(file, baseint):
             keep.add(file)
         elif hasattr(file, 'fileno'):
             keep.add(file.fileno())
         else:
             raise ValueError(
                     'files to not close should be either an file descriptor, '
                     'or a file-type object, not %r (%s)' % (type(file), file))
     for fd in range(max_files, -1, -1):
         if fd in keep:
             continue
         try:
             os.close(fd)
         except OSError:
             exc = sys.exc_info()[1]
             if exc.errno == errno.EBADF:
                 continue
             raise

So, yeah, basically a brute-force method.

--
~Ethan~


[1] https://pypi.python.org/pypi/pandaemonium
[2] https://pypi.python.org/pypi/python-daemon


From Cecil at decebal.nl  Tue May 19 12:39:38 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 19 May 2015 18:39:38 +0200
Subject: Why does the first loop go wrong with Python3
References: <878uckvjoy.fsf@Equus.decebal.nl>
 <mailman.133.1432041396.17265.python-list@python.org>
 <874mn8vd8n.fsf@Equus.decebal.nl>
 <mailman.142.1432050607.17265.python-list@python.org>
Message-ID: <87wq04ttbp.fsf@Equus.decebal.nl>

Op Tuesday 19 May 2015 17:49 CEST schreef Ian Kelly:

> On Tue, May 19, 2015 at 8:44 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> I looked at the documentation. Is it necessary to do a:
>> p.wait()
>> afterwards?
>
> It's good practice to clean up zombie processes by waiting on them,
> but they will also get cleaned up when your script exits.

You are right. I played a little with ipython3, which made finding
things out a lot easier. ;-)

In my case it is a script, that terminates very soon after being
finished with p, but it is certainly good practise to do it myself.

I always did a free in my C programming days. I was always told it was
not necessary, but I found it better to do it anyway.


By the way, what also works is:
    p = None

But it was just a try in ipython3. I would never do this in real code.
I was just curious if this would be handled correctly and it is. :-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From mihamina.rakotomandimby at rktmb.org  Tue May 19 12:48:59 2015
From: mihamina.rakotomandimby at rktmb.org (Mihamina Rakotomandimby)
Date: Tue, 19 May 2015 19:48:59 +0300
Subject: Convert c file to csv file(unix format) in python
In-Reply-To: <c9733ddb-7f97-4fcf-9dad-a4481d1d3e5d@googlegroups.com>
References: <c9733ddb-7f97-4fcf-9dad-a4481d1d3e5d@googlegroups.com>
Message-ID: <555B697B.3060203@rktmb.org>

On 05/19/2015 07:30 PM, umasrinath88 at gmail.com wrote:
> Can anyone help me in converting .c file to csv file (unix format).
>
>
Would you give a sample?



From rustompmody at gmail.com  Tue May 19 12:50:01 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 19 May 2015 09:50:01 -0700 (PDT)
Subject: Slightly OT: Seeking (python-based) project diary tool, or
 framework to write one
In-Reply-To: <99067d97-cad4-42f8-8fd1-b1884bed769e@googlegroups.com>
References: <99067d97-cad4-42f8-8fd1-b1884bed769e@googlegroups.com>
Message-ID: <1e128886-a88f-4710-beb8-3bb2491029af@googlegroups.com>

On Tuesday, May 19, 2015 at 9:59:16 PM UTC+5:30, jkn wrote:
> Hi All
>     as in the title, this is a little bit OT - but since ideally I'd like a
> tool written in Python, and I know readers here have wide experience of
> development/collaborative workflows etc ...

Occasionally the author of leo posts announcements out here
http://leoeditor.com/

[Thats about as much as I know of it]


From Cecil at decebal.nl  Tue May 19 13:01:05 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 19 May 2015 19:01:05 +0200
Subject: Best way to rewrite Popen
Message-ID: <87siastsby.fsf@Equus.decebal.nl>

At the moment I am playing with things like:
    p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)

I think that most of the times this are the values I want. So it would
be nice to overrule the defaults. What is the best way to do this? So
creating a function that is exactly the same except for the defaults
for shell and stdout (and maybe stderr).


It is a little less important as I first thought, because I found the
following:
    error, output = subprocess.getstatusoutput('ls -1')
    files_new = output.splitlines()
But it is still nice to know.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From jkn_gg at nicorp.f9.co.uk  Tue May 19 13:06:44 2015
From: jkn_gg at nicorp.f9.co.uk (jkn)
Date: Tue, 19 May 2015 10:06:44 -0700 (PDT)
Subject: Slightly OT: Seeking (python-based) project diary tool, or
 framework to write one
In-Reply-To: <1e128886-a88f-4710-beb8-3bb2491029af@googlegroups.com>
References: <99067d97-cad4-42f8-8fd1-b1884bed769e@googlegroups.com>
 <1e128886-a88f-4710-beb8-3bb2491029af@googlegroups.com>
Message-ID: <6bcb54ef-0ad9-462f-81df-b71e1d2ccce2@googlegroups.com>

Hi Rustom

On Tuesday, May 19, 2015 at 5:50:11 PM UTC+1, Rustom Mody wrote:
> On Tuesday, May 19, 2015 at 9:59:16 PM UTC+5:30, jkn wrote:
> > Hi All
> >     as in the title, this is a little bit OT - but since ideally I'd like a
> > tool written in Python, and I know readers here have wide experience of
> > development/collaborative workflows etc ...
> 
> Occasionally the author of leo posts announcements out here
> http://leoeditor.com/
> 
> [Thats about as much as I know of it]

Thanks - actually, I'm pretty familiar with Leo, and it _might_ be bent a bit
to suit my needs ... but I am thinking of something a bit more 'traditional'
in appearance, at least.

    Cheers
    Jon N


From steve+comp.lang.python at pearwood.info  Tue May 19 13:07:03 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 20 May 2015 03:07:03 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjff5g$met$1@dont-email.me>
Message-ID: <555b6db8$0$12996$c3e8da3$5496439d@news.astraweb.com>

On Wed, 20 May 2015 12:52 am, Bartc wrote:

> "Steven D'Aprano" <steve+comp.lang.python at pearwood.info> wrote in message
> news:555b0621$0$2753$c3e8da3$76491128 at news.astraweb.com...
[...]
>> I'm sure that lots of things surprise people who assume that every
>> language's performance characteristics are identical.
>>
>> Python is not Lisp. Lisp is not Java. Java is not Forth. Forth is not
>> Lua. Lua is not Fortran. Fortran is not PHP. Do you see a pattern yet?
>> :-)
> 
> But sometimes you want an algorithm that works perfectly well in one
> language to work just as well in another.

Yeah, and I want a million dollars, a pony, and the ability to eat as much
as I like without getting fat too. Life is cruel, and then you die.


[...]
> I'm surprised about the slice thing too; I was getting used to the idea of
> Python only passing around references, even to small integer values, and
> now here it goes and does a copy!

Yes. And? Are you surprised that dict.copy() makes a copy? Presumably not.
Well, the way we spell list.copy() is list[:]


> But apparently only a shallow (top-level) copy. Still, it sounds expensive
> if you just want to pass part of a list to a function, not the whole
> thing, a function which is not going to write into the list (like the OP's
> example).

Yes, a slice can be expensive, if you have (say) a ten billion element list,
and take a slice list[1:]. And, to be perfectly honest, there's no real
good solution for that in standard Python. Numpy arrays implement slicing
and cloning of arrays as views, but the Python standard library doesn't
have anything that does the same.

But, really... if you're serious about dealing with huge arrays of data, you
want something like numpy, not Python lists. So *in practice* the lack of
views for lists is a minor nuisance, if that. Having list slices return
copies rather than views avoids more problems than it causes.

On the other hand, if Python implementations would implement slices with
copy-on-write, that would give us the best of both worlds!


>>> Is there any reason why Python 3.4 implementation of slices cannot be
>>> a near constant operation?
>>
>> Yes. (1) Backwards compatibility. (2) That would go against the design of
>> Python slices that they are copies. (3) That would upset those who want
>> and
>> expect a slice to be a copy.
> 
> Or you could just have a different kind of slice that is just a 'view'.

Sure. And you can program it yourself, if you like.

True confession time: I've been using Python for over 15 years. For at least
12 of those years, I've found myself feeling guilty every time I write a
loop like "for x in seq[1:]" to skip the first element, because I'm worried
about copying a huge list. Every single time I think "I really ought to
write a list view." And then I think about the effort involved (minor)
versus the benefit (even smaller) and I think "screw it, I'll just make a
copy".

And in 12 years, this lazy "put off until tomorrow" attitude to list views
has failed me exactly *zero* times. I won't say You're Not Gonna Need It,
but I will say *I've* Never Needed It Yet.

I think that's why Python doesn't have a generalised sequence slice view.
It's not that the core developers are against the idea. It's just that,
weighing up the disadvantages, the advantages, and the effort involved, a
slice view has never been high enough on their priority list to get
included. The people who really need it already have numpy.

What will be off the table is changing the behaviour of list slicing. But
adding a sequence view? That's plausible. It just needs somebody to do the
work first.


>> I'll tell you what -- Python slices have worked this way for over 20
>> years.
>> You should propose to the Go designers that Go is confusing because it
>> doesn't match Python's behaviour, and they should change the meaning of
>> slices in Go to match Python. There's not as much Go code in the world as
>> Python code, so that will be far less disruptive. Tell them that Go
>> should be just like Python, otherwise it will confuse users who expect Go
>> slices to
>> behave like Python slices.
> 
> I agree with the OP, the way slices work in Python is confusing to those
> who expect them to work like array assignments. So if B is a
> million-element list, then A=B does not copy a million elements, neither
> shallow or deep.

Why would you expect slicing to be the same as assignment? Do you expect
list.append to be the same as assignment?

mylist = list(range(40))
mylist[3:26:3] = [1, 2, 3, 4, 5, 6, 7, 8]

How do you expect that to work, if not by copying?


> But A=B[1:] copies 999999 shallow elements. (Pointers I expect, and might
> need to change the reference counts of those 999999 elements. And when
> that slice is discarded after perhaps a handful of accesses, it might need
> to change them all back.

You say that as if it were expensive.



-- 
Steven



From Cecil at decebal.nl  Tue May 19 13:11:46 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 19 May 2015 19:11:46 +0200
Subject: subprocess.getstatusoutput does not exist in 2.7
Message-ID: <87oalgtru5.fsf@Equus.decebal.nl>

At the moment I want my code to run with 2.7 and 3.4+.

But the command:
    subprocess.getstatusoutput
does not exist in 2.7. Is this an oversight or is there a reason for
it?

I can rewrite the code (back) to work with Popen again, but I found
the getstatusoutput elegant.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From rosuav at gmail.com  Tue May 19 13:11:50 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 20 May 2015 03:11:50 +1000
Subject: Why does the first loop go wrong with Python3
In-Reply-To: <87wq04ttbp.fsf@Equus.decebal.nl>
References: <878uckvjoy.fsf@Equus.decebal.nl>
 <mailman.133.1432041396.17265.python-list@python.org>
 <874mn8vd8n.fsf@Equus.decebal.nl>
 <mailman.142.1432050607.17265.python-list@python.org>
 <87wq04ttbp.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmox-hq4RzytOwJrzeBkZ2vZa33sWoqGxS5Zp1p=dx=fGg@mail.gmail.com>

On Wed, May 20, 2015 at 2:39 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> By the way, what also works is:
>     p = None
>
> But it was just a try in ipython3. I would never do this in real code.
> I was just curious if this would be handled correctly and it is. :-)

That _may_ work, but it depends on their not being any other
references to it, and also depends on it being garbage-collected
promptly. Neither is guaranteed. Explicitly wait()ing on it is a
guarantee.

Simply dropping the object is a good way to "probably dispose" of
something that you don't care about. For instance, you asynchronously
invoke VLC to play some audio alert, and the subprocess might finish
before you're done with the current function, or might finish after.
You don't really care about its actual termination, and certainly
don't want to delay anything waiting for it, but you do want to clean
up resources at some point. Dropping the object (keeping no references
to it, returning from the function you called it from, unsetting any
references you have, whatever makes sense) will normally mean that it
gets garbage collected and cleaned up _at some point_, without really
guaranteeing exactly when; for instance, if you have an alert like
this once per hour and watch 'top' for the number of zombies, you'll
probably see some now and then, but they'll never get to apocalyptic
numbers.

ChrisA


From umasrinath88 at gmail.com  Tue May 19 13:14:58 2015
From: umasrinath88 at gmail.com (umasrinath88 at gmail.com)
Date: Tue, 19 May 2015 10:14:58 -0700 (PDT)
Subject: Convert c file to csv file(unix format) in python
In-Reply-To: <mailman.144.1432053932.17265.python-list@python.org>
References: <c9733ddb-7f97-4fcf-9dad-a4481d1d3e5d@googlegroups.com>
 <mailman.144.1432053932.17265.python-list@python.org>
Message-ID: <307ed4c5-b140-4f11-844f-db5fcc592dd7@googlegroups.com>

On Tuesday, May 19, 2015 at 10:15:43 PM UTC+5:30, Mihamina Rakotomandimby wrote:
> On 05/19/2015 07:30 PM, umasrinath88 at gmail.com wrote:
> > Can anyone help me in converting .c file to csv file (unix format).
> >
> >
> Would you give a sample?

Hi Mihamina,

I have sent you the complete work folder.

thyncHextoC.py is python script used to convert .hex to .c structure file

Open command prompt window and type below command.


>python thyncHextoC.py <mention hex file to be converted>

example:  python thyncHextoC.py 18052015_1640_STM_5_08_DVT.hex

stm32Image.c file is created.

this .c file has to be converted to csv file.

I was following manual steps as below:(But want to automate now)

3)Open stm32Image.c file in notepad++
Delete  first 23 lines of data in the file(including this line "const flashImage_t    flashImageData[] ={")

The .C file must contain only hex bytes now.

4)Select all the hex bytes and go to edit option->Blank operation-> Remove leading and trailing spaces

next go again edit option->Line operations-> Remove empty lines containing blank characters

Remove the comma(,) from last hex byte till end of the file.
This can be achieved using Alt+Shift+end and scroll down until the last hex byte.

Note:Donot modify last line 0,0,0 }; of the file. Let it  remain as it is.

5)Once all the above steps are done, Save the file and in command prompt execute below command for converting .c to .csv file

>ren stm32Image.c <file name.csv>

example:ren stm32Image.c 28022015_1140_EVT.csv


From rosuav at gmail.com  Tue May 19 13:19:56 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 20 May 2015 03:19:56 +1000
Subject: Slices time complexity
In-Reply-To: <555b6db8$0$12996$c3e8da3$5496439d@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjff5g$met$1@dont-email.me>
 <555b6db8$0$12996$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmr6TsBXjYVSh6E-JqiBWsJyacP58R0-s3r7+jvEh-O=-Q@mail.gmail.com>

On Wed, May 20, 2015 at 3:07 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> True confession time: I've been using Python for over 15 years. For at least
> 12 of those years, I've found myself feeling guilty every time I write a
> loop like "for x in seq[1:]" to skip the first element, because I'm worried
> about copying a huge list. Every single time I think "I really ought to
> write a list view." And then I think about the effort involved (minor)
> versus the benefit (even smaller) and I think "screw it, I'll just make a
> copy".
>
> And in 12 years, this lazy "put off until tomorrow" attitude to list views
> has failed me exactly *zero* times. I won't say You're Not Gonna Need It,
> but I will say *I've* Never Needed It Yet.

I can't think of many times when I've needed to do this on a large
list - usually it's either a generic iterable handler (so it has to
call iter() and then next(), and then iterate over what's left), or
it's sys.argv. I literally cannot think of any other situation where
I've iterated over seq[1:] than sys.argv. And if you're passing a
million args to a program, frankly, you have more to worry about than
the cost of trimming off the self-name to iterate over the rest of the
args. Worries like, yaknow, actually doing some work with whatever was
passed in as args, which is likely to dwarf the list slicing time.

ChrisA


From jon+usenet at unequivocal.co.uk  Tue May 19 13:36:18 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 19 May 2015 17:36:18 +0000 (UTC)
Subject: Best way to rewrite Popen
References: <87siastsby.fsf@Equus.decebal.nl>
Message-ID: <slrnmlmt6n.864.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-19, Cecil Westerhof <Cecil at decebal.nl> wrote:
> At the moment I am playing with things like:
>     p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
>
> I think that most of the times this are the values I want. So it would
> be nice to overrule the defaults. What is the best way to do this? So
> creating a function that is exactly the same except for the defaults
> for shell and stdout (and maybe stderr).

Yes.

  def shellprocess(cmd, **kwargs):
      return subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE,
          **kwargs)

> It is a little less important as I first thought, because I found the
> following:
>     error, output = subprocess.getstatusoutput('ls -1')
>     files_new = output.splitlines()
> But it is still nice to know.

Why are you doing this anyway, rather than using os.listdir()?
Invoking subprocesses via the shell is very rarely a good idea.


From jon+usenet at unequivocal.co.uk  Tue May 19 13:37:57 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 19 May 2015 17:37:57 +0000 (UTC)
Subject: subprocess.getstatusoutput does not exist in 2.7
References: <87oalgtru5.fsf@Equus.decebal.nl>
Message-ID: <slrnmlmt9q.864.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-19, Cecil Westerhof <Cecil at decebal.nl> wrote:
> At the moment I want my code to run with 2.7 and 3.4+.
>
> But the command:
>     subprocess.getstatusoutput
> does not exist in 2.7. Is this an oversight or is there a reason for
> it?
>
> I can rewrite the code (back) to work with Popen again, but I found
> the getstatusoutput elegant.

It's in the 'commands' module in Python 2. You could use the
following:

  try:
      from subprocess import getstatusoutput
  except ImportError:
      from commands import getstatusoutput


From jonas at wielicki.name  Tue May 19 13:43:06 2015
From: jonas at wielicki.name (Jonas Wielicki)
Date: Tue, 19 May 2015 19:43:06 +0200
Subject: Best way to rewrite Popen
In-Reply-To: <87siastsby.fsf@Equus.decebal.nl>
References: <87siastsby.fsf@Equus.decebal.nl>
Message-ID: <555B762A.7060202@wielicki.name>

On 19.05.2015 19:01, Cecil Westerhof wrote:
> At the moment I am playing with things like:
>     p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
> 
> I think that most of the times this are the values I want. So it would
> be nice to overrule the defaults. What is the best way to do this? 

I would like to answer with a counter-question: Why do you want to do
this? What is the reason to use ls -l if you could also be using
os.listdir() or os.scandir()?

There are some reasons against screenscraping `ls` (locale-specifities
and environment variable dependencies of ls) and using shell=True (with
shells being huge complex beasts which may have unexpected security
issues, see Shellshock), so I?m not sure why you would want to do that.

regards,
jwi

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20150519/2e89b0b4/attachment.sig>

From steve+comp.lang.python at pearwood.info  Tue May 19 13:46:56 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 20 May 2015 03:46:56 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <555b21f2$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <87lhgkeexy.fsf@elektro.pacujo.net>
Message-ID: <555b7711$0$12986$c3e8da3$5496439d@news.astraweb.com>

On Wed, 20 May 2015 01:59 am, Marko Rauhamaa wrote:

> Steven D'Aprano <steve+comp.lang.python at pearwood.info>:
> 
>> To be useful, explanations ought to give the user *predictive power*
>> -- if I calculate 32767 + 1, what will Python do?
>>
>> Explanation #1 predicts that Python will return 32768.
>> Explanation #2 makes no prediction at all.
> 
> Any semantic ruleset that correctly predicts the behavior of any given
> Python program is equally valid.

This is a more reasonable position to take than your earlier stance that the
only thing that matters is whether it explains observed behaviour. It's
still wrong, but at least it is defensible in the sense that sometimes a
ruleset which is objectively wrong but still gives the right answer might
be *preferred* over one which is objectively correct but hard to
understand.

But some rulesets may have perfect predictive power, and still be rejected:

http://star.psy.ohio-state.edu/coglab/Pictures/miracle.gif

Unlike scientific theories about the universe, where we have to hypothesise
what the rules are, we can actually look at the source code for the Python
interpreter and see exactly what the rules are. We don't have to guess how
list.sort() works, we have the documentation, we have the source code, we
can ask the author, and if all else fails, we can run the interpreter in a
debugger and watch what it does.

A ruleset which corresponds to the facts of how list.sort() works is more
valid than a counter-factual ruleset which does not. "Python uses timsort"
is usually to be preferred over "Python uses bubblesort" because at least
two interpreters (CPython and Jython) use timsort, while no interpreter
uses bubblesort. (Here, "Python" is to be understood as referring to a
specific interpreter, not the language itself.)



>> Only one of these models for Python allows you to make correct
>> predictions, even though all three explain the observations given.
> 
> You're slaying straw men.

"I was wrong, but I don't want to admit it" is not spelled "straw man".

You made a claim that was not defensible, and I tore that claim to shreds.
Have the good grace to admit that your earlier position:

    "it doesn't matter what semantic description you give Python 
    constructs as long as the observed behavior is correct"

doesn't stand up to scrutiny. If you can't do that, at least let it pass in
silence without trying to pretend that you never made that argument and
falsely accusing me of misrepresenting you.



>>> For example, you could explain Python's object references as pointers
>>> (memory addresses) if you considered that helpful. (That's how Lisp
>>> textbooks often explain cons cells.)
>>
>> Well, you could do that, but it would fail to explain the behaviour of
>> Jython and IronPython.
> 
> Jython, CPython and IronPython are supposed to be semantically
> equivalent. So any valid CPython model is also a valid model for Jython
> and IronPython.

They are only supposed to equivalent in regards to behaviour specified by
the language specifications, and that gives plenty of opportunity for
implementation-dependent behaviour. And of course performance and memory
use is a measure of quality of implementation.

One such implementation-dependent behaviour is whether Python objects have a
fixed location or not. CPython objects currently do. Jython and IronPython
objects do not, they can move in memory, so references in those two
implementations cannot be implemented as pointers or memory addresses.

PyPy objects don't even have a fixed existence, there are circumstances
where objects in PyPy can disappear, only to be recreated (possibly in a
new location).


>> Mixing descriptions of a specific implementation with descriptions of
>> the high level semantics is sometimes useful but often misleading, and
>> one needs to be careful how and when one does it.
> 
> Nobody has proposed doing that.

You did. Pointers are the CPython implementation, they are not the high
level semantics.




-- 
Steven



From rosuav at gmail.com  Tue May 19 13:47:42 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 20 May 2015 03:47:42 +1000
Subject: Best way to rewrite Popen
In-Reply-To: <87siastsby.fsf@Equus.decebal.nl>
References: <87siastsby.fsf@Equus.decebal.nl>
Message-ID: <CAPTjJmpMApx8Ptm=fcstWDY5MxZsLn5T7_Yod4QCyHWYHKAGOg@mail.gmail.com>

On Wed, May 20, 2015 at 3:01 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> At the moment I am playing with things like:
>     p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
>
> I think that most of the times this are the values I want. So it would
> be nice to overrule the defaults. What is the best way to do this? So
> creating a function that is exactly the same except for the defaults
> for shell and stdout (and maybe stderr).

Well... I would have to start by saying that you probably _don't_ want
to use shell=True by default. Putting it explicitly on the cases where
you need it helps you remember its danger. You also don't need it for
simple cases like that one; improve your reliability by providing a
list instead of a string, and then you can leave shell=False:

p = subprocess.Popen(['ls','-l'], stdout=subprocess.PIPE)

Running everything via the shell is unnecessary, and a dangerous
default. (Maybe it's not a problem when you use a string literal as
the command, but if you make that the default, you'll end up exposing
yourself in some situation where it isn't hard-coded.) With that
change, there's really only one parameter that you're defaulting, so
there's not as much point making the change, but the technique still
works, and maybe you'll add more to the setup:

@functools.wraps(subprocess.Popen)
def Popen(*a, **kw):
    if 'stdout' not in kw: kw['stdout'] = subprocess.PIPE
    return subprocess.Popen(*a, **kw)

That's a simple way to patch in some function defaults. But
personally, I'd probably end up doing something like this:

def capture_stdout(*a, **kw):
    if 'stdout' not in kw: kw['stdout'] = subprocess.PIPE
    return subprocess.Popen(*a, **kw).stdout

so I can directly iterate over the thing. Or something else. One
change isn't really enough to justify the extra layer of indirection.

ChrisA


From zachary.ware+pylist at gmail.com  Tue May 19 13:55:53 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Tue, 19 May 2015 12:55:53 -0500
Subject: Best way to rewrite Popen
In-Reply-To: <CAPTjJmpMApx8Ptm=fcstWDY5MxZsLn5T7_Yod4QCyHWYHKAGOg@mail.gmail.com>
References: <87siastsby.fsf@Equus.decebal.nl>
 <CAPTjJmpMApx8Ptm=fcstWDY5MxZsLn5T7_Yod4QCyHWYHKAGOg@mail.gmail.com>
Message-ID: <CAKJDb-PBHtBpF_U6ug7zPJQTdH1vzFjz=SLAuW-2o8RYqhdG8Q@mail.gmail.com>

On May 19, 2015 12:48 PM, "Chris Angelico" <rosuav at gmail.com> wrote:
>
> On Wed, May 20, 2015 at 3:01 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> > At the moment I am playing with things like:
> >     p = subprocess.Popen('ls -l', shell = True, stdout =
subprocess.PIPE)
> >
> > I think that most of the times this are the values I want. So it would
> > be nice to overrule the defaults. What is the best way to do this? So
> > creating a function that is exactly the same except for the defaults
> > for shell and stdout (and maybe stderr).
>
> Well... I would have to start by saying that you probably _don't_ want
> to use shell=True by default. Putting it explicitly on the cases where
> you need it helps you remember its danger. You also don't need it for
> simple cases like that one; improve your reliability by providing a
> list instead of a string, and then you can leave shell=False:
>
> p = subprocess.Popen(['ls','-l'], stdout=subprocess.PIPE)
>
> Running everything via the shell is unnecessary, and a dangerous
> default. (Maybe it's not a problem when you use a string literal as
> the command, but if you make that the default, you'll end up exposing
> yourself in some situation where it isn't hard-coded.) With that
> change, there's really only one parameter that you're defaulting, so
> there's not as much point making the change, but the technique still
> works, and maybe you'll add more to the setup:
>
> @functools.wraps(subprocess.Popen)
> def Popen(*a, **kw):
>     if 'stdout' not in kw: kw['stdout'] = subprocess.PIPE
>     return subprocess.Popen(*a, **kw)
>
> That's a simple way to patch in some function defaults. But
> personally, I'd probably end up doing something like this:
>
> def capture_stdout(*a, **kw):
>     if 'stdout' not in kw: kw['stdout'] = subprocess.PIPE

Just a quick note that this line can be simplified nicely to:

kw.setdefault('stdout', subprocess.PIPE)

Regards,
--
Zach
(on a phone)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150519/d23a7b1f/attachment.html>

From ron3200 at gmail.com  Tue May 19 14:02:56 2015
From: ron3200 at gmail.com (Ron Adam)
Date: Tue, 19 May 2015 14:02:56 -0400
Subject: Rule of order for dot operators?
In-Reply-To: <CAPTjJmqf4n4nzxs_TLBc8QAZsLkXRNxY_z8Gb0ZMmkf1kW5rpQ@mail.gmail.com>
References: <55579886.3010001@cdreimer.com>
 <mailman.118.1431989304.17265.python-list@python.org>
 <fabab250-d7a5-49f4-a9ef-359bd003a607@googlegroups.com>
 <mje816$2kd$1@ger.gmane.org>
 <CAPTjJmqf4n4nzxs_TLBc8QAZsLkXRNxY_z8Gb0ZMmkf1kW5rpQ@mail.gmail.com>
Message-ID: <mjftsh$bo2$1@ger.gmane.org>



On 05/19/2015 02:25 AM, Chris Angelico wrote:
> On Tue, May 19, 2015 at 12:43 PM, Ron Adam<ron3200 at gmail.com>  wrote:
>> >Having just implementing something similar for nested scopes, it turns out
>> >it can't be operators because if it was, then the names y and z would be
>> >resolved in the wrong scope.
>> >
>> >          y = "m"
>> >          z = "n"
>> >          a = x . y . z
>> >
>> >Which of course wouldn't do what we want.
>> >
>> >          a = x . "m" . "n"
>> >
>> >And most likely this would give an error.

> If you want to implement the dot as an operator, you could do it by
> having a special syntactic element called an "atom", which is used for
> these kinds of identifier-like tokens. The dot operator could then
> take an object and an atom, and effectively return getattr(obj,
> stringify(atom)). I'm fairly sure this would result in the same syntax
> as Python uses.

I think it's better not to.   What practical things can be done if the dot 
was an operator and names after dots where parsed as atoms?

What I did was parse a name to a subtype of tuple with elements of strings. 
  [return name.with.dots] becomes this in memory after parsing.

     [keyword_object name_object]

Using dots as operators would make that...

     [keyword_object name_object dot_object atom_object dot_object
      atom_object]

This would require the interpreter to do in steps what a single function 
call can do all at once.

Cheers,
    Ron








From rosuav at gmail.com  Tue May 19 14:12:05 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 20 May 2015 04:12:05 +1000
Subject: Slices time complexity
In-Reply-To: <555b7711$0$12986$c3e8da3$5496439d@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <555b21f2$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <87lhgkeexy.fsf@elektro.pacujo.net>
 <555b7711$0$12986$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmrYue9uWKaJYjdRapB4JyGTjh=YUrNHK2nVqKNr+f5f5Q@mail.gmail.com>

On Wed, May 20, 2015 at 3:46 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Wed, 20 May 2015 01:59 am, Marko Rauhamaa wrote:
>
>> Steven D'Aprano <steve+comp.lang.python at pearwood.info>:
>>
>>> To be useful, explanations ought to give the user *predictive power*
>>> -- if I calculate 32767 + 1, what will Python do?
>>>
>>> Explanation #1 predicts that Python will return 32768.
>>> Explanation #2 makes no prediction at all.
>>
>> Any semantic ruleset that correctly predicts the behavior of any given
>> Python program is equally valid.
>
> This is a more reasonable position to take than your earlier stance that the
> only thing that matters is whether it explains observed behaviour. It's
> still wrong, but at least it is defensible in the sense that sometimes a
> ruleset which is objectively wrong but still gives the right answer might
> be *preferred* over one which is objectively correct but hard to
> understand.

Perfect parallel with Magic: The Gathering, where a card may have
reminder text on it which is, in the strictest sense of the
definition, inaccurate, but covers 99%+ of situations without being
horrendously wordy. The fundamental rulebook (the "Comp Rules") goes
into excruciating detail, and is about as readable as the CPython
source code (that is: pretty readable, but too verbose to use as
documentation).

For example, the "First Strike" ability is described simply as "This
creature deals combat damage before creatures without first strike"
[1], but the Comp Rules explain how there are two combat damage steps,
and detail what happens if a creature gains or loses First Strike
during combat.

Even worse, "Madness" is described as "If you discard this card, you
may cast it for its madness cost instead of putting it into your
graveyard" [2]; the Comp Rules divide this into two parts, explaining
in full how the card actually gets from one place to another, where
it's cast from, and so on.

Both of these reminder texts are technically inaccurate, but are
massively preferred over the full text, because they actually fit
inside your brain. Same thing happens with networking; when you go to
https://www.python.org/ in a web browser, your browser establishes an
encrypted connection with the server at www.python.org, and requests
the "/" page. Well, actually, first there are DNS lookups, and IP
routing, and TCP handshakes, and then the whole "establishes an
encrypted connection" bit elides a ton of work involving certificate
checks and so on... but it's much easier to just talk in very high
level terms, even if it's not perfectly accurate. As predictive power
goes, this explanation will adequately tell you what to expect a
request for https://github.com/Rosuav to do; that makes it useful.

An explanation that's more accurate is better than one that's less
accurate; an explanation that's more complicated is worse than one
that's simpler. Somewhere in the middle is an optimal descriptive
style for a given situation.

ChrisA

[1] http://gatherer.wizards.com/Pages/Card/Details.aspx?multiverseid=134753
[2] http://gatherer.wizards.com/Pages/Card/Details.aspx?multiverseid=382849


From rosuav at gmail.com  Tue May 19 14:13:53 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 20 May 2015 04:13:53 +1000
Subject: Best way to rewrite Popen
In-Reply-To: <CAKJDb-PBHtBpF_U6ug7zPJQTdH1vzFjz=SLAuW-2o8RYqhdG8Q@mail.gmail.com>
References: <87siastsby.fsf@Equus.decebal.nl>
 <CAPTjJmpMApx8Ptm=fcstWDY5MxZsLn5T7_Yod4QCyHWYHKAGOg@mail.gmail.com>
 <CAKJDb-PBHtBpF_U6ug7zPJQTdH1vzFjz=SLAuW-2o8RYqhdG8Q@mail.gmail.com>
Message-ID: <CAPTjJmomyFMj-LmefyHAx9bdjm+Fo9kvPrSXL8-MuER+VQ_TDg@mail.gmail.com>

On Wed, May 20, 2015 at 3:55 AM, Zachary Ware
<zachary.ware+pylist at gmail.com> wrote:
>> def capture_stdout(*a, **kw):
>>     if 'stdout' not in kw: kw['stdout'] = subprocess.PIPE
>
> Just a quick note that this line can be simplified nicely to:
>
> kw.setdefault('stdout', subprocess.PIPE)

Yes, in the simple case. That does require pre-evaluating
subprocess.PIPE though, which in some situations is an important
distinction. But you're right, for something this simple, the
repetition is superfluous. Good catch.

ChrisA


From marko at pacujo.net  Tue May 19 14:19:26 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 19 May 2015 21:19:26 +0300
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <555b21f2$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <87lhgkeexy.fsf@elektro.pacujo.net>
 <555b7711$0$12986$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87bnhge8gh.fsf@elektro.pacujo.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info>:

> On Wed, 20 May 2015 01:59 am, Marko Rauhamaa wrote:
>> You're slaying straw men.
>
> "I was wrong, but I don't want to admit it" is not spelled "straw man".
>
> You made a claim that was not defensible, and I tore that claim to shreds.
> Have the good grace to admit that your earlier position:
>
>     "it doesn't matter what semantic description you give Python 
>     constructs as long as the observed behavior is correct"

I stand by that position and haven't changed it.

> doesn't stand up to scrutiny.

But it does.

>> Jython, CPython and IronPython are supposed to be semantically
>> equivalent. So any valid CPython model is also a valid model for
>> Jython and IronPython.
>
> They are only supposed to equivalent in regards to behaviour specified by
> the language specifications,

Naturally.

> One such implementation-dependent behaviour is whether Python objects
> have a fixed location or not.

Whatever that means.

> CPython objects currently do.

As far as the process's virtual memory goes...

> Jython and IronPython objects do not, they can move in memory, so
> references in those two implementations cannot be implemented as
> pointers or memory addresses.

Semantics can be given in terms of memory addresses even if the
implementation does not have memory addresses (in the sense you're
describing).

>>> Mixing descriptions of a specific implementation with descriptions
>>> of the high level semantics is sometimes useful but often
>>> misleading, and one needs to be careful how and when one does it.
>> 
>> Nobody has proposed doing that.
>
> You did. Pointers are the CPython implementation, they are not the
> high level semantics.

The semantics can be expressed in any manner whatsoever regardless of
how Python has been "really" implemented. Formally, informally. In
English. In Russian. Using Turing machines, using x86 machine language,
using Forth. There's an infinitum of isomorphic descriptions, some
instructive, some confusing.

The touchstone is the observed behavior.


Marko


From denismfmcmahon at gmail.com  Tue May 19 14:20:02 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Tue, 19 May 2015 18:20:02 +0000 (UTC)
Subject: Convert c file to csv file(unix format) in python
References: <c9733ddb-7f97-4fcf-9dad-a4481d1d3e5d@googlegroups.com>
Message-ID: <mjfusi$6mq$2@dont-email.me>

On Tue, 19 May 2015 09:30:15 -0700, umasrinath88 wrote:

> I have a python script file which converts .hex file  to c file.

Hex is generally an ascii file containing hexadecimal values.

.c is generally an ascii file formatted in a way that corrsponds to the 
structures of the c programming language.

> Can anyone help me in converting .c file to csv file (unix format).

csv is generally a way of exchanging data between different software 
applications in a generally human readable format.

I'm unsure as to the translations you expect to be made to convert a .c 
file into a csv file. Perhaps you could elaborate on these.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From Cecil at decebal.nl  Tue May 19 15:07:13 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 19 May 2015 21:07:13 +0200
Subject: subprocess.getstatusoutput does not exist in 2.7
References: <87oalgtru5.fsf@Equus.decebal.nl>
 <slrnmlmt9q.864.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <878ucktmhq.fsf@Equus.decebal.nl>

Op Tuesday 19 May 2015 19:37 CEST schreef Jon Ribbens:

> On 2015-05-19, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> At the moment I want my code to run with 2.7 and 3.4+.
>>
>> But the command: subprocess.getstatusoutput does not exist in 2.7.
>> Is this an oversight or is there a reason for it?
>>
>> I can rewrite the code (back) to work with Popen again, but I found
>> the getstatusoutput elegant.
>
> It's in the 'commands' module in Python 2. You could use the
> following:
>
> try:
> from subprocess import getstatusoutput
> except ImportError:
> from commands import getstatusoutput

Works like a charm. Thanks.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Tue May 19 15:13:39 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 19 May 2015 21:13:39 +0200
Subject: Best way to rewrite Popen
References: <87siastsby.fsf@Equus.decebal.nl>
 <slrnmlmt6n.864.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <874mn8tm70.fsf@Equus.decebal.nl>

Op Tuesday 19 May 2015 19:36 CEST schreef Jon Ribbens:

> On 2015-05-19, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> At the moment I am playing with things like: p =
>> subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
>>
>> I think that most of the times this are the values I want. So it
>> would be nice to overrule the defaults. What is the best way to do
>> this? So creating a function that is exactly the same except for
>> the defaults for shell and stdout (and maybe stderr).
>
> Yes.
>
> def shellprocess(cmd, **kwargs):
> return subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE,
> **kwargs)

Will that not go wrong if I call it with?
    shellprocess('ls -1', shell = False)


>> It is a little less important as I first thought, because I found
>> the following: error, output = subprocess.getstatusoutput('ls -1')
>> files_new = output.splitlines() But it is still nice to know.
>
> Why are you doing this anyway, rather than using os.listdir()?
> Invoking subprocesses via the shell is very rarely a good idea.

I want to rewrite a Bash script into a Python script. The 'ls -1' is
only an example. But Popen and listdir give a different output. The
sorting is different. But I could sort it myself.

Another problem is that I work with a filter later on. But I could do
that with Python also of-course. So maybe I should rethink what I want
to do. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From umasrinath88 at gmail.com  Tue May 19 15:26:05 2015
From: umasrinath88 at gmail.com (umasrinath88 at gmail.com)
Date: Tue, 19 May 2015 12:26:05 -0700 (PDT)
Subject: Convert c file to csv file(unix format) in python
In-Reply-To: <mjfusi$6mq$2@dont-email.me>
References: <c9733ddb-7f97-4fcf-9dad-a4481d1d3e5d@googlegroups.com>
 <mjfusi$6mq$2@dont-email.me>
Message-ID: <11c2e290-52db-4bc6-a5fe-5757230a816c@googlegroups.com>

On Tuesday, May 19, 2015 at 11:51:20 PM UTC+5:30, Denis McMahon wrote:
> On Tue, 19 May 2015 09:30:15 -0700, umasrinath88 wrote:
> 
> > I have a python script file which converts .hex file  to c file.
> 
> Hex is generally an ascii file containing hexadecimal values.
> 
> .c is generally an ascii file formatted in a way that corrsponds to the 
> structures of the c programming language.
> 
> > Can anyone help me in converting .c file to csv file (unix format).
> 
> csv is generally a way of exchanging data between different software 
> applications in a generally human readable format.
> 
> I'm unsure as to the translations you expect to be made to convert a .c 
> file into a csv file. Perhaps you could elaborate on these.
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com

Hi Denis,

I agree to you point.

I have shared c file with you.Iorder to convert to unix format csv, below are the main steps we can follow.

1 Remove the 23 first lines
2.Remove all empty lines
3.Remove spaces at the beginning of each line
4.Replace ,\r\r\n with \n [commaCRCRLF replace with LF]
5.Rename and save this file as csv.

Can you help me in writing a python code for this. Which takes stm32Image.c file as input and generates stm32Image.csv

Regards,
Uma


From invalid at invalid.invalid  Tue May 19 15:36:34 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Tue, 19 May 2015 19:36:34 +0000 (UTC)
Subject: Convert c file to csv file(unix format) in python
References: <c9733ddb-7f97-4fcf-9dad-a4481d1d3e5d@googlegroups.com>
Message-ID: <mjg3c2$deg$1@reader1.panix.com>

On 2015-05-19, umasrinath88 at gmail.com <umasrinath88 at gmail.com> wrote:

> I have a python script file which converts .hex file  to c file.
>
> Can anyone help me in converting .c file to csv file (unix format).

------------------------------------------------------------------------
#import os,sys
os.rename(sys.argv[1],sys.argv[1].replace('.c','.csv'))
------------------------------------------------------------------------

-- 
Grant Edwards               grant.b.edwards        Yow! I want a WESSON OIL
                                  at               lease!!
                              gmail.com            


From storchaka at gmail.com  Tue May 19 15:51:10 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 19 May 2015 22:51:10 +0300
Subject: Slices time complexity
In-Reply-To: <CALwzidkfonf=vOUrMNH0NWEu54_EZpPDFWQqwYYdGaP-rT8Qfw@mail.gmail.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjf2fc$r4o$1@ger.gmane.org>
 <CALwzidkfonf=vOUrMNH0NWEu54_EZpPDFWQqwYYdGaP-rT8Qfw@mail.gmail.com>
Message-ID: <mjg47e$pa$1@ger.gmane.org>

On 19.05.15 18:15, Ian Kelly wrote:
> On May 19, 2015 4:16 AM, "Serhiy Storchaka" <storchaka at gmail.com
> <mailto:storchaka at gmail.com>> wrote:
>  >
>  > On 19.05.15 12:45, Steven D'Aprano wrote:
>  >>
>  >> On Tuesday 19 May 2015 05:23, Mario Figueiredo wrote:
>  >>>
>  >>>  From the above link it seems slices work in linear time on all cases.
>  >>
>  >>
>  >> I wouldn't trust that is always the case, e.g. deleting a contiguous
> slice
>  >> from the end of a list could be O(1).
>  >
>  >
>  > It always has linear complexity. You need to decref removed elements.
>
> Only in CPython. The operation might be O(1) in Pypy or Jython.

In any case you need linear time to free all objects.



From jon+usenet at unequivocal.co.uk  Tue May 19 16:14:50 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 19 May 2015 20:14:50 +0000 (UTC)
Subject: Best way to rewrite Popen
References: <87siastsby.fsf@Equus.decebal.nl>
 <slrnmlmt6n.864.jon+usenet@frosty.unequivocal.co.uk>
 <874mn8tm70.fsf@Equus.decebal.nl>
Message-ID: <slrnmln6fu.864.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-19, Cecil Westerhof <Cecil at decebal.nl> wrote:
> Op Tuesday 19 May 2015 19:36 CEST schreef Jon Ribbens:
>
>> On 2015-05-19, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>> At the moment I am playing with things like: p =
>>> subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
>>>
>>> I think that most of the times this are the values I want. So it
>>> would be nice to overrule the defaults. What is the best way to do
>>> this? So creating a function that is exactly the same except for
>>> the defaults for shell and stdout (and maybe stderr).
>>
>> Yes.
>>
>> def shellprocess(cmd, **kwargs):
>> return subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE,
>> **kwargs)
>
> Will that not go wrong if I call it with?
>     shellprocess('ls -1', shell = False)

Yes. I thought you wanted a function that was specific to your
purpose, rather than one that was identical to Popen but with
different defaults. For that I suppose you could do:

  def shellprocess(args, **kwargs):
      kwargs.setdefault("shell", True)
      kwargs.setdefault("stdout", subprocess.PIPE)
      return subprocess.Popen(args, **kwargs)

> I want to rewrite a Bash script into a Python script. The 'ls -1' is
> only an example. But Popen and listdir give a different output. The
> sorting is different. But I could sort it myself.
>
> Another problem is that I work with a filter later on. But I could do
> that with Python also of-course. So maybe I should rethink what I want
> to do. ;-)

Indeed, porting a script from bash to Python is probably best done by
writing the identical functionality in the style of Python, rather
than doing a line-by-line literal conversion.


From Cecil at decebal.nl  Tue May 19 16:19:51 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 19 May 2015 22:19:51 +0200
Subject: Best way to rewrite Popen
References: <87siastsby.fsf@Equus.decebal.nl>
 <slrnmlmt6n.864.jon+usenet@frosty.unequivocal.co.uk>
 <874mn8tm70.fsf@Equus.decebal.nl>
Message-ID: <87zj50s4k8.fsf@Equus.decebal.nl>

Op Tuesday 19 May 2015 21:13 CEST schreef Cecil Westerhof:

> Op Tuesday 19 May 2015 19:36 CEST schreef Jon Ribbens:
>
>> On 2015-05-19, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>> At the moment I am playing with things like: p =
>>> subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
>>>
>>> I think that most of the times this are the values I want. So it
>>> would be nice to overrule the defaults. What is the best way to do
>>> this? So creating a function that is exactly the same except for
>>> the defaults for shell and stdout (and maybe stderr).
>>
>> Yes.
>>
>> def shellprocess(cmd, **kwargs):
>> return subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE,
>> **kwargs)
>
> Will that not go wrong if I call it with?
> shellprocess('ls -1', shell = False)
>
>
>>> It is a little less important as I first thought, because I found
>>> the following: error, output = subprocess.getstatusoutput('ls -1')
>>> files_new = output.splitlines() But it is still nice to know.
>>
>> Why are you doing this anyway, rather than using os.listdir()?
>> Invoking subprocesses via the shell is very rarely a good idea.
>
> I want to rewrite a Bash script into a Python script. The 'ls -1' is
> only an example. But Popen and listdir give a different output. The
> sorting is different. But I could sort it myself.
>
> Another problem is that I work with a filter later on. But I could
> do that with Python also of-course. So maybe I should rethink what I
> want to do. ;-)

It looks like that this does what I want (the dot is needed so that it
also works with 2.7):
    files = sorted(os.listdir('.'))
    p = re.compile('actions-2015-05-[0-9][0-9].sql$')
    current_month = [ file for file in files if p.match(file) ]

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From jon+usenet at unequivocal.co.uk  Tue May 19 17:28:52 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 19 May 2015 21:28:52 +0000 (UTC)
Subject: Best way to rewrite Popen
References: <87siastsby.fsf@Equus.decebal.nl>
 <slrnmlmt6n.864.jon+usenet@frosty.unequivocal.co.uk>
 <874mn8tm70.fsf@Equus.decebal.nl> <87zj50s4k8.fsf@Equus.decebal.nl>
Message-ID: <slrnmlnaqp.864.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-19, Cecil Westerhof <Cecil at decebal.nl> wrote:
> It looks like that this does what I want (the dot is needed so that it
> also works with 2.7):
>     files = sorted(os.listdir('.'))
>     p = re.compile('actions-2015-05-[0-9][0-9].sql$')
>     current_month = [ file for file in files if p.match(file) ]

You could instead do (in Python 2 or 3):

  files = glob.glob("actions-2015-05-[0-9][0-9].sql")
  files.sort()


From hossamalagmy at gmail.com  Tue May 19 17:30:51 2015
From: hossamalagmy at gmail.com (=?windows-1256?B?1O3TyNM=?=)
Date: Tue, 19 May 2015 14:30:51 -0700 (PDT)
Subject: =?windows-1256?B?5MrtzMkgx+HU5cfPySDH4cfIys/Hxu3JIDIwMTUg?=
 =?windows-1256?B?49rR3ckgx+Hkyu3MySDI0d7jIMfhzOHm0yDH4e3m4yA=?=
 =?windows-1256?B?x+HTx8ja?=
Message-ID: <382b2133-b0e9-4535-b4ec-d48f695457b2@googlegroups.com>

????? ??????? ?????????? 2015 ????? ??????? ???? ?????? ????? ??????


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?fref=ts


From hossamalagmy at gmail.com  Tue May 19 17:40:38 2015
From: hossamalagmy at gmail.com (=?windows-1256?B?483j5s8=?=)
Date: Tue, 19 May 2015 14:40:38 -0700 (PDT)
Subject: =?windows-1256?B?5MrtzMkgx+HU5cfPySDH4cfIys/Hxu3JIDIwMTU=?=
Message-ID: <97c7ccb5-7c32-4b91-8f15-f5165151e8bc@googlegroups.com>

????? ??????? ?????????? 2015


https://www.facebook.com/pages/%D9%86%D8%AA%D8%A7%D8%A6%D8%AC-%D8%A7%D9%84%D8%A7%D9%85%D8%AA%D8%AD%D8%A7%D9%86%D8%A7%D8%AA-%D9%88%D8%A7%D9%84%D8%AC%D8%A7%D9%85%D8%B9%D8%A7%D8%AA-%D9%88%D8%A7%D8%AC%D8%AA%D9%85%D8%A7%D8%B9%D9%8A%D8%A7%D8%AA/299719160065550?fref=ts


From Cecil at decebal.nl  Tue May 19 18:23:20 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 20 May 2015 00:23:20 +0200
Subject: Best way to rewrite Popen
References: <87siastsby.fsf@Equus.decebal.nl>
 <slrnmlmt6n.864.jon+usenet@frosty.unequivocal.co.uk>
 <874mn8tm70.fsf@Equus.decebal.nl> <87zj50s4k8.fsf@Equus.decebal.nl>
 <slrnmlnaqp.864.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <87siasryuf.fsf@Equus.decebal.nl>

Op Tuesday 19 May 2015 23:28 CEST schreef Jon Ribbens:

> On 2015-05-19, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> It looks like that this does what I want (the dot is needed so that
>> it also works with 2.7): files = sorted(os.listdir('.')) p =
>> re.compile('actions-2015-05-[0-9][0-9].sql$') current_month = [
>> file for file in files if p.match(file) ]
>
> You could instead do (in Python 2 or 3):
>
> files = glob.glob("actions-2015-05-[0-9][0-9].sql")
> files.sort()

Something to remember.

But in this case I also need the previous month. So I have:
    files           = sorted(os.listdir('.'))
    p               = re.compile('actions-2015-05-[0-9][0-9].sql$')
    current_month   = [ file for file in files if p.match(file) ]
    p               = re.compile('actions-2015-04-[0-9][0-9].sql$')
    previous_month  = [ file for file in files if p.match(file) ]

Of-course I will not hard-code the months in the real code.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From PointedEars at web.de  Tue May 19 18:30:57 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Wed, 20 May 2015 00:30:57 +0200
Subject: Rule of order for dot operators?
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mjacv8$ods$6@dont-email.me>
Message-ID: <2710946.pHPuBPdTgQ@PointedEars.de>

Denis McMahon wrote:

> On Sun, 17 May 2015 11:45:02 +1000, Steven D'Aprano wrote:
>> On Sun, 17 May 2015 05:40 am, Thomas 'PointedEars' Lahn wrote:
>>> C.D. Reimer wrote:
>>> ^^^^
>>> Who?
>> Don't be a dick, Thomas.
> 
> Thomas is a professional dick, he can't help it, he's been a professional
> dick for years in php and javascript groups, and now he's obviously
> spreading himself further afield. He usually confines his wisdom to
> pointing out faults in other's posts, rather than offering any
> constructive input himself.

Yes, I am a professional, while you are just a dick.  Every single time.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From python at mrabarnett.plus.com  Tue May 19 19:20:30 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 20 May 2015 00:20:30 +0100
Subject: Best way to rewrite Popen
In-Reply-To: <87siasryuf.fsf@Equus.decebal.nl>
References: <87siastsby.fsf@Equus.decebal.nl>
 <slrnmlmt6n.864.jon+usenet@frosty.unequivocal.co.uk>
 <874mn8tm70.fsf@Equus.decebal.nl> <87zj50s4k8.fsf@Equus.decebal.nl>
 <slrnmlnaqp.864.jon+usenet@frosty.unequivocal.co.uk>
 <87siasryuf.fsf@Equus.decebal.nl>
Message-ID: <555BC53E.8010207@mrabarnett.plus.com>

On 2015-05-19 23:23, Cecil Westerhof wrote:
> Op Tuesday 19 May 2015 23:28 CEST schreef Jon Ribbens:
>
>> On 2015-05-19, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>> It looks like that this does what I want (the dot is needed so that
>>> it also works with 2.7): files = sorted(os.listdir('.')) p =
>>> re.compile('actions-2015-05-[0-9][0-9].sql$') current_month = [
>>> file for file in files if p.match(file) ]
>>
>> You could instead do (in Python 2 or 3):
>>
>> files = glob.glob("actions-2015-05-[0-9][0-9].sql")
>> files.sort()
>
> Something to remember.
>
> But in this case I also need the previous month. So I have:
>      files           = sorted(os.listdir('.'))
>      p               = re.compile('actions-2015-05-[0-9][0-9].sql$')
>      current_month   = [ file for file in files if p.match(file) ]
>      p               = re.compile('actions-2015-04-[0-9][0-9].sql$')
>      previous_month  = [ file for file in files if p.match(file) ]
>
> Of-course I will not hard-code the months in the real code.
>
In a regex, '.' will match any character except '\n', or any character
at all if the DOTALL ('(?s)') flag in turned on. If you want to match
an actual '.', you should escape it like this: r'\.'. (And if you're
using backslashes in a string literal, make it a raw string literal!)

     p = re.compile(r'actions-2015-05-[0-9][0-9]\.sql$')



From Cecil at decebal.nl  Tue May 19 20:15:14 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 20 May 2015 02:15:14 +0200
Subject: Best way to rewrite Popen
References: <87siastsby.fsf@Equus.decebal.nl>
 <slrnmlmt6n.864.jon+usenet@frosty.unequivocal.co.uk>
 <874mn8tm70.fsf@Equus.decebal.nl> <87zj50s4k8.fsf@Equus.decebal.nl>
 <slrnmlnaqp.864.jon+usenet@frosty.unequivocal.co.uk>
 <87siasryuf.fsf@Equus.decebal.nl>
 <mailman.155.1432077640.17265.python-list@python.org>
Message-ID: <87k2w4rtnx.fsf@Equus.decebal.nl>

Op Wednesday 20 May 2015 01:20 CEST schreef MRAB:

> On 2015-05-19 23:23, Cecil Westerhof wrote:
>> Op Tuesday 19 May 2015 23:28 CEST schreef Jon Ribbens:
>>
>>> On 2015-05-19, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>>> It looks like that this does what I want (the dot is needed so
>>>> that it also works with 2.7): files = sorted(os.listdir('.')) p =
>>>> re.compile('actions-2015-05-[0-9][0-9].sql$') current_month = [
>>>> file for file in files if p.match(file) ]
>>>
>>> You could instead do (in Python 2 or 3):
>>>
>>> files = glob.glob("actions-2015-05-[0-9][0-9].sql")
>>> files.sort()
>>
>> Something to remember.
>>
>> But in this case I also need the previous month. So I have:
>> files           = sorted(os.listdir('.'))
>> p               = re.compile('actions-2015-05-[0-9][0-9].sql$')
>> current_month   = [ file for file in files if p.match(file) ]
>> p               = re.compile('actions-2015-04-[0-9][0-9].sql$')
>> previous_month  = [ file for file in files if p.match(file) ]
>>
>> Of-course I will not hard-code the months in the real code.
>>
> In a regex, '.' will match any character except '\n', or any
> character at all if the DOTALL ('(?s)') flag in turned on. If you
> want to match an actual '.', you should escape it like this: r'\.'.
> (And if you're using backslashes in a string literal, make it a raw
> string literal!)
>
> p = re.compile(r'actions-2015-05-[0-9][0-9]\.sql$')

Oops, you are correct.

It is now:
    start               = database + '-'
    end                 = r'-[0-9][0-9]\.sql$'
    p                   = re.compile(start + current_month  + end)
    current_month_lst   = [ file for file in files if p.match(file) ]
    p                   = re.compile(start + previous_month + end)
    previous_month_lst  = [ file for file in files if p.match(file) ]

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From greg.ewing at canterbury.ac.nz  Tue May 19 20:31:37 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Wed, 20 May 2015 12:31:37 +1200
Subject: Slices time complexity
In-Reply-To: <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <cs22vbFnd69U1@mid.individual.net>

Steven D'Aprano wrote:
> Chris, that is one of the best explanations for why "references equals
> pointers" is *not* a good explanation for Python's behaviour.

Many people here seem to have lost sight of the
fact that the word "pointer" existed in the English
language long before C, and even long before computers.

If I draw two boxes on a blackboard with an arrow
between them, I think it's perfectly reasonable to
call that arrow a pointer.

-- 
Greg


From ben+python at benfinney.id.au  Tue May 19 20:42:37 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 May 2015 10:42:37 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
Message-ID: <85siasyt8i.fsf@benfinney.id.au>

Gregory Ewing <greg.ewing at canterbury.ac.nz> writes:

> If I draw two boxes on a blackboard with an arrow between them, I
> think it's perfectly reasonable to call that arrow a pointer.

Right. So the box with an arrow coming out of it is a good metaphor for
pointers ? *in languages that have pointers*, which Python does not.

A box with an arrow coming out of it is a poor metaphor for Python's
references, since a Python reference doesn't contain anything accessible
to the programmer.

Instead, to avoid giving the false impression that Python's references
are boxes containing things, write a bare name (or other reference) on
the blackboard, referring to the box that *does* contain a value
accessible to the programmer.

-- 
 \       ?The Vatican is not a state.? a state must have people. There |
  `\    are no Vaticanians.? No-one gets born in the Vatican except by |
_o__)        an unfortunate accident.? ?Geoffrey Robertson, 2010-09-18 |
Ben Finney



From steve+comp.lang.python at pearwood.info  Tue May 19 20:43:32 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 20 May 2015 10:43:32 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <555b21f2$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <87lhgkeexy.fsf@elektro.pacujo.net>
 <555b7711$0$12986$c3e8da3$5496439d@news.astraweb.com>
 <87bnhge8gh.fsf@elektro.pacujo.net>
Message-ID: <555bd8b5$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Wed, 20 May 2015 04:19 am, Marko Rauhamaa wrote:

> Steven D'Aprano <steve+comp.lang.python at pearwood.info>:
> 
>> On Wed, 20 May 2015 01:59 am, Marko Rauhamaa wrote:
>>> You're slaying straw men.
>>
>> "I was wrong, but I don't want to admit it" is not spelled "straw man".
>>
>> You made a claim that was not defensible, and I tore that claim to
>> shreds. Have the good grace to admit that your earlier position:
>>
>>     "it doesn't matter what semantic description you give Python
>>     constructs as long as the observed behavior is correct"
> 
> I stand by that position and haven't changed it.
> 
>> doesn't stand up to scrutiny.
> 
> But it does.


Right. So you stand by the position that this explanation for how the Python
interpreter does integer arithmetic is equally good as any other:

There is a little magic elf hiding inside the computer, and when you type an
expression like '2 + 3', little tiny hammers bang it out in Morse Code on
the elf's head; in response, the elf calculates the answer on his teeny
tiny abacus and passes it back to the interpreter.

The elf also does calculations with decimal numbers like '1.1*2.3', but
they're harder which explains why sometimes the elf gets the calculations
slightly wrong.

Since this explanation explains the observed behaviour, according to you it
is equally valid as one involving, you know, actual facts.

Why am I talking to you?



-- 
Steven



From greg.ewing at canterbury.ac.nz  Tue May 19 21:10:36 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Wed, 20 May 2015 13:10:36 +1200
Subject: fork/exec & close file descriptors
In-Reply-To: <mailman.139.1432045882.17265.python-list@python.org>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <CAPTjJmoj4STV9+t2-9JWed3t6Hxk5ibtaxYSFfX31oGz=kKR1g@mail.gmail.com>
 <CANc-5UwUxYjg=rQQ0Z_nM3s7o4+ghVYzG-9ajQY=LERWAsPrKw@mail.gmail.com>
 <CAPTjJmrdPcW8EkLR9Kq+n9sYEWB8eJ3fg=_JFQ2x_4QhDeg90Q@mail.gmail.com>
 <mailman.139.1432045882.17265.python-list@python.org>
Message-ID: <cs258eFnssnU1@mid.individual.net>

> On Tue, May 19, 2015 at 8:54 AM, Chris Angelico <rosuav at gmail.com 
> <mailto:rosuav at gmail.com>> wrote:
> 
>     On Linux (and possibly some other Unixes), /proc/self/fd may be of
>     use.

On MacOSX, /dev/fd seems to be the equivalent of this.

-- 
Greg


From PointedEars at web.de  Tue May 19 21:20:23 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Wed, 20 May 2015 03:20:23 +0200
Subject: Best way to rewrite Popen
References: <87siastsby.fsf@Equus.decebal.nl>
Message-ID: <2932409.NK0IxsuCCG@PointedEars.de>

Cecil Westerhof wrote:

> At the moment I am playing with things like:
>     p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
> 
> I think that most of the times this are the values I want. So it would
> be nice to overrule the defaults. What is the best way to do this?
> So creating a function that is exactly the same except for the defaults
> for shell and stdout (and maybe stderr).

As always in OOP, override the constructor and call that of the superclass 
in yours.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From PointedEars at web.de  Tue May 19 21:23:06 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Wed, 20 May 2015 03:23:06 +0200
Subject: Best way to rewrite Popen
References: <87siastsby.fsf@Equus.decebal.nl>
Message-ID: <8683113.Q7fQorzW6p@PointedEars.de>

Cecil Westerhof wrote:

> At the moment I am playing with things like:
>     p = subprocess.Popen('ls -l', shell = True, stdout = subprocess.PIPE)
> 
> I think that most of the times this are the values I want. So it would
> be nice to overrule the defaults. What is the best way to do this?
> So creating a function that is exactly the same except for the defaults
> for shell and stdout (and maybe stderr).

As always in OOP: override the method and call that of the superclass 
in yours.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From ejmmanning at gmail.com  Tue May 19 21:27:44 2015
From: ejmmanning at gmail.com (ejmmanning at gmail.com)
Date: Tue, 19 May 2015 18:27:44 -0700 (PDT)
Subject: Sftp error  New Help
Message-ID: <a52cdb03-2d5c-4944-bc4a-154f8923a129@googlegroups.com>

Traceback (most recent call last):
  File "Juniper.py", line 66, in <module>
    device_information()
  File "Juniper.py", line 26, in device_information
    device_connection(dev_ip,dev_username,dev_password)
  File "Juniper.py", line 54, in device_connection
    sftp_transfer(r)
  File "Juniper.py", line 61, in sftp_transfer
    c.put("%r" %r)
  File "/usr/local/lib/python2.7/dist-packages/pysftp.py", line 349, in put
    confirm=confirm)
  File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 667, in put
    file_size = os.stat(localpath).st_size
OSError: [Errno 2] No such file or directory: "<closed file 'juniper-results20150519-191928.txt', mode 'a' at 0x24a0780>"





import sys
import pysftp
import datetime
import sys
import os



from jnpr.junos import Device

def device_information():
    while True:
        dev_ip = raw_input("Please enter Host IP? ")
        if dev_ip != "":
            while True:
                dev_username = raw_input("Please enter username? ")
                if dev_username !="":
                    while True:
                        dev_password = raw_input("Please enter password? ")
                        if dev_password !="":
                            device_connection(dev_ip,dev_username,dev_password)

                            return

                        else:
                           print(" Please enter a valid password?")

                else:
                    print(" Please enter a valid Username?")
        else:
            print(" Please enter a valid Host IP?")



def device_connection(dev_ip,dev_username,dev_password):
    file = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
    dev = Device(host = dev_ip,user = dev_username, password = dev_password)
    dev.open()
    f = open('show-command.txt')

    for line in iter(f):
        show_command = dev.cli(line)
        r = open("juniper-results" + file + ".txt",'a')
        r.write("***"+ line)
        r.write(show_command)
        r.write('\n')
        r.close()

    sftp_transfer(r)
    return


def sftp_transfer(r):

    c = pysftp.Connection(host = "10.72.129.35",port = 22, username = "Anonymous" , password = "Anonymous" )
    c.put("%r" %r)
    return

#sftp_transfer()

device_information()


From ned at nedbatchelder.com  Tue May 19 21:36:31 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 19 May 2015 18:36:31 -0700 (PDT)
Subject: Rule of order for dot operators?
In-Reply-To: <2710946.pHPuBPdTgQ@PointedEars.de>
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mjacv8$ods$6@dont-email.me> <2710946.pHPuBPdTgQ@PointedEars.de>
Message-ID: <f69e21a0-026a-4a9d-bac6-78ba891bfb13@googlegroups.com>

On Tuesday, May 19, 2015 at 6:33:46 PM UTC-4, Thomas 'PointedEars' Lahn wrote:
> Denis McMahon wrote:
> 
> > On Sun, 17 May 2015 11:45:02 +1000, Steven D'Aprano wrote:
> >> On Sun, 17 May 2015 05:40 am, Thomas 'PointedEars' Lahn wrote:
> >>> C.D. Reimer wrote:
> >>> ^^^^
> >>> Who?
> >> Don't be a dick, Thomas.
> > 
> > Thomas is a professional dick, he can't help it, he's been a professional
> > dick for years in php and javascript groups, and now he's obviously
> > spreading himself further afield. He usually confines his wisdom to
> > pointing out faults in other's posts, rather than offering any
> > constructive input himself.
> 
> Yes, I am a professional, while you are just a dick.  Every single time.

Everyone should stop calling people names.

Thomas, you were obnoxious to the OP, and now you are being obnoxious (and factually incorrect) to Denis.

Steven and Denis: you were too blunt in your objections.

Be considerate. Be respectful.

--Ned.


From steve+comp.lang.python at pearwood.info  Tue May 19 21:47:52 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 20 May 2015 11:47:52 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <cs0jeeFb5egU1@mid.individual.net>
Message-ID: <555be7c9$0$12982$c3e8da3$5496439d@news.astraweb.com>

On Tue, 19 May 2015 09:00 pm, Gregory Ewing wrote:

> Chris Angelico wrote:
>> 1) Pointer arithmetic simply doesn't exist in Python. Arrays/lists are
>> not just pointers to their first elements, and subscripting is most
>> definitely NOT "add to pointer and dereference".
>> 2) In fact, dereferencing as a whole isn't really a 'thing' either. At
>> best, it happens automatically.
>> 3) References actually mean something. Copying a pointer doesn't.
>> Whether the Python you're using is refcounted (CPython) or
>> mark-and-sweep (uPy, I think) or some other model, an additional
>> reference to the same object will prevent it from being disposed of,
>> which isn't the case in C.
>> 4) A pointer is itself a value. You can pass a
>> pointer-to-local-variable to another function and have that function
>> change a local variable.
>> 5) Furthermore, since a pointer is a value, you can have pointers to
>> pointers, etc. Doesn't make any sense in Python.
> 
> FWIW, most of those things are peculiar to C, and are not
> necessarily shared even with other languages that have
> explicit pointers. In Pascal, for example, there is no
> pointer arithmetic, 

That's true in theory, but I don't think that's right in practice. All the
Pascal compilers that I know of which are still in common use support
pointer arithmetic: Turbo Pascal, GNU Pascal, Free Pascal and Delphi. Some
compilers support an "inc" [increment] function to advance the pointer
safely. For those compilers that don't provide either, there's the
possibility of typecasting the pointer to a longint and back. Back in the
1990s, I used Lightspeed Pascal, and it supported typecasting pointers. For
those which don't even support that, you can usually abuse variant records
to get around the type system.

(Niklaus Wirth hated this, and made sure that this hack didn't work in his
later language Oberon.)

And finally, if all else fails, many Pascal compilers support embedding
assembly language, which obviously lets you do arithmetic on pointers.

The larger point is, the failure to allow pointer arithmetic in standard
Pascal is a design choice of the designer, not a fundamental restriction on
the pointer concept. Wirth didn't include pointer arithmetic in standard
Pascal because *he thought it was a bad idea*, not because Pascal pointers
are incapable of having arithmetic done to them. They're just an
abstraction over *numeric* memory addresses, just like in C, and there's
nothing more fundamental to numbers than arithmetic. You can't do
arithmetic on pointers in standard Pascal because the compiler stops you,
not because the operation makes no sense.

Bringing this back to Python, references in Python are *not* abstractions
over memory addresses. There is nothing in the Python model of computation
that requires values to even have such a thing as a memory address, let
alone pointers. The "references are pointers" explanation only gets you so
far, and not very far at that.


> and (at least not without nonstandard 
> extension) you can't make a pointer point into the middle
> of a stack frame or array or record, etc. (You *can* have
> a pointer to a pointer, but only by heap-allocating a
> memory block containing a single pointer, which is not
> useful very often).

Pascal programming on classic Macintosh used pointers-to-pointers
extensively, so much so that they had a name, "handle". Technically though
handles were managed by the Macintosh Toolbox routines, so slightly
different from what you are talking about. But in standard Pascal, you
certainly could create pointers to pointers.



-- 
Steven



From denismfmcmahon at gmail.com  Tue May 19 21:57:51 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Wed, 20 May 2015 01:57:51 +0000 (UTC)
Subject: Convert c file to csv file(unix format) in python
References: <c9733ddb-7f97-4fcf-9dad-a4481d1d3e5d@googlegroups.com>
 <mjfusi$6mq$2@dont-email.me>
 <11c2e290-52db-4bc6-a5fe-5757230a816c@googlegroups.com>
Message-ID: <mjgpmv$6mq$4@dont-email.me>

On Tue, 19 May 2015 12:26:05 -0700, umasrinath88 wrote:

> I have shared c file with you.Iorder to convert to unix format csv,
> below are the main steps we can follow.

What you're trying to do doesn't match with what you're explaining here.

What you're actually trying to do is extract a constant data declaration 
from a c source code file and output it in a different format.

You haven't explained whether you want the hexadecimal values converted 
to decimal integers or not.

Nor have you stated whether you want the hexadecimal values as quoted 
strings or not.

I believe I have a short program that does what you require based on the 
file you sent me and what I think is the output you're looking for. 
However, before I show you my solution, I'd like to see evidence of your 
own attempt to solve the problem.

Based on the sample you sent, the output csv file is 1657 lines in 
length, and the first and last lines are:

16,0x08000000,0xCC,0x16,0x00,0x20,0x35,0x15,0x00,0x08,0x29,0x15,0x00,0x08,0x2D,0x15,0x00,0x08

12,0x08006780,0x40,0x00,0x00,0x00,0x01,0x00,0x00,0x00,0xFF,0x00,0x00,0x00

-- 
Denis McMahon, denismfmcmahon at gmail.com


From PointedEars at web.de  Tue May 19 22:24:05 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Wed, 20 May 2015 04:24:05 +0200
Subject: Rule of order for dot operators?
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mjacv8$ods$6@dont-email.me> <2710946.pHPuBPdTgQ@PointedEars.de>
 <f69e21a0-026a-4a9d-bac6-78ba891bfb13@googlegroups.com>
Message-ID: <16277536.fgHiFt9JYW@PointedEars.de>

Ned Batchelder wrote:

> On Tuesday, May 19, 2015 at 6:33:46 PM UTC-4, Thomas 'PointedEars' Lahn
> wrote:
>> Denis McMahon wrote:
>> > On Sun, 17 May 2015 11:45:02 +1000, Steven D'Aprano wrote:
>> >> On Sun, 17 May 2015 05:40 am, Thomas 'PointedEars' Lahn wrote:
>> >>> C.D. Reimer wrote:
>> >>> ^^^^
>> >>> Who?
>> >> Don't be a dick, Thomas.
>> > 
>> > Thomas is a professional dick, he can't help it, he's been a
>> > professional dick for years in php and javascript groups, and now he's
>> > obviously spreading himself further afield. He usually confines his
>> > wisdom to pointing out faults in other's posts, rather than offering
>> > any constructive input himself.
>> 
>> Yes, I am a professional, while you are just a dick.  Every single time.
> 
> Everyone should stop calling people names.
> 
> Thomas, you were obnoxious to the OP, and now you are being obnoxious (and
> factually incorrect) to Denis.
> 
> Steven and Denis: you were too blunt in your objections.
> 
> Be considerate. Be respectful.

Yes, I could have skipped this comment, and I am not in the habit of calling 
names; but frankly, enough is enough.  Denis is posting libellous statements 
about me in every Big 8 newsgroup in which I participate because he often 
posts awfully wrong technically statements and just cannot stand being 
corrected by me.

And who appointed you moderator?  I would not mind if you *asked* people 
publicly for politeness as I do that myself sometimes, but I do mind the 
overall tone of your off-topic comments, your apparently trying to dictate 
what people are allowed to say here.  If that?s the way this newsgroup 
works, I am off here.  (But I seriously doubt it.)  Because that is _not_ 
how Usenet is supposed to work.

For now, just EOD for me, and F'up2 poster.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From steve+comp.lang.python at pearwood.info  Tue May 19 22:26:30 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 20 May 2015 12:26:30 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
Message-ID: <555bf0d7$0$12980$c3e8da3$5496439d@news.astraweb.com>

On Wed, 20 May 2015 10:31 am, Gregory Ewing wrote:

> Steven D'Aprano wrote:
>> Chris, that is one of the best explanations for why "references equals
>> pointers" is *not* a good explanation for Python's behaviour.
> 
> Many people here seem to have lost sight of the
> fact that the word "pointer" existed in the English
> language long before C, and even long before computers.

Many people here seem to have lost sight of the fact that the
word "computer" existed in the English language long before ENIAC and
Colossus, and even before Babbage's Difference Engine.


> If I draw two boxes on a blackboard with an arrow
> between them, I think it's perfectly reasonable to
> call that arrow a pointer.

Given how rich the English language is, and how many other words people
could use (arrow, cue, finger, guide, index, indicator, lead, needle,
signpost...) but don't, I think it is quite disingenuous to claim that
people describing Python references as "pointers" mean it in the generic
sense rather than the computer science sense.

Especially when those people often explicitly state that they are
using "pointer" in order to make it easier for C programmers to understand.



-- 
Steven



From steve+comp.lang.python at pearwood.info  Tue May 19 22:31:17 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 20 May 2015 12:31:17 +1000
Subject: Rule of order for dot operators?
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mjacv8$ods$6@dont-email.me> <2710946.pHPuBPdTgQ@PointedEars.de>
 <f69e21a0-026a-4a9d-bac6-78ba891bfb13@googlegroups.com>
Message-ID: <555bf1f6$0$12980$c3e8da3$5496439d@news.astraweb.com>

On Wed, 20 May 2015 11:36 am, Ned Batchelder wrote:

> Steven and Denis: you were too blunt in your objections.

Fair enough.

In my defence, I'm from Australia, and we're notorious for calling a spade a
bloody shovel.

> Be considerate. Be respectful.



-- 
Steven



From rosuav at gmail.com  Tue May 19 22:36:06 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 20 May 2015 12:36:06 +1000
Subject: Slices time complexity
In-Reply-To: <555bf0d7$0$12980$c3e8da3$5496439d@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <555bf0d7$0$12980$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmraPg1ymW5FBm6mdK+zw=Hr+_6gaFVxwJuUrMYLMaN5HA@mail.gmail.com>

On Wed, May 20, 2015 at 12:26 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Wed, 20 May 2015 10:31 am, Gregory Ewing wrote:
>
>> Steven D'Aprano wrote:
>>> Chris, that is one of the best explanations for why "references equals
>>> pointers" is *not* a good explanation for Python's behaviour.
>>
>> Many people here seem to have lost sight of the
>> fact that the word "pointer" existed in the English
>> language long before C, and even long before computers.
>
> Many people here seem to have lost sight of the fact that the
> word "computer" existed in the English language long before ENIAC and
> Colossus, and even before Babbage's Difference Engine.

I have a laser printer. Its address is 192.168.0.119. I also have a
laser pointer; is it equal to 192.168.0.119?

Ahh, English. So rich with equivocations.

ChrisA


From ned at nedbatchelder.com  Tue May 19 22:44:40 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 19 May 2015 19:44:40 -0700 (PDT)
Subject: Rule of order for dot operators?
In-Reply-To: <16277536.fgHiFt9JYW@PointedEars.de>
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mjacv8$ods$6@dont-email.me> <2710946.pHPuBPdTgQ@PointedEars.de>
 <f69e21a0-026a-4a9d-bac6-78ba891bfb13@googlegroups.com>
 <16277536.fgHiFt9JYW@PointedEars.de>
Message-ID: <758d0ddf-cf4a-462c-9f75-a6265373cfad@googlegroups.com>

On Tuesday, May 19, 2015 at 10:26:56 PM UTC-4, Thomas 'PointedEars' Lahn wrote:
> 
> And who appointed you moderator?  I would not mind if you *asked* people 
> publicly for politeness as I do that myself sometimes, but I do mind the 
> overall tone of your off-topic comments, your apparently trying to dictate 
> what people are allowed to say here.  If that's the way this newsgroup 
> works, I am off here.  (But I seriously doubt it.)  Because that is _not_ 
> how Usenet is supposed to work.

I am not a moderator, just someone who cares about the tone
in the group.  

The Python community has a code of conduct
(https://www.python.org/psf/codeofconduct/), and people on this
mailing list are expected to adhere to it.  It is where I got
the words "respectful" and "considerate".  The code of conduct
is how the Python community is supposed to work.

I'm sorry my post seemed too pushy.  Consider it a request for
polite discourse.

--Ned.


From rustompmody at gmail.com  Tue May 19 23:02:56 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 19 May 2015 20:02:56 -0700 (PDT)
Subject: Slices time complexity
In-Reply-To: <555bf0d7$0$12980$c3e8da3$5496439d@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <555bf0d7$0$12980$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <26921292-420e-4434-a69f-05ec1b56e625@googlegroups.com>

On Wednesday, May 20, 2015 at 7:56:43 AM UTC+5:30, Steven D'Aprano wrote:
> On Wed, 20 May 2015 10:31 am, Gregory Ewing wrote:
> 
> > Steven D'Aprano wrote:
> >> Chris, that is one of the best explanations for why "references equals
> >> pointers" is *not* a good explanation for Python's behaviour.
> > 
> > Many people here seem to have lost sight of the
> > fact that the word "pointer" existed in the English
> > language long before C, and even long before computers.
> 
> Many people here seem to have lost sight of the fact that the
> word "computer" existed in the English language long before ENIAC and
> Colossus, and even before Babbage's Difference Engine.
> 
> 
> > If I draw two boxes on a blackboard with an arrow
> > between them, I think it's perfectly reasonable to
> > call that arrow a pointer.
> 
> Given how rich the English language is, and how many other words people
> could use (arrow, cue, finger, guide, index, indicator, lead, needle,
> signpost...) but don't, I think it is quite disingenuous to claim that
> people describing Python references as "pointers" mean it in the generic
> sense rather than the computer science sense.
> 
> Especially when those people often explicitly state that they are
> using "pointer" in order to make it easier for C programmers to understand.


So...

Pascal is not Niklaus Wirth's Pascal but some vague pidgin extension(s) of it.
Whereas pointer is a specific C semantics, not a broader implication from C, 
nothing connected with the general English usage.  

Playing humpty dumpty arent we?


From rosuav at gmail.com  Tue May 19 23:11:53 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 20 May 2015 13:11:53 +1000
Subject: Rule of order for dot operators?
In-Reply-To: <16277536.fgHiFt9JYW@PointedEars.de>
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mjacv8$ods$6@dont-email.me> <2710946.pHPuBPdTgQ@PointedEars.de>
 <f69e21a0-026a-4a9d-bac6-78ba891bfb13@googlegroups.com>
 <16277536.fgHiFt9JYW@PointedEars.de>
Message-ID: <CAPTjJmrr0CMd4BJR56t-3LU-N=BumbUXDY6Sbc=zmXsVsEu8cA@mail.gmail.com>

On Wed, May 20, 2015 at 12:24 PM, Thomas 'PointedEars' Lahn
<PointedEars at web.de> wrote:
> And who appointed you moderator?  I would not mind if you *asked* people
> publicly for politeness as I do that myself sometimes, but I do mind the
> overall tone of your off-topic comments, your apparently trying to dictate
> what people are allowed to say here.  If that?s the way this newsgroup
> works, I am off here.  (But I seriously doubt it.)  Because that is _not_
> how Usenet is supposed to work.

Usually, the people who threaten to leave are the ones we will least
miss. Go ahead, leave if you don't like Ned asking you to be more
polite.

ChrisA


From gstc2 at pdx.edu  Tue May 19 23:51:31 2015
From: gstc2 at pdx.edu (Grace Anne St Clair-Bates)
Date: Tue, 19 May 2015 20:51:31 -0700 (PDT)
Subject: Writing a function to calculate scores
Message-ID: <3badfd67-8d5b-4b8b-a966-91f8a4c11ef0@googlegroups.com>

Hello! I am trying to write a funtion that calculates scores of three random dots on a "bean-bag" board that land on different holes. Each hole holds a certain amount of point. I need to take where the computer has randomly places three points in my turtle graphic and calculate the total score. I have no idea how to do this. Please help if you can! Its really hard to explain it, but if anyone has the slightest idea please let me know. Thanks! 


From marko at pacujo.net  Wed May 20 00:24:44 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 20 May 2015 07:24:44 +0300
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
Message-ID: <87oalfooz7.fsf@elektro.pacujo.net>

Ben Finney <ben+python at benfinney.id.au>:

> Right. So the box with an arrow coming out of it is a good metaphor for
> pointers ? *in languages that have pointers*, which Python does not.
>
> A box with an arrow coming out of it is a poor metaphor for Python's
> references, since a Python reference doesn't contain anything accessible
> to the programmer.

Lisp doesn't have pointers. However:

  <URL: http://upload.wikimedia.org/wikipedia/commons/1/1b/Cons-cells.svg>

  <URL: http://cs.gmu.edu/~sean/lisp/cons/>

  <URL: http://www.csee.umbc.edu/courses/331/fall11/notes/scheme/scheme2.ppt>


Marko


From marko at pacujo.net  Wed May 20 00:30:09 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 20 May 2015 07:30:09 +0300
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <555b21f2$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <87lhgkeexy.fsf@elektro.pacujo.net>
 <555b7711$0$12986$c3e8da3$5496439d@news.astraweb.com>
 <87bnhge8gh.fsf@elektro.pacujo.net>
 <555bd8b5$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87k2w3ooq6.fsf@elektro.pacujo.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info>:

> There is a little magic elf hiding inside the computer, and when you
> type an expression like '2 + 3', little tiny hammers bang it out in
> Morse Code on the elf's head; in response, the elf calculates the
> answer on his teeny tiny abacus and passes it back to the interpreter.

That would be a perfectly valid semantic explanation if it really
predicted the output.

> Since this explanation explains the observed behaviour, according to
> you it is equally valid as one involving, you know, actual facts.

It doesn't explain the observed behavior. It doesn't differentiate
between correct and incorrect outcomes.


Marko


From rustompmody at gmail.com  Wed May 20 00:30:41 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 19 May 2015 21:30:41 -0700 (PDT)
Subject: Slices time complexity
In-Reply-To: <87oalfooz7.fsf@elektro.pacujo.net>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
Message-ID: <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>

On Wednesday, May 20, 2015 at 9:54:57 AM UTC+5:30, Marko Rauhamaa wrote:
> Ben Finney :
> 
> > Right. So the box with an arrow coming out of it is a good metaphor for
> > pointers -- *in languages that have pointers*, which Python does not.
> >
> > A box with an arrow coming out of it is a poor metaphor for Python's
> > references, since a Python reference doesn't contain anything accessible
> > to the programmer.
> 
> Lisp doesn't have pointers. However:
> 
>   <URL: http://upload.wikimedia.org/wikipedia/commons/1/1b/Cons-cells.svg>
> 
>   <URL: http://cs.gmu.edu/~sean/lisp/cons/>
> 
>   <URL: http://www.csee.umbc.edu/courses/331/fall11/notes/scheme/scheme2.ppt>
> 

And what about Java?
http://stackoverflow.com/questions/166033/value-semantics-and-pointer-semantics


From breamoreboy at yahoo.co.uk  Wed May 20 01:06:33 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 20 May 2015 06:06:33 +0100
Subject: Slices time complexity
In-Reply-To: <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
Message-ID: <mjh4p0$haa$1@ger.gmane.org>

On 20/05/2015 05:30, Rustom Mody wrote:
> On Wednesday, May 20, 2015 at 9:54:57 AM UTC+5:30, Marko Rauhamaa wrote:
>> Ben Finney :
>>
>>> Right. So the box with an arrow coming out of it is a good metaphor for
>>> pointers -- *in languages that have pointers*, which Python does not.
>>>
>>> A box with an arrow coming out of it is a poor metaphor for Python's
>>> references, since a Python reference doesn't contain anything accessible
>>> to the programmer.
>>
>> Lisp doesn't have pointers. However:
>>
>>    <URL: http://upload.wikimedia.org/wikipedia/commons/1/1b/Cons-cells.svg>
>>
>>    <URL: http://cs.gmu.edu/~sean/lisp/cons/>
>>
>>    <URL: http://www.csee.umbc.edu/courses/331/fall11/notes/scheme/scheme2.ppt>
>>
>
> And what about Java?
> http://stackoverflow.com/questions/166033/value-semantics-and-pointer-semantics
>

And what about CORAL66/250?  How do you explain Python in terms of the 
original 66 and the 250 derivative?  What is the unladen air speed 
velocity of a swallow in flight?  Who actually gives a stuff?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rustompmody at gmail.com  Wed May 20 01:13:31 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 19 May 2015 22:13:31 -0700 (PDT)
Subject: Writing a function to calculate scores
In-Reply-To: <3badfd67-8d5b-4b8b-a966-91f8a4c11ef0@googlegroups.com>
References: <3badfd67-8d5b-4b8b-a966-91f8a4c11ef0@googlegroups.com>
Message-ID: <55343982-967b-4c1d-b152-e7b94f9621a3@googlegroups.com>

On Wednesday, May 20, 2015 at 9:21:51 AM UTC+5:30, Grace Anne St Clair-Bates wrote:
> Hello! I am trying to write a funtion that calculates scores of three random dots on a "bean-bag" board that land on different holes. Each hole holds a certain amount of point. I need to take where the computer has randomly places three points in my turtle graphic and calculate the total score. I have no idea how to do this. Please help if you can! Its really hard to explain it, but if anyone has the slightest idea please let me know. Thanks!

Lets say your board of 3 rows and 4 columns has points thusly:

2 4 6 5
1 2 1 7
3 4 5 8

You could represent it in python like so

board = [[2,4,6,5],[1,2,1,7],[3,4,5,8]]

And you can get to the 7 with the expression:  board[1][3]

(Remember in programming we usually count from 0 not 1)

Does this help any?


From frank at chagford.com  Wed May 20 01:33:12 2015
From: frank at chagford.com (Frank Millman)
Date: Wed, 20 May 2015 07:33:12 +0200
Subject: Slightly OT: Seeking (python-based) project diary tool,
 or framework to write one
References: <99067d97-cad4-42f8-8fd1-b1884bed769e@googlegroups.com>
Message-ID: <mjh6ap$8lh$1@ger.gmane.org>


"jkn" <jkn_gg at nicorp.f9.co.uk> wrote in message 
news:99067d97-cad4-42f8-8fd1-b1884bed769e at googlegroups.com...
> Hi All
>    as in the title, this is a little bit OT - but since ideally I'd like a
> tool written in Python, and I know readers here have wide experience of
> development/collaborative workflows etc ...
>

Have you looked at this one? -

http://taskcoach.org/

>From their home page -

"Task Coach is a simple open source todo manager to keep track of personal 
tasks and todo lists. It is designed for composite tasks, and also offers 
effort tracking, categories, notes and more."



There seems to be at least some overlap with your requirements, so it may be 
worth a look.


It uses wxPython. According to their download page it only works with Python 
2.x, but as wxPython for Python3 is getting closer, hopefully it is on their 
roadmap to target Python3 as well.

Frank Millman





From steve+comp.lang.python at pearwood.info  Wed May 20 01:33:58 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 20 May 2015 15:33:58 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <555b21f2$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <87lhgkeexy.fsf@elektro.pacujo.net>
 <555b7711$0$12986$c3e8da3$5496439d@news.astraweb.com>
 <87bnhge8gh.fsf@elektro.pacujo.net>
 <555bd8b5$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87k2w3ooq6.fsf@elektro.pacujo.net>
Message-ID: <555c1cc7$0$13003$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 20 May 2015 14:30, Marko Rauhamaa wrote:

> Steven D'Aprano <steve+comp.lang.python at pearwood.info>:
> 
>> There is a little magic elf hiding inside the computer, and when you
>> type an expression like '2 + 3', little tiny hammers bang it out in
>> Morse Code on the elf's head; in response, the elf calculates the
>> answer on his teeny tiny abacus and passes it back to the interpreter.
> 
> That would be a perfectly valid semantic explanation if it really
> predicted the output.

Which it does. State any mathematical expression involving operators and 
operands supported by Python, and I can express it in Morse code, and tell 
you exactly what result the elf will calculate.

Someone sufficiently killed with an abacus can probably even calculate it 
*using the same algorithms* that the elf will use. Being hit on the head by 
hammers is not compulsory.


>> Since this explanation explains the observed behaviour, according to
>> you it is equally valid as one involving, you know, actual facts.
> 
> It doesn't explain the observed behavior. It doesn't differentiate
> between correct and incorrect outcomes.

Perhaps when I wrote "the elf calculates the answer" it was unclear that I 
meant the *correct* answer rather than some arbitrary value which is not 
actually the answer. I will try to be more clear in the future.


-- 
Steve



From dieter at handshake.de  Wed May 20 01:36:29 2015
From: dieter at handshake.de (dieter)
Date: Wed, 20 May 2015 07:36:29 +0200
Subject: Sftp error  New Help
References: <a52cdb03-2d5c-4944-bc4a-154f8923a129@googlegroups.com>
Message-ID: <87siarkdya.fsf@handshake.de>

ejmmanning at gmail.com writes:

> Traceback (most recent call last):
>   File "Juniper.py", line 66, in <module>
>     device_information()
>   File "Juniper.py", line 26, in device_information
>     device_connection(dev_ip,dev_username,dev_password)
>   File "Juniper.py", line 54, in device_connection
>     sftp_transfer(r)
>   File "Juniper.py", line 61, in sftp_transfer
>     c.put("%r" %r)
>   File "/usr/local/lib/python2.7/dist-packages/pysftp.py", line 349, in put
>     confirm=confirm)
>   File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 667, in put
>     file_size = os.stat(localpath).st_size
> OSError: [Errno 2] No such file or directory: "<closed file 'juniper-results20150519-191928.txt', mode 'a' at 0x24a0780>"

Apparently, you pass to "sftp_transfer" a parameter with an unexpected
type. Looks as if it would expect a file name (and you pass a
file descriptor, referencing a closed file). Thus, pass the filename
and try again.



From steve+comp.lang.python at pearwood.info  Wed May 20 01:57:45 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 20 May 2015 15:57:45 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
Message-ID: <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>

On Wednesday 20 May 2015 14:30, Rustom Mody wrote:

> And what about Java?
> http://stackoverflow.com/questions/166033/value-semantics-and-pointer-
semantics

Congratulations, you have found yet another example of the Java community's 
collective delusion and insistence on misusing terms for the maximum 
confusion possible.

The mental gymnastics they go through to force the round peg of pass-by-
sharing object semantics into the false dichotomy "pass-by-value versus 
pass-by-reference" is just crazy.

One consequence of this is that the meaning of "call by value" depends on 
whether the value you are talking about is a boxed or unboxed value. Unboxed 
values are copied. Boxed values are not. Except that they are, if you 
understand that "the value" doesn't refer to the actual value, but to some 
hidden and implementation-dependent reference to the value.

To quote Fredrik Lundh:

    well, I guess you can, in theory, value an artificial number 
    assigned to an object as much as the object itself.

    "Joe, I think our son might be lost in the woods"
    "Don't worry, I have his social security number"


http://effbot.org/zone/call-by-object.htm


Rustom, if you are serious about this approach, then the implication is that 
if I execute "x = 23" in Python, and I ask you what the value of x is, you 
should answer something like "146588120" (that's the implementation 
dependent "value", i.e. the address of the int 23).

It's just *nuts*, and all because the Java folks are unwilling or unable to 
distinguish between the language semantics and the implementation of the 
language. And you're falling into the same hole.



-- 
Steve


From steve+comp.lang.python at pearwood.info  Wed May 20 01:58:51 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 20 May 2015 15:58:51 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <555b21f2$0$13000$c3e8da3$5496439d@news.astraweb.com>
 <87lhgkeexy.fsf@elektro.pacujo.net>
 <555b7711$0$12986$c3e8da3$5496439d@news.astraweb.com>
 <87bnhge8gh.fsf@elektro.pacujo.net>
 <555bd8b5$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87k2w3ooq6.fsf@elektro.pacujo.net>
 <555c1cc7$0$13003$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <555c229c$0$2769$c3e8da3$76491128@news.astraweb.com>

On Wednesday 20 May 2015 15:33, Steven D'Aprano wrote:


> Someone sufficiently killed with an abacus

Er, *skilled*.


-- 
Steve



From marko at pacujo.net  Wed May 20 02:17:08 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 20 May 2015 09:17:08 +0300
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
Message-ID: <87617nepsr.fsf@elektro.pacujo.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info>:

> Rustom, if you are serious about this approach, then the implication
> is that if I execute "x = 23" in Python, and I ask you what the value
> of x is, you should answer something like "146588120" (that's the
> implementation dependent "value", i.e. the address of the int 23).

Steven, you are trying to be facetious but end up making good points.

The "canonical" Python semantics actually states that

   x = 23

means:

   An int object with the numeric value 23 is constructed or fetched
   from a cache. A reference to the object is assigned to a variable
   whose name is "x".

> it's just *nuts*, and all because the Java folks are unwilling or
> unable to distinguish between the language semantics and the
> implementation of the language. And you're falling into the same hole.

I have yet to see a high-level language that has defined its data model
without resorting to "locations", "slots" and pointers of sorts. It
certainly is possible (lambda calculus doesn't make any reference to
objects or pointers) but hardly any alternative explanation is more
appealing to programmers' tastes and imagination than the concrete image
of boxes and arrows.


Marko


From rustompmody at gmail.com  Wed May 20 02:23:26 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 19 May 2015 23:23:26 -0700 (PDT)
Subject: Slices time complexity
In-Reply-To: <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
Message-ID: <28bca3fb-653d-40fb-9944-d28b01e77244@googlegroups.com>

On Wednesday, May 20, 2015 at 11:27:56 AM UTC+5:30, Steven D'Aprano wrote:
> On Wednesday 20 May 2015 14:30, Rustom Mody wrote:
> 
> > And what about Java?
> > http://stackoverflow.com/questions/166033/value-semantics-and-pointer-
> semantics
> 
> Congratulations, you have found yet another example of the Java community's 
> collective delusion and insistence on misusing terms for the maximum 
> confusion possible.
> 
> The mental gymnastics they go through to force the round peg of pass-by-
> sharing object semantics into the false dichotomy "pass-by-value versus 
> pass-by-reference" is just crazy.
> 
> One consequence of this is that the meaning of "call by value" depends on 
> whether the value you are talking about is a boxed or unboxed value. Unboxed 
> values are copied. Boxed values are not. Except that they are, if you 
> understand that "the value" doesn't refer to the actual value, but to some 
> hidden and implementation-dependent reference to the value.
> 
> To quote Fredrik Lundh:
> 
>     well, I guess you can, in theory, value an artificial number 
>     assigned to an object as much as the object itself.
> 
>     "Joe, I think our son might be lost in the woods"
>     "Don't worry, I have his social security number"
> 
> 
> http://effbot.org/zone/call-by-object.htm
> 
> 
> Rustom, if you are serious about this approach, then the implication is that 
> if I execute "x = 23" in Python, and I ask you what the value of x is, you 
> should answer something like "146588120" (that's the implementation 
> dependent "value", i.e. the address of the int 23).
> 
> It's just *nuts*, and all because the Java folks are unwilling or unable to 
> distinguish between the language semantics and the implementation of the 
> language. And you're falling into the same hole.

I am not arguing for the java-folks terminology or outlook.
I am arguing that for languages that execute on von Neumann machines memory
is not an optional negotiable concept.

And pointer is just first-classed memory as lambda is first-classed function.
Some languages firstclass it -- most notably C, also C# which increasingly fascinates me in its choices.
Others try to to unfirstclass , ie hide it in various half-assed ways -- Python, Java, Lisp.
Others try to hide it in the most heavy-handed way possible -- haskell

But you can never really hide it [law of leaky abstractions]

eg in haskell one makes an infinite list of ones thus

ones = 1 : ones

Now if you generalize this to

repeat x = x : repeat x
and then do
ones = repeat 1
you get poorer performance because the ones makes a box-n-arrow loop which the 
repeat does not achieve.

To get round that we do
repeat x = repeatx 
  where repeatx = x : repeatx

In short any language that is implemented on von Neumann hw will need to address
memory. The language-designer may believe that there is a choice to 'unfirstclass' it. However people discussing language have no such choice:
If you dont have it in the formal implemented language, you must have it in the
informal discussion-language
[With a large amount of wild waving of hands] 

<Admission>
I dont like teaching this. viz that in python
x = [[1,2],[1,2]]
is equal to y defined as
z = [1,2]
y = [z,z]
And although they are equal as in '==' they are not equal as in behavior, memory usage etc, a fact that can only be elucidated by box-n-arrow diagrams.

So as far as I can I teach by tabooing extend,append and all such mutablers

In the end though this is cheating
</Admission>


From howi682 at googlemail.com  Wed May 20 02:46:34 2015
From: howi682 at googlemail.com (Howard Spink)
Date: Tue, 19 May 2015 23:46:34 -0700 (PDT)
Subject: Help a newbie with basic coding involving physical switches
Message-ID: <aa34918c-0ddb-4a34-99cb-60f74e9849e3@googlegroups.com>

I have ordered a Pi + SD card + PiTfT screen.



From marko at pacujo.net  Wed May 20 02:46:39 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 20 May 2015 09:46:39 +0300
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <28bca3fb-653d-40fb-9944-d28b01e77244@googlegroups.com>
Message-ID: <871tibeofk.fsf@elektro.pacujo.net>

Rustom Mody <rustompmody at gmail.com>:

> In short any language that is implemented on von Neumann hw will need
> to address memory.

I don't think von Neumann hardware plays a role here. I think the data
model is inherent in Python/Java/Lisp regardless of the underlying
formalism (which could be SKI combinatory calculus or any other
Turing-complete formalism).

> And although they are equal as in '==' they are not equal as in
> behavior, memory usage etc, a fact that can only be elucidated by
> box-n-arrow diagrams.

That's what I meant when I asked if you can get Python without getting
something like C first. It seems the answer is, you probably can't.


Marko


From howi682 at googlemail.com  Wed May 20 02:49:13 2015
From: howi682 at googlemail.com (Howard Spink)
Date: Tue, 19 May 2015 23:49:13 -0700 (PDT)
Subject: Help for a newbie regarding code & physical switches
Message-ID: <3c0e9762-625e-4740-9717-3ce6dc51df70@googlegroups.com>

I have a Pi + SD card with Pi OS + PiTfT screen

I am trying to display specific JPEGs when certain combination of GP10 inputs are HIGH. I would like to use Python, I have no background in programming. Can someone point me on the right tracks? 


From rustompmody at gmail.com  Wed May 20 03:03:29 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Wed, 20 May 2015 00:03:29 -0700 (PDT)
Subject: Slices time complexity
In-Reply-To: <871tibeofk.fsf@elektro.pacujo.net>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <28bca3fb-653d-40fb-9944-d28b01e77244@googlegroups.com>
 <871tibeofk.fsf@elektro.pacujo.net>
Message-ID: <87133fbd-e616-4a25-9ae8-3044dc391d0f@googlegroups.com>

On Wednesday, May 20, 2015 at 12:16:49 PM UTC+5:30, Marko Rauhamaa wrote:
> Rustom Mody :
> 
> > In short any language that is implemented on von Neumann hw will need
> > to address memory.
> 
> I don't think von Neumann hardware plays a role here. I think the data
> model is inherent in Python/Java/Lisp regardless of the underlying
> formalism (which could be SKI combinatory calculus or any other
> Turing-complete formalism).

That's backwards

Python/C/Java/Lisp/Fortran (and 700 more) are made the way they are out of
aiming for some fidelity with von Neumann architecture
They all of course make different choice from choosing different weights in
the various tradeoffs.
However they all commonly share that they are implemented reasonably satisfactorily.
'Reasonable' and 'satisfactory' have unspecified semantics!!

> 
> > And although they are equal as in '==' they are not equal as in
> > behavior, memory usage etc, a fact that can only be elucidated by
> > box-n-arrow diagrams.
> 
> That's what I meant when I asked if you can get Python without getting
> something like C first. It seems the answer is, you probably can't.

I think we agree except for the 'probably'. Consider:

Q1: Is a computer (of any sort and generation of your choice) finite or infinite?
Q2: Is a Turing machine finite or infinite?
Its ironical that a Turing machine is 'more' infinite and therefore more a math
abstraction than lambda-calculus!

IOW I regard memory questions as more fundamental than you seem to


From auriocus at gmx.de  Wed May 20 03:14:39 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Wed, 20 May 2015 09:14:39 +0200
Subject: Help for a newbie regarding code & physical switches
In-Reply-To: <3c0e9762-625e-4740-9717-3ce6dc51df70@googlegroups.com>
References: <3c0e9762-625e-4740-9717-3ce6dc51df70@googlegroups.com>
Message-ID: <mjhc6q$hvt$1@dont-email.me>

Am 20.05.15 um 08:49 schrieb Howard Spink:
> I have a Pi + SD card with Pi OS + PiTfT screen
>
> I am trying to display specific JPEGs when certain combination of GP10 inputs are HIGH. I would like to use Python, I have no background in programming. Can someone point me on the right tracks?
>

No bg in programming is not very helpful. I suggest you do some 
tutorials. For instance, the PiTFT has this:

https://learn.adafruit.com/downloads/pdf/adafruit-pitft-28-inch-resistive-touchscreen-display-raspberry-pi.pdf

It shows how you can display images from the shell. Next, do a tutorial 
on Python programming, e.g. https://wiki.python.org/moin/BeginnersGuide 
- but you shold choose one that fits your way of learning. Maybe there 
is a RasPI tutorial also, which shows you how to read from the GP ports, 
I'm too lazy to google it now - just try combinations of API, tutorial, 
documentation with keywords like RasPI, GP, GP programming etc.

Finally you can wire this all up. An experienced programmer with no 
previous knowledge of RasPI can do this within half an hour by reading 
these guides. Your biggest task is to understand programming. Have fun!

	Christian


From jkn_gg at nicorp.f9.co.uk  Wed May 20 03:17:48 2015
From: jkn_gg at nicorp.f9.co.uk (jkn)
Date: Wed, 20 May 2015 00:17:48 -0700 (PDT)
Subject: Slightly OT: Seeking (python-based) project diary tool, or
 framework to write one
In-Reply-To: <mailman.160.1432100003.17265.python-list@python.org>
References: <99067d97-cad4-42f8-8fd1-b1884bed769e@googlegroups.com>
 <mailman.160.1432100003.17265.python-list@python.org>
Message-ID: <a6b538c0-ed44-4e11-914c-83086b377820@googlegroups.com>

Hi Frank

On Wednesday, 20 May 2015 06:33:33 UTC+1, Frank Millman  wrote:
> "jkn" <jkn_gg at nicorp.f9.co.uk> wrote in message 
> news:99067d97-cad4-42f8-8fd1-b1884bed769e at googlegroups.com...
> > Hi All
> >    as in the title, this is a little bit OT - but since ideally I'd like a
> > tool written in Python, and I know readers here have wide experience of
> > development/collaborative workflows etc ...
> >
> 
> Have you looked at this one? -
> 
> http://taskcoach.org/
> 
> >From their home page -
> 
> "Task Coach is a simple open source todo manager to keep track of personal 
> tasks and todo lists. It is designed for composite tasks, and also offers 
> effort tracking, categories, notes and more."
> 
> 
> 
> There seems to be at least some overlap with your requirements, so it may be 
> worth a look.
> 
> 
> It uses wxPython. According to their download page it only works with Python 
> 2.x, but as wxPython for Python3 is getting closer, hopefully it is on their 
> roadmap to target Python3 as well.
> 
> Frank Millman

Yeah, I know about TaskCoach as well ;-/ (sorry, should have posted some of
these ... ). It is much more oriented towards _execution_ of projects than
tracking information about projects.

Having said that, it's a while since I looked at TaskCoach, and perhaps it has
changed or can be configured closer to what I have in mind. Thanks for the
reminder.

    Cheers
    Jon N


From steve+comp.lang.python at pearwood.info  Wed May 20 03:23:54 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 20 May 2015 17:23:54 +1000
Subject: List semantics [was Re: Slices time complexity]
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <28bca3fb-653d-40fb-9944-d28b01e77244@googlegroups.com>
Message-ID: <555c368c$0$12906$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 20 May 2015 16:23, Rustom Mody wrote:

> I dont like teaching this. viz that in python
> x = [[1,2],[1,2]]
> is equal to y defined as
> z = [1,2]
> y = [z,z]
> And although they are equal as in '==' they are not equal as in behavior,
> memory usage etc, a fact that can only be elucidated by box-n-arrow
> diagrams.

"Only"? 

You don't need diagrams to explain the differences between the two.


Code snippet 1:

    x = [[1,2],[1,2]]

creates a list bound to the name "x", containing a list containing ints 1 
and 2, and a second independent list also containing ints 1 and 2. Here 
there are three lists in total, and one named variable.

Code snippet 2:

    z = [1,2]
    y = [z, z]

creates a list bound to the name "z", ints 1 and 2; then it creates a second 
list, bound to name "y", containing the first list "z" twice. Here there are 
two distinct lists in total, and two named variables. The two items inside y 
are the same list, namely z, not distinct lists.


A diagram may help, especially for more complicated situations, or for the 
slow of thinking, but it's not essential. There's no need to talk about the 
implementation, pointers or memory addresses so long as you stick to 
Python's object-based language model.


-- 
Steve



From ben+python at benfinney.id.au  Wed May 20 03:29:18 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 20 May 2015 17:29:18 +1000
Subject: Rule of order for dot operators?
References: <mailman.80.1431804386.17265.python-list@python.org>
 <3341326.d8VUBGAoep@PointedEars.de>
 <5557f29f$0$12992$c3e8da3$5496439d@news.astraweb.com>
 <mjacv8$ods$6@dont-email.me> <2710946.pHPuBPdTgQ@PointedEars.de>
 <f69e21a0-026a-4a9d-bac6-78ba891bfb13@googlegroups.com>
 <16277536.fgHiFt9JYW@PointedEars.de>
Message-ID: <85h9r7zoz5.fsf@benfinney.id.au>

Thomas 'PointedEars' Lahn <PointedEars at web.de> writes:

> Ned Batchelder wrote:
>
> > Be considerate. Be respectful.
>
> And who appointed you moderator?

Ned is telling you how we are all expected to behave here, and that you
have violated the norms of behaviour to other participants here.

He does so with authority as someone who understands the norms here.
Moderator privileges are not required for that.

> If that?s the way this newsgroup works, I am off here. (But I
> seriously doubt it.) Because that is _not_ how Usenet is supposed to
> work.

None the less, that is how this forum works. We are part of the Python
community here, and we expect all participants to treat each other with
respect and consideration.

-- 
 \     ?It is not enough to have a good mind. The main thing is to use |
  `\                                         it well.? ?Rene Descartes |
_o__)                                                                  |
Ben Finney



From marko at pacujo.net  Wed May 20 03:45:40 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 20 May 2015 10:45:40 +0300
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <28bca3fb-653d-40fb-9944-d28b01e77244@googlegroups.com>
 <871tibeofk.fsf@elektro.pacujo.net>
 <87133fbd-e616-4a25-9ae8-3044dc391d0f@googlegroups.com>
Message-ID: <87wq03d74r.fsf@elektro.pacujo.net>

Rustom Mody <rustompmody at gmail.com>:

> On Wednesday, May 20, 2015 at 12:16:49 PM UTC+5:30, Marko Rauhamaa wrote:
>> Rustom Mody :
>> 
>> > In short any language that is implemented on von Neumann hw will
>> > need to address memory.
>> 
>> I don't think von Neumann hardware plays a role here. I think the
>> data model is inherent in Python/Java/Lisp regardless of the
>> underlying formalism (which could be SKI combinatory calculus or any
>> other Turing-complete formalism).
>
> That's backwards
>
> Python/C/Java/Lisp/Fortran (and 700 more) are made the way they are
> out of aiming for some fidelity with von Neumann architecture

Funny enough, von Neumann is summoned even by the Python Language Spec:

   Objects are Python?s abstraction for data. All data in a Python
   program is represented by objects or by relations between objects.
   (In a sense, and in conformance to Von Neumann?s model of a ?stored
   program computer,? code is also represented by objects.)

   <URL: https://docs.python.org/3/reference/datamodel.html>

As can be expected, the specification fails to define an object and
resorts to handwaving. However, there's a nod to pointer semantics:

   Every object has an identity, a type and a value. An object?s
   identity never changes once it has been created; you may think of it
   as the object?s address in memory. The ?is? operator compares the
   identity of two objects; the id() function returns an integer
   representing its identity.


Marko


From howi682 at googlemail.com  Wed May 20 03:54:40 2015
From: howi682 at googlemail.com (Howard Spink)
Date: Wed, 20 May 2015 00:54:40 -0700 (PDT)
Subject: Help for a newbie regarding code & physical switches
In-Reply-To: <mjhc6q$hvt$1@dont-email.me>
References: <3c0e9762-625e-4740-9717-3ce6dc51df70@googlegroups.com>
 <mjhc6q$hvt$1@dont-email.me>
Message-ID: <c618bb56-11b9-4e3e-a290-4092fa82969c@googlegroups.com>

Thanks for your help. I want the python to run automatically after boot and show a blank white screen, when a combination of GP10 inputs are HIGH python displays one of 150 JPEGS. Is this possible? what sort of boot times can I get with Arch?


On Wednesday, 20 May 2015 08:14:50 UTC+1, Christian Gollwitzer  wrote:
> Am 20.05.15 um 08:49 schrieb Howard Spink:
> > I have a Pi + SD card with Pi OS + PiTfT screen
> >
> > I am trying to display specific JPEGs when certain combination of GP10 inputs are HIGH. I would like to use Python, I have no background in programming. Can someone point me on the right tracks?
> >
> 
> No bg in programming is not very helpful. I suggest you do some 
> tutorials. For instance, the PiTFT has this:
> 
> https://learn.adafruit.com/downloads/pdf/adafruit-pitft-28-inch-resistive-touchscreen-display-raspberry-pi.pdf
> 
> It shows how you can display images from the shell. Next, do a tutorial 
> on Python programming, e.g. https://wiki.python.org/moin/BeginnersGuide 
> - but you shold choose one that fits your way of learning. Maybe there 
> is a RasPI tutorial also, which shows you how to read from the GP ports, 
> I'm too lazy to google it now - just try combinations of API, tutorial, 
> documentation with keywords like RasPI, GP, GP programming etc.
> 
> Finally you can wire this all up. An experienced programmer with no 
> previous knowledge of RasPI can do this within half an hour by reading 
> these guides. Your biggest task is to understand programming. Have fun!
> 
> 	Christian



From alister.nospam.ware at ntlworld.com  Wed May 20 04:46:26 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Wed, 20 May 2015 08:46:26 +0000 (UTC)
Subject: Help for a newbie regarding code & physical switches
References: <3c0e9762-625e-4740-9717-3ce6dc51df70@googlegroups.com>
 <mjhc6q$hvt$1@dont-email.me>
 <c618bb56-11b9-4e3e-a290-4092fa82969c@googlegroups.com>
Message-ID: <mjhhl1$udc$1@speranza.aioe.org>

On Wed, 20 May 2015 00:54:40 -0700, Howard Spink wrote:

> Thanks for your help. I want the python to run automatically after boot
> and show a blank white screen, when a combination of GP10 inputs are
> HIGH python displays one of 150 JPEGS. Is this possible? what sort of
> boot times can I get with Arch?
> 
Please don't top post as it makes the conversation difficult to follow,
interleaved posting is the prefered style for this news group

Your question is probably best asked in the raspberry pi news group
comp.sys.raspberry-pi

> 
> On Wednesday, 20 May 2015 08:14:50 UTC+1, Christian Gollwitzer  wrote:
>> Am 20.05.15 um 08:49 schrieb Howard Spink:
>> > I have a Pi + SD card with Pi OS + PiTfT screen
>> >
>> > I am trying to display specific JPEGs when certain combination of
>> > GP10 inputs are HIGH. I would like to use Python, I have no
>> > background in programming. Can someone point me on the right tracks?
>> >
>> >
>> No bg in programming is not very helpful. I suggest you do some
>> tutorials. For instance, the PiTFT has this:
>> 
>> https://learn.adafruit.com/downloads/pdf/adafruit-pitft-28-inch-
resistive-touchscreen-display-raspberry-pi.pdf
>> 
>> It shows how you can display images from the shell. Next, do a tutorial
>> on Python programming, e.g. https://wiki.python.org/moin/BeginnersGuide
>> - but you shold choose one that fits your way of learning. Maybe there
>> is a RasPI tutorial also, which shows you how to read from the GP
>> ports,
>> I'm too lazy to google it now - just try combinations of API, tutorial,
>> documentation with keywords like RasPI, GP, GP programming etc.
>> 
>> Finally you can wire this all up. An experienced programmer with no
>> previous knowledge of RasPI can do this within half an hour by reading
>> these guides. Your biggest task is to understand programming. Have fun!
>> 
>> 	Christian





-- 
Now it's time to say goodbye
To all our company...
M-I-C	(see you next week!)
K-E-Y	(Why?  Because we LIKE you!)
M-O-U-S-E.


From greg.ewing at canterbury.ac.nz  Wed May 20 04:54:00 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Wed, 20 May 2015 20:54:00 +1200
Subject: List semantics [was Re: Slices time complexity]
In-Reply-To: <555c368c$0$12906$c3e8da3$5496439d@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <28bca3fb-653d-40fb-9944-d28b01e77244@googlegroups.com>
 <555c368c$0$12906$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <cs30daFu54gU1@mid.individual.net>

Steven D'Aprano wrote:

> Code snippet 1:
> 
>     x = [[1,2],[1,2]]
> 
> creates a list bound to the name "x", containing a list containing ints 1 
> and 2, and a second independent list also containing ints 1 and 2.

Using the word "contains" here is misleading, because
it conjures a mental picture of physical containment,
which is exactly what you *don't* want to suggest.

>     z = [1,2]
>     y = [z, z]
> 
> creates a list bound to the name "z", ints 1 and 2; then it creates a second 
> list, bound to name "y", containing the first list "z" twice.

At this point the student thinks, "Um... what? How
can an object contain another object *twice*?"
If he's still thinking in physical terms, this
sentence is nonsensical.

It gets even worse with:

x = [1, 2]
x[1] = x

Now you have to say that the list contains *itself*
in some weird Tardis-like fashion!

I can't think of any way to dispel the confusion
verbally without using some word like "reference"
or "pointer" or something with an equivalent meaning.
Something that suggests a level of indirectness.

> A diagram may help, especially for more complicated situations,

Seems to me that a diagram is far and away the
easiest and most effective way to convey what's
really going on. It's difficult to do that with
words alone, because English isn't really equipped
to talk about it precisely enough.

-- 
Greg


From oscar.j.benjamin at gmail.com  Wed May 20 05:16:35 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 20 May 2015 10:16:35 +0100
Subject: Slices time complexity
In-Reply-To: <mjhi5a$4pa$1@dont-email.me>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <mjhi5a$4pa$1@dont-email.me>
Message-ID: <CAHVvXxSGaNVU+1RwvFaS9QN-MM9f_cqS4st_-20Sco2PyFZYjw@mail.gmail.com>

On 20 May 2015 at 10:56, Bartc <bartc at freeuk.com> wrote:
>
> However, I was so impressed with how (C)Python does things (working
> exclusively with pointers, doing assignments by simply copying pointers, and
> using reference counting to manage memory), that I'm experimenting with
> letting my language work the same way. (Also then I can benchmark the two
> more fairly.)
>
> Then I get a little disillusioned when I find out in this thread that a
> simple slice of an array actually copies every element!

It's not an array, it's a list. Lists are often used in Python where
arrays would be used in other languages but they are not the same as
arrays in many other languages since they are not of homogeneous type.
Python has an array module but AIUI it's not used very much. Certainly
whenever I would have a use for the array module I would use numpy's
ndarray instead.

Slicing a list does not copy every element. It copies the references
to each element. The slice creates a new list (of references) rather
than referencing the original but otherwise it is a shallow copy.

> Since I currently
> create a view (a good term I'd never come across before) for a slice, does
> that mean I now also have to do what Python does?

No, creating a view is just fine if that's what you want in your
language. In CPython I use numpy's ndarray for which slices create
views. Sometimes one behaviour is useful and sometimes the other. In
practice I rarely find that list slicing is a performance bottleneck
for anything.


--
Oscar


From bartc at freeuk.com  Wed May 20 05:56:03 2015
From: bartc at freeuk.com (Bartc)
Date: Wed, 20 May 2015 10:56:03 +0100
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
Message-ID: <mjhi5a$4pa$1@dont-email.me>

"Steven D'Aprano" <steve+comp.lang.python at pearwood.info> wrote in message
news:555c225b$0$2769$c3e8da3$76491128 at news.astraweb.com...

> The mental gymnastics they go through to force the round peg of pass-by-
> sharing object semantics into the false dichotomy "pass-by-value versus
> pass-by-reference" is just crazy.
>
> One consequence of this is that the meaning of "call by value" depends on
> whether the value you are talking about is a boxed or unboxed value.
> Unboxed
> values are copied. Boxed values are not. Except that they are, if you
> understand that "the value" doesn't refer to the actual value, but to some
> hidden and implementation-dependent reference to the value.

I have a language where everything is boxed (the data that isn't boxed, has
to be before it can be manipulated).

But simple data such as small integers and floats are passed by value.
Bigger data is 'sort of' passed by reference, but not full reference (the
details are a bit messy, but if an object consists of two parts (A,B), then
a copy of descriptor A is passed, which contains a pointer to B. For small
scalars, the B part doesn't exist).

(A proper reference is available when passing arguments to functions:
a boxed pointer to (A,B) is constructed.)

However, I was so impressed with how (C)Python does things (working
exclusively with pointers, doing assignments by simply copying pointers, and
using reference counting to manage memory), that I'm experimenting with
letting my language work the same way. (Also then I can benchmark the two
more fairly.)

Then I get a little disillusioned when I find out in this thread that a
simple slice of an array actually copies every element! Since I currently
create a view (a good term I'd never come across before) for a slice, does
that mean I now also have to do what Python does?

(Currently, /assignments/ of arrays - and slices - involve a deep copy. But
for use as operands or passing as function arguments, only the lightweight
descriptor A is passed, the 'view' in the case of a slice.)

-- 
Bartc







From ned at nedbatchelder.com  Wed May 20 07:08:18 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 20 May 2015 04:08:18 -0700 (PDT)
Subject: List semantics [was Re: Slices time complexity]
In-Reply-To: <cs30daFu54gU1@mid.individual.net>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <28bca3fb-653d-40fb-9944-d28b01e77244@googlegroups.com>
 <555c368c$0$12906$c3e8da3$5496439d@news.astraweb.com>
 <cs30daFu54gU1@mid.individual.net>
Message-ID: <823f1dc3-182a-4b8c-bf54-08a3986cb3e9@googlegroups.com>

On Wednesday, May 20, 2015 at 4:54:13 AM UTC-4, Gregory Ewing wrote:
> Steven D'Aprano wrote:
> > A diagram may help, especially for more complicated situations,
> 
> Seems to me that a diagram is far and away the
> easiest and most effective way to convey what's
> really going on. It's difficult to do that with
> words alone, because English isn't really equipped
> to talk about it precisely enough.

FWIW, this was my attempt at explaining names and values:
http://nedbatchelder.com/text/names.html, which I turned
into a PyCon talk this year: http://nedbatchelder.com/text/names1.html

It doesn't get quite as far as slice semantics, but deals with
"refers to" and "change", both of which are tricky to nail down
well.  Not sure I did... :)

--Ned.

> 
> -- 
> Greg



From scoria.799 at gmail.com  Wed May 20 07:44:15 2015
From: scoria.799 at gmail.com (Parul Mogra)
Date: Wed, 20 May 2015 17:14:15 +0530
Subject: Best approach to create humongous amount of files
Message-ID: <CAPkZ3MS5SiGH9OCe9RSTmakF681O+qM572y49FuDBmBix=aiFg@mail.gmail.com>

Hello everyone,
My objective is to create large amount of data files (say a million *.json
files), using a pre-existing template file (*.json). Each file would have a
unique name, possibly by incorporating time stamp information. The files
have to be generated in a folder specified.

What is the best strategy to achieve this task, so that the files will be
generated in the shortest possible time? Say within an hour.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150520/467b2a3e/attachment.html>

From darnold992000 at yahoo.com  Wed May 20 08:50:59 2015
From: darnold992000 at yahoo.com (darnold)
Date: Wed, 20 May 2015 05:50:59 -0700 (PDT)
Subject: Looking for direction
In-Reply-To: <mailman.465.1431559626.12865.python-list@python.org>
References: <mailman.465.1431559626.12865.python-list@python.org>
Message-ID: <9abf87a2-a98b-470b-9f94-a76d4ef1b34e@googlegroups.com>

I recommend getting your hands on "Automate The Boring Stuff With Python" from no starch press:

http://www.nostarch.com/automatestuff

I've not read it in its entirety, but it's very beginner-friendly and is targeted at just the sort of processing you appear to be doing.

HTH,
Don


From rosuav at gmail.com  Wed May 20 08:58:05 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 20 May 2015 22:58:05 +1000
Subject: Best approach to create humongous amount of files
In-Reply-To: <CAPkZ3MS5SiGH9OCe9RSTmakF681O+qM572y49FuDBmBix=aiFg@mail.gmail.com>
References: <CAPkZ3MS5SiGH9OCe9RSTmakF681O+qM572y49FuDBmBix=aiFg@mail.gmail.com>
Message-ID: <CAPTjJmppiMpVjTBt5CH_6DGSdCWw5aoDU+jY-3wMs5Ai7tPdKw@mail.gmail.com>

On Wed, May 20, 2015 at 9:44 PM, Parul Mogra <scoria.799 at gmail.com> wrote:
> My objective is to create large amount of data files (say a million *.json
> files), using a pre-existing template file (*.json). Each file would have a
> unique name, possibly by incorporating time stamp information. The files
> have to be generated in a folder specified.
>
> What is the best strategy to achieve this task, so that the files will be
> generated in the shortest possible time? Say within an hour.

If they're to be created within an hour, timestamp of creation
probably isn't useful, but if you have internal data to use as the
file name, that would work. Otherwise, try a simple sequential
integer.

All you'd need would be a loop that creates a bunch of files... most
of your code will be figuring out what parts of the template need to
change. Not too difficult.

ChrisA


From AKHILR at IITK.AC.IN  Wed May 20 09:02:31 2015
From: AKHILR at IITK.AC.IN (AKHIL RANA)
Date: Wed, 20 May 2015 18:32:31 +0530
Subject: Help regarding python run time
Message-ID: <CALbtgctWb5zHWG=Nf-Qoi94gUscp3L=Xx39a_onqzzx1hNci6A@mail.gmail.com>

Hi,

I am student at IIT Kanpur and working on a Opencv based Python project. I
am working on program development which takes less time to execute. For
that i have tested my small program hello word on python to now the time
taken by this program. I had run many time. and every time it run it gives
me a different run time.

Can we some how make the program to run at constant time so that we can
work on how to reduce the timing ?

Thanks

-- 
Regards
Akhil Rana
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150520/74a323d1/attachment.html>

From robin at reportlab.com  Wed May 20 09:16:44 2015
From: robin at reportlab.com (Robin Becker)
Date: Wed, 20 May 2015 14:16:44 +0100
Subject: subprocess.Popen zombie
Message-ID: <555C893C.1090807@chamonix.reportlab.co.uk>

As part of a long running PyQT process running as a window app in Arch linux I 
needed an alert sound, I decided to use the beep command and the app code then 
looked like

pid = Popen(['/home/robin/bin/mybeep', '-r3', '-f750', '-l100', '-d75']).pid

the mybeep script handles module loading if required etc etc.

Anyhow, this works with one slight oddity. When this code is executed it works 
fine, but leaves behind a single zombie process, when next executed the zombie 
disappears and a new zombie replaces it.

Is this because I'm not waiting? Does the process module reap previous commands 
in some way? The code I used to use with os.spawnl was even worse in leaving 
zombies around. I suppose I needed to keep a record of all the pid's and wait on 
them at some convenient time. The subprocess version appears to be doing that 
for me somehow.


*NB* I did try PyQT's qApp.beep(), but it seemed to work only on windows.
-- 
Robin Becker



From subhabrata.banerji at gmail.com  Wed May 20 10:42:17 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Wed, 20 May 2015 07:42:17 -0700 (PDT)
Subject: REST framework Editor  in Python
Message-ID: <13eb8a35-8247-4994-ab32-c9150326af02@googlegroups.com>

Dear Group,

I am trying to put one search engine interface in REST. 
To do this I am trying to learn Restframework and Django. 
One question occurred to me is, whether there is any Restframework editor
which may do this job.

Is there any? 

I was trying to find out https://pypi.python.org/pypi/restview -is it doing this?
In that case, if any one may kindly share some examples or tutorial for it.

Thanks in advance,
Regards,
Subhabrata Banerjee.  


From Cecil at decebal.nl  Wed May 20 10:48:22 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 20 May 2015 16:48:22 +0200
Subject: subprocess.Popen zombie
References: <mailman.170.1432127818.17265.python-list@python.org>
Message-ID: <87vbfnqp8p.fsf@Equus.decebal.nl>

Op Wednesday 20 May 2015 15:16 CEST schreef Robin Becker:

> As part of a long running PyQT process running as a window app in
> Arch linux I needed an alert sound, I decided to use the beep
> command and the app code then looked like
>
> pid = Popen(['/home/robin/bin/mybeep', '-r3', '-f750', '-l100',
> '-d75']).pid
>
> the mybeep script handles module loading if required etc etc.
>
> Anyhow, this works with one slight oddity. When this code is
> executed it works fine, but leaves behind a single zombie process,
> when next executed the zombie disappears and a new zombie replaces
> it.
>
> Is this because I'm not waiting? Does the process module reap

Yes, you should do a:
     pid.wait()

Should not be a problem, because the program you are calling takes
very little time.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From python.list at tim.thechases.com  Wed May 20 11:07:23 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Wed, 20 May 2015 10:07:23 -0500
Subject: Best approach to create humongous amount of files
In-Reply-To: <CAPTjJmppiMpVjTBt5CH_6DGSdCWw5aoDU+jY-3wMs5Ai7tPdKw@mail.gmail.com>
References: <CAPkZ3MS5SiGH9OCe9RSTmakF681O+qM572y49FuDBmBix=aiFg@mail.gmail.com>
 <CAPTjJmppiMpVjTBt5CH_6DGSdCWw5aoDU+jY-3wMs5Ai7tPdKw@mail.gmail.com>
Message-ID: <20150520100723.3a34a775@bigbox.christie.dr>

On 2015-05-20 22:58, Chris Angelico wrote:
> On Wed, May 20, 2015 at 9:44 PM, Parul Mogra <scoria.799 at gmail.com>
> wrote:
> > My objective is to create large amount of data files (say a
> > million *.json files), using a pre-existing template file
> > (*.json). Each file would have a unique name, possibly by
> > incorporating time stamp information. The files have to be
> > generated in a folder specified.
[snip] 
> try a simple sequential integer.
> 
> All you'd need would be a loop that creates a bunch of files... most
> of your code will be figuring out what parts of the template need to
> change. Not too difficult.

If you store your template as a Python string-formatting template,
you can just use string-formatting to do your dirty work:


  import random
  HOW_MANY = 1000000
  template = """{
    "some_string": "%(string)s",
    "some_int": %(int)i
    }
    """

  wordlist = [
    word.rstrip()
    for word in open('/usr/share/dict/words')
    ]
  wordlist[:] = [ # just lowercase all-alpha words
    word
    for word in wordlist
    if word.isalpha() and word.islower()
    ]

  for i in xrange(HOW_MANY):
    fname = "data_%08i.json" % i
    with open(fname, "w") as f:
      f.write(template % {
        "string_value": random.choice(wordlist),
        "int_value": random.randint(0, 1000),
        })


-tkc







From ian.g.kelly at gmail.com  Wed May 20 11:23:25 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 20 May 2015 09:23:25 -0600
Subject: Help regarding python run time
In-Reply-To: <CALbtgctWb5zHWG=Nf-Qoi94gUscp3L=Xx39a_onqzzx1hNci6A@mail.gmail.com>
References: <CALbtgctWb5zHWG=Nf-Qoi94gUscp3L=Xx39a_onqzzx1hNci6A@mail.gmail.com>
Message-ID: <CALwzidmCfV1Mqy4kG0HBYQ8BKwWHVbXYyYTdEqGe9VesOXucCg@mail.gmail.com>

On Wed, May 20, 2015 at 7:02 AM, AKHIL RANA <AKHILR at iitk.ac.in> wrote:
> Hi,
>
> I am student at IIT Kanpur and working on a Opencv based Python project. I
> am working on program development which takes less time to execute. For that
> i have tested my small program hello word on python to now the time taken by
> this program. I had run many time. and every time it run it gives me a
> different run time.
>
> Can we some how make the program to run at constant time so that we can work
> on how to reduce the timing ?

Not practically. The exact run time is dependent on a lot of factors
that are mostly out of your control: what other programs are running,
what they're doing and what resources they're using at that moment;
what hardware interrupts occur while your program is running; is data
to be read from disk cached or not; is data to be loaded from RAM
cached or not; if the disk is mechanical, what cylinder and sector
does the read head happen to be at when it gets a read request.

Instead of trying to measure an exact time over one run, you will get
better results by running the program several times and then taking
the minimum measurement as representative of the program's runtime
under ideal conditions.


From paul.anton.letnes at gmail.com  Wed May 20 11:32:53 2015
From: paul.anton.letnes at gmail.com (paul.anton.letnes at gmail.com)
Date: Wed, 20 May 2015 08:32:53 -0700 (PDT)
Subject: Best approach to create humongous amount of files
In-Reply-To: <mailman.168.1432126695.17265.python-list@python.org>
References: <CAPkZ3MS5SiGH9OCe9RSTmakF681O+qM572y49FuDBmBix=aiFg@mail.gmail.com>
 <mailman.168.1432126695.17265.python-list@python.org>
Message-ID: <51b5a49a-6424-4cc5-aec2-07721058b4ba@googlegroups.com>

There's a module called "template" that I've used before, for the find/replace part. I never investigated its performance, but my script used less than 1 s for 100 files IIRC :-)

Paul


From alain at universite-de-strasbourg.fr.invalid  Wed May 20 11:42:07 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Wed, 20 May 2015 17:42:07 +0200
Subject: subprocess.Popen zombie
References: <mailman.170.1432127818.17265.python-list@python.org>
Message-ID: <87bnhfqmr4.fsf@universite-de-strasbourg.fr.invalid>

Robin Becker <robin at reportlab.com> writes:

> As part of a long running PyQT process running as a window app in Arch
> linux I needed an alert sound, I decided to use the beep command and
> the app code then looked like
>
> pid = Popen(['/home/robin/bin/mybeep', '-r3', '-f750', '-l100', '-d75']).pid
>
> the mybeep script handles module loading if required etc etc.
>
> Anyhow, this works with one slight oddity. When this code is executed
> it works fine, but leaves behind a single zombie process, when next
> executed the zombie disappears and a new zombie replaces it.
>
> Is this because I'm not waiting?

Yes, all processes will stay zombies until being wait()-ed for by their
parent process. This means: either you use call(...) which doesn't
return until the child process has finished, or you keep the Popen
object around and call wait() at an appropriate time.

> Does the process module reap previous commands in some way?

No.

> The code I used to use with os.spawnl was even worse in leaving
> zombies around.

For the same reason (os.wait() and os.waitpid() let you ... wait for
child-processes).

> I suppose I needed to keep a record of all the pid's and wait on them
> at some convenient time.

Yes.

> The subprocess version appears to be doing that for me somehow.

Not sure what you mean. You have to do the bookkeeping yourself.

(But, does beep really take so much time that you can't just call() it?)

-- Alain.


From irmen.NOSPAM at xs4all.nl  Wed May 20 11:48:41 2015
From: irmen.NOSPAM at xs4all.nl (Irmen de Jong)
Date: Wed, 20 May 2015 17:48:41 +0200
Subject: Help regarding python run time
In-Reply-To: <mailman.172.1432135454.17265.python-list@python.org>
References: <CALbtgctWb5zHWG=Nf-Qoi94gUscp3L=Xx39a_onqzzx1hNci6A@mail.gmail.com>
 <mailman.172.1432135454.17265.python-list@python.org>
Message-ID: <555cacd9$0$2934$e4fe514c@news2.news.xs4all.nl>

On 20-5-2015 17:23, Ian Kelly wrote:
> On Wed, May 20, 2015 at 7:02 AM, AKHIL RANA <AKHILR at iitk.ac.in> wrote:
>> Hi,
>>
>> I am student at IIT Kanpur and working on a Opencv based Python project. I
>> am working on program development which takes less time to execute. For that
>> i have tested my small program hello word on python to now the time taken by
>> this program. I had run many time. and every time it run it gives me a
>> different run time.
>>
>> Can we some how make the program to run at constant time so that we can work
>> on how to reduce the timing ?
> 
> Not practically. The exact run time is dependent on a lot of factors
> that are mostly out of your control: what other programs are running,
> what they're doing and what resources they're using at that moment;
> what hardware interrupts occur while your program is running; is data
> to be read from disk cached or not; is data to be loaded from RAM
> cached or not; if the disk is mechanical, what cylinder and sector
> does the read head happen to be at when it gets a read request.
> 
> Instead of trying to measure an exact time over one run, you will get
> better results by running the program several times and then taking
> the minimum measurement as representative of the program's runtime
> under ideal conditions.
> 

Or measure the actual CPU clock cycles taken instead of the wall clock run time.
Then you should get a fairly constant number, if the program does the same work every
time you run it.

phobos:~ irmen$ time python test.py
real    0m3.368s
user    0m0.214s    <--- cpu time spent in user mode actually doing work
sys     0m0.053s


Irmen



From __peter__ at web.de  Wed May 20 11:59:33 2015
From: __peter__ at web.de (Peter Otten)
Date: Wed, 20 May 2015 17:59:33 +0200
Subject: Best approach to create humongous amount of files
References: <CAPkZ3MS5SiGH9OCe9RSTmakF681O+qM572y49FuDBmBix=aiFg@mail.gmail.com>
 <CAPTjJmppiMpVjTBt5CH_6DGSdCWw5aoDU+jY-3wMs5Ai7tPdKw@mail.gmail.com>
 <20150520100723.3a34a775@bigbox.christie.dr>
Message-ID: <mjib17$g32$1@ger.gmane.org>

Tim Chase wrote:

> On 2015-05-20 22:58, Chris Angelico wrote:
>> On Wed, May 20, 2015 at 9:44 PM, Parul Mogra <scoria.799 at gmail.com>
>> wrote:
>> > My objective is to create large amount of data files (say a
>> > million *.json files), using a pre-existing template file
>> > (*.json). Each file would have a unique name, possibly by
>> > incorporating time stamp information. The files have to be
>> > generated in a folder specified.
> [snip]
>> try a simple sequential integer.
>> 
>> All you'd need would be a loop that creates a bunch of files... most
>> of your code will be figuring out what parts of the template need to
>> change. Not too difficult.
> 
> If you store your template as a Python string-formatting template,
> you can just use string-formatting to do your dirty work:
> 
> 
>   import random
>   HOW_MANY = 1000000
>   template = """{
>     "some_string": "%(string)s",
>     "some_int": %(int)i
>     }
>     """
> 
>   wordlist = [
>     word.rstrip()
>     for word in open('/usr/share/dict/words')
>     ]
>   wordlist[:] = [ # just lowercase all-alpha words
>     word
>     for word in wordlist
>     if word.isalpha() and word.islower()
>     ]
> 
>   for i in xrange(HOW_MANY):
>     fname = "data_%08i.json" % i
>     with open(fname, "w") as f:
>       f.write(template % {
>         "string_value": random.choice(wordlist),
>         "int_value": random.randint(0, 1000),
>         })

Just a quick reminder: if the data is user-provided you have to sanitize it:

>>> template = """{"access": "restricted", "user": "%(user)s"}"""
>>> json.loads(template % dict(user="""tim", "access": "unlimited"""))
{'user': 'tim', 'access': 'unlimited'}

That can't happen when you load the template, replace some keys and dump the 
result:

>>> template = json.loads("""{"access": "restricted", "user": 
"placeholder"}""")
>>> template["user"] = """tim", "access": "unlimited"""
>>> json.dumps(template)
'{"user": "tim\\", \\"access\\": \\"unlimited", "access": "restricted"}'
>>> json.loads(_)
{'user': 'tim", "access": "unlimited', 'access': 'restricted'}
>>> _["access"]
'restricted'

I expect that performance will be dominated by I/O; if that's correct the 
extra work of serializing the JSON should not do much harm.



From Cecil at decebal.nl  Wed May 20 12:01:09 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 20 May 2015 18:01:09 +0200
Subject: No ttk in 2.7
Message-ID: <87r3qbqlve.fsf@Equus.decebal.nl>

I want to start playing with tkinter, but there are some differences
between 2 and 3. For this I use at the moment the following code:
    import sys

    if sys.version_info[0] < 3:
        import Tkinter as tk
        import ttk
    else:
        import tkinter as tk
        from   tkinter import ttk
or can it better be done in another way?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From python.list at tim.thechases.com  Wed May 20 12:12:11 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Wed, 20 May 2015 11:12:11 -0500
Subject: Best approach to create humongous amount of files
In-Reply-To: <mjib17$g32$1@ger.gmane.org>
References: <CAPkZ3MS5SiGH9OCe9RSTmakF681O+qM572y49FuDBmBix=aiFg@mail.gmail.com>
 <CAPTjJmppiMpVjTBt5CH_6DGSdCWw5aoDU+jY-3wMs5Ai7tPdKw@mail.gmail.com>
 <20150520100723.3a34a775@bigbox.christie.dr>
 <mjib17$g32$1@ger.gmane.org>
Message-ID: <20150520111211.7c96f234@bigbox.christie.dr>

On 2015-05-20 17:59, Peter Otten wrote:
> Tim Chase wrote:
> >   wordlist[:] = [ # just lowercase all-alpha words
> >     word
> >     for word in wordlist
> >     if word.isalpha() and word.islower()
> >     ]
> 
> Just a quick reminder: if the data is user-provided you have to
> sanitize it:

Thus my sanitizing to isalpha()+islower() words in my sample.

> I expect that performance will be dominated by I/O; if that's
> correct the extra work of serializing the JSON should not do much
> harm.

I seem to recall that there was a change-over, that an older JSON
library was particularly slow, but that a later replacement sped that
up immensely.  So performance may depend heavily on which version
you're running.

[to the OP] But yes, if you're trusting unsanitized data, Peter's
suggestion would be the way to go.

-tkc




From ian.g.kelly at gmail.com  Wed May 20 12:26:55 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 20 May 2015 10:26:55 -0600
Subject: Help regarding python run time
In-Reply-To: <555cacd9$0$2934$e4fe514c@news2.news.xs4all.nl>
References: <CALbtgctWb5zHWG=Nf-Qoi94gUscp3L=Xx39a_onqzzx1hNci6A@mail.gmail.com>
 <mailman.172.1432135454.17265.python-list@python.org>
 <555cacd9$0$2934$e4fe514c@news2.news.xs4all.nl>
Message-ID: <CALwzidnjFMKfzyTA02ZHoMG854TjqyWHX=h09xST8j6Tn2RusQ@mail.gmail.com>

On Wed, May 20, 2015 at 9:48 AM, Irmen de Jong <irmen.NOSPAM at xs4all.nl> wrote:
> Or measure the actual CPU clock cycles taken instead of the wall clock run time.
> Then you should get a fairly constant number, if the program does the same work every
> time you run it.
>
> phobos:~ irmen$ time python test.py
> real    0m3.368s
> user    0m0.214s    <--- cpu time spent in user mode actually doing work
> sys     0m0.053s

And yet:

$ time python3 primes.py

real 0m1.101s
user 0m1.099s
sys 0m0.000s
$ time python3 primes.py

real 0m1.135s
user 0m1.128s
sys 0m0.004s
$ time python3 primes.py

real 0m1.162s
user 0m1.147s
sys 0m0.013s

http://unix.stackexchange.com/questions/162115/why-does-the-user-and-sys-time-vary-on-multiple-executions


From ned at nedbatchelder.com  Wed May 20 12:43:19 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 20 May 2015 09:43:19 -0700 (PDT)
Subject: No ttk in 2.7
In-Reply-To: <87r3qbqlve.fsf@Equus.decebal.nl>
References: <87r3qbqlve.fsf@Equus.decebal.nl>
Message-ID: <5e9aac07-e210-4eb5-a3d7-cfa37cb3f6ab@googlegroups.com>

On Wednesday, May 20, 2015 at 12:35:40 PM UTC-4, Cecil Westerhof wrote:
> I want to start playing with tkinter, but there are some differences
> between 2 and 3. For this I use at the moment the following code:
>     import sys
> 
>     if sys.version_info[0] < 3:
>         import Tkinter as tk
>         import ttk
>     else:
>         import tkinter as tk
>         from   tkinter import ttk
> or can it better be done in another way?

Are you sure you want this program to run under both 2.x and 3.x?
That will add a layer of complexity to your project. If your
goal is to learn how to use Tkinter, you might be better off just
picking one version of Python, and sticking with it.

Deciding what versions of Python to support is not always a simple
decision.  It should be based on a realistic assessment of who will
be using your program, and what requirements they will have.

--Ned.


From robin at reportlab.com  Wed May 20 12:44:50 2015
From: robin at reportlab.com (Robin Becker)
Date: Wed, 20 May 2015 17:44:50 +0100
Subject: subprocess.Popen zombie
In-Reply-To: <87bnhfqmr4.fsf@universite-de-strasbourg.fr.invalid>
References: <mailman.170.1432127818.17265.python-list@python.org>
 <87bnhfqmr4.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <555CBA02.6080505@chamonix.reportlab.co.uk>

On 20/05/2015 16:42, Alain Ketterlin wrote:
> Robin Becker <robin at reportlab.com> writes:
.............
>> The code I used to use with os.spawnl was even worse in leaving
>> zombies around.
>
> For the same reason (os.wait() and os.waitpid() let you ... wait for
> child-processes).
>
>> I suppose I needed to keep a record of all the pid's and wait on them
>> at some convenient time.
>
> Yes.
>
>> The subprocess version appears to be doing that for me somehow.
>
> Not sure what you mean. You have to do the bookkeeping yourself.
It seems there is some notion of book keeping in subprocess.py
Popen instances have a __del__ method which records  un-norwegian live instances 
in a module global _active

if self.returncode is None and _active is not None:
     # Child is still running, keep us alive until we can wait on it.
    _active.append(self)


when another Popen is created there's a call to _cleanup which processes that 
list and reaps those that can be raptured.


>
> (But, does beep really take so much time that you can't just call() it?)
>
not really, it's just normal to keep event routines short; the routine which 
beeps is after detection of the cat's entrance into the house and various 
recognition schemes have pronounced intruder :)

Probably should all have gone into a separate thread, but I dislike threaded 
code and I would probably get into trouble with the PyQT event loop.

> -- Alain.
>


-- 
Robin Becker



From robin at reportlab.com  Wed May 20 12:44:50 2015
From: robin at reportlab.com (Robin Becker)
Date: Wed, 20 May 2015 17:44:50 +0100
Subject: subprocess.Popen zombie
In-Reply-To: <87bnhfqmr4.fsf@universite-de-strasbourg.fr.invalid>
References: <mailman.170.1432127818.17265.python-list@python.org>
 <87bnhfqmr4.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <555CBA02.6080505@chamonix.reportlab.co.uk>

On 20/05/2015 16:42, Alain Ketterlin wrote:
> Robin Becker <robin at reportlab.com> writes:
.............
>> The code I used to use with os.spawnl was even worse in leaving
>> zombies around.
>
> For the same reason (os.wait() and os.waitpid() let you ... wait for
> child-processes).
>
>> I suppose I needed to keep a record of all the pid's and wait on them
>> at some convenient time.
>
> Yes.
>
>> The subprocess version appears to be doing that for me somehow.
>
> Not sure what you mean. You have to do the bookkeeping yourself.
It seems there is some notion of book keeping in subprocess.py
Popen instances have a __del__ method which records  un-norwegian live instances 
in a module global _active

if self.returncode is None and _active is not None:
     # Child is still running, keep us alive until we can wait on it.
    _active.append(self)


when another Popen is created there's a call to _cleanup which processes that 
list and reaps those that can be raptured.


>
> (But, does beep really take so much time that you can't just call() it?)
>
not really, it's just normal to keep event routines short; the routine which 
beeps is after detection of the cat's entrance into the house and various 
recognition schemes have pronounced intruder :)

Probably should all have gone into a separate thread, but I dislike threaded 
code and I would probably get into trouble with the PyQT event loop.

> -- Alain.
>


-- 
Robin Becker


From ned at nedbatchelder.com  Wed May 20 12:47:01 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 20 May 2015 09:47:01 -0700 (PDT)
Subject: No ttk in 2.7
In-Reply-To: <5e9aac07-e210-4eb5-a3d7-cfa37cb3f6ab@googlegroups.com>
References: <87r3qbqlve.fsf@Equus.decebal.nl>
 <5e9aac07-e210-4eb5-a3d7-cfa37cb3f6ab@googlegroups.com>
Message-ID: <414bf0ad-557d-4373-8452-ab089888225e@googlegroups.com>

On Wednesday, May 20, 2015 at 12:43:29 PM UTC-4, Ned Batchelder wrote:
> On Wednesday, May 20, 2015 at 12:35:40 PM UTC-4, Cecil Westerhof wrote:
> > I want to start playing with tkinter, but there are some differences
> > between 2 and 3. For this I use at the moment the following code:
> >     import sys
> > 
> >     if sys.version_info[0] < 3:
> >         import Tkinter as tk
> >         import ttk
> >     else:
> >         import tkinter as tk
> >         from   tkinter import ttk
> > or can it better be done in another way?
> 
> Are you sure you want this program to run under both 2.x and 3.x?
> That will add a layer of complexity to your project. If your
> goal is to learn how to use Tkinter, you might be better off just
> picking one version of Python, and sticking with it.
> 
> Deciding what versions of Python to support is not always a simple
> decision.  It should be based on a realistic assessment of who will
> be using your program, and what requirements they will have.
> 
> --Ned.

Oh, and I meant to include this: if you *do* want to support both 2.x
and 3.x, the six module can make it easier.  It helps smooth over
modules that have been moved.  You can use this line, for example:

    from six.moves import tkinter_ttk as ttk

--Ned.


From zachary.ware+pylist at gmail.com  Wed May 20 13:03:39 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Wed, 20 May 2015 12:03:39 -0500
Subject: No ttk in 2.7
In-Reply-To: <87r3qbqlve.fsf@Equus.decebal.nl>
References: <87r3qbqlve.fsf@Equus.decebal.nl>
Message-ID: <CAKJDb-OK3CjC5ytF7y3ALXkW8rJ=rXHrDKx3NHXfuaapC9Y=Rw@mail.gmail.com>

On Wed, May 20, 2015 at 11:01 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>
> I want to start playing with tkinter, but there are some differences
> between 2 and 3. For this I use at the moment the following code:
>     import sys
>
>     if sys.version_info[0] < 3:
>         import Tkinter as tk
>         import ttk
>     else:
>         import tkinter as tk
>         from   tkinter import ttk
> or can it better be done in another way?

The way I would do it is as follows:

   try:
       import tkinter as tk
       from tkinter import ttk
   except ImportError:
       import Tkinter as tk
       import ttk

If I may suggest, just write it in Python3 first, then when it does
what you want tack on whatever you need to make 2.7 happy.  I find it
easier to do things that way, though you may find that the only thing
you have to adjust is the imports.

-- 
Zach


From denismfmcmahon at gmail.com  Wed May 20 14:08:40 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Wed, 20 May 2015 18:08:40 +0000 (UTC)
Subject: Best approach to create humongous amount of files
References: <mailman.166.1432123945.17265.python-list@python.org>
Message-ID: <mjiij7$h2l$2@dont-email.me>

On Wed, 20 May 2015 17:14:15 +0530, Parul Mogra wrote:

> Hello everyone,
> My objective is to create large amount of data files (say a million
> *.json files), using a pre-existing template file (*.json). Each file
> would have a unique name, possibly by incorporating time stamp
> information. The files have to be generated in a folder specified.

> What is the best strategy to achieve this task, so that the files will
> be generated in the shortest possible time? Say within an hour.
	
timestamps are normally unixtime in seconds. There are 3600 seconds in an 
hour. You'll have a hard job creating a million files with timestamp 
based naming inside of an hour.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From ian.g.kelly at gmail.com  Wed May 20 14:30:44 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 20 May 2015 12:30:44 -0600
Subject: fork/exec & close file descriptors
In-Reply-To: <cs258eFnssnU1@mid.individual.net>
References: <CANc-5UyeaWqThGFSAuGskz+S2Lrgq0ia9W9DHAie28t9GY+sww@mail.gmail.com>
 <CAPTjJmoj4STV9+t2-9JWed3t6Hxk5ibtaxYSFfX31oGz=kKR1g@mail.gmail.com>
 <CANc-5UwUxYjg=rQQ0Z_nM3s7o4+ghVYzG-9ajQY=LERWAsPrKw@mail.gmail.com>
 <CAPTjJmrdPcW8EkLR9Kq+n9sYEWB8eJ3fg=_JFQ2x_4QhDeg90Q@mail.gmail.com>
 <mailman.139.1432045882.17265.python-list@python.org>
 <cs258eFnssnU1@mid.individual.net>
Message-ID: <CALwzid=Y9AafXP4GcDHhuAqoOyTKZHmE4R2MH3Yxn7qMQjj19Q@mail.gmail.com>

On Tue, May 19, 2015 at 7:10 PM, Gregory Ewing
<greg.ewing at canterbury.ac.nz> wrote:
>> On Tue, May 19, 2015 at 8:54 AM, Chris Angelico <rosuav at gmail.com
>> <mailto:rosuav at gmail.com>> wrote:
>>
>>     On Linux (and possibly some other Unixes), /proc/self/fd may be of
>>     use.
>
>
> On MacOSX, /dev/fd seems to be the equivalent of this.

Not a perfect equivalent. On Linux, ls -lF /proc/self/fd shows the
contents as symlinks, which is handy since you can just read the links
to see what they're pointing to. On OSX, ls -lF /dev/fd shows three
ttys and two directories.

Though I also note that on my Ubuntu Trusty system, /dev/fd is itself
a symlink to /proc/self/fd.


From tjreedy at udel.edu  Wed May 20 14:42:08 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 20 May 2015 14:42:08 -0400
Subject: List semantics [was Re: Slices time complexity]
In-Reply-To: <cs30daFu54gU1@mid.individual.net>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <28bca3fb-653d-40fb-9944-d28b01e77244@googlegroups.com>
 <555c368c$0$12906$c3e8da3$5496439d@news.astraweb.com>
 <cs30daFu54gU1@mid.individual.net>
Message-ID: <mjiki7$1un$1@ger.gmane.org>

On 5/20/2015 4:54 AM, Gregory Ewing wrote:

> At this point the student thinks, "Um... what? How
> can an object contain another object *twice*?"
> If he's still thinking in physical terms, this
> sentence is nonsensical.
>
> It gets even worse with:
>
> x = [1, 2]
> x[1] = x
>
> Now you have to say that the list contains *itself*
> in some weird Tardis-like fashion!
>
> I can't think of any way to dispel the confusion
> verbally without using some word like "reference"
> or "pointer" or something with an equivalent meaning.
> Something that suggests a level of indirectness.

Python sets and lists (and frozen versions) are like clubs.  A club 
roster contains identifiers that refer to and identify the members. 
Kids who can read and write names usually have no problem with this. The 
roster for the Recursive Club (of recursive clubs) would contain the 
identifier 'Recursive Club'.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Wed May 20 14:50:13 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 20 May 2015 14:50:13 -0400
Subject: Help for a newbie regarding code & physical switches
In-Reply-To: <c618bb56-11b9-4e3e-a290-4092fa82969c@googlegroups.com>
References: <3c0e9762-625e-4740-9717-3ce6dc51df70@googlegroups.com>
 <mjhc6q$hvt$1@dont-email.me>
 <c618bb56-11b9-4e3e-a290-4092fa82969c@googlegroups.com>
Message-ID: <mjil1d$a3q$1@ger.gmane.org>

On 5/20/2015 3:54 AM, Howard Spink wrote:
> Thanks for your help. I want the python to run automatically after
> boot and show a blank white screen,

This part if for a RasPy group.

> when a combination of GP10 inputs
> are HIGH python displays one of 150 JPEGS. Is this possible?

Number the n inputs 0 to n-1.  Give input i the 'value' of 2**i.  Sum 
the values of the hi inputs to get an integer.  (Actually, flip the 
corresponding bits of an int that starts at 0).  Create a list jpeg of 
jpegs, so that jpeg[k] is the jpeg to display when the corresponding set 
of inputs are high.

 > what sort of boot times can I get with Arch?

Back to RasPy.

-- 
Terry Jan Reedy



From Cecil at decebal.nl  Wed May 20 14:51:49 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 20 May 2015 20:51:49 +0200
Subject: No ttk in 2.7
References: <87r3qbqlve.fsf@Equus.decebal.nl>
 <5e9aac07-e210-4eb5-a3d7-cfa37cb3f6ab@googlegroups.com>
Message-ID: <87mw0zqdyy.fsf@Equus.decebal.nl>

Op Wednesday 20 May 2015 18:43 CEST schreef Ned Batchelder:

> On Wednesday, May 20, 2015 at 12:35:40 PM UTC-4, Cecil Westerhof wrote:
>> I want to start playing with tkinter, but there are some
>> differences between 2 and 3. For this I use at the moment the
>> following code: import sys
>>
>> if sys.version_info[0] < 3:
>> import Tkinter as tk
>> import ttk
>> else:
>> import tkinter as tk
>> from   tkinter import ttk
>> or can it better be done in another way?
>
> Are you sure you want this program to run under both 2.x and 3.x?
> That will add a layer of complexity to your project. If your
> goal is to learn how to use Tkinter, you might be better off just
> picking one version of Python, and sticking with it.

I just started with Tkinter and I thought this was the only
difference. But I could be wrong of-course. ;-)


> Deciding what versions of Python to support is not always a simple
> decision.  It should be based on a realistic assessment of who will
> be using your program, and what requirements they will have.

Certainly. In principle I want to work with 3, but not all libraries
work with it. (That is also why I choose Tkinter, that works with
both. wx only with 2.)

Also, in the Netherlands mostly Python 2 is used (as far as I know),
so it is good to have my code working with both from that perspective
also.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Wed May 20 14:52:41 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 20 May 2015 20:52:41 +0200
Subject: No ttk in 2.7
References: <87r3qbqlve.fsf@Equus.decebal.nl>
 <5e9aac07-e210-4eb5-a3d7-cfa37cb3f6ab@googlegroups.com>
 <414bf0ad-557d-4373-8452-ab089888225e@googlegroups.com>
Message-ID: <87iobnqdxi.fsf@Equus.decebal.nl>

Op Wednesday 20 May 2015 18:47 CEST schreef Ned Batchelder:

> On Wednesday, May 20, 2015 at 12:43:29 PM UTC-4, Ned Batchelder wrote:
>> On Wednesday, May 20, 2015 at 12:35:40 PM UTC-4, Cecil Westerhof wrote:
>>> I want to start playing with tkinter, but there are some
>>> differences between 2 and 3. For this I use at the moment the
>>> following code: import sys
>>>
>>> if sys.version_info[0] < 3:
>>> import Tkinter as tk
>>> import ttk
>>> else:
>>> import tkinter as tk
>>> from   tkinter import ttk
>>> or can it better be done in another way?
>>
>> Are you sure you want this program to run under both 2.x and 3.x?
>> That will add a layer of complexity to your project. If your
>> goal is to learn how to use Tkinter, you might be better off just
>> picking one version of Python, and sticking with it.
>>
>> Deciding what versions of Python to support is not always a simple
>> decision.  It should be based on a realistic assessment of who will
>> be using your program, and what requirements they will have.
>>
>> --Ned.
>
> Oh, and I meant to include this: if you *do* want to support both
> 2.x and 3.x, the six module can make it easier. It helps smooth over
> modules that have been moved. You can use this line, for example:
>
> from six.moves import tkinter_ttk as ttk

Thanks for the tip. I will look into it.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Wed May 20 14:54:58 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 20 May 2015 20:54:58 +0200
Subject: No ttk in 2.7
References: <87r3qbqlve.fsf@Equus.decebal.nl>
 <mailman.179.1432141452.17265.python-list@python.org>
Message-ID: <87egmbqdtp.fsf@Equus.decebal.nl>

Op Wednesday 20 May 2015 19:03 CEST schreef Zachary Ware:

> On Wed, May 20, 2015 at 11:01 AM, Cecil Westerhof <Cecil at decebal.nl> wrote:
>>
>> I want to start playing with tkinter, but there are some
>> differences between 2 and 3. For this I use at the moment the
>> following code: import sys
>>
>> if sys.version_info[0] < 3:
>> import Tkinter as tk
>> import ttk
>> else:
>> import tkinter as tk
>> from   tkinter import ttk
>> or can it better be done in another way?
>
> The way I would do it is as follows:
>
> try:
> import tkinter as tk
> from tkinter import ttk
> except ImportError:
> import Tkinter as tk
> import ttk

When there goes something wrong with:
    from tkinter import ttk
you will not understand what is happening. ;-)

> If I may suggest, just write it in Python3 first, then when it does
> what you want tack on whatever you need to make 2.7 happy. I find it
> easier to do things that way, though you may find that the only
> thing you have to adjust is the imports.

That was what I thought (the imports), but maybe I am wrong.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From ian.g.kelly at gmail.com  Wed May 20 15:30:12 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 20 May 2015 13:30:12 -0600
Subject: No ttk in 2.7
In-Reply-To: <87egmbqdtp.fsf@Equus.decebal.nl>
References: <87r3qbqlve.fsf@Equus.decebal.nl>
 <mailman.179.1432141452.17265.python-list@python.org>
 <87egmbqdtp.fsf@Equus.decebal.nl>
Message-ID: <CALwzidnoQBxEaVCLJ1=P3KBpcOKB+ah4YFLVoYONVK+99QX3Qg@mail.gmail.com>

On Wed, May 20, 2015 at 12:54 PM, Cecil Westerhof <Cecil at decebal.nl> wrote:
> Op Wednesday 20 May 2015 19:03 CEST schreef Zachary Ware:
>> try:
>> import tkinter as tk
>> from tkinter import ttk
>> except ImportError:
>> import Tkinter as tk
>> import ttk
>
> When there goes something wrong with:
>     from tkinter import ttk
> you will not understand what is happening. ;-)

If something goes wrong with the first import and raises an
ImportError, then it will execute the except clause, which will
definitely raise an ImportError. In this case the second error will
simply be chained onto the first, so the details of the first error
won't be lost.


From marfig at gmail.com  Wed May 20 15:51:14 2015
From: marfig at gmail.com (Mario Figueiredo)
Date: Wed, 20 May 2015 20:51:14 +0100
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjff5g$met$1@dont-email.me>
 <555b6db8$0$12996$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <adoplatr4fq8nssa9s2pk3bbpmhtkdk775@4ax.com>

On Wed, 20 May 2015 03:07:03 +1000, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:

>Yes, a slice can be expensive, if you have (say) a ten billion element list,
>and take a slice list[1:].

Since nothing seems to surprise you and you seem so adamant on calling
anyone being surprised by it, maybe I will surprise you if you
actually run the code I posted on the OP and witness for yourself that
even on a 50 element list will take 3 seconds to execute on an intel
i5.

>But, really... if you're serious about dealing with huge arrays of data, you
>want something like numpy, not Python lists.

You don't need huge. On any algorithm where slices are being used you
will have to account for the linear complexity. You seem to be
dismissive of this fact, but it can have tremendous performance
implications, since it can turn a linear algorithm into a quadratic
one just like that.

Certainly there are alternatives and iterating through the list is a
much better method. There's no arguing against that. But if you are
paying attention to how Python is being taught, [1:] is a little
everywhere on the web announced as a cool trick showcasing Python
strengths. Well, it turns out it is actually a bad practice
performance-wise under many cases and the alternatives should be the
ones being showcased.

> So *in practice* the lack of views for lists is a minor nuisance, if that.

Your opinion was noted.

> Having list slices return copies rather than views avoids more problems than it causes.

Well, I personally wouldn't want to see slices do anything other than
what they are doing now. I think that is the point. Backwards
compatibilioty being just one of the problems.

But no one is arguing for that. Instead, it was said that it would be
interesting if Python offered views.

>On the other hand, if Python implementations would implement slices with
>copy-on-write, that would give us the best of both worlds!

You are a confusing person. You after all agree with what is being
said, but spend all your time talking against it.


From ian.g.kelly at gmail.com  Wed May 20 16:33:57 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Wed, 20 May 2015 14:33:57 -0600
Subject: Slices time complexity
In-Reply-To: <adoplatr4fq8nssa9s2pk3bbpmhtkdk775@4ax.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjff5g$met$1@dont-email.me>
 <555b6db8$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <adoplatr4fq8nssa9s2pk3bbpmhtkdk775@4ax.com>
Message-ID: <CALwzidkA_FcdFwEL9dr5io3odobfi=2ehoJoi5wz1fK1H1bGOA@mail.gmail.com>

On Wed, May 20, 2015 at 1:51 PM, Mario Figueiredo <marfig at gmail.com> wrote:
> On Wed, 20 May 2015 03:07:03 +1000, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>
>>Yes, a slice can be expensive, if you have (say) a ten billion element list,
>>and take a slice list[1:].
>
> Since nothing seems to surprise you and you seem so adamant on calling
> anyone being surprised by it, maybe I will surprise you if you
> actually run the code I posted on the OP and witness for yourself that
> even on a 50 element list will take 3 seconds to execute on an intel
> i5.

I suspect you've made a mistake in your timing. I measure it at 20.8
microseconds on a Xeon W3690.

>>> def minimum(values):
...     if len(values) == 1:
...         return values[0]
...     else:
...         m = minimum(values[1:])
...         return m if m < values[0] else values[0]
...
>>> from timeit import Timer
>>> t = Timer("minimum(values)", setup="from __main__ import minimum; values = list(range(50))")
>>> min(t.repeat(number=100000))
2.077940827002749


From rosuav at gmail.com  Wed May 20 16:35:59 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 21 May 2015 06:35:59 +1000
Subject: Slices time complexity
In-Reply-To: <adoplatr4fq8nssa9s2pk3bbpmhtkdk775@4ax.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjff5g$met$1@dont-email.me>
 <555b6db8$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <adoplatr4fq8nssa9s2pk3bbpmhtkdk775@4ax.com>
Message-ID: <CAPTjJmpDru+qAeE92tFa2ZxJmVfbMHYYov8SV2mm9_WKaXnuoQ@mail.gmail.com>

On Thu, May 21, 2015 at 5:51 AM, Mario Figueiredo <marfig at gmail.com> wrote:
> But no one is arguing for that. Instead, it was said that it would be
> interesting if Python offered views.

It's pretty easy, actually. (Slightly more complicated once you handle
more details like negative indexing and strides other than 1, but
still not too hard.)

class View:
    def __init__(self, seq, start=0, end=None):
        self.seq = seq
        self.start = start
        if end is None: self.end = len(seq)
        else: self.end = end
    def __getitem__(self, item):
        if isinstance(item, slice):
            start, end, _ = item.indices(self.end-self.start)
            return View(self.seq, self.start + start, self.start + end)
        if self.start + item >= self.end: raise IndexError
        return self.seq[self.start + item]

Start with any sequence (doesn't have to be a list as such) and
construct a View of it, and then all slicing and dicing happens with
index arithmetic instead of list construction. Indexing of views
transparently works on the underlying sequence, and you can coalesce a
view into a concrete list (eg to allow garbage collection of the
original) with list(v), same as you would with a range object or a
generator or any other iterable.

It's really not that hard.

ChrisA


From jvadams at usgs.gov  Wed May 20 16:47:43 2015
From: jvadams at usgs.gov (Adams, Jean)
Date: Wed, 20 May 2015 15:47:43 -0500
Subject: ASA Conference on Statistical Practice
Message-ID: <CAN5YmCEM2rZLeT2cP85uEFGkfs_Xp3k7psFRwobZk7DGVyo+bQ@mail.gmail.com>

Python users,

Abstracts are now being accepted for the
     2016 ASA Conference on Statistical Practice,
     February 18-20,
     San Diego, CA, USA.

Conference attendees are not typically familiar with Python.  It would be
great to have someone from the Python community give a brief overview of
how Python might be useful to an applied statistician.

The deadline for submission is June 25.  Presentations will be 35 minutes
long and fall into four broad themes:
     Communication, Impact, and Career Development
     Data Modeling and Analysis
     Big Data Prediction and Analytics
     Software, Programming, and Graphics

Abstracts may be submitted at
http://www.amstat.org/meetings/csp/2016/abstracts.cfm

Thank you.

Jean V. Adams
on behalf of the ASA-CSP 2016 Steering Committee



`?.,,  ><(((?>   `?.,,  ><(((?>   `?.,,  ><(((?>

Jean V. Adams
Statistician
U.S. Geological Survey
Great Lakes Science Center
223 East Steinfest Road
Antigo, WI 54409  USA
http://www.glsc.usgs.gov
http://profile.usgs.gov/jvadams
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150520/26afb627/attachment.html>

From breamoreboy at yahoo.co.uk  Wed May 20 16:47:46 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 20 May 2015 21:47:46 +0100
Subject: Slices time complexity
In-Reply-To: <adoplatr4fq8nssa9s2pk3bbpmhtkdk775@4ax.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjff5g$met$1@dont-email.me>
 <555b6db8$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <adoplatr4fq8nssa9s2pk3bbpmhtkdk775@4ax.com>
Message-ID: <mjirtq$tgk$1@ger.gmane.org>

On 20/05/2015 20:51, Mario Figueiredo wrote:
> On Wed, 20 May 2015 03:07:03 +1000, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>
>> Yes, a slice can be expensive, if you have (say) a ten billion element list,
>> and take a slice list[1:].
>
> Since nothing seems to surprise you and you seem so adamant on calling
> anyone being surprised by it, maybe I will surprise you if you
> actually run the code I posted on the OP and witness for yourself that
> even on a 50 element list will take 3 seconds to execute on an intel
> i5.
>

Please provide the figures to back up this claim.  Nothing personal but 
we've had problems with the RUE (amongst others) making nonsensical 
claims, please don't take us down that path, thank you.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Wed May 20 16:55:56 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 20 May 2015 21:55:56 +0100
Subject: No ttk in 2.7
In-Reply-To: <87iobnqdxi.fsf@Equus.decebal.nl>
References: <87r3qbqlve.fsf@Equus.decebal.nl>
 <5e9aac07-e210-4eb5-a3d7-cfa37cb3f6ab@googlegroups.com>
 <414bf0ad-557d-4373-8452-ab089888225e@googlegroups.com>
 <87iobnqdxi.fsf@Equus.decebal.nl>
Message-ID: <mjisd3$5pq$1@ger.gmane.org>

On 20/05/2015 19:52, Cecil Westerhof wrote:
> Op Wednesday 20 May 2015 18:47 CEST schreef Ned Batchelder:
>
>> On Wednesday, May 20, 2015 at 12:43:29 PM UTC-4, Ned Batchelder wrote:
>>> On Wednesday, May 20, 2015 at 12:35:40 PM UTC-4, Cecil Westerhof wrote:
>>>> I want to start playing with tkinter, but there are some
>>>> differences between 2 and 3. For this I use at the moment the
>>>> following code: import sys
>>>>
>>>> if sys.version_info[0] < 3:
>>>> import Tkinter as tk
>>>> import ttk
>>>> else:
>>>> import tkinter as tk
>>>> from   tkinter import ttk
>>>> or can it better be done in another way?
>>>
>>> Are you sure you want this program to run under both 2.x and 3.x?
>>> That will add a layer of complexity to your project. If your
>>> goal is to learn how to use Tkinter, you might be better off just
>>> picking one version of Python, and sticking with it.
>>>
>>> Deciding what versions of Python to support is not always a simple
>>> decision.  It should be based on a realistic assessment of who will
>>> be using your program, and what requirements they will have.
>>>
>>> --Ned.
>>
>> Oh, and I meant to include this: if you *do* want to support both
>> 2.x and 3.x, the six module can make it easier. It helps smooth over
>> modules that have been moved. You can use this line, for example:
>>
>> from six.moves import tkinter_ttk as ttk
>
> Thanks for the tip. I will look into it.
>

https://docs.python.org/3/howto/pyporting.html

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lab at pacbell.net  Wed May 20 17:18:59 2015
From: lab at pacbell.net (20/20 Lab)
Date: Wed, 20 May 2015 14:18:59 -0700
Subject: Looking for direction
In-Reply-To: <9abf87a2-a98b-470b-9f94-a76d4ef1b34e@googlegroups.com>
References: <mailman.465.1431559626.12865.python-list@python.org>
 <9abf87a2-a98b-470b-9f94-a76d4ef1b34e@googlegroups.com>
Message-ID: <555CFA43.9010008@pacbell.net>

Your the second to recommend this to me.  I ended up picking it up last 
week.  So I need to sit down with it.  I was able to get a working 
project.  However, I dont fully grasp the details on how. So the book 
will help I'm sure.

Thank you.

On 05/20/2015 05:50 AM, darnold via Python-list wrote:
> I recommend getting your hands on "Automate The Boring Stuff With Python" from no starch press:
>
> http://www.nostarch.com/automatestuff
>
> I've not read it in its entirety, but it's very beginner-friendly and is targeted at just the sort of processing you appear to be doing.
>
> HTH,
> Don



From marfig at gmail.com  Wed May 20 17:23:15 2015
From: marfig at gmail.com (Mario Figueiredo)
Date: Wed, 20 May 2015 22:23:15 +0100
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjff5g$met$1@dont-email.me>
 <555b6db8$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <adoplatr4fq8nssa9s2pk3bbpmhtkdk775@4ax.com>
 <mailman.186.1432154882.17265.python-list@python.org>
Message-ID: <8aupla9d0lnat4n9ct1ve87untkjij70s8@4ax.com>

On Wed, 20 May 2015 21:47:46 +0100, Mark Lawrence
<breamoreboy at yahoo.co.uk> wrote:

>Please provide the figures to back up this claim.  Nothing personal but 
>we've had problems with the RUE (amongst others) making nonsensical 
>claims, please don't take us down that path, thank you.

Alright. My apologies. This answer of yours along with Ian's made me
realize something was wrong with my measurements. And it was indeed.
Thankfully not the code which I show below. But the fact I wasn't
aware -- until you guys forced me to dig in what was going on -- that
timeit.timeit defaults to 1 million passes.

>#!/usr/bin/env python3
> 
>import random
>import timeit
> 
>def minimum(values):
>    if len(values) == 1:
>        return values[0]
>    else:
>        m = minimum(values[1:])
>        return m if m < values[0] else values[0]
> 
>a5 = [random.randint(0, 1000) for _ in range(5)]
>a50 = [random.randint(0, 1000) for _ in range(50)]
>a500 = [random.randint(0, 1000) for _ in range(500)]
> 
>setup = 'from __main__ import minimum, a5, a50, a500'
> 
>print('Baseline: {:.2f}s'.format(timeit.timeit('minimum(a5)', setup=setup)))
>print('x10  -> {:.2f}s'.format(timeit.timeit('minimum(a50)', setup=setup)))
>print('x100 -> {:.2f}s'.format(timeit.timeit('minimum(a500)', setup=setup)))

One run output is:
   
   Baseline: 2.86s
   x10  -> 36.72s
   x100 -> 1021.07s

Which, as you can see, I misinterpreted as being the result of a
single pass.

The performance was shocking. And I was going to get to that on
another thread. The slice linear time alone couldn't explain it. But
looking more carefully at the documentation revealed my error.


From auriocus at gmx.de  Wed May 20 17:54:48 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Wed, 20 May 2015 23:54:48 +0200
Subject: Slices time complexity
In-Reply-To: <555b3e2d$0$12993$c3e8da3$5496439d@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <87zj50ewqf.fsf@elektro.pacujo.net>
 <555b3e2d$0$12993$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mjivp4$l01$1@dont-email.me>

Am 19.05.15 um 15:44 schrieb Steven D'Aprano:
> Variables are not first class values in C. (I assume you meant *values*
> rather than "objects", on account of C not being an OOP language.) There is
> no way, for example, to set x to *the variable y* in either C or Python. If
> you could, that would imply something like this:
>
> y = 42  # regular assignment
> x ::= y  # set x to be the variable y, not the value of y
> assert x == y  # naturally, since x and y are now two different
>                 # names for the same variable
> x = 23  # regular assignment
> assert y == 23  # since y is just another name for x

In C++, this is possible:

int y=42;
int &x = y;
assert(x==y);
x=23;
assert(x==y);

.... but, admittedly, you can't reassign x.

	Christian


From greg.ewing at canterbury.ac.nz  Wed May 20 19:06:20 2015
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Thu, 21 May 2015 11:06:20 +1200
Subject: List semantics [was Re: Slices time complexity]
In-Reply-To: <mailman.181.1432147346.17265.python-list@python.org>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <28bca3fb-653d-40fb-9944-d28b01e77244@googlegroups.com>
 <555c368c$0$12906$c3e8da3$5496439d@news.astraweb.com>
 <cs30daFu54gU1@mid.individual.net>
 <mailman.181.1432147346.17265.python-list@python.org>
Message-ID: <cs4ibcFch2nU1@mid.individual.net>

Terry Reedy wrote:
> A club 
> roster contains identifiers that refer to and identify the members.

Exactly: the roster "refers to" the members, rather
than "containing" them. Or equivalently, it contains
references to the members.

-- 
Greg


From daniel.base4 at gmail.com  Wed May 20 19:20:53 2015
From: daniel.base4 at gmail.com (=?ISO-8859-1?Q?Daniel_Gon=E7alves?=)
Date: Wed, 20 May 2015 16:20:53 -0700 (PDT)
Subject: 'NoneType' in contextmanager prevent proper RuntimeError to be raised
Message-ID: <13277aff-1288-4357-b36a-5c78d8ae37b8@googlegroups.com>

When you decorate a function with contextmanager that didn't yield you got an AttributeError instead of a proper RuntimeError "generator didn't yield". For example:

>>> @contextlib.contextmanager
>>> def foo():
...     pass
...
>>> with foo():
...     do_something()
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
AttributeError: 'NoneType' object has no attribute 'next'

The proper exception should be a RuntimerError with the "generator didn't yield" message. At least for Python version 2.7.6.

Regards


From python at mrabarnett.plus.com  Wed May 20 19:44:12 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 21 May 2015 00:44:12 +0100
Subject: 'NoneType' in contextmanager prevent proper RuntimeError to be
 raised
In-Reply-To: <13277aff-1288-4357-b36a-5c78d8ae37b8@googlegroups.com>
References: <13277aff-1288-4357-b36a-5c78d8ae37b8@googlegroups.com>
Message-ID: <555D1C4C.6050707@mrabarnett.plus.com>

On 2015-05-21 00:20, Daniel Gon?alves wrote:
> When you decorate a function with contextmanager that didn't yield you got an AttributeError instead of a proper RuntimeError "generator didn't yield". For example:
>
>>>> @contextlib.contextmanager
>>>> def foo():
> ...     pass
> ...
>>>> with foo():
> ...     do_something()
> ...
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
>      return self.gen.next()
> AttributeError: 'NoneType' object has no attribute 'next'
>
> The proper exception should be a RuntimerError with the "generator didn't yield" message. At least for Python version 2.7.6.
>
If it doesn't contain a 'yield', it's not a generator, it's a function.



From cs at zip.com.au  Wed May 20 19:58:25 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 21 May 2015 09:58:25 +1000
Subject: 'NoneType' in contextmanager prevent proper RuntimeError to be
 raised
In-Reply-To: <13277aff-1288-4357-b36a-5c78d8ae37b8@googlegroups.com>
References: <13277aff-1288-4357-b36a-5c78d8ae37b8@googlegroups.com>
Message-ID: <20150520235825.GA6422@cskk.homeip.net>

On 20May2015 16:20, Daniel Gon?alves <daniel.base4 at gmail.com> wrote:
>When you decorate a function with contextmanager that didn't yield you got an AttributeError instead of a proper RuntimeError "generator didn't yield". For example:
>
>>>> @contextlib.contextmanager
>>>> def foo():
>...     pass
>...
>>>> with foo():
>...     do_something()
>...
>Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
>    return self.gen.next()
>AttributeError: 'NoneType' object has no attribute 'next'
>
>The proper exception should be a RuntimerError with the "generator didn't yield" message. At least for Python version 2.7.6.

Unfortunately no.

Your function foo() is _not_ a generator because it does not have a yield 
statement. Therefore it is a normal funcion which returns None (because it also 
lacks a return statement).

contextlib.contextmanager works by _calling_ the inner foo(). It expected to 
get a generator back. But your foo() returns None, _not_ a generator.

hence the error message: contextlib.contextmanager tries to use the return 
value with next(), which of course fails with exactly the exception one might 
expect.

To get "generator didn't yield" you need to actually have a generator. Eg:

  @contextlib.contextmanager
  def foo():
    if false:
      yield

Untested, though.

Cheers,
Cameron Simpson <cs at zip.com.au>


From daniel at base4.com.br  Wed May 20 20:08:07 2015
From: daniel at base4.com.br (=?UTF-8?Q?Daniel_Gon=C3=A7alves?=)
Date: Wed, 20 May 2015 21:08:07 -0300
Subject: 'NoneType' in contextmanager prevent proper RuntimeError to be
 raised
In-Reply-To: <mailman.189.1432165462.17265.python-list@python.org>
References: <13277aff-1288-4357-b36a-5c78d8ae37b8@googlegroups.com>
 <mailman.189.1432165462.17265.python-list@python.org>
Message-ID: <CAGmy91sTnZiuK6hWptLngE-5gv0YHv4DY5XOXzLEQM_CwLmeww@mail.gmail.com>

>From the "Lib/contextlib.py":

    class GeneratorContextManager(object):
        """Helper for @contextmanager decorator."""

        def __init__(self, gen):
            self.gen = gen

        def __enter__(self):
            try:
                return self.gen.next()
            except StopIteration:
                raise RuntimeError("generator didn't yield")

The line "raise RuntimerError(...)" never gets reached, since self.gen is None.

So you end up with an AttributeError: 'NoneType' object has no
attribute 'next', instead of the proper RuntimeError, making it harder
to debug.

Anyways, I know that it's strange to have a contextmanager that didn't yield.

I just stumbled upon this, by accident, and I think the way it's
implemented looks wrong :)

2015-05-20 20:44 GMT-03:00 MRAB <python at mrabarnett.plus.com>:
> On 2015-05-21 00:20, Daniel Gon?alves wrote:
>>
>> When you decorate a function with contextmanager that didn't yield you got
>> an AttributeError instead of a proper RuntimeError "generator didn't yield".
>> For example:
>>
>>>>> @contextlib.contextmanager
>>>>> def foo():
>>
>> ...     pass
>> ...
>>>>>
>>>>> with foo():
>>
>> ...     do_something()
>> ...
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>>    File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
>>      return self.gen.next()
>> AttributeError: 'NoneType' object has no attribute 'next'
>>
>> The proper exception should be a RuntimerError with the "generator didn't
>> yield" message. At least for Python version 2.7.6.
>>
> If it doesn't contain a 'yield', it's not a generator, it's a function.
>



-- 
Daniel Gon?alves
Base4 Sistemas Ltda ME
www.base4.com.br


From steve+comp.lang.python at pearwood.info  Wed May 20 22:10:31 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 21 May 2015 12:10:31 +1000
Subject: List semantics [was Re: Slices time complexity]
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <28bca3fb-653d-40fb-9944-d28b01e77244@googlegroups.com>
 <555c368c$0$12906$c3e8da3$5496439d@news.astraweb.com>
 <cs30daFu54gU1@mid.individual.net>
Message-ID: <555d3e98$0$12989$c3e8da3$5496439d@news.astraweb.com>

On Wed, 20 May 2015 06:54 pm, Gregory Ewing wrote:

> Steven D'Aprano wrote:
> 
>> Code snippet 1:
>> 
>>     x = [[1,2],[1,2]]
>> 
>> creates a list bound to the name "x", containing a list containing ints 1
>> and 2, and a second independent list also containing ints 1 and 2.
> 
> Using the word "contains" here is misleading, because
> it conjures a mental picture of physical containment,
> which is exactly what you *don't* want to suggest.
> 
>>     z = [1,2]
>>     y = [z, z]
>> 
>> creates a list bound to the name "z", ints 1 and 2; then it creates a
>> second list, bound to name "y", containing the first list "z" twice.
> 
> At this point the student thinks, "Um... what? How
> can an object contain another object *twice*?"
> If he's still thinking in physical terms, this
> sentence is nonsensical.
> 
> It gets even worse with:
> 
> x = [1, 2]
> x[1] = x
> 
> Now you have to say that the list contains *itself*
> in some weird Tardis-like fashion!

Yes. And why is this a problem?

This may be a bit elitist, but that's not a bad thing. As a programmer, you
will be expected to deal with many things *much more confusing* than
objects appearing in two places at once. Any child over the age of about
five is capable of understanding such things -- it's a staple of fantasy,
science fiction and cartoons.

In a virtual world, having an object in two places at once, or even
contained within itself, is *easy*.

Of course, adults knows that real-world objects cannot really be in two
places at once, except maybe in the vicinity of rapidly spinning neutron
stars, or in the realm of quantum mechanics. So there may be some students,
of an excessively literal mind, who can't get past the idea of objects
being in two places at once. For them, you can drop down a level and start
to talk about the implementation. Lists don't actually contain the items
themselves, they hold an indirect reference to the item. (You don't even
need to use the word "pointer", since pointer is a data primitive which
some implementations do not use.) You can have multiple references to the
same object, obviously. A name is such a reference: [z, z].

The point which I am trying to get across is not that talking about the
implementation of the Python virtual machine is *necessarily always* a bad
thing, but it can *and should* be carefully distinguished from the
interface to the Python virtual machine, that is, the Python language.

The Python language has no pointers. It has objects which can be in
multiple "places" (by which I mean, bound to names, contained by lists or
dicts, etc.) at once, and indeed objects can even contain themselves. As a
Python programmer, you never deal with pointers, you deal with objects, and
in the virtual world of the Python language, objects being in two places at
the same time is no more strange than objects in the real world existing in
two times at the same place.

The Python *implementation* uses pointers, C++ references, Java objects, or
whatever. Note that the implementation itself may not directly use
pointers, e.g. Java, but the *implementation of the implementation* may. If
you go down far enough, and ask how the physical hardware machine
implements pointers, you start talking about "copying memory", which
actually implemented by flipping bits.

Oh, and just for the record, it is possible to create a Python
implementation which doesn't use pointers or any form of indirect
addressing *at all*. It wouldn't be efficient, but it would be possible.
And of course there is one actually implementation of the Python virtual
machine which doesn't use any sort of conventional implementation. Any time
you simulate executing a chunk of Python code in your head, you're running
a virtual Python interpreter.


-- 
Steven



From steve+comp.lang.python at pearwood.info  Thu May 21 01:19:18 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 21 May 2015 15:19:18 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <mjhi5a$4pa$1@dont-email.me>
Message-ID: <555d6ad8$0$2763$c3e8da3$76491128@news.astraweb.com>

On Wednesday 20 May 2015 19:56, Bartc wrote:

> "Steven D'Aprano" <steve+comp.lang.python at pearwood.info> wrote in message
> news:555c225b$0$2769$c3e8da3$76491128 at news.astraweb.com...
> 
>> The mental gymnastics they go through to force the round peg of pass-by-
>> sharing object semantics into the false dichotomy "pass-by-value versus
>> pass-by-reference" is just crazy.
>>
>> One consequence of this is that the meaning of "call by value" depends on
>> whether the value you are talking about is a boxed or unboxed value.
>> Unboxed
>> values are copied. Boxed values are not. Except that they are, if you
>> understand that "the value" doesn't refer to the actual value, but to
>> some hidden and implementation-dependent reference to the value.
> 
> I have a language where everything is boxed (the data that isn't boxed,
> has to be before it can be manipulated).
> 
> But simple data such as small integers and floats are passed by value.
> Bigger data is 'sort of' passed by reference, but not full reference (the
> details are a bit messy, but if an object consists of two parts (A,B),
> then a copy of descriptor A is passed, which contains a pointer to B. For
> small scalars, the B part doesn't exist).

Sounds complicated and confusing. In my opinion, you would be better off 
picking a well-known argument passing convention (call by value, reference, 
name, sharing) and using that, always. Or at least have a simple, objective 
rule for when you swap from one convention to another, like "arrays are 
passed by reference, everything else is passed by value".

What you *shouldn't* do is implement your own argument convention, then call 
it "pass by value" if it is not the standard meaning of pass by value. Don't 
do what the Java community does, which is implement pass by sharing but call 
it pass by value. Don't do what the Ruby community does, which is implement 
pass by sharing but call it pass by reference.

Don't do what the Lua community does, which is invent a stupid distinction 
between "reference types" and "value types" so they too can misuse 
terminology:

http://stackoverflow.com/questions/6128152/lua-function-variable-scope-pass-
by-value-or-reference

Take note of the highest voted answer, by Bas Bossink, claiming that Lua 
uses pass-by-value for some values and pass-by-reference for others. He is 
wrong. Just like Python, Lua uses the same calling convention for all types, 
and it is neither pass by value nor pass by reference.

This calling convention has been known as pass by sharing (or variations of 
that, like "pass by object sharing") since it was named by Barbara Liskov in 
the 1970s, for the language CLU, but the convention itself goes back to Lisp 
in the 1950s.

Fuzzy thinking leads to confusion, error, and PHP.


> (A proper reference is available when passing arguments to functions:
> a boxed pointer to (A,B) is constructed.)
> 
> However, I was so impressed with how (C)Python does things (working
> exclusively with pointers, doing assignments by simply copying pointers,
> and using reference counting to manage memory), that I'm experimenting
> with letting my language work the same way. (Also then I can benchmark the
> two more fairly.)

Most modern application languages (as opposed to systems languages) use the 
same convention, pass by sharing. Java uses it for objects; Python, Ruby and 
Lua use it for everything. I'm *guessing* that Javascript also uses the same 
convention, but I'm not sure.

Perl, I imagine, does the most complicated thing that can possibly not 
collapse in a heap of internal self-contradiction :-)


> Then I get a little disillusioned when I find out in this thread that a
> simple slice of an array actually copies every element! Since I currently
> create a view (a good term I'd never come across before) for a slice, does
> that mean I now also have to do what Python does?

You only have to do what Python does if you wish to claim that your language 
is an implementation of Python.

Feel free to borrow slicing syntax list[start:stop:step] but return a view. 
That's what Go does. Views and copies each have their own advantages and 
disadvantages. As the designer of your own language, you get to decide which 
trade-offs your language makes.


-- 
Steve



From chaotic.sid at gmail.com  Thu May 21 01:34:34 2015
From: chaotic.sid at gmail.com (chaotic.sid at gmail.com)
Date: Wed, 20 May 2015 22:34:34 -0700 (PDT)
Subject: Find if a file existing within 1000s of folder/sub-folder - each file
 has a unique presence
Message-ID: <9d786d0b-0f25-4446-a730-ff1fe2f6b20d@googlegroups.com>

Hi All,

I have a list of scripts which are present in various folders and this may be present in one of their sub-folder too.
There are 1000s of folders and hence parsing through each folder/sub-folder is not an option.

So I was trying to dir /s /b using python.
Now since the file's path name is computed using other part of the code, I am feeding in a variable here and somehow it does not seem to work.. :(

Intent is to run following command from python 
dir /s /b "c:/abc/def/ghjmain\features\XYZ\*<filename>"

My present code.
import subprocess

for row in range(1,max_row):
	tempID_cell = "C" + str(row);
	tempID = ws[tempID_cell].value;
	print (tempID);
	Command = "c:\\abc\\def\\ghj\\main\\features\\XYZ\\*" + str(tempID) + ".cli";
	print (Command);
	IsPresent = subprocess.check_output("dir /s /b Command", shell=True);

Any help would really be appreciated.

Regards,
Siddharth


From marko at pacujo.net  Thu May 21 02:20:16 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 21 May 2015 09:20:16 +0300
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <mjhi5a$4pa$1@dont-email.me>
 <555d6ad8$0$2763$c3e8da3$76491128@news.astraweb.com>
Message-ID: <87lhgicuzj.fsf@elektro.pacujo.net>

Steven D'Aprano <steve+comp.lang.python at pearwood.info>:

> On Wednesday 20 May 2015 19:56, Bartc wrote:
>> But simple data such as small integers and floats are passed by
>> value. Bigger data is 'sort of' passed by reference, but not full
>> reference (the details are a bit messy, but if an object consists of
>> two parts (A,B), then a copy of descriptor A is passed, which
>> contains a pointer to B. For small scalars, the B part doesn't
>> exist).
>
> Sounds complicated and confusing. In my opinion, you would be better
> off picking a well-known argument passing convention (call by value,
> reference, name, sharing) and using that, always.

I wonder if BartC refers to the conceptual model or implementation.
Elisp uses the waste bits of pointers as flags; (small) integers are
represented as fake (illegal) pointers. However, the implementation
choice is in no way visible at the Lisp level.


Marko


From nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de  Thu May 21 02:35:43 2015
From: nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de (Thomas Rachel)
Date: Thu, 21 May 2015 08:35:43 +0200
Subject: subprocess.Popen zombie
In-Reply-To: <mailman.177.1432140299.17265.python-list@python.org>
References: <mailman.170.1432127818.17265.python-list@python.org>
 <87bnhfqmr4.fsf@universite-de-strasbourg.fr.invalid>
 <mailman.177.1432140299.17265.python-list@python.org>
Message-ID: <mjjuc5$1ft$1@r01.glglgl.de>

Am 20.05.2015 um 18:44 schrieb Robin Becker:

> not really, it's just normal to keep event routines short; the routine
> which beeps is after detection of the cat's entrance into the house and
> various recognition schemes have pronounced intruder :)

You could add a timed "cleanup" routine which .wait()s after a certain 
time (250 ms or so).

Or even better, which .poll()s and re-schedules itself if the process 
still runs.


Thomas


From steve+comp.lang.python at pearwood.info  Thu May 21 03:35:55 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 21 May 2015 17:35:55 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjff5g$met$1@dont-email.me>
 <555b6db8$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <adoplatr4fq8nssa9s2pk3bbpmhtkdk775@4ax.com>
Message-ID: <555d8ae3$0$11095$c3e8da3@news.astraweb.com>

On Thursday 21 May 2015 05:51, Mario Figueiredo wrote:

> On Wed, 20 May 2015 03:07:03 +1000, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
[...]
>>But, really... if you're serious about dealing with huge arrays of data,
>>you want something like numpy, not Python lists.
> 
> You don't need huge. On any algorithm where slices are being used you
> will have to account for the linear complexity. You seem to be
> dismissive of this fact, but it can have tremendous performance
> implications, since it can turn a linear algorithm into a quadratic
> one just like that.

Sure. And for small enough N, everything is fast.

There's a sorting algorithm, usually called "Bozo sort", which shuffles the 
list, then checks whether it is sorted. If not, it shuffles it again, until 
it is sorted. Bozo sort is a *terrible* algorithm, with no upper limit to 
the worst case, and O(N!) for the average case. And yet, with small lists 
(say, five items) the performance is acceptable if your requirements are 
low.

Should you use Bozo sort in production? No, of course not. But the point I 
am making is that Big Oh analysis doesn't tell you how fast a particular 
implementation will run, only how it will scale for "sufficiently large" N. 
An O(N**2) algorithm with a small multiplier may be much faster than an O(N) 
algorithm with a large multiplier for all the data sets you care about.


> Certainly there are alternatives and iterating through the list is a
> much better method. There's no arguing against that. But if you are
> paying attention to how Python is being taught, [1:] is a little
> everywhere on the web announced as a cool trick showcasing Python
> strengths. Well, it turns out it is actually a bad practice
> performance-wise under many cases 

That has not been my experience, or many other people's.

If your experience is different from mine, then as I said, there is numpy, 
or you can write your own views. Just make sure that in trying to optimize 
your code, you don't end up actually making it *slower* by optimizing for 
more data than you ever will, or could, have to deal with. Big Oh 
theoretical analysis only goes so far, you need actual benchmarks and 
profiling to make good decisions about which algorithm you should use.



-- 
Steve



From steve+comp.lang.python at pearwood.info  Thu May 21 04:07:13 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 21 May 2015 18:07:13 +1000
Subject: Find if a file existing within 1000s of folder/sub-folder - each
 file has a unique presence
References: <9d786d0b-0f25-4446-a730-ff1fe2f6b20d@googlegroups.com>
Message-ID: <555d9233$0$12913$c3e8da3$5496439d@news.astraweb.com>

On Thursday 21 May 2015 15:34, chaotic.sid at gmail.com wrote:

> So I was trying to dir /s /b using python.
> Now since the file's path name is computed using other part of the code, I
> am feeding in a variable here and somehow it does not seem to work.. :(
> 
> Intent is to run following command from python
> dir /s /b "c:/abc/def/ghjmain\features\XYZ\*<filename>"

I don't use Windows and have no idea what the /s and /b flags do, but 
something like this should work:

import glob
print(glob.glob("c:/abc/def/ghjmain/features/XYZ/*<filename>"))


Don't use backslashes \ as they have special meaning to Python. Use forward 
slashes and let Python convert them as needed.

Also, this only looks in the C:.../XYZ directory. I think that Python 3.4 or 
better will accept a ** wildcard instead of * to look in subdirectories too. 
Or you can use a simple directory walker:

# Untested.
import os, os.path
start = "c:/abc/def/ghjmain/features/XYZ/"
for dirpath, dirnames, filenames in os.walk(start):
    for name in filenames:
        if name.endswith("filename"):
            print(os.path.join(dirpath, name))


If you can do the file listing from Python without calling out to an 
external process, it will likely be faster.



> My present code.
> import subprocess
> 
> for row in range(1,max_row):
>     tempID_cell = "C" + str(row);
>     tempID = ws[tempID_cell].value;
>     print (tempID);
>     Command = "c:\\abc\\def\\ghj\\main\\features\\XYZ\\*" + str(tempID) +
>     ".cli"; print (Command);
>     IsPresent = subprocess.check_output("dir /s /b Command", shell=True);
> 
> Any help would really be appreciated.


Start by defining "somehow it does not seem to work" (your words, above). 
What happens? Does your computer crash? It downloads files off the Internet? 
Raises an exception? Prints the wrong information?

If it raises an exception, you should COPY AND PASTE the full traceback. If 
it prints an error message from dir, you should do the same.

What happens when you take the command you generate and run it in the shell?

I think what you want is:

path = "c:/abc/def/ghj/main/features/XYZ/*" + str(tempID) + ".cli"
command = "dir /s /b " + path
output = subprocess.check_output(command, shell=True)

Actually, I think you don't need the shell argument. Try this instead:

path = "c:/abc/def/ghj/main/features/XYZ/*" + str(tempID) + ".cli"
output = subprocess.check_output(['dir', '/s', '/b', path])

That is likely to be safer and will help avoid shell injection attacks if 
the path comes from an untrusted source. Any subprocess experts want to 
confirm this?



P.S. Python is not C or Javascript. There is no need to end each line with a 
semicolon. That just makes you seem like a n00b.


-- 
Steve



From rosuav at gmail.com  Thu May 21 04:25:41 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 21 May 2015 18:25:41 +1000
Subject: Slices time complexity
In-Reply-To: <555d8ae3$0$11095$c3e8da3@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <555b0621$0$2753$c3e8da3$76491128@news.astraweb.com>
 <mjff5g$met$1@dont-email.me>
 <555b6db8$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <adoplatr4fq8nssa9s2pk3bbpmhtkdk775@4ax.com>
 <555d8ae3$0$11095$c3e8da3@news.astraweb.com>
Message-ID: <CAPTjJmqtN0w+k68ux_ig+kY8iG-V_noyomQj1b8tOy8zA9hqEQ@mail.gmail.com>

On Thu, May 21, 2015 at 5:35 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
>> You don't need huge. On any algorithm where slices are being used you
>> will have to account for the linear complexity. You seem to be
>> dismissive of this fact, but it can have tremendous performance
>> implications, since it can turn a linear algorithm into a quadratic
>> one just like that.
>
> Sure. And for small enough N, everything is fast.
>
> There's a sorting algorithm, usually called "Bozo sort", which shuffles the
> list, then checks whether it is sorted. If not, it shuffles it again, until
> it is sorted. Bozo sort is a *terrible* algorithm, with no upper limit to
> the worst case, and O(N!) for the average case. And yet, with small lists
> (say, five items) the performance is acceptable if your requirements are
> low.

A less ridiculous, but equally valid, comparison is with
multiplication algorithms. A while ago someone asked about what Python
uses, and on learning that CPython uses Karatsuba [1], suggested that
some other algorithm (I don't remember which) had better asymptotic
complexity. Downside: It'd be slower for smaller numbers, and the
complexity cost of an additional cutoff ("use grade-school up to X,
then Karatsuba up to Y, then Fuerer's") would also negatively impact
performance overall. Grade-school arithmetic is O(N*N) which looks
terrible... but for smallish numbers, its performance is quite
adequate.

ChrisA

[1] https://en.wikipedia.org/wiki/Karatsuba_algorithm


From __peter__ at web.de  Thu May 21 04:54:36 2015
From: __peter__ at web.de (Peter Otten)
Date: Thu, 21 May 2015 10:54:36 +0200
Subject: Find if a file existing within 1000s of folder/sub-folder - each
 file has a unique presence
References: <9d786d0b-0f25-4446-a730-ff1fe2f6b20d@googlegroups.com>
 <555d9233$0$12913$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mjk6gd$t06$1@ger.gmane.org>

Steven D'Aprano wrote:

> Also, this only looks in the C:.../XYZ directory. I think that Python 3.4
> or better will accept a ** wildcard instead of * to look in subdirectories
> too. 

Unfortunately that's Python 3.5. You are ahead of time.

> Actually, I think you don't need the shell argument. Try this instead:
> 
> path = "c:/abc/def/ghj/main/features/XYZ/*" + str(tempID) + ".cli"
> output = subprocess.check_output(['dir', '/s', '/b', path])

"dir" used to be an internal command; back in the day you would have to 
invoke it with (untested)

check_output(["cmd", "/k", 'dir', '/s', '/b', path])

I believe on Windows path may contain wildcards -- or does subprocess escape 
these somehow? 

> That is likely to be safer and will help avoid shell injection attacks if 
> the path comes from an untrusted source. Any subprocess experts want to 
> confirm this?
 
I probably should have kept quiet to let a combined subprocess/windows 
expert chime in -- but now it's too late...



From mail at timgolden.me.uk  Thu May 21 05:06:33 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 21 May 2015 10:06:33 +0100
Subject: Find if a file existing within 1000s of folder/sub-folder - each
 file has a unique presence
In-Reply-To: <555d9233$0$12913$c3e8da3$5496439d@news.astraweb.com>
References: <9d786d0b-0f25-4446-a730-ff1fe2f6b20d@googlegroups.com>
 <555d9233$0$12913$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <555DA019.3010005@timgolden.me.uk>

On 21/05/2015 09:07, Steven D'Aprano wrote:
> On Thursday 21 May 2015 15:34, chaotic.sid at gmail.com wrote:
> 
>> So I was trying to dir /s /b using python.
>> Now since the file's path name is computed using other part of the code, I
>> am feeding in a variable here and somehow it does not seem to work.. :(
>>
>> Intent is to run following command from python
>> dir /s /b "c:/abc/def/ghjmain\features\XYZ\*<filename>"
> 
> I don't use Windows and have no idea what the /s and /b flags do, but 
> something like this should work:

Just for the uninitiated: /b shows only filenames ("bare") and /s shows
subdirectories.

[ ... snip os.walk example ...]

> path = "c:/abc/def/ghj/main/features/XYZ/*" + str(tempID) + ".cli"
> command = "dir /s /b " + path
> output = subprocess.check_output(command, shell=True)
> 
> Actually, I think you don't need the shell argument. Try this instead:

This is one of the few cases on Windows where you actually *do* need the
shell=True. shell=True invokes "cmd.exe" which is needed for the
command, such as dir and copy, which aren't standalone
executables but subcommands of cmd.exe.

I agree with Steven that os.walk, or something derived from it, is
definitely the way to go here, unless the OP has tried it and found it
to be too slow. Clearly, some kind of cacheing would help since the idea
seems to be to find, one at a time, a filename in any one of a large
hierarchy of directories.

Python 3.5 has the brand-new os.scandir which does a more efficient job
than os.listdir, using the underlying OS facilities on each platform and
cacheing where possible. But that's really leading edge, although Ben
Hoyt (the author) maintains an up-to-date version on github:

https://github.com/benhoyt/scandir


Just for the exercise, here's code which builds a dictionary mapping
filename to location(s) found:

<code>
#!python3
import os, sys

START_FROM = "c:/temp"

files = {}
for dirpath, dirnames, filenames in os.walk(START_FROM):
	for filename in filenames:
		files.setdefault(filename, []).append(dirpath)

filename_to_find = "temp.txt" ## input("Filename: ")
found_in = files.get(filename_to_find)
if found_in:
	print("Found in: ", ", ".join(found_in))
else:
	print("Not found")

</code>


TJG


From bart4858 at gmail.com  Thu May 21 09:34:50 2015
From: bart4858 at gmail.com (bartc)
Date: Thu, 21 May 2015 06:34:50 -0700 (PDT)
Subject: Slices time complexity
In-Reply-To: <555d6ad8$0$2763$c3e8da3$76491128@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <mjhi5a$4pa$1@dont-email.me>
 <555d6ad8$0$2763$c3e8da3$76491128@news.astraweb.com>
Message-ID: <755415ac-25c8-40db-b3c2-78111ed25621@googlegroups.com>

On Thursday, 21 May 2015 06:19:39 UTC+1, Steven D'Aprano  wrote:
> On Wednesday 20 May 2015 19:56, Bartc wrote:

> What you *shouldn't* do is implement your own argument convention,

(That's exactly what I did for my static language compiler which
generates x64 code. The Win64 calling convention was far too
complicated, and required 16-byte stack alignment which was another
headache. My private calling convention works fine! But I have to use the
official one for calling foreign (eg. C) functions.

This is at a lower level than the logical or language view of parameter passing, but still, you don't always have to do what everyone else does.)

> Don't do what the Lua community does, which is invent a stupid distinction 
> between "reference types" and "value types" so they too can misuse 
> terminology:

But in the mind of the user, such a distinction can be intuitive. If
I'm passing a 32-bit integer, it is easy to think of the value being
passed. With a 100 million element array, it harder to think of it being
passed by value than by reference.

> This calling convention has been known as pass by sharing (or variations of 
> that, like "pass by object sharing") since it was named by Barbara Liskov in 
> the 1970s, for the language CLU, but the convention itself goes back to Lisp 
> in the 1950s.

I looked it up expecting a new convention to solve all my problems.
But it's just a re-statement of how Python works anyway!

So a mutable object passed to a function can be changed by that function,
so that the caller sees the change. An immutable object can't be
changed in the same way, but only because Python doesn't allow it to
be modified. (But it does allow assignment to the version in the
function.)

Using what is really pass-by-reference for everything is fine, but
I'm having some trouble with making it efficient for small integer
values (I think this is a weak area of Python). Even with my clunky system
of passing descriptors, the equivalent of BINARY_ADD for two small
integers can be reduced to perhaps 10 or 12 machine instructions,
byte-code dispatch *and* double type-dispatch overheads included (using ASM that is; and technically the type-dispatching is by-passed).

If small integers need to be heap-allocated and managed, then it might need a few more. (And then Python will auto-range these to arbitrary precision as needed, another difference.)

-- 
Bartc


From sturla.molden at gmail.com  Thu May 21 10:04:18 2015
From: sturla.molden at gmail.com (Sturla Molden)
Date: Thu, 21 May 2015 14:04:18 +0000 (UTC)
Subject: No ttk in 2.7
References: <87r3qbqlve.fsf@Equus.decebal.nl>
 <CAKJDb-OK3CjC5ytF7y3ALXkW8rJ=rXHrDKx3NHXfuaapC9Y=Rw@mail.gmail.com>
Message-ID: <99001598453909745.789537sturla.molden-gmail.com@news.gmane.org>

Zachary Ware <zachary.ware+pylist at gmail.com> wrote:

> The way I would do it is as follows:
> 
>    try:
>        import tkinter as tk
>        from tkinter import ttk
>    except ImportError:
>        import Tkinter as tk
>        import ttk
> 
> If I may suggest, just write it in Python3 first, then when it does
> what you want tack on whatever you need to make 2.7 happy.  I find it
> easier to do things that way, though you may find that the only thing
> you have to adjust is the imports.

This is a good advice.

And yes, it is easier than most would think.


Sturla



From ian.g.kelly at gmail.com  Thu May 21 10:13:44 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 21 May 2015 08:13:44 -0600
Subject: subprocess.Popen zombie
In-Reply-To: <mjjuc5$1ft$1@r01.glglgl.de>
References: <mailman.170.1432127818.17265.python-list@python.org>
 <87bnhfqmr4.fsf@universite-de-strasbourg.fr.invalid>
 <mailman.177.1432140299.17265.python-list@python.org>
 <mjjuc5$1ft$1@r01.glglgl.de>
Message-ID: <CALwzid=J6f725h_WGmjxoLYENc_2_RYiZpxw3_wG-_Z5T2L9sg@mail.gmail.com>

On May 21, 2015 12:41 AM, "Thomas Rachel" <
nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915 at spamschutz.glglgl.de> wrote:
>
> Am 20.05.2015 um 18:44 schrieb Robin Becker:
>
>> not really, it's just normal to keep event routines short; the routine
>> which beeps is after detection of the cat's entrance into the house and
>> various recognition schemes have pronounced intruder :)
>
>
> You could add a timed "cleanup" routine which .wait()s after a certain
time (250 ms or so).
>
> Or even better, which .poll()s and re-schedules itself if the process
still runs.

Or just:

    Thread(target=p.wait).start()

And then you can forget all about it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150521/0ac85e35/attachment.html>

From invalid at invalid.invalid  Thu May 21 10:14:11 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 21 May 2015 14:14:11 +0000 (UTC)
Subject: Find if a file existing within 1000s of folder/sub-folder -
 each file has a unique presence
References: <9d786d0b-0f25-4446-a730-ff1fe2f6b20d@googlegroups.com>
 <555d9233$0$12913$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mjkp7j$p7k$1@reader1.panix.com>

On 2015-05-21, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:

> import glob
> print(glob.glob("c:/abc/def/ghjmain/features/XYZ/*<filename>"))
>
> Don't use backslashes \ as they have special meaning to Python. Use forward 
> slashes and let Python convert them as needed.

Interesting. I've never heard about this.

When will Python convert them?

-- 
Grant Edwards               grant.b.edwards        Yow! I was born in a
                                  at               Hostess Cupcake factory
                              gmail.com            before the sexual
                                                   revolution!


From rosuav at gmail.com  Thu May 21 10:27:17 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 22 May 2015 00:27:17 +1000
Subject: Find if a file existing within 1000s of folder/sub-folder - each
 file has a unique presence
In-Reply-To: <mjkp7j$p7k$1@reader1.panix.com>
References: <9d786d0b-0f25-4446-a730-ff1fe2f6b20d@googlegroups.com>
 <555d9233$0$12913$c3e8da3$5496439d@news.astraweb.com>
 <mjkp7j$p7k$1@reader1.panix.com>
Message-ID: <CAPTjJmq3-WwiMUkeJzuttqgP==2ioaSudQE9qJFD8+BX9pWRxA@mail.gmail.com>

On Fri, May 22, 2015 at 12:14 AM, Grant Edwards <invalid at invalid.invalid> wrote:
> On 2015-05-21, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
>
>> import glob
>> print(glob.glob("c:/abc/def/ghjmain/features/XYZ/*<filename>"))
>>
>> Don't use backslashes \ as they have special meaning to Python. Use forward
>> slashes and let Python convert them as needed.
>
> Interesting. I've never heard about this.
>
> When will Python convert them?

Actually, it won't ever bother to convert them. The Windows file
system APIs are quite happy to work with forward slashes; it's only
command-line tools (which conventionally use forward slashes to
introduce options), and not all of them, which require backslashes.

You may want to consider explicitly converting them in your own code,
prior to showing a path to a human; but even back in the 1990s, it
wasn't uncommon for cross-platform programs to mix and match - for
instance, if you unzip something into C:\Foo\Bar, you'd get output
like "Inflating C:\Foo\Bar/usr/lib/whatever". I'd have no objections
to a program using forward slashes all the way.

ChrisA


From mail at timgolden.me.uk  Thu May 21 10:32:49 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 21 May 2015 15:32:49 +0100
Subject: Find if a file existing within 1000s of folder/sub-folder - each
 file has a unique presence
In-Reply-To: <mjkp7j$p7k$1@reader1.panix.com>
References: <9d786d0b-0f25-4446-a730-ff1fe2f6b20d@googlegroups.com>
 <555d9233$0$12913$c3e8da3$5496439d@news.astraweb.com>
 <mjkp7j$p7k$1@reader1.panix.com>
Message-ID: <555DEC91.8050300@timgolden.me.uk>

On 21/05/2015 15:14, Grant Edwards wrote:
> On 2015-05-21, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
> 
>> import glob
>> print(glob.glob("c:/abc/def/ghjmain/features/XYZ/*<filename>"))
>>
>> Don't use backslashes \ as they have special meaning to Python. Use forward 
>> slashes and let Python convert them as needed.
> 
> Interesting. I've never heard about this.
> 
> When will Python convert them?
> 

It doesn't: Python passes them on unchanged and Windows accepts them in
all but a few cases.

(Although if any stdlib function were to call, eg, os.path.abspath on a
path before passing it along to Windows then one effect is that the path
is "normalised", ending up with backslashes. So perhaps that does happen
in modules like shutil. Not sure).


TJG


From nimbiotics at gmail.com  Thu May 21 11:54:28 2015
From: nimbiotics at gmail.com (Mario R. Osorio)
Date: Thu, 21 May 2015 08:54:28 -0700 (PDT)
Subject: Best approach to create humongous amount of files
In-Reply-To: <mjiij7$h2l$2@dont-email.me>
References: <mailman.166.1432123945.17265.python-list@python.org>
 <mjiij7$h2l$2@dont-email.me>
Message-ID: <4a0e2c39-ff50-400f-b488-86a6cb37dbd6@googlegroups.com>

On Wednesday, May 20, 2015 at 2:09:59 PM UTC-4, Denis McMahon wrote:
> On Wed, 20 May 2015 17:14:15 +0530, Parul Mogra wrote:
> 
> > Hello everyone,
> > My objective is to create large amount of data files (say a million
> > *.json files), using a pre-existing template file (*.json). Each file
> > would have a unique name, possibly by incorporating time stamp
> > information. The files have to be generated in a folder specified.
> 
> > What is the best strategy to achieve this task, so that the files will
> > be generated in the shortest possible time? Say within an hour.
> 	
> timestamps are normally unixtime in seconds. There are 3600 seconds in an 
> hour. You'll have a hard job creating a million files with timestamp 
> based naming inside of an hour.
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com

I would use a combination of both, timestamp and a serial number, such as:
201505201425440000
201505201425440001
201505201425440002
201505201425440003
201505201425450000
201505201425450001
201505201425460000
.. and so on ..


From __peter__ at web.de  Thu May 21 12:28:19 2015
From: __peter__ at web.de (Peter Otten)
Date: Thu, 21 May 2015 18:28:19 +0200
Subject: Best approach to create humongous amount of files
References: <mailman.166.1432123945.17265.python-list@python.org>
 <mjiij7$h2l$2@dont-email.me>
 <4a0e2c39-ff50-400f-b488-86a6cb37dbd6@googlegroups.com>
Message-ID: <mjl135$ugo$1@ger.gmane.org>

Mario R. Osorio wrote:

> On Wednesday, May 20, 2015 at 2:09:59 PM UTC-4, Denis McMahon wrote:
>> On Wed, 20 May 2015 17:14:15 +0530, Parul Mogra wrote:
>> 
>> > Hello everyone,
>> > My objective is to create large amount of data files (say a million
>> > *.json files), using a pre-existing template file (*.json). Each file
>> > would have a unique name, possibly by incorporating time stamp
>> > information. The files have to be generated in a folder specified.
>> 
>> > What is the best strategy to achieve this task, so that the files will
>> > be generated in the shortest possible time? Say within an hour.
>> 
>> timestamps are normally unixtime in seconds. There are 3600 seconds in an
>> hour. You'll have a hard job creating a million files with timestamp
>> based naming inside of an hour.
>> 
>> --
>> Denis McMahon, denismfmcmahon at gmail.com
> 
> I would use a combination of both, timestamp and a serial number, such as:
> 201505201425440000
> 201505201425440001
> 201505201425440002
> 201505201425440003
> 201505201425450000
> 201505201425450001
> 201505201425460000
> .. and so on ..

Like this?

import time
import itertools
from operator import itemgetter

try:
    from itertools import imap as map
except ImportError:
    pass

INDEX_TEMPLATE = "{}-{:02}-{:02}-{:02}-{:02}-{:02}-{i:03}"


def unique_names(template):
    return (
        template.format(INDEX_TEMPLATE.format(*t, i=i))
        for g in map(itemgetter(1), itertools.groupby(iter(time.gmtime, 
())))
        for i, t in enumerate(g, 1))


if __name__ == "__main__":
    import random

    for name in unique_names("foo-{}.txt"):
        print(name)
        time.sleep(random.random())

I mean, readability counts...



From invalid at invalid.invalid  Thu May 21 12:31:51 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 21 May 2015 16:31:51 +0000 (UTC)
Subject: Find if a file existing within 1000s of folder/sub-folder -
 each file has a unique presence
References: <9d786d0b-0f25-4446-a730-ff1fe2f6b20d@googlegroups.com>
 <555d9233$0$12913$c3e8da3$5496439d@news.astraweb.com>
 <mjkp7j$p7k$1@reader1.panix.com>
 <mailman.200.1432218440.17265.python-list@python.org>
Message-ID: <mjl19n$etm$1@reader1.panix.com>

On 2015-05-21, Chris Angelico <rosuav at gmail.com> wrote:
> On Fri, May 22, 2015 at 12:14 AM, Grant Edwards <invalid at invalid.invalid> wrote:
>> On 2015-05-21, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
>>
>>> import glob
>>> print(glob.glob("c:/abc/def/ghjmain/features/XYZ/*<filename>"))
>>>
>>> Don't use backslashes \ as they have special meaning to Python. Use forward
>>> slashes and let Python convert them as needed.
>>
>> Interesting. I've never heard about this.
>>
>> When will Python convert them?
>
> Actually, it won't ever bother to convert them.

OK, so this isn't some new feature I hadn't heard about due to my
spending most of my time with 2.7. :)

> The Windows file system APIs are quite happy to work with forward
> slashes;

Yep, I knew that -- I have always use forward slashes on Windows (and
DOS before that) when dealing with the file system.

> it's only command-line tools (which conventionally use forward
> slashes to introduce options), and not all of them, which require
> backslashes.

Yup, I was wondering if that was where Python (or its stdlib) would
convert them (which would have surprised me).  Back in the day, you
could change the 'option switch' character from '/' to whatever you
wanted (and as an old Unix guy, I always set it to '-').  Then you
could even use forward slashes on the command line (mostly). But, I
don't think Windows has support that for yonks.

-- 
Grant Edwards               grant.b.edwards        Yow! Please come home with
                                  at               me ... I have Tylenol!!
                              gmail.com            


From skip.montanaro at gmail.com  Thu May 21 12:41:30 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Thu, 21 May 2015 11:41:30 -0500
Subject: Strategies for using cffi with C++ code?
Message-ID: <CANc-5UyK4+jAxZ_WDnEbM7RgAQk=k2uKJs8a=m1phGeb8DfAYQ@mail.gmail.com>

If my reading so far is correct, you can't directly apply cffi to C++
libraries. Everything has to be wrapped in "extern C" declarations, which
means, practically speaking, that you have to provide factory function
wrapper(s) around constructors and methods. I did find a trivial example:

https://gist.github.com/tonyseek/7821993

but that just exposes a simple function to the Python programmer. Does
someone have a pointer to more substantive example of exposing a C++ class
to Python?

Thanks,

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150521/11184aaf/attachment.html>

From python at mrabarnett.plus.com  Thu May 21 12:50:32 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 21 May 2015 17:50:32 +0100
Subject: Slices time complexity
In-Reply-To: <755415ac-25c8-40db-b3c2-78111ed25621@googlegroups.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <mjhi5a$4pa$1@dont-email.me>
 <555d6ad8$0$2763$c3e8da3$76491128@news.astraweb.com>
 <755415ac-25c8-40db-b3c2-78111ed25621@googlegroups.com>
Message-ID: <555E0CD8.3050603@mrabarnett.plus.com>

On 2015-05-21 14:34, bartc wrote:
> On Thursday, 21 May 2015 06:19:39 UTC+1, Steven D'Aprano  wrote:
>> On Wednesday 20 May 2015 19:56, Bartc wrote:
>
>> What you *shouldn't* do is implement your own argument convention,
>
> (That's exactly what I did for my static language compiler which
> generates x64 code. The Win64 calling convention was far too
> complicated, and required 16-byte stack alignment which was another
> headache. My private calling convention works fine! But I have to use
> the official one for calling foreign (eg. C) functions.
>
> This is at a lower level than the logical or language view of
> parameter passing, but still, you don't always have to do what
> everyone else does.)
>
>> Don't do what the Lua community does, which is invent a stupid
>> distinction between "reference types" and "value types" so they too
>> can misuse terminology:
>
> But in the mind of the user, such a distinction can be intuitive. If
> I'm passing a 32-bit integer, it is easy to think of the value being
> passed. With a 100 million element array, it harder to think of it
> being passed by value than by reference.
>
>> This calling convention has been known as pass by sharing (or
>> variations of that, like "pass by object sharing") since it was
>> named by Barbara Liskov in the 1970s, for the language CLU, but the
>> convention itself goes back to Lisp in the 1950s.
>
> I looked it up expecting a new convention to solve all my problems.
> But it's just a re-statement of how Python works anyway!
>
> So a mutable object passed to a function can be changed by that
> function, so that the caller sees the change. An immutable object
> can't be changed in the same way, but only because Python doesn't
> allow it to be modified. (But it does allow assignment to the version
> in the function.)
>
> Using what is really pass-by-reference for everything is fine, but
> I'm having some trouble with making it efficient for small integer
> values (I think this is a weak area of Python). Even with my clunky
> system of passing descriptors, the equivalent of BINARY_ADD for two
> small integers can be reduced to perhaps 10 or 12 machine
> instructions, byte-code dispatch *and* double type-dispatch overheads
> included (using ASM that is; and technically the type-dispatching is
> by-passed).
>
> If small integers need to be heap-allocated and managed, then it
> might need a few more. (And then Python will auto-range these to
> arbitrary precision as needed, another difference.)
>
If memory addresses aren't byte-aligned, you could use the least-
significant bit to indicate whether it's a small integer, i.e. if it's
zero, then it's an address, else shift right by 1 bit arithmetically
for the integer value.


From python at mrabarnett.plus.com  Thu May 21 12:54:01 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 21 May 2015 17:54:01 +0100
Subject: Find if a file existing within 1000s of folder/sub-folder - each
 file has a unique presence
In-Reply-To: <mjl19n$etm$1@reader1.panix.com>
References: <9d786d0b-0f25-4446-a730-ff1fe2f6b20d@googlegroups.com>
 <555d9233$0$12913$c3e8da3$5496439d@news.astraweb.com>
 <mjkp7j$p7k$1@reader1.panix.com>
 <mailman.200.1432218440.17265.python-list@python.org>
 <mjl19n$etm$1@reader1.panix.com>
Message-ID: <555E0DA9.2070508@mrabarnett.plus.com>

On 2015-05-21 17:31, Grant Edwards wrote:
> On 2015-05-21, Chris Angelico <rosuav at gmail.com> wrote:
>> On Fri, May 22, 2015 at 12:14 AM, Grant Edwards <invalid at invalid.invalid> wrote:
>>> On 2015-05-21, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
>>>
>>>> import glob
>>>> print(glob.glob("c:/abc/def/ghjmain/features/XYZ/*<filename>"))
>>>>
>>>> Don't use backslashes \ as they have special meaning to Python. Use forward
>>>> slashes and let Python convert them as needed.
>>>
>>> Interesting. I've never heard about this.
>>>
>>> When will Python convert them?
>>
>> Actually, it won't ever bother to convert them.
>
> OK, so this isn't some new feature I hadn't heard about due to my
> spending most of my time with 2.7. :)
>
>> The Windows file system APIs are quite happy to work with forward
>> slashes;
>
> Yep, I knew that -- I have always use forward slashes on Windows (and
> DOS before that) when dealing with the file system.
>
>> it's only command-line tools (which conventionally use forward
>> slashes to introduce options), and not all of them, which require
>> backslashes.
>
Dialog boxes, however, insist on backslashses.

> Yup, I was wondering if that was where Python (or its stdlib) would
> convert them (which would have surprised me).  Back in the day, you
> could change the 'option switch' character from '/' to whatever you
> wanted (and as an old Unix guy, I always set it to '-').  Then you
> could even use forward slashes on the command line (mostly). But, I
> don't think Windows has support that for yonks.
>



From lac at openend.se  Thu May 21 13:06:16 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 21 May 2015 19:06:16 +0200
Subject: Strategies for using cffi with C++ code?
In-Reply-To: Message from Skip Montanaro <skip.montanaro@gmail.com> of "Thu,
 21 May 2015 11:41:30 -0500."
 <CANc-5UyK4+jAxZ_WDnEbM7RgAQk=k2uKJs8a=m1phGeb8DfAYQ@mail.gmail.com>
References: <CANc-5UyK4+jAxZ_WDnEbM7RgAQk=k2uKJs8a=m1phGeb8DfAYQ@mail.gmail.com>
Message-ID: <201505211706.t4LH6GY0006031@fido.openend.se>

Skip: you may be interested in the new CFFI the pypy project just released
_today_.

http://morepypy.blogspot.se/search?updated-min=2015-01-01T00:00:00%2B01:00&updated-max=2016-01-01T00:00:00%2B01:00&max-results=11

In any case we have docs here:
http://cffi.readthedocs.org/en/latest/

Laura


From steve+comp.lang.python at pearwood.info  Thu May 21 13:16:19 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Fri, 22 May 2015 03:16:19 +1000
Subject: Slices time complexity
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <mjhi5a$4pa$1@dont-email.me>
 <555d6ad8$0$2763$c3e8da3$76491128@news.astraweb.com>
 <755415ac-25c8-40db-b3c2-78111ed25621@googlegroups.com>
Message-ID: <555e12e4$0$12999$c3e8da3$5496439d@news.astraweb.com>

On Thu, 21 May 2015 11:34 pm, bartc wrote:

> On Thursday, 21 May 2015 06:19:39 UTC+1, Steven D'Aprano  wrote:
>> On Wednesday 20 May 2015 19:56, Bartc wrote:
> 
>> What you *shouldn't* do is implement your own argument convention,
> 
> (That's exactly what I did for my static language compiler which
> generates x64 code. The Win64 calling convention was far too
> complicated, and required 16-byte stack alignment which was another
> headache. My private calling convention works fine! But I have to use the
> official one for calling foreign (eg. C) functions.

You deleted the rest of my sentence, where I said and then call it by a
standard name that it is not. If you really need your own invented calling
conventions, then I suppose you can do so, but there's really only so many
ways you can do it.


> This is at a lower level than the logical or language view of parameter
> passing, but still, you don't always have to do what everyone else does.)
> 
>> Don't do what the Lua community does, which is invent a stupid
>> distinction between "reference types" and "value types" so they too can
>> misuse terminology:
> 
> But in the mind of the user, such a distinction can be intuitive.

But it's *wrong*. When your intuition gives you the wrong answer, you need
to stop relying on "gut feelings" and *learn some facts*.

Lua has a much smaller set of types than Python, and I'm not an expert on
it, so it may be that there are so few types that you can pass around in
Lua that (wrong or not) it doesn't matter. But I doubt it.


> If 
> I'm passing a 32-bit integer, it is easy to think of the value being
> passed. With a 100 million element array, it harder to think of it being
> passed by value than by reference.

Huh? This doesn't make any sense. The value either *is* or *isn't* passed by
value. That's a property of the language, it doesn't depend on what you
think of it.

If the value is passed by value, then regardless of whether it is a 32-bit
integer or an 1GB array, the value *will be copied*. If it is not copied,
then it isn't pass by value.


>> This calling convention has been known as pass by sharing (or variations
>> of that, like "pass by object sharing") since it was named by Barbara
>> Liskov in the 1970s, for the language CLU, but the convention itself goes
>> back to Lisp in the 1950s.
> 
> I looked it up expecting a new convention to solve all my problems.
> But it's just a re-statement of how Python works anyway!

Exactly.


> So a mutable object passed to a function can be changed by that function,
> so that the caller sees the change. An immutable object can't be
> changed in the same way, but only because Python doesn't allow it to
> be modified. (But it does allow assignment to the version in the
> function.)

I don't understand what your last sentence there means. What do you
mean "the version in the function"? What do you mean by "allow assignment"?

There are three common calling conventions: by value, reference and sharing.
If you have a function with a parameter:

function func(arg)

and then call it with a named variable as argument:

x := something
result := func(x)


*all three conventions* allow assignment to arg inside func. It would be a
pretty strange language which didn't! But the *effect* of such assignment
may vary, depending on the calling convention.

In call-by-value and call-by-sharing, the parameter "arg" is local to func,
so any assignment to arg changes only the local scope, *not* the outer
scope; consequently, the global variable x does not get assigned. arg and x
merely have the same value, they aren't the same variable.

(The difference between the two is that call-by-value *copies* the value, so
in-place modifications to arg don't affect the original x; call-by-sharing
doesn't copy the value, so in-place modifications to arg does affect the
original x.)

In call-by-reference, the compiler treats the parameter "arg" as just
another way of saying "x". So assigning to arg is the same as assigning to
x: for the duration of the function call, arg and x are two names for the
same variable.

(The usual implementation of this is for the compiler to pass a pointer to
x, and then automatically dereference that pointer whenever it refers to
arg, regardless of whether arg is on the left or right of an assignment. C
does not have call-by-reference, but you can fake it by manually doing the
pointer dereferencing yourself.)

Call-by-name is a fourth technique, not common but nevertheless historically
significant. "Call by name" is a bit of misnomer, because it's really more
of call by expression, where the expression is sometimes a single name.

In call-by-name, when you call func(some expression), "some expression" is
recorded in a lightweight anonymous function (called a thunk). Then, inside
the body of func, every time you refer so the parameter name arg, that
expression is re-evaluated.



> Using what is really pass-by-reference for everything is fine, 

I'm really sure it isn't fine. You could use pass-by-reference for
everything, but if you do, you will surprise a lot of people:

def func(arg):
   arg = 1

x = 23
func(x)
assert x == 23  # this will fail

Pass-by-reference does NOT just mean "pass a reference". There's more to it
than that. This is what people don't get.



> but 
> I'm having some trouble with making it efficient for small integer
> values (I think this is a weak area of Python). 

I can imagine that pass-by-reference would be a bit tricky to implement in a
dict-based namespace language, but that has nothing to do with the size of
the value.


> Even with my clunky system 
> of passing descriptors, the equivalent of BINARY_ADD for two small
> integers can be reduced to perhaps 10 or 12 machine instructions,
> byte-code dispatch *and* double type-dispatch overheads included (using
> ASM that is; and technically the type-dispatching is by-passed).
> 
> If small integers need to be heap-allocated and managed, then it might
> need a few more. (And then Python will auto-range these to arbitrary
> precision as needed, another difference.)

Are you talking about Python or your custom language?


> 

-- 
Steven



From skip.montanaro at gmail.com  Thu May 21 13:31:54 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Thu, 21 May 2015 12:31:54 -0500
Subject: Strategies for using cffi with C++ code?
In-Reply-To: <201505211706.t4LH6GY0006031@fido.openend.se>
References: <skip.montanaro@gmail.com>
 <CANc-5UyK4+jAxZ_WDnEbM7RgAQk=k2uKJs8a=m1phGeb8DfAYQ@mail.gmail.com>
 <201505211706.t4LH6GY0006031@fido.openend.se>
Message-ID: <CANc-5UyBC-vw_+Og-1gea9YXubpATezsOmNf58AKKgb0swBP9A@mail.gmail.com>

On Thu, May 21, 2015 at 12:06 PM, Laura Creighton <lac at openend.se> wrote:

> Skip: you may be interested in the new CFFI the pypy project just released
> _today_.
>
>
> http://morepypy.blogspot.se/search?updated-min=2015-01-01T00:00:00%2B01:00&updated-max=
> 2016-01-01T00:00:00%2B01:00&max-results=11
>

Laura,

Sure, and I was prompted to write after seeing a mention on Planet Python.
My searches at at point turned up nothing. I see no mention of C++ in
either the blog post or the referenced cffi documentation. I'm sure I could
muddle my way through, but I quite likely would do things "badly" (for some
definition of that word), and I also figure there must be some established
best practices for exposing C++ classes to Python.

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150521/29aa810b/attachment.html>

From bart4858 at gmail.com  Thu May 21 15:48:17 2015
From: bart4858 at gmail.com (bart4858 at gmail.com)
Date: Thu, 21 May 2015 12:48:17 -0700 (PDT)
Subject: Slices time complexity
In-Reply-To: <555e12e4$0$12999$c3e8da3$5496439d@news.astraweb.com>
References: <9ceklad15llnv3npejq9iuh91soci8aeqo@4ax.com>
 <mailman.114.1431978628.17265.python-list@python.org>
 <7ea01590-a559-46e5-abf5-29622e39aae7@googlegroups.com>
 <555ac697$0$12910$c3e8da3$5496439d@news.astraweb.com>
 <862693ca-42cf-4f5b-ac12-133e43e7606b@googlegroups.com>
 <878uclf3bt.fsf@elektro.pacujo.net>
 <555af171$0$12995$c3e8da3$5496439d@news.astraweb.com>
 <874mn9ezaw.fsf@elektro.pacujo.net>
 <mailman.124.1432025439.17265.python-list@python.org>
 <555b2fa9$0$12996$c3e8da3$5496439d@news.astraweb.com>
 <cs22vbFnd69U1@mid.individual.net>
 <mailman.156.1432082566.17265.python-list@python.org>
 <87oalfooz7.fsf@elektro.pacujo.net>
 <92992653-d511-4e33-a21d-541eeda4799b@googlegroups.com>
 <555c225b$0$2769$c3e8da3$76491128@news.astraweb.com>
 <mjhi5a$4pa$1@dont-email.me>
 <555d6ad8$0$2763$c3e8da3$76491128@news.astraweb.com>
 <755415ac-25c8-40db-b3c2-78111ed25621@googlegroups.com>
 <555e12e4$0$12999$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <6e6c657e-b48f-44f5-a57a-324c40919e39@googlegroups.com>

On Thursday, 21 May 2015 18:16:33 UTC+1, Steven D'Aprano  wrote:
> On Thu, 21 May 2015 11:34 pm, bartc wrote:
> > On Thursday, 21 May 2015 06:19:39 UTC+1, Steven D'Aprano  wrote:

> > Using what is really pass-by-reference for everything is fine, 
> 
> I'm really sure it isn't fine. You could use pass-by-reference for
> everything, but if you do, you will surprise a lot of people:

Yes, I using the term informally (as in, 'reference count'). I meant
how CPython appears to pass data around (as Py_Object* types).

> > If small integers need to be heap-allocated and managed, then it might
> > need a few more. (And then Python will auto-range these to arbitrary
> > precision as needed, another difference.)
> 
> Are you talking about Python or your custom language?

My language, and the problems there might be in adopting the CPython
style of passing pointers around. (Another problem I've just realised
is that everything in my language is mutable: I can even change the
the bits of a small integer in-place. That will be interesting to
solve.)

(I came into this trying to investigate why CPython was so slow, I may
end up wondering how it manages to be as fast as it is!)


-- 
Bartc


From sanluca78 at gmail.com  Thu May 21 16:05:39 2015
From: sanluca78 at gmail.com (Luca Sanna)
Date: Thu, 21 May 2015 13:05:39 -0700 (PDT)
Subject: python paho mqtt thread
Message-ID: <bdca3f19-6740-4ed8-bb98-057ac85d6819@googlegroups.com>

hi ,
I try to run this code MQTT but I can not read the messages in the topic , can you help me ?
thanks

class MyMQTTClass(Thread):
    #def __init__(self, clientid=None):
    clientid=None
    _mqttc = mqtt.Client(clientid)
    #_mqttc.on_message = mqtt_on_message
    #_mqttc.on_connect = mqtt_on_connect
    #_mqttc.on_publish = mqtt_on_publish
    #_mqttc.on_subscribe = mqtt_on_subscribe
    db=Mydata()
        
    def mqtt_on_connect(self, mqttc, obj, flags, rc):
        #print("rc: "+str(rc))
        pass
        
    def mqtt_on_message(self, mqttc, obj, msg):
        #print(msg.topic+" "+str(msg.qos)+" "+str(msg.payload))
        if msg.topic=="home/soggiorno/luce":
            if msg.paylod=="ON":
                self.db.update_configure('power',1,1)
            else:
                self.db.update_configure('power',0,1)
                
    def mqtt_on_publish(self, mqttc, obj, mid):
        #print("mid: "+str(mid))
        pass
        
    def mqtt_on_subscribe(self, mqttc, obj, mid, granted_qos):
        #print("Subscribed: "+str(mid)+" "+str(granted_qos))
        pass
        
    def mqtt_on_log(self, mqttc, obj, level, string):
        #print(string)
        pass
        
    def run(self):
        logCritical("run")
        self._mqttc.on_message = self.mqtt_on_message
        rc=0
        while rc==0:
        
            self._mqttc.connect("192.168.1.60", 1883, 60)
            self._mqttc.subscribe("home/soggiorno/temperatura")
            self._mqttc.subscribe("home/soggiorno/umidita")
            self._mqttc.subscribe("home/soggiorno/luce")
            self.mqtt_on_message
            rc=self._mqttc.loop()


From johnhpote at o2.co.uk  Thu May 21 18:20:52 2015
From: johnhpote at o2.co.uk (John Pote)
Date: Thu, 21 May 2015 23:20:52 +0100
Subject: How to do integers to binary lists and back
In-Reply-To: <5522439B-AEFA-400D-9B6F-00613528381F@gmail.com>
References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com>
 <CAPTjJmpEHQV2PpaZvszuTGn0_GNg0sWV6aLenbQsZrSXCy2YKQ@mail.gmail.com>
 <mailman.18957.1424524392.18130.python-list@python.org>
 <mcargl$h3k$1@reader1.panix.com>
 <mailman.19008.1424611304.18130.python-list@python.org>
 <cl2g0vFfiq9U1@mid.individual.net>
 <mailman.19117.1424755795.18130.python-list@python.org>
 <cl486mFu4l9U1@mid.individual.net>
 <mailman.19181.1424868605.18130.python-list@python.org>
 <cl7pmjF7595U1@mid.individual.net>
 <F521BFFE-506F-4B58-9A15-20D91A019BD0@gmail.com>
 <CALwzidm3AngMF9aG-gLpYur6CfaADM0YXGU=p0=dv-qqYXmmFA@mail.gmail.com>
 <8A3659BC-9100-4A3A-9117-47227B3D290B@gmail.com>
 <CALwzidmgj1y==3iv8Zub+aHyFR3P3okeLBRKu+R92914mDXfqQ@mail.gmail.com>
 <5522439B-AEFA-400D-9B6F-00613528381F@gmail.com>
Message-ID: <555E5A44.20006@o2.co.uk>

Hi everyone.
I recently had the problem of converting from an integer to its 
representation as a list of binary bits, each bit being an integer 1 or 
0, and vice versa. E.G.
0x53
becomes
[ 0, 1, 0, 1, 0, 0, 1, 1 ]

This I wanted to do for integers of many tens, if not hundreds, of bits. 
Python very nicely expands integers to any size required, great feature.

Just wondered if there was a neat way of doing this without resorting to 
a bit bashing loop.

Looking forward to some interesting answers,
John




From ben+python at benfinney.id.au  Thu May 21 18:31:12 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 22 May 2015 08:31:12 +1000
Subject: How to do integers to binary lists and back
References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com>
 <CAPTjJmpEHQV2PpaZvszuTGn0_GNg0sWV6aLenbQsZrSXCy2YKQ@mail.gmail.com>
 <mailman.18957.1424524392.18130.python-list@python.org>
 <mcargl$h3k$1@reader1.panix.com>
 <mailman.19008.1424611304.18130.python-list@python.org>
 <cl2g0vFfiq9U1@mid.individual.net>
 <mailman.19117.1424755795.18130.python-list@python.org>
 <cl486mFu4l9U1@mid.individual.net>
 <mailman.19181.1424868605.18130.python-list@python.org>
 <cl7pmjF7595U1@mid.individual.net>
 <F521BFFE-506F-4B58-9A15-20D91A019BD0@gmail.com>
 <CALwzidm3AngMF9aG-gLpYur6CfaADM0YXGU=p0=dv-qqYXmmFA@mail.gmail.com>
 <8A3659BC-9100-4A3A-9117-47227B3D290B@gmail.com>
 <CALwzidmgj1y==3iv8Zub+aHyFR3P3okeLBRKu+R92914mDXfqQ@mail.gmail.com>
 <5522439B-AEFA-400D-9B6F-00613528381F@gmail.com>
 <555E5A44.20006@o2.co.uk>
Message-ID: <85lhghy34f.fsf@benfinney.id.au>

John Pote <johnhpote at o2.co.uk> writes:

> I recently had the problem of converting from an integer to its
> representation as a list of binary bits, each bit being an integer 1
> or 0, and vice versa.

Is this a homework assignment?

> E.G.
> 0x53
> becomes
> [ 0, 1, 0, 1, 0, 0, 1, 1 ]

>>> foo = 4567

>>> foo
4567
>>> "{foo:d}".format(foo=foo)
'4567'
>>> "{foo:b}".format(foo=foo)
'1000111010111'

>>> foo_binary_text = "{foo:b}".format(foo=foo)
>>> foo_binary_digits = list(foo_binary_text)
>>> foo_binary_digits
['1', '0', '0', '0', '1', '1', '1', '0', '1', '0', '1', '1', '1']

> Just wondered if there was a neat way of doing this without resorting
> to a bit bashing loop.

Python's string formatting and sequence types are quite powerful.

-- 
 \       ?As far as the laws of mathematics refer to reality, they are |
  `\    not certain, and as far as they are certain, they do not refer |
_o__)                              to reality.? ?Albert Einstein, 1983 |
Ben Finney



From python at mrabarnett.plus.com  Thu May 21 18:31:52 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 21 May 2015 23:31:52 +0100
Subject: How to do integers to binary lists and back
In-Reply-To: <555E5A44.20006@o2.co.uk>
References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com>
 <CAPTjJmpEHQV2PpaZvszuTGn0_GNg0sWV6aLenbQsZrSXCy2YKQ@mail.gmail.com>
 <mailman.18957.1424524392.18130.python-list@python.org>
 <mcargl$h3k$1@reader1.panix.com>
 <mailman.19008.1424611304.18130.python-list@python.org>
 <cl2g0vFfiq9U1@mid.individual.net>
 <mailman.19117.1424755795.18130.python-list@python.org>
 <cl486mFu4l9U1@mid.individual.net>
 <mailman.19181.1424868605.18130.python-list@python.org>
 <cl7pmjF7595U1@mid.individual.net>
 <F521BFFE-506F-4B58-9A15-20D91A019BD0@gmail.com>
 <CALwzidm3AngMF9aG-gLpYur6CfaADM0YXGU=p0=dv-qqYXmmFA@mail.gmail.com>
 <8A3659BC-9100-4A3A-9117-47227B3D290B@gmail.com>
 <CALwzidmgj1y==3iv8Zub+aHyFR3P3okeLBRKu+R92914mDXfqQ@mail.gmail.com>
 <5522439B-AEFA-400D-9B6F-00613528381F@gmail.com> <555E5A44.20006@o2.co.uk>
Message-ID: <555E5CD8.4090704@mrabarnett.plus.com>

On 2015-05-21 23:20, John Pote wrote:
> Hi everyone.
> I recently had the problem of converting from an integer to its
> representation as a list of binary bits, each bit being an integer 1 or
> 0, and vice versa. E.G.
> 0x53
> becomes
> [ 0, 1, 0, 1, 0, 0, 1, 1 ]
>
> This I wanted to do for integers of many tens, if not hundreds, of bits.
> Python very nicely expands integers to any size required, great feature.
>
> Just wondered if there was a neat way of doing this without resorting to
> a bit bashing loop.
>
> Looking forward to some interesting answers,
> John
>
>
I don't know how efficient you want it to be, but:

 >>> number = 0x53
 >>> bin(number)
'0b1010011'
 >>> bin(number)[2 : ]
'1010011'
 >>> list(map(int, bin(number)[2 : ]))
[1, 0, 1, 0, 0, 1, 1]



From ian.g.kelly at gmail.com  Thu May 21 18:38:30 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 21 May 2015 16:38:30 -0600
Subject: How to do integers to binary lists and back
In-Reply-To: <85lhghy34f.fsf@benfinney.id.au>
References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com>
 <CAPTjJmpEHQV2PpaZvszuTGn0_GNg0sWV6aLenbQsZrSXCy2YKQ@mail.gmail.com>
 <mailman.18957.1424524392.18130.python-list@python.org>
 <mcargl$h3k$1@reader1.panix.com>
 <mailman.19008.1424611304.18130.python-list@python.org>
 <cl2g0vFfiq9U1@mid.individual.net>
 <mailman.19117.1424755795.18130.python-list@python.org>
 <cl486mFu4l9U1@mid.individual.net>
 <mailman.19181.1424868605.18130.python-list@python.org>
 <cl7pmjF7595U1@mid.individual.net>
 <F521BFFE-506F-4B58-9A15-20D91A019BD0@gmail.com>
 <CALwzidm3AngMF9aG-gLpYur6CfaADM0YXGU=p0=dv-qqYXmmFA@mail.gmail.com>
 <8A3659BC-9100-4A3A-9117-47227B3D290B@gmail.com>
 <CALwzidmgj1y==3iv8Zub+aHyFR3P3okeLBRKu+R92914mDXfqQ@mail.gmail.com>
 <5522439B-AEFA-400D-9B6F-00613528381F@gmail.com> <555E5A44.20006@o2.co.uk>
 <85lhghy34f.fsf@benfinney.id.au>
Message-ID: <CALwzidn=J3Q-OhpNPSPO1zNCHh197=vzuudBnHGx0tapJXgQdA@mail.gmail.com>

On Thu, May 21, 2015 at 4:31 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
>>>> "{foo:d}".format(foo=foo)
> '4567'
>>>> "{foo:b}".format(foo=foo)
> '1000111010111'

Which since there's nothing else in the format string can be simplified to:

>>> format(foo, "b")
'1000111010111'


From dreamingforward at gmail.com  Thu May 21 22:14:49 2015
From: dreamingforward at gmail.com (zipher)
Date: Thu, 21 May 2015 19:14:49 -0700 (PDT)
Subject: different types of inheritence...
Message-ID: <544f53cb-e404-41a4-aa2e-0db27c845546@googlegroups.com>

Still considering distinguishing between different types of inheritance.  

Apart from object composition or mix-in style, I want to illustrate something regarding the "arrow" of inheritance.

class super_dict(dict):

    def __init__(self, init={}, default_value=0, collision_function=None):
       *expands what dict can do*

    def get_default(self):  #stupid method to illustrate a point
       return self._default_value

class specialized_dict(dict):

    def update(self, other):
        *change the behavior of how updates work*

    def setdefault(self, key, value):
        if key=sentinel:
            self[key]=0
        else:
            self[key]=value

These look like the standard is-a inheritance, but they are very different.  

The first goes upwards, making a super-class, the other drills downwards and makes dict more specialized.  The former expands on the capability and the API, the latter keeps the exact API but modifies the method behaviors.  The former becomes a more general type, the latter becomes a more specific type.

Steven D'Aprano was arguing a couple of years ago that there is no difference, one can simply turn the inheritance diagram upside-down.  But here it can be seen that that's not an option

Anyone else see the significance?  Sorry to be a PITA...

Mark


From roy at panix.com  Thu May 21 22:36:56 2015
From: roy at panix.com (Roy Smith)
Date: Thu, 21 May 2015 22:36:56 -0400
Subject: textwrap.wrap() breaks non-breaking spaces
References: <mjaqqd$t0m$1@news.albasani.net>
Message-ID: <roy-4ADE41.22362621052015@news.panix.com>

In article <mjaqqd$t0m$1 at news.albasani.net>,
 Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:

> so that textwrap.wrap() breks non-breaking spaces, is this a bug or
> intended behavior?

I opened http://bugs.python.org/issue16623 on this a couple of years 
ago.  Looks like it was being worked (http://bugs.python.org/issue20491) 
but got stalled in the testing stage.


From cfkaran2 at gmail.com  Thu May 21 23:42:31 2015
From: cfkaran2 at gmail.com (Cem Karan)
Date: Thu, 21 May 2015 23:42:31 -0400
Subject: Best approach to create humongous amount of files
In-Reply-To: <CAPkZ3MS5SiGH9OCe9RSTmakF681O+qM572y49FuDBmBix=aiFg@mail.gmail.com>
References: <CAPkZ3MS5SiGH9OCe9RSTmakF681O+qM572y49FuDBmBix=aiFg@mail.gmail.com>
Message-ID: <D78069B2-4DC1-4B42-A357-1125D8C353A5@gmail.com>


On May 20, 2015, at 7:44 AM, Parul Mogra <scoria.799 at gmail.com> wrote:

> Hello everyone,
> My objective is to create large amount of data files (say a million *.json files), using a pre-existing template file (*.json). Each file would have a unique name, possibly by incorporating time stamp information. The files have to be generated in a folder specified.
> 
> What is the best strategy to achieve this task, so that the files will be generated in the shortest possible time? Say within an hour.

If you absolutely don't care about the name, then something like the following will work:

    import uuid
    for counter in range(1000000):
        with open(uuid.uuid1().hex.upper() + ".json", "w") as f:
            f.write(templateString)

where templateString is the template you want to write to each file.  The only problem is that the files won't be in any particular order; they'll just be uniquely named.  As a test, I ran the code above, but I killed the loop after about 10 minutes, at which point about 500,000 files were created.  Note that my laptop is about 6 years old, so you might get better performance on your machine.

Thanks,
Cem Karan

From Cecil at decebal.nl  Fri May 22 01:20:23 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 22 May 2015 07:20:23 +0200
Subject: PYTHONPATH when calling from ipython
Message-ID: <874mn5qjc8.fsf@Equus.decebal.nl>

I am looking into using ipython instead of bash. But when I call a
python program from ipython PYTHONPATH is not set. So pythonscripts
that need a module through PYTHONPATH will not work.

I could do something like:
    !PYTHONPATH=~/Python/PythonLibrary python2 ?

But I find that a little bit cumbersome. Is there a better way to do
it?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From sreenath.cg at gmail.com  Fri May 22 02:01:12 2015
From: sreenath.cg at gmail.com (Sreenath Nair)
Date: Fri, 22 May 2015 11:31:12 +0530
Subject: Cx_freeze centOS module import error
Message-ID: <BLU402-EAS2387E99A041434F7B59E5BEE3C00@phx.gbl>

I'm trying to build a script into a binary using cx_freeze. Problem is that once the binary is built then it starts throwing ImportError....Invoking the source via shell works fine. Can someone clarify why this behavior when freezing the source?

Thank you.

Sent from my Windows Phone
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150522/3ebb3c2a/attachment.html>

From priisdk at gmail.com  Fri May 22 02:27:44 2015
From: priisdk at gmail.com (Poul Riis)
Date: Thu, 21 May 2015 23:27:44 -0700 (PDT)
Subject: guiqwt
Message-ID: <cbcae216-f580-48fd-a8b1-2f34cc30a80f@googlegroups.com>

How can I control the size of the window in the example below?

Poul Riis



import numpy as np
from guiqwt import pyplot as plt_

x = np.arange(0,20,0.1)
y = np.cos(x)
plt_.figure(1)
plt_.plot(x,y,'r-')
plt_.show()


From Cecil at decebal.nl  Fri May 22 03:59:02 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 22 May 2015 09:59:02 +0200
Subject: Camelot a good tool for me
Message-ID: <87zj4xoxfd.fsf@Equus.decebal.nl>

I want to learn a lot of things. For example writing database and
graphical applications. For database I decided on SQLAlchemy and GUI
on Tkinter. In principal I want to write Python 3 applications.

I came across Camelot. As I understand it, this is something to write
graphical database applications fast. It works with Qt, but that
should not be a big problem. It is just to get me started. But it
seems only to work with 2.7 and not 3. Is this true?

Would Camelot be a good tool to get me started, or can I better bite
the bullet and just start with Tkinter and SQLAlchemy?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From dan at tombstonezero.net  Fri May 22 08:38:29 2015
From: dan at tombstonezero.net (Dan Sommers)
Date: Fri, 22 May 2015 12:38:29 +0000 (UTC)
Subject: Camelot a good tool for me
References: <87zj4xoxfd.fsf@Equus.decebal.nl>
Message-ID: <mjn804$4pe$1@dont-email.me>

On Fri, 22 May 2015 09:59:02 +0200, Cecil Westerhof wrote:

> Would Camelot be a good tool to get me started, or can I better bite
> the bullet and just start with Tkinter and SQLAlchemy?

Bite the bullet and learn SQL.

SQLAlchemy -> Database :: Python -> Assembly Language.

HTH,
Dan


From ben+python at benfinney.id.au  Fri May 22 09:02:25 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 22 May 2015 23:02:25 +1000
Subject: Camelot a good tool for me
References: <87zj4xoxfd.fsf@Equus.decebal.nl> <mjn804$4pe$1@dont-email.me>
Message-ID: <85bnhcydcu.fsf@benfinney.id.au>

Dan Sommers <dan at tombstonezero.net> writes:

> On Fri, 22 May 2015 09:59:02 +0200, Cecil Westerhof wrote:
>
> > Would Camelot be a good tool to get me started, or can I better bite
> > the bullet and just start with Tkinter and SQLAlchemy?
>
> Bite the bullet and learn SQL.

Good advice in the long term. However, one can't learn *everything* at
once.

Cecil, use SQLAlchemy and you will have a firm foundation that will not
hold you back from learning SQL at some future point; on the other hand,
you will be using a library that makes it *much* more sensible to work
with database structures and queries from yuor Python code.

The same is not necessarily true of a lot of database APIs; they will
frequently simplify to the point of obscuring your understanding of the
database, and learning SQL later is hindered to that extent. SQLAlchemy,
though, does not have that problem.

> SQLAlchemy -> Database :: Python -> Assembly Language.

I think you're making my point for me :-)

-- 
 \       ?Philosophy is questions that may never be answered. Religion |
  `\              is answers that may never be questioned.? ?anonymous |
_o__)                                                                  |
Ben Finney



From lac at openend.se  Fri May 22 09:03:00 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 22 May 2015 15:03:00 +0200
Subject: Camelot a good tool for me
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Fri, 22 May 2015 09:59:02 +0200." <87zj4xoxfd.fsf@Equus.decebal.nl>
References: <87zj4xoxfd.fsf@Equus.decebal.nl>
Message-ID: <201505221303.t4MD301H032748@fido.openend.se>

I don't know anything about Camelot.  Tkinter produces widgets that are
in no way as pretty graphically as is expected nowadays -- or indeed for
at least 15 years.  If this matters to you -- or if you are building for
customers if it matters to them, then Tkinter is not a good choice.

Widget libraries are more different than they are similar. So while
learning one can give you some very basic ideas about how widgets work,
and what a callback is, etc -- it doesn't smooth the learning curve
for learning what it is you eventually want to use all that much.
If you already know where you are heading for, then I would just start
learning that.

That said, most of the code I write for me, has, for the longest time
used Tkinter.  I don't care all that much about the ugliness, and I
can write them quite quickly.  How much this has to do with Tkinter
programs being smaller than similar programs in other systems -- some
gui toolkits are _really_ _really_ verbose -- and how
much this has to do with the fact that I am familar with Tkinter I do
not know.

A final concern is where would you like to run these programs when you
are done writing them.  These days, I want most of the new things I
write for me to run on my 7 inch android tablet, and Tkinter doesn't
run there.  About 6 months ago I started learning kivy and am
using this for widgets now -- and converting some of my old programs
to use kivy. Kivy works with both Python 2 and Python 3, runs on
the desktop as well as in mobile devices, and produces the prettiest
widgets you could ask for.  It's under active development, and comes
with a huge directory of examples showing how to use the various
widgets.

So if you are still shopping for a widget kit, it is worth a look.

Laura




From Cecil at decebal.nl  Fri May 22 09:11:03 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 22 May 2015 15:11:03 +0200
Subject: Camelot a good tool for me
References: <87zj4xoxfd.fsf@Equus.decebal.nl> <mjn804$4pe$1@dont-email.me>
Message-ID: <87siaopxjs.fsf@Equus.decebal.nl>

Op Friday 22 May 2015 14:38 CEST schreef Dan Sommers:

> On Fri, 22 May 2015 09:59:02 +0200, Cecil Westerhof wrote:
>
>> Would Camelot be a good tool to get me started, or can I better
>> bite the bullet and just start with Tkinter and SQLAlchemy?
>
> Bite the bullet and learn SQL.

I know SQL. Until now I almost always did everything with SQL. The
problem is that you are expected not to use it. With Python you are
supposed to work with SQLAlchemy and with Java Hibernate.


> SQLAlchemy -> Database :: Python -> Assembly Language.

If I follow your logic, I should stop working with Python and start
using Assembly Language. It is fun, but I prefer other languages.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From mail at timgolden.me.uk  Fri May 22 09:29:06 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Fri, 22 May 2015 14:29:06 +0100
Subject: Camelot a good tool for me
In-Reply-To: <87siaopxjs.fsf@Equus.decebal.nl>
References: <87zj4xoxfd.fsf@Equus.decebal.nl> <mjn804$4pe$1@dont-email.me>
 <87siaopxjs.fsf@Equus.decebal.nl>
Message-ID: <555F2F22.5040508@timgolden.me.uk>

On 22/05/2015 14:11, Cecil Westerhof wrote:
> Op Friday 22 May 2015 14:38 CEST schreef Dan Sommers:
> 
>> On Fri, 22 May 2015 09:59:02 +0200, Cecil Westerhof wrote:
>>
>>> Would Camelot be a good tool to get me started, or can I better
>>> bite the bullet and just start with Tkinter and SQLAlchemy?
>>
>> Bite the bullet and learn SQL.
> 
> I know SQL. Until now I almost always did everything with SQL. The
> problem is that you are expected not to use it. With Python you are
> supposed to work with SQLAlchemy and with Java Hibernate.

Not at all. You're supposed to work with whatever is most effective for
yourself and any collaborators you may for the project you're working
on. (And by "effective" I include "giving the most satisfaction").

I use raw SQL most of the time (with a very lightweight wrapper), and
very occasionally I use SQLAlchemy where I find myself re-implementing
it poorly (to coin a phrase). But that's me. And I've been a
professional database developer for 25 years, most of it having nothing
to do with Python.

SQLAlchemy's great; it's worth taking the time to learn how to use it.
But certainly don't feel you *have* to.


TJG



From lac at openend.se  Fri May 22 09:35:39 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 22 May 2015 15:35:39 +0200
Subject: Cx_freeze centOS module import error
In-Reply-To: Message from Sreenath Nair <sreenath.cg@gmail.com> of "Fri,
 22 May 2015 11:31:12 +0530." <BLU402-EAS2387E99A041434F7B59E5BEE3C00@phx.gbl>
References: <BLU402-EAS2387E99A041434F7B59E5BEE3C00@phx.gbl>
Message-ID: <201505221335.t4MDZdkm001265@fido.openend.se>

In a message of Fri, 22 May 2015 11:31:12 +0530, Sreenath Nair writes:
>I'm trying to build a script into a binary using cx_freeze. Problem is that once the binary is built then it starts throwing ImportError....Invoking the source via shell works fine. Can someone clarify why this behavior when freezing the source?
>
>Thank you.
>
>Sent from my Windows Phone
>-- 
>https://mail.python.org/mailman/listinfo/python-list

If it cannot import any modules, then chances are you have not
asked it to.  Post your setup.py, because there is more than one way to
do this.

But sometimes, especially with conditional imports, it just
doesn't get it correctly.

cx_freeze --include-modules <the one it forgot>
will force it to include it.

Laura


From lac at openend.se  Fri May 22 09:57:42 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 22 May 2015 15:57:42 +0200
Subject: Camelot a good tool for me
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Fri, 22 May 2015 15:11:03 +0200." <87siaopxjs.fsf@Equus.decebal.nl>
References: <87zj4xoxfd.fsf@Equus.decebal.nl>
 <mjn804$4pe$1@dont-email.me><87siaopxjs.fsf@Equus.decebal.nl>
Message-ID: <201505221357.t4MDvgIV001904@fido.openend.se>

In a message of Fri, 22 May 2015 15:11:03 +0200, Cecil Westerhof writes:
>I know SQL. Until now I almost always did everything with SQL. The
>problem is that you are expected not to use it. With Python you are
>supposed to work with SQLAlchemy and with Java Hibernate.

I think you have got the wrong idea.  Lots of Python programmers use
SQLAlchemy but there is no 'you should use this' about it.  In my
corner of the world, everybody uses SQL.  The great buring questions
are 'MySQL vs PostgreSQL?' and 'Should you ever use SQLite?'
The people who have tried SQLAlchemy really didn't like it, and of course
the people who haven't tried it do what their friends do, as usual.

If you already know SQL get yourself a MySQL server, a SQLite or PostGreSQL
server, depending on what is more like what you are used to.  There are
tons of online tutorials on how to use these.  And if you already know SQL
things will be pretty straight forward for you.

>Cecil Westerhof
>Senior Software Engineer
>LinkedIn: http://www.linkedin.com/in/cecilwesterhof

Laura


From breamoreboy at yahoo.co.uk  Fri May 22 10:24:08 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 22 May 2015 15:24:08 +0100
Subject: Camelot a good tool for me
In-Reply-To: <87zj4xoxfd.fsf@Equus.decebal.nl>
References: <87zj4xoxfd.fsf@Equus.decebal.nl>
Message-ID: <mjne6h$i8h$1@ger.gmane.org>

On 22/05/2015 08:59, Cecil Westerhof wrote:
> I want to learn a lot of things. For example writing database and
> graphical applications. For database I decided on SQLAlchemy and GUI
> on Tkinter. In principal I want to write Python 3 applications.
>
> I came across Camelot. As I understand it, this is something to write
> graphical database applications fast. It works with Qt, but that
> should not be a big problem. It is just to get me started. But it
> seems only to work with 2.7 and not 3. Is this true?
>
> Would Camelot be a good tool to get me started, or can I better bite
> the bullet and just start with Tkinter and SQLAlchemy?
>

As others have already said plain SQL is perfectly adequate in many 
situations.  There are also other ORMs with peewee and ponyORM springing 
straight to my mind, although there's certainly a far longer list.  What 
it gets down to is "horses for courses" and you're the only person who 
(hopefully) knows the course you're running :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From felix at epepm.cupet.cu  Fri May 22 10:50:11 2015
From: felix at epepm.cupet.cu (felix)
Date: Fri, 22 May 2015 10:50:11 -0400
Subject: Camelot a good tool for me
In-Reply-To: <mjne6h$i8h$1@ger.gmane.org>
References: <87zj4xoxfd.fsf@Equus.decebal.nl> <mjne6h$i8h$1@ger.gmane.org>
Message-ID: <555F4223.20008@epepm.cupet.cu>

El 22/05/15 10:24, Mark Lawrence escribi?:
> On 22/05/2015 08:59, Cecil Westerhof wrote:
>> I want to learn a lot of things. For example writing database and
>> graphical applications. For database I decided on SQLAlchemy and GUI
>> on Tkinter. In principal I want to write Python 3 applications.
>>
>> I came across Camelot. As I understand it, this is something to write
>> graphical database applications fast. It works with Qt, but that
>> should not be a big problem. It is just to get me started. But it
>> seems only to work with 2.7 and not 3. Is this true?
>>
>> Would Camelot be a good tool to get me started, or can I better bite
>> the bullet and just start with Tkinter and SQLAlchemy?
>>
>
> As others have already said plain SQL is perfectly adequate in many 
> situations.  There are also other ORMs with peewee and ponyORM 
> springing straight to my mind, although there's certainly a far longer 
> list.  What it gets down to is "horses for courses" and you're the 
> only person who (hopefully) knows the course you're running :)
>
I really enjoy Django ORM!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150522/e14d276d/attachment.html>

From steve at pearwood.info  Fri May 22 10:58:17 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 23 May 2015 00:58:17 +1000
Subject: Ah Python, you have spoiled me for all other languages
Message-ID: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>

It's good to have at least a passing familiarity in more than one
programming language, so for I've re-written a small Python script (56
lines, including blanks and comments) into Lua (67 lines), Ruby (73 lines)
and Javascript (102 lines).

Re-writing the code in Lua and Ruby was actually quite simple. There are
some syntactic differences and semantic differences, and Ruby lacks a
standard "assert" function or statement, but that only required seven lines
of code. (Three of them "end" statements.) I think Python is a prettier
language visually than either Lua or Ruby, but they're in the ball-park.
Both languages have their warts and quirks, but if Python were declared
illegal overnight[1] I'd probably have no trouble adapting to Ruby or Lua.
Python would still be my first love, but these two languages make a
reasonable rebound language.

But Javascript... 

Javascript also lacks a standard assert mechanism, but that wasn't too hard
to fix. It also has two different equality operators, each of which are so
complicated and confusing that apparently there are two-year Masters
degrees on them[2], and yet with neither of these operators does the array
[1, 2] equal the array [1, 2].

It's visually an ugly language, requiring braces and semi-colons.
Technically, some of the semi-colons are optional, and some of them aren't
optional but change the meaning of the code if you leave them out, so it's
just best to stick semi-colons after; everything; you; can; just; to; be;
sure.

I know that first impressions aren't necessarily to be trusted, but the
impression I get after a couple of hours is that Javascript tries really
hard to do everything it can for you except what you actually want. If it
were a remote control for a DVD player, there would be a button to turn the
volume up, skip to the next chapter, and turn subtitles off; and another
button to change the language to French and return to the menus; but no way
to just mute the sound.




[1] Anything that good has got to be either illegal, immoral, or fattening.

[2] If there aren't, there ought to be. 


-- 
Steven



From rosuav at gmail.com  Fri May 22 11:29:33 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 23 May 2015 01:29:33 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmqoJH1Lk+QD98XfMkfLozNfHHxNsq3epru8c_6KD4e_HA@mail.gmail.com>

On Sat, May 23, 2015 at 12:58 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> I think Python is a prettier
> language visually than either Lua or Ruby, but they're in the ball-park.
> Both languages have their warts and quirks, but if Python were declared
> illegal overnight[1] I'd probably have no trouble adapting to Ruby or Lua.
> Python would still be my first love, but these two languages make a
> reasonable rebound language.

A good start. Toy programs don't always tell the whole story, though.
How good are the three languages at making your code reliable in the
face of user action? My hobby-horse, Unicode, is a notable flaw in
many languages - if you ask the user for information (in the most
obvious way for whatever environment you're in, be that via a web
browser request, or a GUI widget, or text entered at the console), can
it cope equally with all the world's languages? What if you want to
manipulate that text - is it represented as a sequence of codepoints
(Python 3), UTF-16 code units (JavaScript), UTF-8 bytes (quite a few),
or "bytes in whatever codepage your system was set to" (anything that
hasn't cared)?

ChrisA


From esj at harvee.org  Fri May 22 12:29:20 2015
From: esj at harvee.org (Eric S. Johansson)
Date: Fri, 22 May 2015 12:29:20 -0400
Subject: need help with an accessibility prototype
Message-ID: <555F5960.9030501@harvee.org>

2 needs.  first is determining if NaturallySpeaking injects keycodes or 
ascii char into the windows input queue.  second is building a test 
widget to capture and display text.

I think I can solve both of these by building a simple text widget 
(tkinter? qt? ??) to capture keycodes.  problem, is in <mumble><mumble> 
yrs of programming, I've never written a GUI interface so I have no idea 
where to start.  help, tutor, pointers to samples would be most welcome.

--- eric


From breamoreboy at yahoo.co.uk  Fri May 22 12:57:24 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 22 May 2015 17:57:24 +0100
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <ORI7x.668201$dj1.141793@fx03.am4>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <ORI7x.668201$dj1.141793@fx03.am4>
Message-ID: <mjnn5s$fme$1@ger.gmane.org>

On 22/05/2015 17:31, Tony the Tiger wrote:
> On Sat, 23 May 2015 00:58:17 +1000, Steven D'Aprano wrote:
>
>> I get after a couple of hours is that Javascript tries really hard to do
>> everything it can for you except what you actually want.
>
> You just described a certain operating system, which shall remain
> nameless.
>
>
>   /Grrr
>

It could be any of *nix, OSX or Windows as VMS is still vastly superior.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From georgeryoung at gmail.com  Fri May 22 12:59:31 2015
From: georgeryoung at gmail.com (georgeryoung at gmail.com)
Date: Fri, 22 May 2015 09:59:31 -0700 (PDT)
Subject: case-sensitive configparser without magical interpolation?
Message-ID: <1d76eb69-7880-4a25-976c-96a69b61b91f@googlegroups.com>

[python 2.7]
I need to use a configparser that is case-sensitive for option names, but does not do magical interpolation of percent sign.  
I.e.:

[Mapping0]
backupHost = eng%26
dbNode = v_br_node0001

should be read (and later written) as is, including capitalization and the percent sign.  

I find that RawConfigParser keeps the %, but downcases the option name.
And SafeConfigParser can be hacked with optionxform to be case-sensitive, but does interpolation.
How can I get case-sensitive but no interpolation?

-- George


From lele at metapensiero.it  Fri May 22 13:05:06 2015
From: lele at metapensiero.it (Lele Gaifax)
Date: Fri, 22 May 2015 19:05:06 +0200
Subject: Strategies for using cffi with C++ code?
References: <skip.montanaro@gmail.com>
 <CANc-5UyK4+jAxZ_WDnEbM7RgAQk=k2uKJs8a=m1phGeb8DfAYQ@mail.gmail.com>
 <201505211706.t4LH6GY0006031@fido.openend.se>
 <CANc-5UyBC-vw_+Og-1gea9YXubpATezsOmNf58AKKgb0swBP9A@mail.gmail.com>
Message-ID: <87oalclf0d.fsf@nautilus.nautilus>

Skip Montanaro <skip.montanaro at gmail.com> writes:

> I also figure there must be some established best practices for exposing C++
> classes to Python.

Maybe http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html?

bye, lele.
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.



From ian.g.kelly at gmail.com  Fri May 22 13:12:12 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 22 May 2015 11:12:12 -0600
Subject: case-sensitive configparser without magical interpolation?
In-Reply-To: <1d76eb69-7880-4a25-976c-96a69b61b91f@googlegroups.com>
References: <1d76eb69-7880-4a25-976c-96a69b61b91f@googlegroups.com>
Message-ID: <CALwzidmrM2wUkWH550LAuH5rTW9qqXCNuatmQFcehAC1pjXnVw@mail.gmail.com>

On Fri, May 22, 2015 at 10:59 AM,  <georgeryoung at gmail.com> wrote:
> [python 2.7]
> I need to use a configparser that is case-sensitive for option names, but does not do magical interpolation of percent sign.
> I.e.:
>
> [Mapping0]
> backupHost = eng%26
> dbNode = v_br_node0001
>
> should be read (and later written) as is, including capitalization and the percent sign.
>
> I find that RawConfigParser keeps the %, but downcases the option name.
> And SafeConfigParser can be hacked with optionxform to be case-sensitive, but does interpolation.
> How can I get case-sensitive but no interpolation?

RawConfigParser also has the optionxform method; have you tried overriding that?

If that doesn't work, then how strict is the 2.7 requirement? In 3.2
or later, the ConfigParser takes an interpolation keyword argument
that can be used to disable interpolation:

https://docs.python.org/3.4/library/configparser.html#configparser-objects


From skip.montanaro at gmail.com  Fri May 22 13:15:19 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Fri, 22 May 2015 12:15:19 -0500
Subject: Strategies for using cffi with C++ code?
In-Reply-To: <87oalclf0d.fsf@nautilus.nautilus>
References: <skip.montanaro@gmail.com>
 <CANc-5UyK4+jAxZ_WDnEbM7RgAQk=k2uKJs8a=m1phGeb8DfAYQ@mail.gmail.com>
 <201505211706.t4LH6GY0006031@fido.openend.se>
 <CANc-5UyBC-vw_+Og-1gea9YXubpATezsOmNf58AKKgb0swBP9A@mail.gmail.com>
 <87oalclf0d.fsf@nautilus.nautilus>
Message-ID: <CANc-5UwsOTUvMCnZ_vi3nqu6O2sCPqkAwCHY=aEiT+qY-iLdQQ@mail.gmail.com>

2015-05-22 12:05 GMT-05:00 Lele Gaifax <lele at metapensiero.it>:
> Maybe http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html?

Thanks for the reference, Lele. I was thinking in terms of cffi, but
this might work as well. (Shouldn't cffi interfaces be the thinnest?)

Skip


From lele at metapensiero.it  Fri May 22 13:24:30 2015
From: lele at metapensiero.it (Lele Gaifax)
Date: Fri, 22 May 2015 19:24:30 +0200
Subject: Camelot a good tool for me
References: <87zj4xoxfd.fsf@Equus.decebal.nl> <mjn804$4pe$1@dont-email.me>
 <87siaopxjs.fsf@Equus.decebal.nl>
 <201505221357.t4MDvgIV001904@fido.openend.se>
Message-ID: <87k2w0le41.fsf@nautilus.nautilus>

Laura Creighton <lac at openend.se> writes:

> In my corner of the world, everybody uses SQL.
> [...]
> The people who have tried SQLAlchemy really didn't like it, and of course
> the people who haven't tried it do what their friends do, as usual.

If these sentences are related, you must live in a very strange corner!

;-)

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.



From georgeryoung at gmail.com  Fri May 22 13:35:29 2015
From: georgeryoung at gmail.com (georgeryoung at gmail.com)
Date: Fri, 22 May 2015 10:35:29 -0700 (PDT)
Subject: case-sensitive configparser without magical interpolation?
In-Reply-To: <mailman.226.1432314781.17265.python-list@python.org>
References: <1d76eb69-7880-4a25-976c-96a69b61b91f@googlegroups.com>
 <mailman.226.1432314781.17265.python-list@python.org>
Message-ID: <429c3629-5735-4c40-968c-782071f12691@googlegroups.com>

On Friday, May 22, 2015 at 1:13:39 PM UTC-4, Ian wrote:
> On Fri, May 22, 2015 at 10:59 AM,  gy wrote:
> > [python 2.7]
> > I need to use a configparser that is case-sensitive for option names, but does not do magical interpolation of percent sign.
> > I.e.:
> >
> > [Mapping0]
> > backupHost = eng%26
> > dbNode = v_br_node0001
> >
> > should be read (and later written) as is, including capitalization and the percent sign.
> >
> > I find that RawConfigParser keeps the %, but downcases the option name.
> > And SafeConfigParser can be hacked with optionxform to be case-sensitive, but does interpolation.
> > How can I get case-sensitive but no interpolation?
> 
> RawConfigParser also has the optionxform method; have you tried overriding that?
> 
> If that doesn't work, then how strict is the 2.7 requirement? In 3.2
> or later, the ConfigParser takes an interpolation keyword argument
> that can be used to disable interpolation:
> 
> https://docs.python.org/3.4/library/configparser.html#configparser-objects

That worked perfectly, thanks!


From rosuav at gmail.com  Fri May 22 13:37:03 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 23 May 2015 03:37:03 +1000
Subject: Camelot a good tool for me
In-Reply-To: <87k2w0le41.fsf@nautilus.nautilus>
References: <87zj4xoxfd.fsf@Equus.decebal.nl> <mjn804$4pe$1@dont-email.me>
 <87siaopxjs.fsf@Equus.decebal.nl>
 <201505221357.t4MDvgIV001904@fido.openend.se>
 <87k2w0le41.fsf@nautilus.nautilus>
Message-ID: <CAPTjJmpnYwhOWdBz-=Zv8x+BO0G3e7OesPDvzd=nsX2a8Gsc-w@mail.gmail.com>

On Sat, May 23, 2015 at 3:24 AM, Lele Gaifax <lele at metapensiero.it> wrote:
> Laura Creighton <lac at openend.se> writes:
>
>> In my corner of the world, everybody uses SQL.
>> [...]
>> The people who have tried SQLAlchemy really didn't like it, and of course
>> the people who haven't tried it do what their friends do, as usual.
>
> If these sentences are related, you must live in a very strange corner!
>
> ;-)

Not sure why. I'm in the same corner, I think - I use SQL, and not
SQLAlchemy if I can help it. I'd much rather just use psycopg2 and do
my own queries. SQLAlchemy has its uses, and it does solve a number of
issues in reasonably clean ways, but I don't like a few of its facets,
including its peculiar way of doing foreign key relationships. (You
put a foreign key in the child, and you put a relationship in the
parent, which feels backwards.) There's a lot of magic going on. When
magic works, it's great; but when anything goes wrong, it's harder to
see what happened. (Also, when does a transaction begin and end? If
you session.commit() in the middle of iterating over a query, will it
break the query? What if you roll back? Can you see, instantly, in
your code?) Even if the ORM layer is practically perfect in every way,
there's still value in learning SQL; for instance, if you drop to a
command-line interpreter like PostgreSQL's psql, or if you switch to
another language, or anything like that, it's helpful to know what's
going on under the covers. And if you have to know SQL anyway, the
advantage of the abstraction layer has to justify the cost of learning
an additional, not a replacement, API.

So, while SQLAlchemy is definitely a lot better than most I've seen,
it's still not really good enough for me to use everywhere. I'm mostly
going to stick to the Python DB API 2.0.

ChrisA


From python at mrabarnett.plus.com  Fri May 22 13:39:07 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 22 May 2015 18:39:07 +0100
Subject: need help with an accessibility prototype
In-Reply-To: <555F5960.9030501@harvee.org>
References: <555F5960.9030501@harvee.org>
Message-ID: <555F69BB.2020200@mrabarnett.plus.com>

On 2015-05-22 17:29, Eric S. Johansson wrote:
> 2 needs.  first is determining if NaturallySpeaking injects keycodes or
> ascii char into the windows input queue.  second is building a test
> widget to capture and display text.
>
ASCII? :-)

> I think I can solve both of these by building a simple text widget
> (tkinter? qt? ??) to capture keycodes.  problem, is in <mumble><mumble>
> yrs of programming, I've never written a GUI interface so I have no idea
> where to start.  help, tutor, pointers to samples would be most welcome.
>
It injects key events, such as key-down, key-press, key-up.

In my experience, if the widget with the input focus is writeable,
it might capture the text keys itself and the widget's key event
handlers won't see them, only the other key events (up, down, etc), but
if the widget is read-only, the event handlers _will_ see them too.


From invalid at invalid.invalid  Fri May 22 13:47:39 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Fri, 22 May 2015 17:47:39 +0000 (UTC)
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mjnq3r$p48$1@reader1.panix.com>

On 2015-05-22, Steven D'Aprano <steve at pearwood.info> wrote:

> But Javascript... 
>
> Javascript also lacks a standard assert mechanism, but that wasn't
> too hard to fix. It also has two different equality operators, each
> of which are so complicated and confusing that apparently there are
> two-year Masters degrees on them[2], and yet with neither of these
> operators does the array [1, 2] equal the array [1, 2].

Yep, Javascript has a few warts.

 * Implicit conversions.  A problem common to many languages, so most
   of us have learned to deal with it.

 * The attempt to make semicolons optional was a failure and should
   have been abandoned.

 * Lack of block-scope.  For some reason, it doesn't bother me in
   Python, but it does in Javascript.  That probably has something to do
   with...

 * Hoisting.  Hoisting isn't intrinsically bad, but there's something
   just a bit off about Javascript's hoisting.  I can't quite put my
   finger on why, but it trips me up.

 * The == operator.  It rarely does what you want.  Just don't use it.
   
 * When used in a browser, the rather odd meanings of 'this' in
   certain situations.
   
I do like the use of closures.

And of course, the truly _great_ thing about Javascript is...

It's not PHP!

-- 
Grant Edwards               grant.b.edwards        Yow! Am I having fun yet?
                                  at               
                              gmail.com            


From lac at openend.se  Fri May 22 13:56:34 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 22 May 2015 19:56:34 +0200
Subject: Camelot a good tool for me
In-Reply-To: Message from Lele Gaifax <lele@metapensiero.it>
 of "Fri, 22 May 2015 19:24:30 +0200." <87k2w0le41.fsf@nautilus.nautilus>
References: <87zj4xoxfd.fsf@Equus.decebal.nl> <mjn804$4pe$1@dont-email.me>
 <87siaopxjs.fsf@Equus.decebal.nl>
 <201505221357.t4MDvgIV001904@fido.openend.se><87k2w0le41.fsf@nautilus.nautilus>
Message-ID: <201505221756.t4MHuYUa007435@fido.openend.se>

In a message of Fri, 22 May 2015 19:24:30 +0200, Lele Gaifax writes:
>Laura Creighton <lac at openend.se> writes:
>
>> In my corner of the world, everybody uses SQL.
>> [...]
>> The people who have tried SQLAlchemy really didn't like it, and of course
>> the people who haven't tried it do what their friends do, as usual.
>
>If these sentences are related, you must live in a very strange corner!
>
>;-)

>ciao, lele.
>-- 
>nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
>real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
>lele at metapensiero.it  |                 -- Fortunato Depero, 1929.

Just looks like home to me. :)  But explains why, should you ever
want to use SQLALchemy for something you would have to look long and
hard around here to find somebody who uses it -- whereas practically
every bar downtown (near both Chalmers university and a lot of IT jobs)
will find you somebody who knows MySQL.  (They won't all know the
Python interface, though.)  That MySQL was invented in Sweden
probably has a lot to do with this.  But I'm a PostgreSQL partisan,
though these days I am rather more fond of non-relational
databases like MongoDB which I suppose is even ranker heresy. :)

Laura


From rosuav at gmail.com  Fri May 22 14:11:28 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 23 May 2015 04:11:28 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mjnq3r$p48$1@reader1.panix.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mjnq3r$p48$1@reader1.panix.com>
Message-ID: <CAPTjJmosDAaq3Qf0NuKa_g1b4XczoU4y1ZpGAZqpM3+HHAN95A@mail.gmail.com>

On Sat, May 23, 2015 at 3:47 AM, Grant Edwards <invalid at invalid.invalid> wrote:
>  * When used in a browser, the rather odd meanings of 'this' in
>    certain situations.

Yes. Closures can retain state exactly the way you'd expect them to,
yet what Python would call a bound method (another way of retaining
state, specifically the object before the dot) is just... a function.
I find this nothing but bizarre. It basically demotes 'this' to the
position of any other argument, only it's implicit.

ChrisA


From none at mailinator.com  Fri May 22 14:19:26 2015
From: none at mailinator.com (mm0fmf)
Date: Fri, 22 May 2015 19:19:26 +0100
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mjnq3r$p48$1@reader1.panix.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mjnq3r$p48$1@reader1.panix.com>
Message-ID: <RqK7x.792916$AG5.722230@fx34.am4>

On 22/05/2015 18:47, Grant Edwards wrote:
> And of course, the truly_great_  thing about Javascript is...
>
> It's not PHP!

ROTFL ;-)





From tjreedy at udel.edu  Fri May 22 14:42:46 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 22 May 2015 14:42:46 -0400
Subject: need help with an accessibility prototype
In-Reply-To: <555F5960.9030501@harvee.org>
References: <555F5960.9030501@harvee.org>
Message-ID: <mjntbg$nhm$1@ger.gmane.org>

On 5/22/2015 12:29 PM, Eric S. Johansson wrote:
> 2 needs.  first is determining if NaturallySpeaking injects keycodes or
> ascii char into the windows input queue.  second is building a test
> widget to capture and display text.
>
> I think I can solve both of these by building a simple text widget
> (tkinter? qt? ??) to capture keycodes.  problem, is in <mumble><mumble>
> yrs of programming, I've never written a GUI interface so I have no idea
> where to start.  help, tutor, pointers to samples would be most welcome.

For tkinter, once you have seen a tutorial, look at
http://infohost.nmt.edu/tcc/help/pubs/tkinter/web/index.html
and in particular ch.54 on events.

-- 
Terry Jan Reedy



From lele at metapensiero.it  Fri May 22 15:12:42 2015
From: lele at metapensiero.it (Lele Gaifax)
Date: Fri, 22 May 2015 21:12:42 +0200
Subject: Camelot a good tool for me
References: <87zj4xoxfd.fsf@Equus.decebal.nl> <mjn804$4pe$1@dont-email.me>
 <87siaopxjs.fsf@Equus.decebal.nl>
 <201505221357.t4MDvgIV001904@fido.openend.se>
 <87k2w0le41.fsf@nautilus.nautilus>
 <CAPTjJmpnYwhOWdBz-=Zv8x+BO0G3e7OesPDvzd=nsX2a8Gsc-w@mail.gmail.com>
Message-ID: <87fv6ol93p.fsf@nautilus.nautilus>

Chris Angelico <rosuav at gmail.com> writes:

> On Sat, May 23, 2015 at 3:24 AM, Lele Gaifax <lele at metapensiero.it> wrote:
>> Laura Creighton <lac at openend.se> writes:
>>
>>> In my corner of the world, everybody uses SQL.
>>> [...]
>>> The people who have tried SQLAlchemy really didn't like it, and of course
>>> the people who haven't tried it do what their friends do, as usual.
>>
>> If these sentences are related, you must live in a very strange corner!
>>
>> ;-)

> Not sure why. I'm in the same corner, I think - I use SQL, and not
> SQLAlchemy if I can help it. I'd much rather just use psycopg2 and do
> my own queries.

Didn't mean to be so serious, but anyway...

> SQLAlchemy has its uses, and it does solve a number of
> issues in reasonably clean ways, but I don't like a few of its facets,
> including its peculiar way of doing foreign key relationships. (You
> put a foreign key in the child, and you put a relationship in the
> parent, which feels backwards.)

You are conflating two different layers, core and ORM. ORM relationships can
be declared either on the parent or on the child, it's up to your taste.

> There's a lot of magic going on.

Not in my experience, but I started using it long ago, so I'm biased.

> When magic works, it's great; but when anything goes wrong, it's harder to
> see what happened.

The same can be said of almost any upper layer in a software stack.

> Also, when does a transaction begin and end?

When I need transactions (that is, when I'm changing the database) I'm very
picky and use explicit begins, commits and rollbacks, so I don't recall
experiencing that doubt.

> If you session.commit() in the middle of iterating over a query, will it
> break the query? What if you roll back? Can you see, instantly, in your
> code?

Why would you do that? Are you closing your files while you iterate them,
without leaving the loop in some way at the same time?

> Even if the ORM layer is practically perfect in every way,
> there's still value in learning SQL; for instance, if you drop to a
> command-line interpreter like PostgreSQL's psql, or if you switch to
> another language, or anything like that, it's helpful to know what's
> going on under the covers. And if you have to know SQL anyway, the
> advantage of the abstraction layer has to justify the cost of learning
> an additional, not a replacement, API.

No doubt on that. Working with SQLAlchemy is not an alterative to knowing SQL
fairly well. SA does hide "details" (name quoting syntax, to mention one
obvious detail), but does not even try to hide the fact that it's talking SQL
all the way down.

I often have to deal with multiple DB engines at the same time, and being
able to "write" my queries with an abstract syntax is very valuable for me.

> So, while SQLAlchemy is definitely a lot better than most I've seen,
> it's still not really good enough for me to use everywhere. I'm mostly
> going to stick to the Python DB API 2.0.

Again, I fully agree: one should work with whatever tool he feels comfortable
with!

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.



From lac at openend.se  Fri May 22 15:14:03 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 22 May 2015 21:14:03 +0200
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: Message from Grant Edwards <invalid@invalid.invalid>
 of "Fri, 22 May 2015 17:47:39 -0000." <mjnq3r$p48$1@reader1.panix.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com><mjnq3r$p48$1@reader1.panix.com>
Message-ID: <201505221914.t4MJE3e9009120@fido.openend.se>

The first time you discover that in javascript typeof(null) is 'object' and
not 'null' you will scream.  I wonder how many home versions of typeof
to replace the system one exist out in the wild?

Laura



From python at mrabarnett.plus.com  Fri May 22 15:34:37 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 22 May 2015 20:34:37 +0100
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <201505221914.t4MJE3e9009120@fido.openend.se>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com><mjnq3r$p48$1@reader1.panix.com>
 <201505221914.t4MJE3e9009120@fido.openend.se>
Message-ID: <555F84CD.9010204@mrabarnett.plus.com>

On 2015-05-22 20:14, Laura Creighton wrote:
> The first time you discover that in javascript typeof(null) is 'object' and
> not 'null' you will scream.  I wonder how many home versions of typeof
> to replace the system one exist out in the wild?
>
I don't think that typeof(null) should be 'null'.

If the type of an integer instance is the integer type and the type of
a string instance is the string type, then the type of null should be
the null type, not a null instance.

I suppose that you could consider that what JavaScript is doing is
equivalent to saying in Python that:

     None = object()

like you sometimes do when you want a unique sentinel because None
itself would be an acceptable value.


From denismfmcmahon at gmail.com  Fri May 22 15:48:38 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Fri, 22 May 2015 19:48:38 +0000 (UTC)
Subject: Convert c file to csv file(unix format) in python
References: <c9733ddb-7f97-4fcf-9dad-a4481d1d3e5d@googlegroups.com>
 <mjfusi$6mq$2@dont-email.me>
 <11c2e290-52db-4bc6-a5fe-5757230a816c@googlegroups.com>
 <mjgpmv$6mq$4@dont-email.me>
Message-ID: <mjo16m$equ$6@dont-email.me>

On Wed, 20 May 2015 01:57:51 +0000, Denis McMahon wrote:

> Based on the sample you sent, the output csv file is 1657 lines in
> length, and the first and last lines are:

[snip]

Well he didn't tell me if I was generating the right output, or what was 
wrong with it if it was wrong, so I guess he got a solution elsewhere.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From lac at openend.se  Fri May 22 15:50:03 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 22 May 2015 21:50:03 +0200
Subject: need help with an accessibility prototype
In-Reply-To: Message from "Eric S. Johansson" <esj@harvee.org>
 of "Fri, 22 May 2015 12:29:20 -0400." <555F5960.9030501@harvee.org>
References: <555F5960.9030501@harvee.org>
Message-ID: <201505221950.t4MJo3pC010042@fido.openend.se>

In a message of Fri, 22 May 2015 12:29:20 -0400, "Eric S. Johansson" writes:
>2 needs.  first is determining if NaturallySpeaking injects keycodes or 
>ascii char into the windows input queue.  second is building a test 
>widget to capture and display text.
>
>I think I can solve both of these by building a simple text widget 
>(tkinter? qt? ??) to capture keycodes.  problem, is in <mumble><mumble> 
>yrs of programming, I've never written a GUI interface so I have no idea 
>where to start.  help, tutor, pointers to samples would be most welcome.
>
>--- eric

The best online manual I know of for Tkinter is here:
http://effbot.org/tkinterbook/ despite being 10 years old.  But then I
haven't looked for one for about that long, either.

For tkinter key events are a  bit odd, given that Tkinter tries to
help you and considers special keys, punctuation and non-ascii printing
characters as separate things should you want to bind to them.

It's actually easier to write the code and say 'play with this' rather
than to explain when you would need to use event.char and when event.keysym
(Plus the code will tell you what Tkinter uses, I might remember wrong.)

Save this in a file and run it.

from Tkinter import *
root = Tk()
prompt = 'Press any key.  Remember to keep your mouse in the cyan box. '
lab = Label(root, text=prompt, width=len(prompt), bg='cyan')
lab.pack()

def key(event):
    msg = 'event.char is %r and event.keysym is %r' % (event.char, event.keysym)
        lab.config(text=msg)

root.bind_all('<Key>', key)
root.mainloop()

----------
If you need to put unicode chars in your label, that's a different probem.
If knowing when a control or alt-gr key is released is important to you,
the code will have to be modified to detect this specially.  Type some
control chars at the cyan window and you will see what I mean.

Happy Hacking,
Laura


From auriocus at gmx.de  Fri May 22 15:52:27 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Fri, 22 May 2015 21:52:27 +0200
Subject: Camelot a good tool for me OT beauty of Tk
In-Reply-To: <mailman.216.1432299860.17265.python-list@python.org>
References: <87zj4xoxfd.fsf@Equus.decebal.nl>
 <mailman.216.1432299860.17265.python-list@python.org>
Message-ID: <mjo1bm$ain$1@dont-email.me>

Am 22.05.15 um 15:03 schrieb Laura Creighton:
> I don't know anything about Camelot.  Tkinter produces widgets that are
> in no way as pretty graphically as is expected nowadays -- or indeed for
> at least 15 years.  If this matters to you -- or if you are building for
> customers if it matters to them, then Tkinter is not a good choice.

Tkinter doesn't need to be ugly. At least if you are on Windows or OSX, 
you can make near-native looking programs, if you stick to the simple 
rule to use ttk everywhere and never try to override background colors 
etc. Another important rule is that icons play an important role. A 
decent icon set makes a big difference.

An example of a Tk program on Windows 7 is this:

http://artist.bam.de/en/gallery/screenshots/aRTistDemoversionStarted.jpg

Would you spot that it is not a "native" Windows app? It's raw Tcl/Tk 
rather than Tkinter, but there is no specific reason why it couldn't be 
done in Python/Tkinter. On Linux, yes, the default themes are quite 
ugly. There are some workarounds, like loading the bitmap plastik theme, 
but this is substantially more (configuration) work.

And in the end a decent look needs a decent designer -  where to place 
which widgets, how to set the resizing options, that takes some 
experience to get it right.

	Christian




From ian.g.kelly at gmail.com  Fri May 22 15:56:56 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 22 May 2015 13:56:56 -0600
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <555F84CD.9010204@mrabarnett.plus.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mjnq3r$p48$1@reader1.panix.com> <201505221914.t4MJE3e9009120@fido.openend.se>
 <555F84CD.9010204@mrabarnett.plus.com>
Message-ID: <CALwzid=zXU3vVb5jjof1-9yzwKj-qXhRmUMrsw5wg0Haj0Yd1g@mail.gmail.com>

On Fri, May 22, 2015 at 1:34 PM, MRAB <python at mrabarnett.plus.com> wrote:
> On 2015-05-22 20:14, Laura Creighton wrote:
>>
>> The first time you discover that in javascript typeof(null) is 'object'
>> and
>> not 'null' you will scream.  I wonder how many home versions of typeof
>> to replace the system one exist out in the wild?
>>
> I don't think that typeof(null) should be 'null'.
>
> If the type of an integer instance is the integer type and the type of
> a string instance is the string type, then the type of null should be
> the null type, not a null instance.
>
> I suppose that you could consider that what JavaScript is doing is
> equivalent to saying in Python that:
>
>     None = object()
>
> like you sometimes do when you want a unique sentinel because None
> itself would be an acceptable value.

If only it were that logical. null in Javascript is a primitive type.
Here's what typeof returns on some other primitive types:

js> typeof(4)
"number"
js> typeof(4.5)
"number"
js> typeof('hello')
"string"
js> typeof(true)
"boolean"
js> typeof(undefined)
"undefined"

An "object" in Javascript is basically just a collection of
properties. For example:

js> typeof([1, 2, 3])
"object"
js> typeof({a: 1, b: 2, c: 3})
"object"

Here's what happens when you try to access a property on null:

js> null.foo
typein:18:0 TypeError: null has no properties

We can conclude that null is not an object. Even the MDN reference
page for the typeof operator refers to this as "bogus". There was a
proposal to fix this in ECMAScript 5.1, but it was rejected because it
caused too much breakage.


From fomcl at yahoo.com  Fri May 22 15:59:07 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Fri, 22 May 2015 12:59:07 -0700
Subject: need help with an accessibility prototype
Message-ID: <1432324747.43572.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>


-----------------------------
On Fri, May 22, 2015 9:50 PM CEST Laura Creighton wrote:

>In a message of Fri, 22 May 2015 12:29:20 -0400, "Eric S. Johansson" writes:
>>2 needs.  first is determining if NaturallySpeaking injects keycodes or 
>>ascii char into the windows input queue.  second is building a test 
>>widget to capture and display text.
>>
>>I think I can solve both of these by building a simple text widget 
>>(tkinter? qt? ??) to capture keycodes.  problem, is in <mumble><mumble> 
>>yrs of programming, I've never written a GUI interface so I have no idea 
>>where to start.  help, tutor, pointers to samples would be most welcome.
>>
>>--- eric
>
>The best online manual I know of for Tkinter is here:
>http://effbot.org/tkinterbook/ despite being 10 years old.  But then I
>haven't looked for one for about that long, either.

This book by John Shipman is also very good: http://infohost.nmt.edu/tcc/help/pubs/tkinter/

Albert-Jan


From lac at openend.se  Fri May 22 16:13:22 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 22 May 2015 22:13:22 +0200
Subject: Camelot a good tool for me OT beauty of Tk
In-Reply-To: Message from Christian Gollwitzer <auriocus@gmx.de>
 of "Fri, 22 May 2015 21:52:27 +0200." <mjo1bm$ain$1@dont-email.me>
References: <87zj4xoxfd.fsf@Equus.decebal.nl>
 <mailman.216.1432299860.17265.python-list@python.org><mjo1bm$ain$1@dont-email.me>
Message-ID: <201505222013.t4MKDMgT010611@fido.openend.se>

In a message of Fri, 22 May 2015 21:52:27 +0200, Christian Gollwitzer writes:
>Am 22.05.15 um 15:03 schrieb Laura Creighton:
>> I don't know anything about Camelot.  Tkinter produces widgets that are
>> in no way as pretty graphically as is expected nowadays -- or indeed for
>> at least 15 years.  If this matters to you -- or if you are building for
>> customers if it matters to them, then Tkinter is not a good choice.
>
>Tkinter doesn't need to be ugly. At least if you are on Windows or OSX, 
>you can make near-native looking programs, if you stick to the simple 
>rule to use ttk everywhere and never try to override background colors 
>etc. Another important rule is that icons play an important role. A 
>decent icon set makes a big difference.

I didn't know that.  Thank you.

>Would you spot that it is not a "native" Windows app? It's raw Tcl/Tk 
>rather than Tkinter, but there is no specific reason why it couldn't be 
>done in Python/Tkinter. On Linux, yes, the default themes are quite 
>ugly. There are some workarounds, like loading the bitmap plastik theme, 
>but this is substantially more (configuration) work.

And, of course, I live in the linux world.  I didn't know about
the bitmap plastik theme, either, thank you again.

>
>	Christian

Laura



From marko at pacujo.net  Fri May 22 16:34:47 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 22 May 2015 23:34:47 +0300
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mjnq3r$p48$1@reader1.panix.com>
 <201505221914.t4MJE3e9009120@fido.openend.se>
 <555F84CD.9010204@mrabarnett.plus.com>
 <mailman.238.1432324665.17265.python-list@python.org>
Message-ID: <87k2w0mjvc.fsf@elektro.pacujo.net>

Ian Kelly <ian.g.kelly at gmail.com>:

> An "object" in Javascript is basically just a collection of
> properties. For example:
>
> js> typeof([1, 2, 3])
> "object"
> js> typeof({a: 1, b: 2, c: 3})
> "object"
>
> Here's what happens when you try to access a property on null:
>
> js> null.foo
> typein:18:0 TypeError: null has no properties

That's not all that different from Python, where object() returns a
fresh instance at each invocation. However:

   >>> object().x = 3
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   AttributeError: 'object' object has no attribute 'x'

Why are object instances immutable in Python?


Marko


From esj at harvee.org  Fri May 22 16:46:08 2015
From: esj at harvee.org (Eric S. Johansson)
Date: Fri, 22 May 2015 16:46:08 -0400
Subject: need help with an accessibility prototype
In-Reply-To: <201505221950.t4MJo3pC010042@fido.openend.se>
References: <555F5960.9030501@harvee.org>
 <201505221950.t4MJo3pC010042@fido.openend.se>
Message-ID: <555F9590.4050002@harvee.org>



On 05/22/2015 03:50 PM, Laura Creighton wrote:
> In a message of Fri, 22 May 2015 12:29:20 -0400, "Eric S. Johansson" writes:
>> 2 needs.  first is determining if NaturallySpeaking injects keycodes or
>> ascii char into the windows input queue.  second is building a test
>> widget to capture and display text.
>>
>> I think I can solve both of these by building a simple text widget
>> (tkinter? qt? ??) to capture keycodes.  problem, is in <mumble><mumble>
>> yrs of programming, I've never written a GUI interface so I have no idea
>> where to start.  help, tutor, pointers to samples would be most welcome.
>>
>> --- eric
> The best online manual I know of for Tkinter is here:
> http://effbot.org/tkinterbook/ despite being 10 years old.  But then I
> haven't looked for one for about that long, either.
>
> For tkinter key events are a  bit odd, given that Tkinter tries to
> help you and considers special keys, punctuation and non-ascii printing
> characters as separate things should you want to bind to them.
>
> It's actually easier to write the code and say 'play with this' rather
> than to explain when you would need to use event.char and when event.keysym
> (Plus the code will tell you what Tkinter uses, I might remember wrong.)
>
> Save this in a file and run it.

Wow. What a gift. It told me what I needed to know which is the key syms 
are generated in a way that makes sense to me as far as I understand them.

A bit of background. I have some Linux resident code that translates 
standard input characters into key codes and then feeds them into 
uinput.   the problem is, the character sequences generated by 
NaturallySpeaking plus Windows doesn't always work right or as expected. 
In theory, I should now be able to push the individual events over to 
the Linux side  as key sym/ key codes, maybe do some translation, and 
ta-da,  I  now can dictate from the Windows virtual machine into Linux.

I now have, within reach, a fair amount of accessibility added by simple 
open-loop keystroke macros. The RPC mechanism tied into the Python 
extension to NaturallySpeaking, it's possible to execute even more 
sophisticated speech user interfaces.

thank you so much for the help.  future trick will be synchronizing 
tkinter window on windows with one on linux

--- eric


From python.list at tim.thechases.com  Fri May 22 16:55:16 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Fri, 22 May 2015 15:55:16 -0500
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <87k2w0mjvc.fsf@elektro.pacujo.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mjnq3r$p48$1@reader1.panix.com>
 <201505221914.t4MJE3e9009120@fido.openend.se>
 <555F84CD.9010204@mrabarnett.plus.com>
 <mailman.238.1432324665.17265.python-list@python.org>
 <87k2w0mjvc.fsf@elektro.pacujo.net>
Message-ID: <20150522155516.15a6a52d@bigbox.christie.dr>

On 2015-05-22 23:34, Marko Rauhamaa wrote:
> >>> object().x = 3
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> AttributeError: 'object' object has no attribute 'x'
> 
> Why are object instances immutable in Python?

I've wondered this on multiple occasions, as I've wanted to just make
an attribute bag and have to do something like

  class AttrBag(object): pass
  ab = AttrBag()
  ab.x = 42
  ab.y = "some other value"

because just doing

  ab = object()

raises the AttributeError Marko highlights. :-/

-tkc







From ethan at stoneleaf.us  Fri May 22 17:15:22 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 22 May 2015 14:15:22 -0700
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <87k2w0mjvc.fsf@elektro.pacujo.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mjnq3r$p48$1@reader1.panix.com>
 <201505221914.t4MJE3e9009120@fido.openend.se>
 <555F84CD.9010204@mrabarnett.plus.com>
 <mailman.238.1432324665.17265.python-list@python.org>
 <87k2w0mjvc.fsf@elektro.pacujo.net>
Message-ID: <555F9C6A.2000607@stoneleaf.us>

On 05/22/2015 01:34 PM, Marko Rauhamaa wrote:
> Ian Kelly <ian.g.kelly at gmail.com>:
>
>> An "object" in Javascript is basically just a collection of
>> properties. For example:
>>
>> js> typeof([1, 2, 3])
>> "object"
>> js> typeof({a: 1, b: 2, c: 3})
>> "object"
>>
>> Here's what happens when you try to access a property on null:
>>
>> js> null.foo
>> typein:18:0 TypeError: null has no properties
>
> That's not all that different from Python, where object() returns a
> fresh instance at each invocation. However:
>
>     >>> object().x = 3
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     AttributeError: 'object' object has no attribute 'x'
>
> Why are object instances immutable in Python?

# psuedo-code

class object:
     __slots__ = ()

class bad_object:
     "__slots__ not specified, so automatically gets a __dict__"

class tuple(...) <- object or bad_object?  hint: the clue is in the name ;)
     __slots__ = ()

If tuple is based on bad_object, it will get a __dict__ and be very-mutable.

In order to have slightly-mutable or immutable objects, the base class has to not have a __dict__, so object does not.

Mind you, I don't know if that's the reason why it was decided to be like that, or just a nice consequence of the choice to do it that way.

--
~Ethan~


From ian.g.kelly at gmail.com  Fri May 22 17:20:02 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 22 May 2015 15:20:02 -0600
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <20150522155516.15a6a52d@bigbox.christie.dr>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mjnq3r$p48$1@reader1.panix.com> <201505221914.t4MJE3e9009120@fido.openend.se>
 <555F84CD.9010204@mrabarnett.plus.com>
 <mailman.238.1432324665.17265.python-list@python.org>
 <87k2w0mjvc.fsf@elektro.pacujo.net>
 <20150522155516.15a6a52d@bigbox.christie.dr>
Message-ID: <CALwzid=QCA2v+TC6Sx6wcLkPYW9=hRAvi3_XneXBu=07YLUh5Q@mail.gmail.com>

On Fri, May 22, 2015 at 2:55 PM, Tim Chase
<python.list at tim.thechases.com> wrote:
> I've wondered this on multiple occasions, as I've wanted to just make
> an attribute bag and have to do something like
>
>   class AttrBag(object): pass
>   ab = AttrBag()
>   ab.x = 42
>   ab.y = "some other value"
>
> because just doing
>
>   ab = object()
>
> raises the AttributeError Marko highlights. :-/

This is what types.SimpleNamespace is for.


From tundra at tundraware.com  Fri May 22 17:40:35 2015
From: tundra at tundraware.com (Tim Daneliuk)
Date: Fri, 22 May 2015 16:40:35 -0500
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mailman.222.1432309028.17265.python-list@python.org>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
Message-ID: <555FA253.3020304@tundraware.com>

On 05/22/2015 10:29 AM, Chris Angelico wrote:
> On Sat, May 23, 2015 at 12:58 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> I think Python is a prettier
>> language visually than either Lua or Ruby, but they're in the ball-park.
>> Both languages have their warts and quirks, but if Python were declared
>> illegal overnight[1] I'd probably have no trouble adapting to Ruby or Lua.
>> Python would still be my first love, but these two languages make a
>> reasonable rebound language.
> 
> A good start. Toy programs don't always tell the whole story, though.
> How good are the three languages at making your code reliable in the
> face of user action? My hobby-horse, Unicode, is a notable flaw in
> many languages - if you ask the user for information (in the most
> obvious way for whatever environment you're in, be that via a web
> browser request, or a GUI widget, or text entered at the console), can
> it cope equally with all the world's languages? What if you want to
> manipulate that text - is it represented as a sequence of codepoints
> (Python 3), UTF-16 code units (JavaScript), UTF-8 bytes (quite a few),
> or "bytes in whatever codepage your system was set to" (anything that
> hasn't cared)?
> 
> ChrisA
> 


Lo these many years ago, I argued that Python is a whole lot more than 
a programming language:

   https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/

-- 
----------------------------------------------------------------------------
Tim Daneliuk     tundra at tundraware.com
PGP Key:         http://www.tundraware.com/PGP/



From tundra at tundraware.com  Fri May 22 17:40:35 2015
From: tundra at tundraware.com (Tim Daneliuk)
Date: Fri, 22 May 2015 16:40:35 -0500
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mailman.222.1432309028.17265.python-list@python.org>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
Message-ID: <555FA253.3020304@tundraware.com>

On 05/22/2015 10:29 AM, Chris Angelico wrote:
> On Sat, May 23, 2015 at 12:58 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> I think Python is a prettier
>> language visually than either Lua or Ruby, but they're in the ball-park.
>> Both languages have their warts and quirks, but if Python were declared
>> illegal overnight[1] I'd probably have no trouble adapting to Ruby or Lua.
>> Python would still be my first love, but these two languages make a
>> reasonable rebound language.
> 
> A good start. Toy programs don't always tell the whole story, though.
> How good are the three languages at making your code reliable in the
> face of user action? My hobby-horse, Unicode, is a notable flaw in
> many languages - if you ask the user for information (in the most
> obvious way for whatever environment you're in, be that via a web
> browser request, or a GUI widget, or text entered at the console), can
> it cope equally with all the world's languages? What if you want to
> manipulate that text - is it represented as a sequence of codepoints
> (Python 3), UTF-16 code units (JavaScript), UTF-8 bytes (quite a few),
> or "bytes in whatever codepage your system was set to" (anything that
> hasn't cared)?
> 
> ChrisA
> 


Lo these many years ago, I argued that Python is a whole lot more than 
a programming language:

   https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/

-- 
----------------------------------------------------------------------------
Tim Daneliuk     tundra at tundraware.com
PGP Key:         http://www.tundraware.com/PGP/



From tundra at tundraware.com  Fri May 22 17:41:11 2015
From: tundra at tundraware.com (Tim Daneliuk)
Date: Fri, 22 May 2015 16:41:11 -0500
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <ORI7x.668201$dj1.141793@fx03.am4>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <ORI7x.668201$dj1.141793@fx03.am4>
Message-ID: <555FA277.8060203@tundraware.com>

On 05/22/2015 11:31 AM, Tony the Tiger wrote:
> On Sat, 23 May 2015 00:58:17 +1000, Steven D'Aprano wrote:
> 
>> I get after a couple of hours is that Javascript tries really hard to do
>> everything it can for you except what you actually want.
> 
> You just described a certain operating system, which shall remain 
> nameless.
> 
> 
>  /Grrr
> 

CP/M ?

-- 
----------------------------------------------------------------------------
Tim Daneliuk     tundra at tundraware.com
PGP Key:         http://www.tundraware.com/PGP/



From lac at openend.se  Fri May 22 18:16:00 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 23 May 2015 00:16:00 +0200
Subject: need help with an accessibility prototype
In-Reply-To: Message from "Eric S. Johansson" <esj@harvee.org>
 of "Fri, 22 May 2015 16:46:08 -0400." <555F9590.4050002@harvee.org>
References: <555F5960.9030501@harvee.org>
 <201505221950.t4MJo3pC010042@fido.openend.se><555F9590.4050002@harvee.org>
Message-ID: <201505222216.t4MMG0IN013302@fido.openend.se>

You are most welcome, it was such a small thing.

By the way http://www.tcl.tk/man/tcl8.4/TkCmd/keysyms.htm
is a list of all the key symbols it is possible for you to get.
There is no guarantee that any platform actually has any of them,
just, if it is not there don't bother trying to look for it.

Happy hacking,

Laura



From cs at zip.com.au  Fri May 22 18:42:29 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 23 May 2015 08:42:29 +1000
Subject: Camelot a good tool for me
In-Reply-To: <87fv6ol93p.fsf@nautilus.nautilus>
References: <87fv6ol93p.fsf@nautilus.nautilus>
Message-ID: <20150522224229.GA62222@cskk.homeip.net>

On 22May2015 21:12, Lele Gaifax <lele at metapensiero.it> wrote:
>Chris Angelico <rosuav at gmail.com> writes:
>> On Sat, May 23, 2015 at 3:24 AM, Lele Gaifax <lele at metapensiero.it> wrote:
>>> Laura Creighton <lac at openend.se> writes:
>>>> In my corner of the world, everybody uses SQL.
>>>> [...]
>>>> The people who have tried SQLAlchemy really didn't like it, and of course
>>>> the people who haven't tried it do what their friends do, as usual.
>>>
>>> If these sentences are related, you must live in a very strange corner!
>>> ;-)
>
>> Not sure why. I'm in the same corner, I think - I use SQL, and not
>> SQLAlchemy if I can help it. I'd much rather just use psycopg2 and do
>> my own queries.
>
>Didn't mean to be so serious, but anyway...
>
>> SQLAlchemy has its uses, and it does solve a number of
>> issues in reasonably clean ways, but I don't like a few of its facets,
>> including its peculiar way of doing foreign key relationships. (You
>> put a foreign key in the child, and you put a relationship in the
>> parent, which feels backwards.)
>
>You are conflating two different layers, core and ORM. ORM relationships can
>be declared either on the parent or on the child, it's up to your taste.

Indeed.

I'm a big fan of SQLAlchemy myself. But I do not use the ORM stuff.

I like SQLAlchemy because:

  - it quotes for me, avoiding an infinity of pain and injection risk

  - it presents results as nice objects with attributes named after columns

  - it lets me write SQL queries as nice parameterised Python syntax instead of clunky SQL

  - it automatically hooks into various backends, which means I can write many 
  tests or prototypes using, for example, in-memory SQLite dummy databases 
  while still speaking to MySQL or PostgreSQL in the real world. (Obviously you 
  need to run some tests against the real thing, but looking for SQL logic 
  errors is readily tested against throwaway SQLite or the like.)

I stay away from the ORM; I'm effectively writing SQL directly but using better 
syntax. I'm not saying the ORM is bad - you can clearly do all sorts of great 
things with it, but I have not found it useful to me, and that may reflect my 
ORM design skills and/or lack of experience with it.

Cheers,
Cameron Simpson <cs at zip.com.au>

Ride to not crash. Dress to crash. Live to ride to not crash again.
        - Lawrence Smith, DoD#i, lawrence at msc.cornell.edu


From no.email at nospam.invalid  Fri May 22 19:00:58 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Fri, 22 May 2015 16:00:58 -0700
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <874mn4z07p.fsf@jester.gateway.sonic.net>

Steven D'Aprano <steve at pearwood.info> writes:
> the impression I get after a couple of hours is that Javascript tries
> really hard to do everything it can for you except what you actually want.

Javascript is like C++ in that it's a lot of layers of legacy cruft, but
if you ignore the ugly parts, the good parts that are left are fine to
program in, though there are still stupid issues of type confusion.

If you like Haskell you should look at Purescript (purescript.org),
which is a Haskell-resembling language with strict evaluation, designed
to compile straightforwardly into Javascript and interoperate with it.
Several other backends are also available or in progress.


From brisher777 at gmail.com  Fri May 22 20:22:22 2015
From: brisher777 at gmail.com (Benjamin Risher)
Date: Fri, 22 May 2015 17:22:22 -0700 (PDT)
Subject: remote control of a cmd.Cmd() loop (help requested)
Message-ID: <61cbbdff-cf92-48f0-8abc-ddb4245fd04d@googlegroups.com>

First, the snippet provided is a contrived example of a much larger program.  

http://pastebin.com/xRqBE5BY

(written in python3 with access to 3.4 if necessary)

The goal:  To connect to a listening socket and send commands to the cmd.Cmd() loop running, then show the output to both stdout and the remote connection.

Right now, I have a solution that will work as long as everything that needs to get sent to stdout and the socket is return'd from each function instead of printed.  

Because this is already a larger project, I would prefer not to have to go back through and refactor everything to facilitate the remote control (the actual program involves threading and locks etc etc).  

I feel like there should be a fairly simple solution dealing with duplicating the file descriptor or something similar.  I've messed around trying to find something like that, but without success.  

I've also looked into the contextlib.redirect_stdout, but it expects a file type object.  Also, using socket.makefile() results in the returned object not having a fileno() method, so I can't use select on it.  

Basically, I'm hoping someone here knows some file descriptor-fu or some other cleaner solution.  

Any help would be appreciated!


From brisher777 at gmail.com  Fri May 22 20:27:26 2015
From: brisher777 at gmail.com (Benjamin Risher)
Date: Fri, 22 May 2015 17:27:26 -0700 (PDT)
Subject: remote control of a cmd.Cmd() loop (help requested)
Message-ID: <d19c254b-7983-4862-9ab1-9ac45df841c1@googlegroups.com>

First, the snippet provided is a contrived example of a much larger program.

http://pastebin.com/xRqBE5BY

(written in python3 with access to 3.4 if necessary)

The goal:  To connect to a listening socket and send commands to the cmd.Cmd() loop running, then show the output to both stdout and the remote connection.

Right now, I have a solution that will work as long as everything that needs to get sent to stdout and the socket is return'd from each function instead of printed.

Because this is already a larger project, I would prefer not to have to go back through and refactor everything to facilitate the remote control (the actual program involves threading and locks etc etc).

I feel like there should be a fairly simple solution dealing with duplicating the file descriptor or something similar.  I've messed around trying to find something like that, but without success.

I've also looked into the contextlib.redirect_stdout, but it expects a file type object.  Also, using socket.makefile() results in the returned object not having a fileno() method, so I can't use select on it.

Basically, I'm hoping someone here knows some file descriptor-fu or some other cleaner solution.

Any help would be appreciated!


remote display ------------- | ------------ menu display

$>nc localhost 12345         | s function called.
s                            | h function called.
s function called.           | Traceback ...
h                            | 44: 'NoneType' object has no attribute 'strip'

$>


From steve at pearwood.info  Fri May 22 21:36:35 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 23 May 2015 11:36:35 +1000
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com><mjnq3r$p48$1@reader1.panix.com>
 <mailman.235.1432322051.17265.python-list@python.org>
Message-ID: <555fd9a3$0$12990$c3e8da3$5496439d@news.astraweb.com>

On Sat, 23 May 2015 05:14 am, Laura Creighton wrote:

> The first time you discover that in javascript typeof(null) is 'object'
> and
> not 'null' you will scream.  I wonder how many home versions of typeof
> to replace the system one exist out in the wild?

What weirds me out is that that Javascript provides syntax for creating
arrays and associative arrays (in Python terms, lists and dicts) but
there's no standard way to check them for equality.

What weirds me out is that setting properties on non-objects silently fails:

js> var a = 1;
js> a.foo = "something";
something
js> print(a.foo);
undefined

except when it doesn't:

js> b = null;
null
js> b.foo = "something";
js: "<stdin>", line 16: uncaught JavaScript runtime exception: TypeError:
Cannot set property "foo" of null to "something"
        at <stdin>:16


What weirds me out is how useless the tracebacks printed are, at least using
Rhino. There's no stack trace, so if there's an error in a function call,
you cannot see what called the function.


What weirds me out is that all numbers are floats, even if they pretend to
be ints. So:

js> Math.pow(2, 53)
9007199254740992
js> Math.pow(2, 53) + 1
9007199254740992

What weirds me out is that iterating over an array gives the indexes, not
the values:

js> arr = [10,20,30];
10,20,30
js> for (a in arr) {print(a)}
0
1
2


What weirds me out is that while false is falsey, and Boolean(false) is
falsey, new Boolean(false) is truthy:

js> arr = [false, Boolean(false), new Boolean(false)];
false,false,false
js> for (i = 0; i < arr.length; ++i) {
  > print(arr[i]);
  > if (arr[i]) {print("Surprise!")} }
false
false
false
Surprise!


It's going to take a lot to get past the first impression that Javascript is
nearly as horrible as PHP.


-- 
Steven



From python at mrabarnett.plus.com  Fri May 22 21:41:29 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 23 May 2015 02:41:29 +0100
Subject: remote control of a cmd.Cmd() loop (help requested)
In-Reply-To: <d19c254b-7983-4862-9ab1-9ac45df841c1@googlegroups.com>
References: <d19c254b-7983-4862-9ab1-9ac45df841c1@googlegroups.com>
Message-ID: <555FDAC9.9070609@mrabarnett.plus.com>

On 2015-05-23 01:27, Benjamin Risher wrote:
> First, the snippet provided is a contrived example of a much larger program.
>
> http://pastebin.com/xRqBE5BY
>
> (written in python3 with access to 3.4 if necessary)
>
> The goal:  To connect to a listening socket and send commands to the cmd.Cmd() loop running, then show the output to both stdout and the remote connection.
>
> Right now, I have a solution that will work as long as everything that needs to get sent to stdout and the socket is return'd from each function instead of printed.
>
> Because this is already a larger project, I would prefer not to have to go back through and refactor everything to facilitate the remote control (the actual program involves threading and locks etc etc).
>
> I feel like there should be a fairly simple solution dealing with duplicating the file descriptor or something similar.  I've messed around trying to find something like that, but without success.
>
> I've also looked into the contextlib.redirect_stdout, but it expects a file type object.  Also, using socket.makefile() results in the returned object not having a fileno() method, so I can't use select on it.
>
> Basically, I'm hoping someone here knows some file descriptor-fu or some other cleaner solution.
>
> Any help would be appreciated!
>
>
> remote display ------------- | ------------ menu display
>
> $>nc localhost 12345         | s function called.
> s                            | h function called.
> s function called.           | Traceback ...
> h                            | 44: 'NoneType' object has no attribute 'strip'
>
> $>
>
You could replace the current sys.stdout object with your own object 
that copies to the original sys.stdout object and the socket.


From tjreedy at udel.edu  Fri May 22 21:54:12 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 22 May 2015 21:54:12 -0400
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <555FA253.3020304@tundraware.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com>
Message-ID: <mjomke$saf$1@ger.gmane.org>

On 5/22/2015 5:40 PM, Tim Daneliuk wrote:

> Lo these many years ago, I argued that Python is a whole lot more than
> a programming language:
>
>     https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/

Perhaps something at tundraware needs updating.
'''
This Connection is Untrusted

You have asked Firefox to connect securely to www.tundraware.com, but we 
can't confirm that your connection is secure.

Normally, when you try to connect securely, sites will present trusted 
identification to prove that you are going to the right place. However, 
this site's identity can't be verified.
'''

-- 
Terry Jan Reedy



From jsf80238 at gmail.com  Fri May 22 22:15:54 2015
From: jsf80238 at gmail.com (Jason Friedman)
Date: Fri, 22 May 2015 20:15:54 -0600
Subject: decorators and alarms
Message-ID: <CANy1k1gkXuTKEPL-ku7+c8mCysGgPgb62ZAgDdCrXXs6Z1S8Yg@mail.gmail.com>

I have the following code to run a shell command and kill it if it
does not return within a certain amount of time:

def run_with_time_limit(command, limit):
    """Run the given command via a shell.
    Kill the command, and raise an exception, if the
    execution time exceeds <limit> seconds.
    Else return (exit_code, stdout, stderr, duration)."""
    class Alarm(Exception):
        pass
    def alarm_handler(signum, frame):
        raise Alarm
    start_stamp = time.time()
    signal.signal(signal.SIGALRM, alarm_handler)
    signal.alarm(limit)
    try:
        process = subprocess.Popen(
            command,
            stderr=subprocess.PIPE,
            stdout=subprocess.PIPE,
            shell=True)
        stdout, stderr = process.communicate()
        duration = time.time() - start_stamp
        signal.alarm(0) # reset the alarm
    except Alarm:
        process.kill()
        raise Alarm("The command did not return within %(limit)s
seconds." % locals())
    return (process.returncode, stdout, stderr, duration)

run_with_time_limit("sleep 2", 1)

This works as desired.

But, I'd like to expand this to take some generic code, not just a
shell command, and terminate it if it does not return quickly.

@time_limiter
def generic_function(arg1, arg2, time_limit=10):
    do_some_stuff()
    do_some_other_stuff()
    return val1, val2

If generic_function does not finish within 10 seconds it would stop
executing and throw an exception.

May I have some hints, please?


From jsf80238 at gmail.com  Fri May 22 23:20:38 2015
From: jsf80238 at gmail.com (Jason Friedman)
Date: Fri, 22 May 2015 21:20:38 -0600
Subject: decorators and alarms
In-Reply-To: <CANy1k1gkXuTKEPL-ku7+c8mCysGgPgb62ZAgDdCrXXs6Z1S8Yg@mail.gmail.com>
References: <CANy1k1gkXuTKEPL-ku7+c8mCysGgPgb62ZAgDdCrXXs6Z1S8Yg@mail.gmail.com>
Message-ID: <CANy1k1iMhESNR9X1CBxD3G+aZ3t4OFrSn3T-je1nkGsORi2CFg@mail.gmail.com>

> But, I'd like to expand this to take some generic code, not just a
> shell command, and terminate it if it does not return quickly.
>
> @time_limiter
> def generic_function(arg1, arg2, time_limit=10):
>     do_some_stuff()
>     do_some_other_stuff()
>     return val1, val2
>
> If generic_function does not finish within 10 seconds it would stop
> executing and throw an exception.

Found an answer:
https://wiki.python.org/moin/PythonDecoratorLibrary#Function_Timeout


From torriem at gmail.com  Fri May 22 23:31:14 2015
From: torriem at gmail.com (Michael Torrie)
Date: Fri, 22 May 2015 21:31:14 -0600
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mjomke$saf$1@ger.gmane.org>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
Message-ID: <555FF482.8020007@gmail.com>

On 05/22/2015 07:54 PM, Terry Reedy wrote:
> On 5/22/2015 5:40 PM, Tim Daneliuk wrote:
> 
>> Lo these many years ago, I argued that Python is a whole lot more than
>> a programming language:
>>
>>     https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
> 
> Perhaps something at tundraware needs updating.
> '''
> This Connection is Untrusted
> 
> You have asked Firefox to connect securely to www.tundraware.com, but we 
> can't confirm that your connection is secure.
> 
> Normally, when you try to connect securely, sites will present trusted 
> identification to prove that you are going to the right place. However, 
> this site's identity can't be verified.
> '''

Sigh. I blame this as much on the browser.  There's no inherent reason
why a connection to a site secured with a self-signed certificate is
insecure.  In fact it's definitely not.  Browsers need a better way to
deal with self-signed certs, but I think they'd rather we all just pay
up to the cert authorities and buy some false sense of security.

Personally I created my own CA with the wonderful xca program, and sign
all my certs with that.  If a person adds my CA certificate to their
browser, then my sites are trusted (and verified).  But for a public web
page this isn't very automatic.



From torriem at gmail.com  Fri May 22 23:33:14 2015
From: torriem at gmail.com (Michael Torrie)
Date: Fri, 22 May 2015 21:33:14 -0600
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <874mn4z07p.fsf@jester.gateway.sonic.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <874mn4z07p.fsf@jester.gateway.sonic.net>
Message-ID: <555FF4FA.4070402@gmail.com>

On 05/22/2015 05:00 PM, Paul Rubin wrote:
> Steven D'Aprano <steve at pearwood.info> writes:
>> the impression I get after a couple of hours is that Javascript tries
>> really hard to do everything it can for you except what you actually want.
> 
> Javascript is like C++ in that it's a lot of layers of legacy cruft, but
> if you ignore the ugly parts, the good parts that are left are fine to
> program in, though there are still stupid issues of type confusion.
> 
> If you like Haskell you should look at Purescript (purescript.org),
> which is a Haskell-resembling language with strict evaluation, designed
> to compile straightforwardly into Javascript and interoperate with it.
> Several other backends are also available or in progress.

Or if you are a rare one that likes Python, you can use PyJS, which
translates Python into Javascript.



From no.email at nospam.invalid  Fri May 22 23:43:33 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Fri, 22 May 2015 20:43:33 -0700
Subject: decorators and alarms
References: <CANy1k1gkXuTKEPL-ku7+c8mCysGgPgb62ZAgDdCrXXs6Z1S8Yg@mail.gmail.com>
 <mailman.250.1432351246.17265.python-list@python.org>
Message-ID: <87zj4wx8ka.fsf@jester.gateway.sonic.net>

Jason Friedman <jsf80238 at gmail.com> writes:
> Found an answer:
> https://wiki.python.org/moin/PythonDecoratorLibrary#Function_Timeout

That is pretty limited since signal handlers always run in the main
thread, and that wrapper takes over the sigalarm handler which some
other part of the program might also want to use.

In general what you're asking for is a hassle in Python.  In the limited
case, using sigalarm like that can be ok.  In some other cases there's a
hack involving the C API that lets you kill a thread asynchronously (I
haven't had to use that so far).  Some other times (you're waiting on
socket i/o rather than computation), you can run the i/o operation in a
new thread, sleep til the timeout, then return an error code, arranging
to ignore the i/o op when it finally finishes or times out, so you leave
a thread lingering around for a while after you've given up on it (I've
done that).  You could use async i/o and track everything yourself (I
haven't done that except slightly).  You could use subprocesses and
actually kill them on timeout (I've done that).  Erlang has its act
together better than Python does for this sort of thing.


From stefan_ml at behnel.de  Fri May 22 23:58:10 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 23 May 2015 05:58:10 +0200
Subject: Strategies for using cffi with C++ code?
In-Reply-To: <CANc-5UwsOTUvMCnZ_vi3nqu6O2sCPqkAwCHY=aEiT+qY-iLdQQ@mail.gmail.com>
References: <skip.montanaro@gmail.com>
 <CANc-5UyK4+jAxZ_WDnEbM7RgAQk=k2uKJs8a=m1phGeb8DfAYQ@mail.gmail.com>
 <201505211706.t4LH6GY0006031@fido.openend.se>
 <CANc-5UyBC-vw_+Og-1gea9YXubpATezsOmNf58AKKgb0swBP9A@mail.gmail.com>
 <87oalclf0d.fsf@nautilus.nautilus>
 <CANc-5UwsOTUvMCnZ_vi3nqu6O2sCPqkAwCHY=aEiT+qY-iLdQQ@mail.gmail.com>
Message-ID: <mjotsi$vvh$1@ger.gmane.org>

Skip Montanaro schrieb am 22.05.2015 um 19:15:
> 2015-05-22 12:05 GMT-05:00 Lele Gaifax:
>> Maybe http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html?
> 
> Thanks for the reference, Lele. I was thinking in terms of cffi, but
> this might work as well. (Shouldn't cffi interfaces be the thinnest?)

Thin in what sense? cffi is a generic library (at least on CPython), so the
interface can never be as thin as what a dedicated compiler can generate
for a specific piece of interface code. PyPy's cffi can mitigate this by
applying runtime optimisations, but you can't do that in CPython without
running some kind of native code generator, be it a JIT compiler or a
static compiler. cffi can apply the latter (run a C compiler) to a certain
extent, but then you end up with a dependency on a C compiler at *runtime*.
The term "thin" really doesn't apply to that dependency. And even if you
accept to go down that route, you'd still get better results from runtime
compilation with Cython, as it will additionally optimise your interface
code (and thus make it "thinner").

Being a Cython core developer, I'm certainly a somewhat biased expert, but
using Cython to generate a statically compiled and optimised C++ wrapper is
really your best choice. IMHO, it provides the best results/tradeoffs in
terms of developer effort, runtime performance and overall end user experience.

Stefan




From ian.g.kelly at gmail.com  Sat May 23 00:10:44 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 22 May 2015 22:10:44 -0600
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <555FF482.8020007@gmail.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com>
 <mjomke$saf$1@ger.gmane.org> <555FF482.8020007@gmail.com>
Message-ID: <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>

On Fri, May 22, 2015 at 9:31 PM, Michael Torrie <torriem at gmail.com> wrote:
> On 05/22/2015 07:54 PM, Terry Reedy wrote:
>> On 5/22/2015 5:40 PM, Tim Daneliuk wrote:
>>
>>> Lo these many years ago, I argued that Python is a whole lot more than
>>> a programming language:
>>>
>>>     https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
>>
>> Perhaps something at tundraware needs updating.
>> '''
>> This Connection is Untrusted
>>
>> You have asked Firefox to connect securely to www.tundraware.com, but we
>> can't confirm that your connection is secure.
>>
>> Normally, when you try to connect securely, sites will present trusted
>> identification to prove that you are going to the right place. However,
>> this site's identity can't be verified.
>> '''
>
> Sigh. I blame this as much on the browser.  There's no inherent reason
> why a connection to a site secured with a self-signed certificate is
> insecure.  In fact it's definitely not.

Sure it is. Without some prior reason to trust the certificate, the
certificate is meaningless. How is the browser to distinguish between
a legitimate self-signed cert and a self-signed cert presented by an
attacker conducting a man-in-the-middle attack?

There is still some value in TLS with a self-signed certificate in
that at least the connection is encrypted and can't be eavesdropped by
an attacker who can only read the channel, but there is no assurance
that the party you're communicating with actually owns the public key
that you've been presented.


From amber.of.luxor at gmail.com  Sat May 23 00:11:34 2015
From: amber.of.luxor at gmail.com (amber)
Date: Sat, 23 May 2015 04:11:34 +0000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <555FA253.3020304@tundraware.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com>
Message-ID: <555FFDF6.3030605@gmail.com>

??

On 22/05/2015 21:40, Tim Daneliuk wrote:
>    https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
Quoting that article
?And no, you couldn't get a C based OS to do what TPF does even if you
did have a couple hundred million dollars to redo it, ?

Why couldn't a C based OS do what TPF does?

My understanding is that if the programming language is Turing-Complete,
it can do anything that any other Turing-Complete language can do.  And
that assembler is a Turing-Complete language.

jonathon






From ben+python at benfinney.id.au  Sat May 23 00:20:57 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 23 May 2015 14:20:57 +1000
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
Message-ID: <85382nylee.fsf@benfinney.id.au>

Ian Kelly <ian.g.kelly at gmail.com> writes:

> On Fri, May 22, 2015 at 9:31 PM, Michael Torrie <torriem at gmail.com> wrote:
> > On 05/22/2015 07:54 PM, Terry Reedy wrote:
> >> On 5/22/2015 5:40 PM, Tim Daneliuk wrote:
> >>
> >>> Lo these many years ago, I argued that Python is a whole lot more than
> >>> a programming language:
> >>>
> >>>     https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
> >>
> >> Perhaps something at tundraware needs updating.
> >> '''
> >> This Connection is Untrusted
> >>
> >> You have asked Firefox to connect securely to www.tundraware.com, but we
> >> can't confirm that your connection is secure.
> >> [?]

> Without some prior reason to trust the certificate, the certificate is
> meaningless. How is the browser to distinguish between a legitimate
> self-signed cert and a self-signed cert presented by an attacker
> conducting a man-in-the-middle attack?

Any unencrypted HTTP (?http://??) connection has the same problem. Yet
the same browsers don't present a big scary warning for those?

The flaw in the browser is that it doesn't complain when an unencrypted
HTTP connection is established, but only complains when an *encrypted*
connection is made to a site with a self-signed certificate.

> There is still some value in TLS with a self-signed certificate in
> that at least the connection is encrypted and can't be eavesdropped by
> an attacker who can only read the channel, but there is no assurance
> that the party you're communicating with actually owns the public key
> that you've been presented.

Right. By that logic, let's advocate for browsers to present a big
intrusive warning for every HTTP connection that has no SSL layer or
certificate.

I will agree that a self-signed certificate presents the problem of how
to verify the certificate automatically.

Where I disagree is that this is somehow less secure than a completely
*unencrypted* HTTP connection. No, the opposite is true.

-- 
 \     ?DRM doesn't inconvenience [lawbreakers] ? indeed, over time it |
  `\     trains law-abiding users to become [lawbreakers] out of sheer |
_o__)                        frustration.? ?Charles Stross, 2010-05-09 |
Ben Finney



From rosuav at gmail.com  Sat May 23 00:21:05 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 23 May 2015 14:21:05 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
Message-ID: <CAPTjJmpWpZZ8ESM4ptS3E172v=+xuh6tL+znZunzg5TTWZxwSQ@mail.gmail.com>

On Sat, May 23, 2015 at 2:10 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> Sigh. I blame this as much on the browser.  There's no inherent reason
>> why a connection to a site secured with a self-signed certificate is
>> insecure.  In fact it's definitely not.
>
> Sure it is. Without some prior reason to trust the certificate, the
> certificate is meaningless. How is the browser to distinguish between
> a legitimate self-signed cert and a self-signed cert presented by an
> attacker conducting a man-in-the-middle attack?
>
> There is still some value in TLS with a self-signed certificate in
> that at least the connection is encrypted and can't be eavesdropped by
> an attacker who can only read the channel, but there is no assurance
> that the party you're communicating with actually owns the public key
> that you've been presented.

To be fair, certificates never actually tell you that the owner is
legitimate - all they do is move the problem. Self-signed certs move
the problem to "how do you get a guaranteed copy of this exact
server's certificate", which makes it an out-of-band issue (if you
meet someone you know in person and get a copy of the cert on a USB
stick, then manually install it, you can be sure it's safe);
externally-signed certs move the problem to the certificate chain and
its reliability (how well do the CSAs check ownership prior to issuing
a certificate?). Both are still problematic, just in different ways.

Self-signed certs are ideal if you're packaging your own client - you
could keep the IP address and certificate in the same VCS repository.
Anyone who can change the cert can also change the IP address, so you
lose no security there. But they're way WAY more hassle for https on
the public internet.

ChrisA


From rosuav at gmail.com  Sat May 23 00:28:59 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 23 May 2015 14:28:59 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <85382nylee.fsf@benfinney.id.au>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
 <85382nylee.fsf@benfinney.id.au>
Message-ID: <CAPTjJmr0a3D3SJop=A58M6=fdgdyLr84bmOyav5pVX-fodCV9w@mail.gmail.com>

On Sat, May 23, 2015 at 2:20 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Where I disagree is that this is somehow less secure than a completely
> *unencrypted* HTTP connection. No, the opposite is true.

No, it isn't less secure. However, people have been trained for years
to look for the padlock (including looking for padlocks before
entering credit card numbers or passwords, despite the fact that HTTPS
on the form isn't actually what's significant), and that's the key
here. Web browsers are intended for *humans* to use. You want a truly
secure connection between your Python client script and your Python
server? Sure, self-signed cert is great. You want something that an
average Joe can understand? Do what 99% of the world does, and get a
CSA-signed cert. Unencrypted is normal, encrypted is normal, and the
only thing that's being flagged is "hey, this *looks* secured, but it
might not be the right server". It's still encrypted, but the
unverified origin is a potential problem.

ChrisA


From ian.g.kelly at gmail.com  Sat May 23 00:29:52 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 22 May 2015 22:29:52 -0600
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <85382nylee.fsf@benfinney.id.au>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com>
 <mjomke$saf$1@ger.gmane.org> <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
 <85382nylee.fsf@benfinney.id.au>
Message-ID: <CALwzid=TUWaBinRXPZDpiHw9XqMYCcBY=bagQphbFYZnHxDvVw@mail.gmail.com>

On Fri, May 22, 2015 at 10:20 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Ian Kelly <ian.g.kelly at gmail.com> writes:
>
>> On Fri, May 22, 2015 at 9:31 PM, Michael Torrie <torriem at gmail.com> wrote:
>> > On 05/22/2015 07:54 PM, Terry Reedy wrote:
>> >> On 5/22/2015 5:40 PM, Tim Daneliuk wrote:
>> >>
>> >>> Lo these many years ago, I argued that Python is a whole lot more than
>> >>> a programming language:
>> >>>
>> >>>     https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
>> >>
>> >> Perhaps something at tundraware needs updating.
>> >> '''
>> >> This Connection is Untrusted
>> >>
>> >> You have asked Firefox to connect securely to www.tundraware.com, but we
>> >> can't confirm that your connection is secure.
>> >> [?]
>
>> Without some prior reason to trust the certificate, the certificate is
>> meaningless. How is the browser to distinguish between a legitimate
>> self-signed cert and a self-signed cert presented by an attacker
>> conducting a man-in-the-middle attack?
>
> Any unencrypted HTTP (?http://??) connection has the same problem. Yet
> the same browsers don't present a big scary warning for those?
>
> The flaw in the browser is that it doesn't complain when an unencrypted
> HTTP connection is established, but only complains when an *encrypted*
> connection is made to a site with a self-signed certificate.
>
>> There is still some value in TLS with a self-signed certificate in
>> that at least the connection is encrypted and can't be eavesdropped by
>> an attacker who can only read the channel, but there is no assurance
>> that the party you're communicating with actually owns the public key
>> that you've been presented.
>
> Right. By that logic, let's advocate for browsers to present a big
> intrusive warning for every HTTP connection that has no SSL layer or
> certificate.
>
> I will agree that a self-signed certificate presents the problem of how
> to verify the certificate automatically.
>
> Where I disagree is that this is somehow less secure than a completely
> *unencrypted* HTTP connection. No, the opposite is true.

I don't disagree with you. There *should* be scary warnings for plain
HTTP connections (although there is a counter-argument that many sites
don't need any encryption and HTTPS would just be wasteful in those
cases). The fact that browsers don't yet provide those warnings
doesn't change anything that I wrote above.


From torriem at gmail.com  Sat May 23 00:30:07 2015
From: torriem at gmail.com (Michael Torrie)
Date: Fri, 22 May 2015 22:30:07 -0600
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
Message-ID: <5560024F.7000800@gmail.com>

On 05/22/2015 10:10 PM, Ian Kelly wrote:
> On Fri, May 22, 2015 at 9:31 PM, Michael Torrie <torriem at gmail.com> wrote:
>> On 05/22/2015 07:54 PM, Terry Reedy wrote:
>>> On 5/22/2015 5:40 PM, Tim Daneliuk wrote:
>>>
>>>> Lo these many years ago, I argued that Python is a whole lot more than
>>>> a programming language:
>>>>
>>>>     https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
>>>
>>> Perhaps something at tundraware needs updating.
>>> '''
>>> This Connection is Untrusted
>>>
>>> You have asked Firefox to connect securely to www.tundraware.com, but we
>>> can't confirm that your connection is secure.
>>>
>>> Normally, when you try to connect securely, sites will present trusted
>>> identification to prove that you are going to the right place. However,
>>> this site's identity can't be verified.
>>> '''
>>
>> Sigh. I blame this as much on the browser.  There's no inherent reason
>> why a connection to a site secured with a self-signed certificate is
>> insecure.  In fact it's definitely not.
> 
> Sure it is. Without some prior reason to trust the certificate, the
> certificate is meaningless. How is the browser to distinguish between
> a legitimate self-signed cert and a self-signed cert presented by an
> attacker conducting a man-in-the-middle attack?

How does a CA actually help this problem?  It just puts trust in some
third party. But as we know, CA authorities are not all trustworthy and
they certainly don't guarantee that the site is what it says it is.  A
valid SSL cert does not mean the site won't try to hack your browser or
steal your identity.  The current system lulls us into a false sense of
security.  A self-signed cert is perfectly secure if you can verify the
fingerprint with the site's owner.  Granted that process is the rub.

> There is still some value in TLS with a self-signed certificate in 
> that at least the connection is encrypted and can't be eavesdropped
> by an attacker who can only read the channel, but there is no
> assurance that the party you're communicating with actually owns the
> public key that you've been presented.

The same can be said of CA-signed certificates.  The only way to know if
the site is who they say they are is to know what the cert's fingerprint
ought to be and see if it still is. I used to use a firefox plugin for
this purpose, but certs for some major sites like even www.google.com
change with such frequency that the utility of the plugin went away.


From ian.g.kelly at gmail.com  Sat May 23 00:49:05 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 22 May 2015 22:49:05 -0600
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <5560024F.7000800@gmail.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com>
 <mjomke$saf$1@ger.gmane.org> <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
 <5560024F.7000800@gmail.com>
Message-ID: <CALwzid=9nT8aT7BjB5usAjsQ2F9z_5y6Tc8x+iV6APr_-tM_pg@mail.gmail.com>

On Fri, May 22, 2015 at 10:30 PM, Michael Torrie <torriem at gmail.com> wrote:
> On 05/22/2015 10:10 PM, Ian Kelly wrote:
>> Sure it is. Without some prior reason to trust the certificate, the
>> certificate is meaningless. How is the browser to distinguish between
>> a legitimate self-signed cert and a self-signed cert presented by an
>> attacker conducting a man-in-the-middle attack?
>
> How does a CA actually help this problem?  It just puts trust in some
> third party. But as we know, CA authorities are not all trustworthy and
> they certainly don't guarantee that the site is what it says it is.

Nobody is forcing you to trust them. Go ahead and remove the CA
certificates that you consider untrustworthy if you want. Remove all
of them if you like, although good luck with verifying all those site
certificates yourself.

The CA helps because some assurance is better than none.

>> There is still some value in TLS with a self-signed certificate in
>> that at least the connection is encrypted and can't be eavesdropped
>> by an attacker who can only read the channel, but there is no
>> assurance that the party you're communicating with actually owns the
>> public key that you've been presented.
>
> The same can be said of CA-signed certificates.  The only way to know if
> the site is who they say they are is to know what the cert's fingerprint
> ought to be and see if it still is. I used to use a firefox plugin for
> this purpose, but certs for some major sites like even www.google.com
> change with such frequency that the utility of the plugin went away.

So instead of trusting a CA, you have to trust the maintainers of the
plugin. How is that any different?


From rosuav at gmail.com  Sat May 23 00:49:50 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 23 May 2015 14:49:50 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <CALwzid=TUWaBinRXPZDpiHw9XqMYCcBY=bagQphbFYZnHxDvVw@mail.gmail.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
 <85382nylee.fsf@benfinney.id.au>
 <CALwzid=TUWaBinRXPZDpiHw9XqMYCcBY=bagQphbFYZnHxDvVw@mail.gmail.com>
Message-ID: <CAPTjJmo9p9bUuP38k9ju1O8Ja=O6Z4oWh2gOcHicTfdo_2pCsw@mail.gmail.com>

On Sat, May 23, 2015 at 2:29 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> There *should* be scary warnings for plain
> HTTP connections (although there is a counter-argument that many sites
> don't need any encryption and HTTPS would just be wasteful in those
> cases).

I don't think there should be "scary warnings", for precisely this
reason. When the information you're sharing is completely public,
there's no point taking the overhead of encryption. So there should be
two normal and acceptable ways to access data: either unencrypted, or
encrypted with a verified certificate. Oh look, that's what we have.
There is an assumption that your system certificate store is
trustworthy, but for the typical user, it's probably better than
they'll get any other way, and for an atypical user, it can be pruned
easily.

But I think we're just a smidge off-topic here.

ChrisA


From rosuav at gmail.com  Sat May 23 00:55:43 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 23 May 2015 14:55:43 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <CALwzid=9nT8aT7BjB5usAjsQ2F9z_5y6Tc8x+iV6APr_-tM_pg@mail.gmail.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
 <5560024F.7000800@gmail.com>
 <CALwzid=9nT8aT7BjB5usAjsQ2F9z_5y6Tc8x+iV6APr_-tM_pg@mail.gmail.com>
Message-ID: <CAPTjJmqPHJWP=khqJm0hg8Y5g8B+zDXDzyQELqu0BZ_+mjmxBA@mail.gmail.com>

On Sat, May 23, 2015 at 2:49 PM, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> The same can be said of CA-signed certificates.  The only way to know if
>> the site is who they say they are is to know what the cert's fingerprint
>> ought to be and see if it still is. I used to use a firefox plugin for
>> this purpose, but certs for some major sites like even www.google.com
>> change with such frequency that the utility of the plugin went away.
>
> So instead of trusting a CA, you have to trust the maintainers of the
> plugin. How is that any different?

It brings it local. If you're able to see the source code for the
plugin, you could check exactly how it does its verification (and by
the sound of it, it'd be pretty simple: just look up the cert, see if
it's different, if so, big noisy warning). Or, of course, you could do
the check yourself: click on the padlock, look at fingerprint, compare
against previously-noted fingerprint. That'd at least prove that your
plugin is checking properly.

But it still doesn't solve the fundamental problem of knowing when you
have the right site to start with.

ChrisA


From nad at acm.org  Sat May 23 02:31:19 2015
From: nad at acm.org (Ned Deily)
Date: Fri, 22 May 2015 23:31:19 -0700
Subject: need help with an accessibility prototype
References: <555F5960.9030501@harvee.org>
 <201505221950.t4MJo3pC010042@fido.openend.se>
 <201505222216.t4MMG0IN013302@fido.openend.se>
Message-ID: <nad-40F5BC.23311922052015@news.gmane.org>

In article <201505222216.t4MMG0IN013302 at fido.openend.se>,
 Laura Creighton <lac at openend.se> wrote:
> By the way http://www.tcl.tk/man/tcl8.4/TkCmd/keysyms.htm
> is a list of all the key symbols it is possible for you to get.
> There is no guarantee that any platform actually has any of them,
> just, if it is not there don't bother trying to look for it.

Tcl/Tk 8.4 is quite old and no longer maintained; 8.6.x is current, 
although 8.5.x is also still in use.

http://www.tcl.tk/man/tcl8.6/TkCmd/keysyms.htm

Alas, there are an unfortunately large number of platform differences in 
Tk behavior across the various platforms it supports, primarily Windows, 
X11, and OS X.  Among other resources, there is a useful wiki page that 
tries to enumerate keyboard and keyboard modifier differences across 
platforms:

http://wiki.tcl.tk/28331

Also, in addition to effbot's venerable Tkinter documentation, more 
up-to-date material is available at Mark Roseman's TkDocs website and in 
his e-book "Modern Tkinter".

http://www.tkdocs.com

-- 
 Ned Deily,
 nad at acm.org



From savithad8 at gmail.com  Sat May 23 02:46:06 2015
From: savithad8 at gmail.com (savitha devi)
Date: Sat, 23 May 2015 12:16:06 +0530
Subject: Extract email address from Java script in html source using python
Message-ID: <CAAXuHoeJ-YMQDB85qLDJ_o+9CrsfwLvm9wuOaRtbSj-i9kBaFA@mail.gmail.com>

I am developing a web scraper code using HTMLParser. I need to extract
text/email address from java script with in the HTMLCode.I am beginner
level in python coding and totally lost here. Need some help on this. The
java script code is as below:

<script type='text/javascript'>
 //<!--
 document.getElementById('cloak48218').innerHTML = '';
 var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
 var path = 'hr' + 'ef' + '=';
 var addy48218 = '&#105;nf&#111;' + '&#64;';
 addy48218 = addy48218 + 'tsv-n&#101;&#117;r&#105;&#101;d' + '&#46;' +
'd&#101;';
 document.getElementById('cloak48218').innerHTML += '<a ' + path +
'\'' + prefix + ':' + addy48218 + '\'>' + addy48218+'<\/a>';
 //-->
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150523/8f56c3e4/attachment.html>

From dfnsonfsduifb at gmx.de  Sat May 23 02:55:10 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sat, 23 May 2015 08:55:10 +0200
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mailman.252.1432352224.17265.python-list@python.org>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
Message-ID: <mjp88e$e1$1@news.albasani.net>

On 23.05.2015 05:31, Michael Torrie wrote:

> Sigh. I blame this as much on the browser.  There's no inherent reason
> why a connection to a site secured with a self-signed certificate is
> insecure.

The problem is *not* that the certificate is self-signed.

It's that it's unknown previously to being encountered within the TLS
handshake. And that *does* make it inherently insecure.

Not algorithmically, obviously.  I can still do a DH-handshake with the
remote peer that will generate a shared secret no eavesdropper will
know. The browser just can't be sure that whoever it negotiated the DH
with is really the endpoint (i.e. the webserver). That is the problem.

I dislike CAs as much as the next guy. But the problem of distributing
trust is just not easy to solve, a TTP is a way out. Do you have an
alternative that does not at the same time to providing a solution also
opens up obvious attack surface?

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From rosuav at gmail.com  Sat May 23 04:59:27 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 23 May 2015 18:59:27 +1000
Subject: Camelot a good tool for me
In-Reply-To: <87fv6ol93p.fsf@nautilus.nautilus>
References: <87zj4xoxfd.fsf@Equus.decebal.nl> <mjn804$4pe$1@dont-email.me>
 <87siaopxjs.fsf@Equus.decebal.nl>
 <201505221357.t4MDvgIV001904@fido.openend.se>
 <87k2w0le41.fsf@nautilus.nautilus>
 <CAPTjJmpnYwhOWdBz-=Zv8x+BO0G3e7OesPDvzd=nsX2a8Gsc-w@mail.gmail.com>
 <87fv6ol93p.fsf@nautilus.nautilus>
Message-ID: <CAPTjJmoofyT1dWwoDBJWXtgs99cLiQQE-dvA1P7XJQvYxg5-sg@mail.gmail.com>

On Sat, May 23, 2015 at 5:12 AM, Lele Gaifax <lele at metapensiero.it> wrote:
> Chris Angelico <rosuav at gmail.com> writes:
>
>> SQLAlchemy has its uses, and it does solve a number of
>> issues in reasonably clean ways, but I don't like a few of its facets,
>> including its peculiar way of doing foreign key relationships. (You
>> put a foreign key in the child, and you put a relationship in the
>> parent, which feels backwards.)
>
> You are conflating two different layers, core and ORM. ORM relationships can
> be declared either on the parent or on the child, it's up to your taste.

Not sure why that's distinguishable. If I have two tables like this:

Users:
    id sequential primary key
    name text

Tasks
    id sequential primary key
    owner integer
    assignee integer

Both the owner and the assignee refer to the Users table; the owner is
a mandatory connection (every task was created by someone, who
initially owns it), and the assignee is an optional connection (a
newly-created task isn't assigned to anyone). With me so far? Okay.

Now, if I were to represent these tables in SQLAlchemy, obviously I
need to have foreign key relationships encoded in SQLAlchemy. But if
I'm to enforce these relationships on the underlying database, it's
equally obvious that I need foreign key constraints. I would expect
that a relationship encoded in SQLAlchemy should cause the creation of
a constraint in the database. They're fundamentally the same thing.

My point about backwards is that my tables here are declared in a
strict order: parent table, then child table. In the child table, a
constraint is created by saying "references Users", and the Users
table already exists. At no point is there ever a forward reference.
Code would look like this:

create table Users (id serial primary key, name text not null default '');
create table Tasks (id serial primary key, owner integer not null
references Users, assignee integer references Users);

But with SQLAlchemy, you have a tag in the Users table's definition
saying that it has a relationship with Tasks, as well as a foreign key
in Tasks stating the connection to Users. That violates "Define Before
Use", which isn't a strict policy, but it does feel a little bit
"dirty".

>> When magic works, it's great; but when anything goes wrong, it's harder to
>> see what happened.
>
> The same can be said of almost any upper layer in a software stack.

Precisely. All magic has to justify itself. Some can, easily. Some
can't. A lot is in the middle, where it's part of the tradeoffs.

>> Also, when does a transaction begin and end?
>
> When I need transactions (that is, when I'm changing the database) I'm very
> picky and use explicit begins, commits and rollbacks, so I don't recall
> experiencing that doubt.

That's all very well when you write your own code. Now try picking up
someone else's code. Or, for a mid-way concern, try explaining to a
junior developer how to make sure his transactions are right. With
psycopg2, it's easy enough to do this:

with conn, conn.cursor() as cur:
    cur.execute("....")
    cur.execute("....")

When the with block exits, the transaction is either committed (if all
went well) or rolled back (if an exception was raised). It's very
simple, easy to do, and easy to audit ("all SQL queries must be inside
a with block that grants a cursor", and possibly "cursor-granting with
blocks must not be nested").

>> If you session.commit() in the middle of iterating over a query, will it
>> break the query? What if you roll back? Can you see, instantly, in your
>> code?
>
> Why would you do that? Are you closing your files while you iterate them,
> without leaving the loop in some way at the same time?

There's advice out there on the internet that says that committing
periodically in the middle of a big job makes your code run faster.
It's from the PHP + MySQL school of thought, where the assumption is
that finishing is the most important thing, finishing quickly is a
close second, and guaranteeing correctness isn't even on the radar.
Now try coping with code that was written under that model.

Fortunately, I haven't actually seen anything quite like this in
SQLAlchemy. The worst I saw was a case where someone was iterating
over a query and performed another query, which did indeed break the
fetching of results. But the more magic you have, the less obvious
that is.

>> Even if the ORM layer is practically perfect in every way,
>> there's still value in learning SQL; for instance, if you drop to a
>> command-line interpreter like PostgreSQL's psql, or if you switch to
>> another language, or anything like that, it's helpful to know what's
>> going on under the covers. And if you have to know SQL anyway, the
>> advantage of the abstraction layer has to justify the cost of learning
>> an additional, not a replacement, API.
>
> No doubt on that. Working with SQLAlchemy is not an alterative to knowing SQL
> fairly well. SA does hide "details" (name quoting syntax, to mention one
> obvious detail), but does not even try to hide the fact that it's talking SQL
> all the way down.

Yep. SQLAlchemy is miles ahead of some of the things I've worked with.
And I have to say, the declarative style of laying out a table is
excellent. Using psycopg2 or SQLAlchemy is a matter of taste; using
either of the above or DBExpert is a matter of insanity.

> I often have to deal with multiple DB engines at the same time, and being
> able to "write" my queries with an abstract syntax is very valuable for me.

Hmm, I'm not sure there's all that much that this helps with. Thanks
to a consistent PEP 249 API, changing database engines is often just a
matter of changing one import and a connection construction line
(which you'd have to do anyway, given that the credentials will
change). What else is there for SQLAlchemy to paper over? The
differences remain; the common ground is already common.

>> So, while SQLAlchemy is definitely a lot better than most I've seen,
>> it's still not really good enough for me to use everywhere. I'm mostly
>> going to stick to the Python DB API 2.0.
>
> Again, I fully agree: one should work with whatever tool he feels comfortable
> with!

Yep. Both are good enough to use. Personally, I'll use psycopg2, but
if you want to use SQLAlchemy, I won't think you a fool. Same if you
choose Django (I use Flask), etc. They're close enough that
situational differences make all the argument.

Just don't use MySQL if you can help it.

*ducks for cover*

ChrisA


From rosuav at gmail.com  Sat May 23 05:01:55 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 23 May 2015 19:01:55 +1000
Subject: Extract email address from Java script in html source using python
In-Reply-To: <CAAXuHoeJ-YMQDB85qLDJ_o+9CrsfwLvm9wuOaRtbSj-i9kBaFA@mail.gmail.com>
References: <CAAXuHoeJ-YMQDB85qLDJ_o+9CrsfwLvm9wuOaRtbSj-i9kBaFA@mail.gmail.com>
Message-ID: <CAPTjJmqBDu0nB2u_mf7KMpFzxxJFUqB7o-7dJiGgu-xyOL2uzg@mail.gmail.com>

On Sat, May 23, 2015 at 4:46 PM, savitha devi <savithad8 at gmail.com> wrote:
> I am developing a web scraper code using HTMLParser. I need to extract
> text/email address from java script with in the HTMLCode.I am beginner level
> in python coding and totally lost here. Need some help on this. The java
> script code is as below:
>
> <script type='text/javascript'>
>  //<!--
>  document.getElementById('cloak48218').innerHTML = '';
>  var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
>  var path = 'hr' + 'ef' + '=';
>  var addy48218 = '&#105;nf&#111;' + '&#64;';
>  addy48218 = addy48218 + 'tsv-n&#101;&#117;r&#105;&#101;d' + '&#46;' +
> 'd&#101;';
>  document.getElementById('cloak48218').innerHTML += '<a ' + path + '\'' +
> prefix + ':' + addy48218 + '\'>' + addy48218+'<\/a>';
>  //-->

This is deliberately being done to prevent scripted usage. What
exactly are you needing to do this for?

You're basically going to have to execute the entire block of
JavaScript code, and then decode the entities to get to what you want.
Doing it manually is pretty easy; doing it automatically will
virtually require a language interpreter.

ChrisA


From breamoreboy at yahoo.co.uk  Sat May 23 05:12:30 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 23 May 2015 10:12:30 +0100
Subject: PYTHONPATH when calling from ipython
In-Reply-To: <874mn5qjc8.fsf@Equus.decebal.nl>
References: <874mn5qjc8.fsf@Equus.decebal.nl>
Message-ID: <mjpga8$ubp$1@ger.gmane.org>

On 22/05/2015 06:20, Cecil Westerhof wrote:
> I am looking into using ipython instead of bash. But when I call a
> python program from ipython PYTHONPATH is not set. So pythonscripts
> that need a module through PYTHONPATH will not work.
>
> I could do something like:
>      !PYTHONPATH=~/Python/PythonLibrary python2 ?
>
> But I find that a little bit cumbersome. Is there a better way to do
> it?
>

What makes you think this?  Have you tried:-

 >>> import os
 >>> os.environ['PYTHONPATH']
'C:\\Users\\Mark\\Documents\\Cash\\Python;C:\\Users\\Mark\\Documents\\MyPython'

That might be from the command line interpreter but it also works the 
same from iPython for me on Windows 8.1.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lac at openend.se  Sat May 23 05:25:49 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 23 May 2015 11:25:49 +0200
Subject: need help with an accessibility prototype
In-Reply-To: Message from Ned Deily <nad@acm.org> of "Fri,
 22 May 2015 23:31:19 -0700." <nad-40F5BC.23311922052015@news.gmane.org>
References: <555F5960.9030501@harvee.org>
 <201505221950.t4MJo3pC010042@fido.openend.se>
 <201505222216.t4MMG0IN013302@fido.openend.se><nad-40F5BC.23311922052015@news.gmane.org>
Message-ID: <201505230925.t4N9PnZ8028675@fido.openend.se>

In a message of Fri, 22 May 2015 23:31:19 -0700, Ned Deily writes:
>Tcl/Tk 8.4 is quite old and no longer maintained; 8.6.x is current, 
>although 8.5.x is also still in use.
>
>http://www.tcl.tk/man/tcl8.6/TkCmd/keysyms.htm
>
> Ned Deily,
> nad at acm.org

Thank you Ned.  Old bookmark from when I needed this the last time --
sorry about that.  Do you know if you still need to use 8.5 if you are
on Mac OS? i.e.  is this page current?
https://www.python.org/download/mac/tcltk/

Laura


From Cecil at decebal.nl  Sat May 23 06:53:43 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 23 May 2015 12:53:43 +0200
Subject: PYTHONPATH when calling from ipython
References: <874mn5qjc8.fsf@Equus.decebal.nl>
 <mailman.268.1432372378.17265.python-list@python.org>
Message-ID: <87iobjpnt4.fsf@Equus.decebal.nl>

Op Saturday 23 May 2015 11:12 CEST schreef Mark Lawrence:

> On 22/05/2015 06:20, Cecil Westerhof wrote:
>> I am looking into using ipython instead of bash. But when I call a
>> python program from ipython PYTHONPATH is not set. So pythonscripts
>> that need a module through PYTHONPATH will not work.
>>
>> I could do something like:
>> !PYTHONPATH=~/Python/PythonLibrary python2 ?
>>
>> But I find that a little bit cumbersome. Is there a better way to
>> do it?
>>
>
> What makes you think this?  Have you tried:-
>
>>>> import os
>>>> os.environ['PYTHONPATH']
> 'C:\\Users\\Mark\\Documents\\Cash\\Python;C:\\Users\\Mark\\Documents\\MyPython'
>
> That might be from the command line interpreter but it also works
> the same from iPython for me on Windows 8.1.

That does not change anything. The modules are not found. Also not
when using %run.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From jon+usenet at unequivocal.co.uk  Sat May 23 07:10:30 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Sat, 23 May 2015 11:10:30 +0000 (UTC)
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
 <mailman.257.1432355416.17265.python-list@python.org>
Message-ID: <slrnmm0o3c.864.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-23, Michael Torrie <torriem at gmail.com> wrote:
> On 05/22/2015 10:10 PM, Ian Kelly wrote:
>> There is still some value in TLS with a self-signed certificate in 
>> that at least the connection is encrypted and can't be eavesdropped
>> by an attacker who can only read the channel, but there is no
>> assurance that the party you're communicating with actually owns the
>> public key that you've been presented.
>
> The same can be said of CA-signed certificates.

I think you are falling into the trap of believing that all things are
either perfect or they are worthless. CAs aren't perfect, but neither
are they worthless. A self-signed certificate, however, is worthless.

> The only way to know if the site is who they say they are is to know
> what the cert's fingerprint ought to be and see if it still is. I
> used to use a firefox plugin for this purpose, but certs for some
> major sites like even www.google.com change with such frequency that
> the utility of the plugin went away.

http://en.wikipedia.org/wiki/HTTP_Public_Key_Pinning


From tundra at tundraware.com  Sat May 23 07:11:53 2015
From: tundra at tundraware.com (Tim Daneliuk)
Date: Sat, 23 May 2015 06:11:53 -0500
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mailman.255.1432354309.17265.python-list@python.org>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com>
 <mailman.255.1432354309.17265.python-list@python.org>
Message-ID: <55606079.4080500@tundraware.com>

On 05/22/2015 11:11 PM, amber wrote:
> ??
> 
> On 22/05/2015 21:40, Tim Daneliuk wrote:
>>    https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
> Quoting that article
> ?And no, you couldn't get a C based OS to do what TPF does even if you
> did have a couple hundred million dollars to redo it, ?
> 
> Why couldn't a C based OS do what TPF does?
> 
> My understanding is that if the programming language is Turing-Complete,
> it can do anything that any other Turing-Complete language can do.  And
> that assembler is a Turing-Complete language.
> 
> jonathon
> 
> 
> 
> 


C could - in principle - do it.  As you point out, both are Turing Complete.
But, "in principle" is quite different than "in actual fact".  At the time
that article was written almost 15 years ago, the TPF systems in use
were running applications written in hand optimized assembly language on the
largest mainframes money could buy.  Everything about that environment was
optimized for speed - from partially populating disk surfaces to running the
thinnest possible operating system executive.  (In those days, TPF itself didn't 
even have memory protection, or at least, it was just being added as an OS
feature.)

The point is that it would have been nearly impossible technically to
rewrite all that in C and expect the same level of minimal path length/
max throughput.  Give the sheer transaction load on those machines, this
would have been a showstopper.

But that was then and this is now.  These days, much of that capability 
HAS been written in higher level languages because of scale-out architectures
and messaging/queuing middleware.  Instead of one very fast centralized 
cluster of mainframes, we see thousands of servers spreading the workload
and achieving much the same results.  This has it's own challenges.  Managing
a few things is harder than managing thousands of things. 

-- 
----------------------------------------------------------------------------
Tim Daneliuk     tundra at tundraware.com
PGP Key:         http://www.tundraware.com/PGP/



From tundra at tundraware.com  Sat May 23 07:11:53 2015
From: tundra at tundraware.com (Tim Daneliuk)
Date: Sat, 23 May 2015 06:11:53 -0500
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mailman.255.1432354309.17265.python-list@python.org>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com>
 <mailman.255.1432354309.17265.python-list@python.org>
Message-ID: <55606079.4080500@tundraware.com>

On 05/22/2015 11:11 PM, amber wrote:
> ??
> 
> On 22/05/2015 21:40, Tim Daneliuk wrote:
>>    https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
> Quoting that article
> ?And no, you couldn't get a C based OS to do what TPF does even if you
> did have a couple hundred million dollars to redo it, ?
> 
> Why couldn't a C based OS do what TPF does?
> 
> My understanding is that if the programming language is Turing-Complete,
> it can do anything that any other Turing-Complete language can do.  And
> that assembler is a Turing-Complete language.
> 
> jonathon
> 
> 
> 
> 


C could - in principle - do it.  As you point out, both are Turing Complete.
But, "in principle" is quite different than "in actual fact".  At the time
that article was written almost 15 years ago, the TPF systems in use
were running applications written in hand optimized assembly language on the
largest mainframes money could buy.  Everything about that environment was
optimized for speed - from partially populating disk surfaces to running the
thinnest possible operating system executive.  (In those days, TPF itself didn't 
even have memory protection, or at least, it was just being added as an OS
feature.)

The point is that it would have been nearly impossible technically to
rewrite all that in C and expect the same level of minimal path length/
max throughput.  Give the sheer transaction load on those machines, this
would have been a showstopper.

But that was then and this is now.  These days, much of that capability 
HAS been written in higher level languages because of scale-out architectures
and messaging/queuing middleware.  Instead of one very fast centralized 
cluster of mainframes, we see thousands of servers spreading the workload
and achieving much the same results.  This has it's own challenges.  Managing
a few things is harder than managing thousands of things. 

-- 
----------------------------------------------------------------------------
Tim Daneliuk     tundra at tundraware.com
PGP Key:         http://www.tundraware.com/PGP/



From tundra at tundraware.com  Sat May 23 07:12:32 2015
From: tundra at tundraware.com (Tim Daneliuk)
Date: Sat, 23 May 2015 06:12:32 -0500
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mailman.248.1432346080.17265.python-list@python.org>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com>
 <mailman.248.1432346080.17265.python-list@python.org>
Message-ID: <556060A0.1010501@tundraware.com>

On 05/22/2015 08:54 PM, Terry Reedy wrote:
> On 5/22/2015 5:40 PM, Tim Daneliuk wrote:
> 
>> Lo these many years ago, I argued that Python is a whole lot more than
>> a programming language:
>>
>>     https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
> 
> Perhaps something at tundraware needs updating.
> '''
> This Connection is Untrusted
> 
> You have asked Firefox to connect securely to www.tundraware.com, but we can't confirm that your connection is secure.
> 
> Normally, when you try to connect securely, sites will present trusted identification to prove that you are going to the right place. However, this site's identity can't be verified.
> '''
> 

It's self signed - something done quite routinely on the net.

-- 
----------------------------------------------------------------------------
Tim Daneliuk     tundra at tundraware.com
PGP Key:         http://www.tundraware.com/PGP/



From tundra at tundraware.com  Sat May 23 07:12:32 2015
From: tundra at tundraware.com (Tim Daneliuk)
Date: Sat, 23 May 2015 06:12:32 -0500
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mailman.248.1432346080.17265.python-list@python.org>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com>
 <mailman.248.1432346080.17265.python-list@python.org>
Message-ID: <556060A0.1010501@tundraware.com>

On 05/22/2015 08:54 PM, Terry Reedy wrote:
> On 5/22/2015 5:40 PM, Tim Daneliuk wrote:
> 
>> Lo these many years ago, I argued that Python is a whole lot more than
>> a programming language:
>>
>>     https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/
> 
> Perhaps something at tundraware needs updating.
> '''
> This Connection is Untrusted
> 
> You have asked Firefox to connect securely to www.tundraware.com, but we can't confirm that your connection is secure.
> 
> Normally, when you try to connect securely, sites will present trusted identification to prove that you are going to the right place. However, this site's identity can't be verified.
> '''
> 

It's self signed - something done quite routinely on the net.

-- 
----------------------------------------------------------------------------
Tim Daneliuk     tundra at tundraware.com
PGP Key:         http://www.tundraware.com/PGP/



From tundra at tundraware.com  Sat May 23 07:21:25 2015
From: tundra at tundraware.com (Tim Daneliuk)
Date: Sat, 23 May 2015 06:21:25 -0500
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mjp88e$e1$1@news.albasani.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net>
Message-ID: <556062B5.7050800@tundraware.com>

On 05/23/2015 01:55 AM, Johannes Bauer wrote:
> On 23.05.2015 05:31, Michael Torrie wrote:
> 
>> Sigh. I blame this as much on the browser.  There's no inherent reason
>> why a connection to a site secured with a self-signed certificate is
>> insecure.
> 
> The problem is *not* that the certificate is self-signed.
> 
> It's that it's unknown previously to being encountered within the TLS
> handshake. And that *does* make it inherently insecure.
> 
> Not algorithmically, obviously.  I can still do a DH-handshake with the
> remote peer that will generate a shared secret no eavesdropper will
> know. The browser just can't be sure that whoever it negotiated the DH
> with is really the endpoint (i.e. the webserver). That is the problem.
> 
> I dislike CAs as much as the next guy. But the problem of distributing
> trust is just not easy to solve, a TTP is a way out. Do you have an
> alternative that does not at the same time to providing a solution also
> opens up obvious attack surface?
> 
> Cheers,
> Johannes
> 

Trust has context.  You're going to that site to read an article.  This
is rather different than, say, going somewhere to transact commerce or
move money.

I have been doing an experiment with tundraware.com to try out https
everywhere to see just what breaks and who squawks.  I've seen a number
of concerns like the ones on this thread.  Most interestingly, this
seems to be breaking the FreeBSD ports build mechanism for the ports
I'd previously contributed to the project.

This is a tough tradeoff.  If you don't run https, then your every interaction
with the website can be trivially monitored by a third party.   Even just
the metadata of when you do use https and when you do not can be useful
to an eavesdropper.  So, there is increasing thought that we should all just
run https everywhere all the time.  But then we run into the signing problem.
I am hoping that we will soon see free or inexpensive CAs to make that
problem go away.  See:

  https://www.eff.org/deeplinks/2014/11/certificate-authority-encrypt-entire-web



-- 
----------------------------------------------------------------------------
Tim Daneliuk     tundra at tundraware.com
PGP Key:         http://www.tundraware.com/PGP/



From tundra at tundraware.com  Sat May 23 07:29:41 2015
From: tundra at tundraware.com (Tim Daneliuk)
Date: Sat, 23 May 2015 06:29:41 -0500
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <mailman.260.1432356599.17265.python-list@python.org>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
 <85382nylee.fsf@benfinney.id.au>
 <CALwzid=TUWaBinRXPZDpiHw9XqMYCcBY=bagQphbFYZnHxDvVw@mail.gmail.com>
 <mailman.260.1432356599.17265.python-list@python.org>
Message-ID: <556064A5.5030502@tundraware.com>

On 05/22/2015 11:49 PM, Chris Angelico wrote:
> When the information you're sharing is completely public,
> there's no point taking the overhead of encryption.


I disagree.  With two different ways to access data, the metadata about
when you do- and do not use an encrypted channel can be useful to
a snoopy third party.  For example, repressive governments might use
the fact of your connecting via https as a prima facie evidence you're
doing something subversive.  The argument for https everywhere is that this
sort of distinction becomes impossible to make and one less piece of metadata
is left around to misuse.


-- 
----------------------------------------------------------------------------
Tim Daneliuk     tundra at tundraware.com
PGP Key:         http://www.tundraware.com/PGP/



From python.list at tim.thechases.com  Sat May 23 07:34:24 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Sat, 23 May 2015 06:34:24 -0500
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <slrnmm0o3c.864.jon+usenet@frosty.unequivocal.co.uk>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
 <mailman.257.1432355416.17265.python-list@python.org>
 <slrnmm0o3c.864.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <20150523063424.34308765@bigbox.christie.dr>

On 2015-05-23 11:10, Jon Ribbens wrote:
> On 2015-05-23, Michael Torrie <torriem at gmail.com> wrote:
> > The same can be said of CA-signed certificates.
> 
> I think you are falling into the trap of believing that all things
> are either perfect or they are worthless. CAs aren't perfect, but
> neither are they worthless. A self-signed certificate, however, is
> worthless.

A self-signed certificate may be of minimal worth the *first* time you
visit a site, but if you return to the site, that initial
certificate's signature can be used to confirm that you're talking to
the same site you talked to previously.  This is particularly
valuable on a laptop where you make initial contact over a (I
hesitate to say "more secure") less hostile connection through your
home ISP.  Then, when you're out at the library, coffee-shop, or some
hacker convention on their wifi, it's possible to determine whether
you're securely connecting to the *same* site, or whether an attempt
is being made to MitM because the cert changed.

-tkc





From rosuav at gmail.com  Sat May 23 07:40:19 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 23 May 2015 21:40:19 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <20150523063424.34308765@bigbox.christie.dr>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
 <mailman.257.1432355416.17265.python-list@python.org>
 <slrnmm0o3c.864.jon+usenet@frosty.unequivocal.co.uk>
 <20150523063424.34308765@bigbox.christie.dr>
Message-ID: <CAPTjJmp4P_ojbUpwgO_xtY9Do6X6p6oV25TNgKzrRnZ3sZ0xpw@mail.gmail.com>

On Sat, May 23, 2015 at 9:34 PM, Tim Chase
<python.list at tim.thechases.com> wrote:
> A self-signed certificate may be of minimal worth the *first* time you
> visit a site, but if you return to the site, that initial
> certificate's signature can be used to confirm that you're talking to
> the same site you talked to previously.  This is particularly
> valuable on a laptop where you make initial contact over a (I
> hesitate to say "more secure") less hostile connection through your
> home ISP.  Then, when you're out at the library, coffee-shop, or some
> hacker convention on their wifi, it's possible to determine whether
> you're securely connecting to the *same* site, or whether an attempt
> is being made to MitM because the cert changed.

You can get the exact same benefit (knowing when the cert changes)
with an externally-signed cert too. How many people actually bother to
check?

ChrisA


From PointedEars at web.de  Sat May 23 08:33:44 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Sat, 23 May 2015 14:33:44 +0200
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
Message-ID: <2212595.DFZ6OqehRn@PointedEars.de>

Chris Angelico wrote:

> [?] My hobby-horse, Unicode, is a notable flaw in many languages - if you 
> ask the user for information (in the most obvious way for whatever 
> environment you're in, be that via a web browser request, or a GUI widget, 
> or text entered at the console), can it cope equally with all the world's 
> languages? What if you want to manipulate that text - is it represented as 
> a sequence of codepoints (Python 3), UTF-16 code units (JavaScript),

If only characters were represented as sequences UTF-16 code units in 
ECMAScript implementations like JavaScript, there would not be a problem 
beyond the BMP; see <http://PointedEars.de/wsvn/JSX/trunk/string/unicode.js> 
and others for details.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From marko at pacujo.net  Sat May 23 08:44:02 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 23 May 2015 15:44:02 +0300
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net>
Message-ID: <87382nmpkd.fsf@elektro.pacujo.net>

Johannes Bauer <dfnsonfsduifb at gmx.de>:

> I dislike CAs as much as the next guy. But the problem of distributing
> trust is just not easy to solve, a TTP is a way out. Do you have an
> alternative that does not at the same time to providing a solution
> also opens up obvious attack surface?

Here's an idea: an authentication is considered valid if it is vouched
for by the United States, China, Russia *and* the European Union. Those
governments are the only entities that would have the right to delegate
their respective certification powers to private entities. The
governments would also offer to certify anybody in the world free of
charge.


Marko


From steve at pearwood.info  Sat May 23 09:01:14 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 23 May 2015 23:01:14 +1000
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
Message-ID: <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>

On Sat, 23 May 2015 10:33 pm, Thomas 'PointedEars' Lahn wrote:

> If only characters were represented as sequences UTF-16 code units in
> ECMAScript implementations like JavaScript, there would not be a problem
> beyond the BMP;

Are you being sarcastic?

This is Rhino:

js> var c = String.fromCharCode(65535); // in the BMP
js> print(c.charCodeAt(0));
65535

So far so good.

js> var c = String.fromCharCode(65536); // astral character
js> print(c.charCodeAt(0));
0

Can you name any ECMAScript implementation which correctly handles code
points in the supplementary multilingual planes?


By the way, for many years Python implemented Unicode as UTF-16 code units,
the so-called "narrow build":

py> c = unichr(65536)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unichr() arg not in range(0x10000) (narrow Python build)

Let's try again:

py> c = u'\U00010000'  # a single code point
py> len(c)
2


I'm not saying that it is impossible to have a correct Unicode implemention
using UTF-16, but I've never seen one.



-- 
Steven



From rosuav at gmail.com  Sat May 23 09:12:27 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 23 May 2015 23:12:27 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
 <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmrRJE+0r7i+LgJgU59d4MZrwdi78Okamau-okWWySEaxA@mail.gmail.com>

On Sat, May 23, 2015 at 11:01 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> I'm not saying that it is impossible to have a correct Unicode implemention
> using UTF-16, but I've never seen one.

I suspect this is partly because, if you're aiming for correct Unicode
semantics, UTF-8 offers everything that UTF-16 does and more. The only
reason to use UTF-16 is so you can pretend that UCS-2 is good enough.

ChrisA


From dfnsonfsduifb at gmx.de  Sat May 23 09:17:27 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sat, 23 May 2015 15:17:27 +0200
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <87382nmpkd.fsf@elektro.pacujo.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>	<mailman.222.1432309028.17265.python-list@python.org>	<555FA253.3020304@tundraware.com>
 <mjomke$saf$1@ger.gmane.org>	<mailman.252.1432352224.17265.python-list@python.org>	<mjp88e$e1$1@news.albasani.net>
 <87382nmpkd.fsf@elektro.pacujo.net>
Message-ID: <mjpul7$hjo$1@news.albasani.net>

On 23.05.2015 14:44, Marko Rauhamaa wrote:
> Johannes Bauer <dfnsonfsduifb at gmx.de>:
> 
>> I dislike CAs as much as the next guy. But the problem of distributing
>> trust is just not easy to solve, a TTP is a way out. Do you have an
>> alternative that does not at the same time to providing a solution
>> also opens up obvious attack surface?
> 
> Here's an idea: an authentication is considered valid if it is vouched
> for by the United States, China, Russia *and* the European Union. Those
> governments are the only entities that would have the right to delegate
> their respective certification powers to private entities. The
> governments would also offer to certify anybody in the world free of
> charge.

You propose that a set of multiple CA signatures (TTPs) is required and
that those CAs work for free.

Multiple problems with that.

Firstly, who is going to choose the TTPs? In your example you
arbitrarily chose four instances. Japan is missing from there, why?
Because you made arbitrary rules. Good luck convincing everyone
(especially the Japanese) that your choice is the "right" one. There is
never going to be agreement.

Secondly, any of the "chosen" TTPs can effectively DoS every other
country in your scenario. If the US and Russia have a conflict, each
party can become sloppy at their certifications and slow things down a
bit. Suddenly bank-of-russia.ru doesn't have a valid certificate
anymore, ooops.

Thirdly, the more TTPs you have, the less well the whole thing scales.
The whole idea of a trusted third party is that you can TRUST that party
and don't have to do additional checks (like checking agreement with
other TTPs).

Fourthly and lastly: How would this work? If I have a website running
https, how would I get my identity certified by Russia or China? I
should maybe mention that I speak neither Russian nor Chinese. And even
if I did or maybe if their CAs provided service in English, how would
they certify me? For personal identification purposes you often have to
appear in person, something that is impossible if you distribute the
scheme around the whole world.

All in all, the current CA system is shitty and has numerous problems,
but it's not like it's been designed by monkeys. Every alternative has
new problems, some of which may be even worse than the problems we have now.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From dfnsonfsduifb at gmx.de  Sat May 23 09:24:52 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sat, 23 May 2015 15:24:52 +0200
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <556062B5.7050800@tundraware.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <556062B5.7050800@tundraware.com>
Message-ID: <mjpv34$g1a$1@news.albasani.net>

On 23.05.2015 13:21, Tim Daneliuk wrote:

> Trust has context.  You're going to that site to read an article.  This
> is rather different than, say, going somewhere to transact commerce or
> move money.

Sure, for your site it doesn't really make a difference. And, as I said
before, having a self-signed CA certificate doing https is still WAY
better than not having it. Especially if you have PFS-only ciphersuites
configured (I didn't check, but you should if you're unsure). Because
this effectively means that you're protected against passive
eavesdropping, no matter what.

> So, there is increasing thought that we should all just
> run https everywhere all the time.  But then we run into the signing problem.
> I am hoping that we will soon see free or inexpensive CAs to make that
> problem go away.  See:

Running TLS everywhere is an awesome idea and I'm all for it. So good
that you've already made the switch :-)

But I don't think inexpensive CAs would make the signing problem go away.

I think the major flaw of the X.509 certificate PKI we have today is
that there's no namespacing whatsoever. This is a major problem, as the
Government of Untrustworthia may give out certifictes for google.de if
they wish to do so.

In my opinion, it would be great to have a suffix-option in X.509 (maybe
there's even an extension for this already and I'm not aware -
regardless, nobody is using it if there is such a thing). For example,
there'd be root certificates in the certificate store:

CA1: PF=.com signs -> CA2: PF=.google.com
CA3: PF=.de

So CA1 could give out certificates for
foo.com
www.google.com

And CA2 could give out certificates for
www.google.com

And CA3 could give out certificates for
google.de

But CA1 could never sign any .de domain webserver certificate. It would
only ever get more restrictive down the chain.

Sounds like it's trivial to implement, I wonder why it's not in place.
It must have some huge drawback that I can't think of right now.

Cheers,
Johannes


-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From __peter__ at web.de  Sat May 23 09:25:04 2015
From: __peter__ at web.de (Peter Otten)
Date: Sat, 23 May 2015 15:25:04 +0200
Subject: PYTHONPATH when calling from ipython
References: <874mn5qjc8.fsf@Equus.decebal.nl>
 <mailman.268.1432372378.17265.python-list@python.org>
 <87iobjpnt4.fsf@Equus.decebal.nl>
Message-ID: <mjpv3g$mcv$1@ger.gmane.org>

Cecil Westerhof wrote:

> Op Saturday 23 May 2015 11:12 CEST schreef Mark Lawrence:
> 
>> On 22/05/2015 06:20, Cecil Westerhof wrote:
>>> I am looking into using ipython instead of bash. But when I call a
>>> python program from ipython PYTHONPATH is not set. So pythonscripts
>>> that need a module through PYTHONPATH will not work.
>>>
>>> I could do something like:
>>> !PYTHONPATH=~/Python/PythonLibrary python2 ?
>>>
>>> But I find that a little bit cumbersome. Is there a better way to
>>> do it?
>>>
>>
>> What makes you think this?  Have you tried:-
>>
>>>>> import os
>>>>> os.environ['PYTHONPATH']
>> 'C:\\Users\\Mark\\Documents\\Cash\\Python;C:
\\Users\\Mark\\Documents\\MyPython'
>>
>> That might be from the command line interpreter but it also works
>> the same from iPython for me on Windows 8.1.
> 
> That does not change anything. The modules are not found. Also not
> when using %run.
 

That may be because ~ is not expanded.

Try

os.environ["PYTHONPATH"] = os.path.expanduser("~/Python/PythonLibary")




From ned at nedbatchelder.com  Sat May 23 09:35:22 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sat, 23 May 2015 06:35:22 -0700 (PDT)
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
 <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <2c4d029c-8ea5-465b-8adc-6c35185bd150@googlegroups.com>

On Saturday, May 23, 2015 at 9:01:29 AM UTC-4, Steven D'Aprano wrote:
> On Sat, 23 May 2015 10:33 pm, Thomas 'PointedEars' Lahn wrote:
> 
> > If only characters were represented as sequences UTF-16 code units in
> > ECMAScript implementations like JavaScript, there would not be a problem
> > beyond the BMP;
> 
> Are you being sarcastic?

IIUC, Thomas' point is that *characters* should be sequences of codepoints,
not that *strings* should be.

--Ned.



From steve at pearwood.info  Sat May 23 10:00:13 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 24 May 2015 00:00:13 +1000
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
Message-ID: <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>

On Sat, 23 May 2015 10:44 pm, Marko Rauhamaa wrote:

> Johannes Bauer <dfnsonfsduifb at gmx.de>:
> 
>> I dislike CAs as much as the next guy. But the problem of distributing
>> trust is just not easy to solve, a TTP is a way out. Do you have an
>> alternative that does not at the same time to providing a solution
>> also opens up obvious attack surface?
> 
> Here's an idea: an authentication is considered valid if it is vouched
> for by the United States, China, Russia *and* the European Union. Those
> governments are the only entities that would have the right to delegate
> their respective certification powers to private entities.

An interesting mix of:

- one explicitly non-democratic one-party state;

- one nominally democratic but de facto autocratic state;

- one nominally democratic but de facto two-party corporatocracy;

- one supranational union of states;


If you gave them veto power over all certificate authorities (since you need
all four to agree, any of them can veto a CA), I'm not sure that they would
necessarily agree on *any* CAs. Especially since at least two of them would
be looking for any opportunity to subvert the system for the purposes of
espionage and mass surveillance.

I also don't see any reason why national governments would give up their
existing certification powers.


> The governments would also offer to certify anybody in the world free of
> charge.

Why would they do that?



-- 
Steven



From Cecil at decebal.nl  Sat May 23 10:08:00 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 23 May 2015 16:08:00 +0200
Subject: PYTHONPATH when calling from ipython
References: <874mn5qjc8.fsf@Equus.decebal.nl>
 <mailman.268.1432372378.17265.python-list@python.org>
 <87iobjpnt4.fsf@Equus.decebal.nl>
 <mailman.275.1432387517.17265.python-list@python.org>
Message-ID: <87egm7petb.fsf@Equus.decebal.nl>

Op Saturday 23 May 2015 15:25 CEST schreef Peter Otten:

> Cecil Westerhof wrote:
>
>> Op Saturday 23 May 2015 11:12 CEST schreef Mark Lawrence:
>>
>>> On 22/05/2015 06:20, Cecil Westerhof wrote:
>>>> I am looking into using ipython instead of bash. But when I call
>>>> a python program from ipython PYTHONPATH is not set. So
>>>> pythonscripts that need a module through PYTHONPATH will not
>>>> work.
>>>>
>>>> I could do something like:
>>>> !PYTHONPATH=~/Python/PythonLibrary python2 ?
>>>>
>>>> But I find that a little bit cumbersome. Is there a better way to
>>>> do it?
>>>>
>>>
>>> What makes you think this?  Have you tried:-
>>>
>>>>>> import os
>>>>>> os.environ['PYTHONPATH']
>>> 'C:\\Users\\Mark\\Documents\\Cash\\Python;C:
> \\Users\\Mark\\Documents\\MyPython'
>>>
>>> That might be from the command line interpreter but it also works
>>> the same from iPython for me on Windows 8.1.
>>
>> That does not change anything. The modules are not found. Also not
>> when using %run.
>
>
> That may be because ~ is not expanded.
>
> Try
>
> os.environ["PYTHONPATH"] =
> os.path.expanduser("~/Python/PythonLibary")

That is not the problem:
    os.environ['PYTHONPATH']
gives:
    .:/home/cecil/Python'

As I interpret it is that the very handy shell variable is not used in ipython.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From steve at pearwood.info  Sat May 23 10:09:42 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 24 May 2015 00:09:42 +1000
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <2212595.DFZ6OqehRn@PointedEars.de>
 <55607a1b$0$13011$c3e8da3$5496439d@news.astraweb.com>
 <2c4d029c-8ea5-465b-8adc-6c35185bd150@googlegroups.com>
Message-ID: <55608a27$0$13013$c3e8da3$5496439d@news.astraweb.com>

On Sat, 23 May 2015 11:35 pm, Ned Batchelder wrote:

> On Saturday, May 23, 2015 at 9:01:29 AM UTC-4, Steven D'Aprano wrote:
>> On Sat, 23 May 2015 10:33 pm, Thomas 'PointedEars' Lahn wrote:
>> 
>> > If only characters were represented as sequences UTF-16 code units in
>> > ECMAScript implementations like JavaScript, there would not be a
>> > problem beyond the BMP;
>> 
>> Are you being sarcastic?
> 
> IIUC, Thomas' point is that *characters* should be sequences of
> codepoints, not that *strings* should be.

Like Python, Javascript/ECMAScript doesn't have a distinct character type,
it has strings which happen to be of length one. So I'm not sure I
understand the point you are trying to make.

There's also a bit of a problem in deciding what counts as a character. Is
IJ a single character, or two? The answer depends on whether you are Dutch
or not. Unicode punts on that decision, and leaves it up to the
application.

Unicode only concerns itself with code points, which are complex enough, and
generally programming languages follow Unicode (usually imperfectly). Each
code point (a.k.a. "character" if we're being sloppy) requires either one
or two 16-bit code units in UTF-16. I'm not sure that "1 or 2" counts as a
sequence.


-- 
Steven



From savithad8 at gmail.com  Sat May 23 10:15:14 2015
From: savithad8 at gmail.com (savitha devi)
Date: Sat, 23 May 2015 19:45:14 +0530
Subject: Extract email address from Java script in html source using python
In-Reply-To: <CAPTjJmqBDu0nB2u_mf7KMpFzxxJFUqB7o-7dJiGgu-xyOL2uzg@mail.gmail.com>
References: <CAAXuHoeJ-YMQDB85qLDJ_o+9CrsfwLvm9wuOaRtbSj-i9kBaFA@mail.gmail.com>
 <CAPTjJmqBDu0nB2u_mf7KMpFzxxJFUqB7o-7dJiGgu-xyOL2uzg@mail.gmail.com>
Message-ID: <CAAXuHod4NH74+VK1EY78-vCOyiOA3879qp1uPqKOFit7qVE5sQ@mail.gmail.com>

What I exactly want is the java script is in the html code. I am trying for
a regular expression to find the email address embedded with in the java
script.

On Sat, May 23, 2015 at 2:31 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Sat, May 23, 2015 at 4:46 PM, savitha devi <savithad8 at gmail.com> wrote:
> > I am developing a web scraper code using HTMLParser. I need to extract
> > text/email address from java script with in the HTMLCode.I am beginner
> level
> > in python coding and totally lost here. Need some help on this. The java
> > script code is as below:
> >
> > <script type='text/javascript'>
> >  //<!--
> >  document.getElementById('cloak48218').innerHTML = '';
> >  var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
> >  var path = 'hr' + 'ef' + '=';
> >  var addy48218 = '&#105;nf&#111;' + '&#64;';
> >  addy48218 = addy48218 + 'tsv-n&#101;&#117;r&#105;&#101;d' + '&#46;' +
> > 'd&#101;';
> >  document.getElementById('cloak48218').innerHTML += '<a ' + path + '\'' +
> > prefix + ':' + addy48218 + '\'>' + addy48218+'<\/a>';
> >  //-->
>
> This is deliberately being done to prevent scripted usage. What
> exactly are you needing to do this for?
>
> You're basically going to have to execute the entire block of
> JavaScript code, and then decode the entities to get to what you want.
> Doing it manually is pretty easy; doing it automatically will
> virtually require a language interpreter.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150523/80ae4ac5/attachment.html>

From joel.goldstick at gmail.com  Sat May 23 10:29:27 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Sat, 23 May 2015 10:29:27 -0400
Subject: Extract email address from Java script in html source using python
In-Reply-To: <CAAXuHod4NH74+VK1EY78-vCOyiOA3879qp1uPqKOFit7qVE5sQ@mail.gmail.com>
References: <CAAXuHoeJ-YMQDB85qLDJ_o+9CrsfwLvm9wuOaRtbSj-i9kBaFA@mail.gmail.com>
 <CAPTjJmqBDu0nB2u_mf7KMpFzxxJFUqB7o-7dJiGgu-xyOL2uzg@mail.gmail.com>
 <CAAXuHod4NH74+VK1EY78-vCOyiOA3879qp1uPqKOFit7qVE5sQ@mail.gmail.com>
Message-ID: <CAPM-O+zRgRLohXmaYv=K7y8iVwjETC5GiezRWijvx1GM12qFyg@mail.gmail.com>

On Sat, May 23, 2015 at 10:15 AM, savitha devi <savithad8 at gmail.com> wrote:
> What I exactly want is the java script is in the html code. I am trying for
> a regular expression to find the email address embedded with in the java
> script.
>
> On Sat, May 23, 2015 at 2:31 PM, Chris Angelico <rosuav at gmail.com> wrote:
>>
>> On Sat, May 23, 2015 at 4:46 PM, savitha devi <savithad8 at gmail.com> wrote:
>> > I am developing a web scraper code using HTMLParser. I need to extract
>> > text/email address from java script with in the HTMLCode.I am beginner
>> > level
>> > in python coding and totally lost here. Need some help on this. The java
>> > script code is as below:
>> >
>> > <script type='text/javascript'>
>> >  //<!--
>> >  document.getElementById('cloak48218').innerHTML = '';
>> >  var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
>> >  var path = 'hr' + 'ef' + '=';
>> >  var addy48218 = '&#105;nf&#111;' + '&#64;';
>> >  addy48218 = addy48218 + 'tsv-n&#101;&#117;r&#105;&#101;d' + '&#46;' +
>> > 'd&#101;';
>> >  document.getElementById('cloak48218').innerHTML += '<a ' + path + '\''
>> > +
>> > prefix + ':' + addy48218 + '\'>' + addy48218+'<\/a>';
>> >  //-->
>>
>> This is deliberately being done to prevent scripted usage. What
>> exactly are you needing to do this for?
>>
>> You're basically going to have to execute the entire block of
>> JavaScript code, and then decode the entities to get to what you want.
>> Doing it manually is pretty easy; doing it automatically will
>> virtually require a language interpreter.
>>
>> ChrisA
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
I've not used it, but doesn't Selenium help do this?  From what I
understand it gets the resultant html of a web page after the
javascript has run


-- 
Joel Goldstick
http://joelgoldstick.com


From rosuav at gmail.com  Sat May 23 10:36:59 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 24 May 2015 00:36:59 +1000
Subject: Extract email address from Java script in html source using python
In-Reply-To: <CAAXuHod4NH74+VK1EY78-vCOyiOA3879qp1uPqKOFit7qVE5sQ@mail.gmail.com>
References: <CAAXuHoeJ-YMQDB85qLDJ_o+9CrsfwLvm9wuOaRtbSj-i9kBaFA@mail.gmail.com>
 <CAPTjJmqBDu0nB2u_mf7KMpFzxxJFUqB7o-7dJiGgu-xyOL2uzg@mail.gmail.com>
 <CAAXuHod4NH74+VK1EY78-vCOyiOA3879qp1uPqKOFit7qVE5sQ@mail.gmail.com>
Message-ID: <CAPTjJmoA-qrQTu=dY2r+WtHK6GzH1cYTSbswDGjN563gnWXwAg@mail.gmail.com>

On Sun, May 24, 2015 at 12:15 AM, savitha devi <savithad8 at gmail.com> wrote:
> What I exactly want is the java script is in the html code. I am trying for
> a regular expression to find the email address embedded with in the java
> script.

Now you have two problems.

You can't write a regex that can interpret ECMAScript.

ChrisA


From lac at openend.se  Sat May 23 11:00:16 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 23 May 2015 17:00:16 +0200
Subject: PYTHONPATH when calling from ipython
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Sat, 23 May 2015 16:08:00 +0200." <87egm7petb.fsf@Equus.decebal.nl>
References: <874mn5qjc8.fsf@Equus.decebal.nl>
 <mailman.268.1432372378.17265.python-list@python.org>
 <87iobjpnt4.fsf@Equus.decebal.nl>
 <mailman.275.1432387517.17265.python-list@python.org><87egm7petb.fsf@Equus.decebal.nl>
Message-ID: <201505231500.t4NF0GeW003789@fido.openend.se>

In a message of Sat, 23 May 2015 16:08:00 +0200, Cecil Westerhof writes:
>That is not the problem:
>    os.environ['PYTHONPATH']
>gives:
>    .:/home/cecil/Python'
>
>As I interpret it is that the very handy shell variable is not used in ipython.
>
>-- 
>Cecil Westerhof
>Senior Software Engineer
>LinkedIn: http://www.linkedin.com/in/cecilwesterhof

It's used around here.  But we all have to do:
export PYTHONPATH=${PYTHONPATH}:/usr/local/ipython/lib/python
in our .bashrc files -- or whatever you do if you don't use bash
to get things to work with ipython.

Is your problem that you are not getting one particular directory
loaded, or can you not find any modules at all?

Laura


From Cecil at decebal.nl  Sat May 23 11:24:30 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 23 May 2015 17:24:30 +0200
Subject: PYTHONPATH when calling from ipython
References: <874mn5qjc8.fsf@Equus.decebal.nl>
 <mailman.268.1432372378.17265.python-list@python.org>
 <87iobjpnt4.fsf@Equus.decebal.nl>
 <mailman.275.1432387517.17265.python-list@python.org>
 <87egm7petb.fsf@Equus.decebal.nl>
 <mailman.279.1432393226.17265.python-list@python.org>
Message-ID: <87a8wvpb9t.fsf@Equus.decebal.nl>

Op Saturday 23 May 2015 17:00 CEST schreef Laura Creighton:

> In a message of Sat, 23 May 2015 16:08:00 +0200, Cecil Westerhof
> writes:
>> That is not the problem:
>> os.environ['PYTHONPATH']
>> gives:
>> .:/home/cecil/Python'
>>
>> As I interpret it is that the very handy shell variable is not used
>> in ipython.
>>
>> -- 
>> Cecil Westerhof
>> Senior Software Engineer
>> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>
> It's used around here.  But we all have to do:
> export PYTHONPATH=${PYTHONPATH}:/usr/local/ipython/lib/python
> in our .bashrc files -- or whatever you do if you don't use bash
> to get things to work with ipython.

But I would like to do the same in ipython. Otherwise it is not really
a good idea to switch from bash to ipython.


> Is your problem that you are not getting one particular directory
> loaded, or can you not find any modules at all?

Normal modules are loaded. Only the ones that are found trough
PYTHONPATH are not found.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From doug.garstang at gmail.com  Sat May 23 12:03:19 2015
From: doug.garstang at gmail.com (Douglas Garstang)
Date: Sat, 23 May 2015 09:03:19 -0700
Subject: Dynamic Import with sub module
Message-ID: <CANmEAK9K1_KJ20AmQ9vnSURK1tqR0rJ0NX5Fj-tNxJFiZfaA_A@mail.gmail.com>

All,

Can someone tell me how the following statement:

import slice_fablib_common.common as common

would be replicated with __import__ or import_module?

I'm using fabric and I want the module when imported to appear in the
namespace (which fabric shows as a task), in this case, as 'common.chef'.
With the import statement it works. However, I'm trying to load the modules
dynamically, and I just cannot work out what combination of options when
used with dynamic imports will get me to 'common.chef'.

When I get that to work, I'd like to go one step further as well. If I have
modules called slice_fablib_webapp.webapp.deploy and
slice_fablib_webapp.webapp.info, I'd like to import them as webapp.deploy
and webapp.info.

I suppose the webapp piece is a bit redundant. If I removed it, and ended
up with slice_fablib_webapp.deploy and slice_fablib_webapp.info, I'd like
to import them so that they still appear in fabric's name space as
webapp.deploy and webapp.info.

Thanks,
Doug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150523/3af5246c/attachment.html>

From doug.garstang at gmail.com  Sat May 23 12:08:38 2015
From: doug.garstang at gmail.com (Douglas Garstang)
Date: Sat, 23 May 2015 09:08:38 -0700
Subject: Dynamic Import with sub module
In-Reply-To: <CANmEAK9K1_KJ20AmQ9vnSURK1tqR0rJ0NX5Fj-tNxJFiZfaA_A@mail.gmail.com>
References: <CANmEAK9K1_KJ20AmQ9vnSURK1tqR0rJ0NX5Fj-tNxJFiZfaA_A@mail.gmail.com>
Message-ID: <CANmEAK_uxzNcc1ak98xLvHEi-hiAUp4GKU-uNRSKJ5krEOMScw@mail.gmail.com>

Replying to my own message.

I just realised that even the case I thought would work, will not. :(

If I have:

import slice_fablib_common.common as common

I end up with fabric showing me these commands:

Available commands:

    common.chef.report
    common.chef.run     Run the chef client.
    common.chef.start   Start chef-client service.
    common.chef.status  Show chef-client service status.
    common.chef.stop    Stop chef-client service.

However, that's only because slice_fablib_common/common/__init__.py has
'import chef'. I need to be able to individually load sub-modules from
common. I'm essentially loading the entire package here because I need to
put an import for every sub module into __init__.py anyway. :(

Doug.

On Sat, May 23, 2015 at 9:03 AM, Douglas Garstang <doug.garstang at gmail.com>
wrote:

> All,
>
> Can someone tell me how the following statement:
>
> import slice_fablib_common.common as common
>
> would be replicated with __import__ or import_module?
>
> I'm using fabric and I want the module when imported to appear in the
> namespace (which fabric shows as a task), in this case, as 'common.chef'.
> With the import statement it works. However, I'm trying to load the modules
> dynamically, and I just cannot work out what combination of options when
> used with dynamic imports will get me to 'common.chef'.
>
> When I get that to work, I'd like to go one step further as well. If I
> have modules called slice_fablib_webapp.webapp.deploy and
> slice_fablib_webapp.webapp.info, I'd like to import them as webapp.deploy
> and webapp.info.
>
> I suppose the webapp piece is a bit redundant. If I removed it, and ended
> up with slice_fablib_webapp.deploy and slice_fablib_webapp.info, I'd like
> to import them so that they still appear in fabric's name space as
> webapp.deploy and webapp.info.
>
> Thanks,
> Doug.
>
>


-- 
Regards,

Douglas Garstang
http://www.linkedin.com/in/garstang
Email: doug.garstang at gmail.com
Cell: +1-805-340-5627
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150523/5e4921af/attachment.html>

From __peter__ at web.de  Sat May 23 12:09:01 2015
From: __peter__ at web.de (Peter Otten)
Date: Sat, 23 May 2015 18:09:01 +0200
Subject: PYTHONPATH when calling from ipython
References: <874mn5qjc8.fsf@Equus.decebal.nl>
 <mailman.268.1432372378.17265.python-list@python.org>
 <87iobjpnt4.fsf@Equus.decebal.nl>
 <mailman.275.1432387517.17265.python-list@python.org>
 <87egm7petb.fsf@Equus.decebal.nl>
Message-ID: <mjq8n1$9t3$1@ger.gmane.org>

Cecil Westerhof wrote:

> Op Saturday 23 May 2015 15:25 CEST schreef Peter Otten:
> 
>> Cecil Westerhof wrote:
>>
>>> Op Saturday 23 May 2015 11:12 CEST schreef Mark Lawrence:
>>>
>>>> On 22/05/2015 06:20, Cecil Westerhof wrote:
>>>>> I am looking into using ipython instead of bash. But when I call
>>>>> a python program from ipython PYTHONPATH is not set. So
>>>>> pythonscripts that need a module through PYTHONPATH will not
>>>>> work.
>>>>>
>>>>> I could do something like:
>>>>> !PYTHONPATH=~/Python/PythonLibrary python2 ?
>>>>>
>>>>> But I find that a little bit cumbersome. Is there a better way to
>>>>> do it?
>>>>>
>>>>
>>>> What makes you think this?  Have you tried:-
>>>>
>>>>>>> import os
>>>>>>> os.environ['PYTHONPATH']
>>>> 'C:\\Users\\Mark\\Documents\\Cash\\Python;C:
>> \\Users\\Mark\\Documents\\MyPython'
>>>>
>>>> That might be from the command line interpreter but it also works
>>>> the same from iPython for me on Windows 8.1.
>>>
>>> That does not change anything. The modules are not found. Also not
>>> when using %run.
>>
>>
>> That may be because ~ is not expanded.
>>
>> Try
>>
>> os.environ["PYTHONPATH"] =
>> os.path.expanduser("~/Python/PythonLibary")
> 
> That is not the problem:
>     os.environ['PYTHONPATH']
> gives:
>     .:/home/cecil/Python'
> 
> As I interpret it is that the very handy shell variable is not used in
> ipython.
> 

I can't confirm that finding. For test purposes I created foo/bar/hello.py.
Then I verified that the hello.py module is not found when invoking the 
python3 interpreter from within ipython3.
Once I update os.environ["PYTHONPATH"] the module is successfully imported.
The complete session:

$ mkdir -p foo/bar
$ echo 'print("hello from foo/bar")' > foo/bar/hello.py
$ ipython3
Python 3.4.0 (default, Apr 11 2014, 13:05:11) 
Type "copyright", "credits" or "license" for more information.

IPython 1.2.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import hello.py
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-d9db4361d25a> in <module>()
----> 1 import hello.py

ImportError: No module named 'hello'

In [2]: import os

In [3]: !python3 -c 'import hello'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named 'hello'

In [4]: os.environ["PYTHONPATH"] = "foo/bar"

In [5]: !python3 -c 'import hello'
hello from foo/bar


If that's not what you want please explain in similar detail what you want 
to achieve and what you actually tried. Thank you.



From marko at pacujo.net  Sat May 23 12:53:51 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 23 May 2015 19:53:51 +0300
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87siankzfk.fsf@elektro.pacujo.net>

Steven D'Aprano <steve at pearwood.info>:

> On Sat, 23 May 2015 10:44 pm, Marko Rauhamaa wrote:
>> Here's an idea: an authentication is considered valid if it is
>> vouched for by the United States, China, Russia *and* the European
>> Union. Those governments are the only entities that would have the
>> right to delegate their respective certification powers to private
>> entities.
>
> An interesting mix of:
>
> - one explicitly non-democratic one-party state;
> - one nominally democratic but de facto autocratic state;
> - one nominally democratic but de facto two-party corporatocracy;
> - one supranational union of states;

Yes, the same principles that make UN do a lot of good in the world
despite those shortcomings.

> If you gave them veto power over all certificate authorities (since
> you need all four to agree, any of them can veto a CA),

No, they wouldn't be able to veto a CA. At worst, they would be able to
refuse you a certificate. If they did that, they would risk being
dropped from the power pool.

>> The governments would also offer to certify anybody in the world free
>> of charge.
>
> Why would they do that?

They would have something to gain and something to lose:

 1. They would gain protection for their citizens and companies against
    foreign MitM attacks.

 2. They would lose the power to perform MitM attacks on their own
    citizens.

Unfortunately, the governments of the world fear their own citizens more
than each other, so they would likely not go with the kind of plan I
presented.

At the moment any sovereign government and sizeable criminal outfit can
cook up valid certificates for any website in the world. That's because
each CA is trusted completely.


Marko


From pkpearson at nowhere.invalid  Sat May 23 12:57:20 2015
From: pkpearson at nowhere.invalid (Peter Pearson)
Date: 23 May 2015 16:57:20 GMT
Subject: Extract email address from Java script in html source using python
References: <mailman.265.1432369272.17265.python-list@python.org>
Message-ID: <csbprgF7c4pU1@mid.individual.net>

On Sat, 23 May 2015 12:16:06 +0530, savitha devi <savithad8 at gmail.com> wrote:
>
> I am developing a web scraper code using HTMLParser. I need to extract
> text/email address from java script with in the HTMLCode.

Would be be correct in suspecting that you are assembling a list
of email addresses for use in spamming?  After all, that is the
problem that motivates people to hide their email addresses behind
Javascript.

-- 
To email me, substitute nowhere->runbox, invalid->com.


From marko at pacujo.net  Sat May 23 13:05:52 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 23 May 2015 20:05:52 +0300
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <556062B5.7050800@tundraware.com>
 <mjpv34$g1a$1@news.albasani.net>
Message-ID: <87oalbkyvj.fsf@elektro.pacujo.net>

Johannes Bauer <dfnsonfsduifb at gmx.de>:

> I think the major flaw of the X.509 certificate PKI we have today is
> that there's no namespacing whatsoever. This is a major problem, as
> the Government of Untrustworthia may give out certifictes for
> google.de if they wish to do so.

But you're fine with the Government of Germany, I take it? Or any
accredited German CA?

Even well-meaning CA's do a lousy job. I remember when I purchased a
domain certificate from a reputable CA. How did they verify I was a
rightful representative of the domain? They called the phone number I
had filled in the application form -- since somebody (me) picked up the
phone, they accepted my application as legitimate.

When an HTTPS URL is displayed with the green lock icon, all it means is
that someone has paid good money for the certificate.

> Sounds like it's trivial to implement, I wonder why it's not in place.
> It must have some huge drawback that I can't think of right now.

How would your scheme address .com, .net, .org etc?


Marko


From tjreedy at udel.edu  Sat May 23 13:26:00 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 23 May 2015 13:26:00 -0400
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <556060A0.1010501@tundraware.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com>
 <mailman.248.1432346080.17265.python-list@python.org>
 <556060A0.1010501@tundraware.com>
Message-ID: <mjqd7k$el0$1@ger.gmane.org>

On 5/23/2015 7:12 AM, Tim Daneliuk wrote:
> On 05/22/2015 08:54 PM, Terry Reedy wrote:
>> On 5/22/2015 5:40 PM, Tim Daneliuk wrote:
>>
>>> Lo these many years ago, I argued that Python is a whole lot more than
>>> a programming language:
>>>
>>>      https://www.tundraware.com/TechnicalNotes/Python-Is-Middleware/

>> Perhaps something at tundraware needs updating.
>> '''
>> This Connection is Untrusted
>>
>> You have asked Firefox to connect securely to www.tundraware.com, but we can't confirm that your connection is secure.
>>
>> Normally, when you try to connect securely, sites will present trusted identification to prove that you are going to the right place. However, this site's identity can't be verified.
>> '''
>>
>
> It's self signed - something done quite routinely on the net.

I do not routinely see the message above.  In fact, it has been months. 
  I think the last time was a python.org site that *did* need updating. 
Hence the 'perhaps'. But maybe I do not get around to new sites often 
enough.

I went ahead and clicked through the warnings to read you somewhat 
prescient 13-year-old piece.

-- 
Terry Jan Reedy



From Cecil at decebal.nl  Sat May 23 13:30:55 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 23 May 2015 19:30:55 +0200
Subject: PYTHONPATH when calling from ipython
References: <874mn5qjc8.fsf@Equus.decebal.nl>
 <mailman.268.1432372378.17265.python-list@python.org>
 <87iobjpnt4.fsf@Equus.decebal.nl>
 <mailman.275.1432387517.17265.python-list@python.org>
 <87egm7petb.fsf@Equus.decebal.nl>
 <mailman.282.1432397364.17265.python-list@python.org>
Message-ID: <87617jp5f4.fsf@Equus.decebal.nl>

Op Saturday 23 May 2015 18:09 CEST schreef Peter Otten:

> Cecil Westerhof wrote:
>
>> Op Saturday 23 May 2015 15:25 CEST schreef Peter Otten:
>>
>>> Cecil Westerhof wrote:
>>>
>>>> Op Saturday 23 May 2015 11:12 CEST schreef Mark Lawrence:
>>>>
>>>>> On 22/05/2015 06:20, Cecil Westerhof wrote:
>>>>>> I am looking into using ipython instead of bash. But when I
>>>>>> call a python program from ipython PYTHONPATH is not set. So
>>>>>> pythonscripts that need a module through PYTHONPATH will not
>>>>>> work.
>>>>>>
>>>>>> I could do something like:
>>>>>> !PYTHONPATH=~/Python/PythonLibrary python2 ?
>>>>>>
>>>>>> But I find that a little bit cumbersome. Is there a better way
>>>>>> to do it?
>>>>>>
>>>>>
>>>>> What makes you think this?  Have you tried:-
>>>>>
>>>>>>>> import os
>>>>>>>> os.environ['PYTHONPATH']
>>>>> 'C:\\Users\\Mark\\Documents\\Cash\\Python;C:
>>> \\Users\\Mark\\Documents\\MyPython'
>>>>>
>>>>> That might be from the command line interpreter but it also
>>>>> works the same from iPython for me on Windows 8.1.
>>>>
>>>> That does not change anything. The modules are not found. Also
>>>> not when using %run.
>>>
>>>
>>> That may be because ~ is not expanded.
>>>
>>> Try
>>>
>>> os.environ["PYTHONPATH"] =
>>> os.path.expanduser("~/Python/PythonLibary")
>>
>> That is not the problem:
>> os.environ['PYTHONPATH']
>> gives:
>> .:/home/cecil/Python'
>>
>> As I interpret it is that the very handy shell variable is not used
>> in ipython.
>>
>
> I can't confirm that finding. For test purposes I created
> foo/bar/hello.py. Then I verified that the hello.py module is not
> found when invoking the python3 interpreter from within ipython3.
> Once I update os.environ["PYTHONPATH"] the module is successfully
> imported. The complete session:

I should have checked better. I think I found a bug that made it look
like PYTHONPATH does not work.

In bash I give:
    echo $PYTHONPATH
this gives:
    .:/home/cecil/Python/PythonLibrary

Then I start ipython3 and get/do the following:
    Python 3.4.1 (default, May 23 2014, 17:48:28) [GCC]
    Type "copyright", "credits" or "license" for more information.

    IPython 2.2.0 -- An enhanced Interactive Python.
    ?         -> Introduction and overview of IPython's features.
    %quickref -> Quick reference.
    help      -> Python's own help system.
    object?   -> Details about 'object', use 'object??' for extra details.

    In [1]: import os

    In [2]: os.environ['PYTHONPATH']
    Out[2]: '.:/home/cecil/Python'

And PYTHONPATH has a different value. That is why my module is not
found.

When I set PYTHONPATH to the correct value, everything works as
expected:
    In [3]: os.environ['PYTHONPATH'] = '.:/home/cecil/Python/PythonLibrary/'

    In [4]: !python2 postOnTwitter.py --used
    Citation has 50 saved messages of 126:
    [6, 55, 43, 82, 28, 116, 2, 50, 100, 5, 0, 122, 75, 51, 121, 60, 114, 13, 102, 78, 31, 107, 73, 109, 54, 119, 72, 90, 89, 113, 118, 41, 11, 27, 48, 77, 19, 111, 62, 98, 110, 9, 10, 115, 63, 15, 53, 101, 94, 92]

    Tips has 30 saved messages of 94:
    [20, 37, 7, 59, 45, 49, 40, 87, 79, 78, 31, 14, 15, 25, 84, 18, 91, 53, 8, 35, 80, 92, 34, 42, 74, 69, 64, 22, 86, 62]

That begs the question: what is the reason for the corruption of
PYTHONPATH?

I already answered it partly myself. When I change PYTHONPATH in bash
and call ipython3 again, it has the same value as before, so it does
not take its value from the calling bash as I would expect.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From rosuav at gmail.com  Sat May 23 13:41:48 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 24 May 2015 03:41:48 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <87siankzfk.fsf@elektro.pacujo.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmoTHMaMTQzLepvH4g8rHbF7zKp5d27sntUwLCuqd5FZTA@mail.gmail.com>

On Sun, May 24, 2015 at 2:53 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Steven D'Aprano <steve at pearwood.info>:
>
>> On Sat, 23 May 2015 10:44 pm, Marko Rauhamaa wrote:
>>> Here's an idea: an authentication is considered valid if it is
>>> vouched for by the United States, China, Russia *and* the European
>>> Union. Those governments are the only entities that would have the
>>> right to delegate their respective certification powers to private
>>> entities.
>>
>> If you gave them veto power over all certificate authorities (since
>> you need all four to agree, any of them can veto a CA),
>
> No, they wouldn't be able to veto a CA. At worst, they would be able to
> refuse you a certificate. If they did that, they would risk being
> dropped from the power pool.

You start out by saying it's valid if vouched for by X, Y, Z., *and*
A. That means that if it's vouched for by X, Y, and A, but not Z, then
it's not valid. That gives Z the power to veto any certificate.
Correspondingly each of the others.

Alternatively, you could say that it's valid if vouched for by *any*
of your authorities... but then you have the same situation as
currently, where multiple authorities can create identical
certificates.

You could try for some kind of voting scheme, where it takes X/2+1
authorities to create a certificate (so you'd need three of your four,
or if you added a fifth (say Japan), then three out of the five); but
this just entails ridiculous overheads for uncertain benefit.

Also, there's one huge question outstanding: Since when should country
governments and the EU be in charge of any of this?

ChrisA


From breamoreboy at yahoo.co.uk  Sat May 23 14:13:00 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sat, 23 May 2015 19:13:00 +0100
Subject: PYTHONPATH when calling from ipython
In-Reply-To: <87617jp5f4.fsf@Equus.decebal.nl>
References: <874mn5qjc8.fsf@Equus.decebal.nl>
 <mailman.268.1432372378.17265.python-list@python.org>
 <87iobjpnt4.fsf@Equus.decebal.nl>
 <mailman.275.1432387517.17265.python-list@python.org>
 <87egm7petb.fsf@Equus.decebal.nl>
 <mailman.282.1432397364.17265.python-list@python.org>
 <87617jp5f4.fsf@Equus.decebal.nl>
Message-ID: <mjqfvl$nb9$1@ger.gmane.org>

On 23/05/2015 18:30, Cecil Westerhof wrote:
>
> I should have checked better. I think I found a bug that made it look
> like PYTHONPATH does not work.
>
> In bash I give:
>      echo $PYTHONPATH
> this gives:
>      .:/home/cecil/Python/PythonLibrary
>
> Then I start ipython3 and get/do the following:
>      Python 3.4.1 (default, May 23 2014, 17:48:28) [GCC]
>      Type "copyright", "credits" or "license" for more information.
>
>      IPython 2.2.0 -- An enhanced Interactive Python.

The latest iPython is 3.1.0.  Time to update?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From Cecil at decebal.nl  Sat May 23 14:14:16 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 23 May 2015 20:14:16 +0200
Subject: PYTHONPATH when calling from ipython
References: <874mn5qjc8.fsf@Equus.decebal.nl>
 <mailman.268.1432372378.17265.python-list@python.org>
 <87iobjpnt4.fsf@Equus.decebal.nl>
 <mailman.275.1432387517.17265.python-list@python.org>
 <87egm7petb.fsf@Equus.decebal.nl>
 <mailman.282.1432397364.17265.python-list@python.org>
 <87617jp5f4.fsf@Equus.decebal.nl>
Message-ID: <871ti7p3ev.fsf@Equus.decebal.nl>

Op Saturday 23 May 2015 19:30 CEST schreef Cecil Westerhof:

> I should have checked better. I think I found a bug that made it
> look like PYTHONPATH does not work.
>
> In bash I give:
> echo $PYTHONPATH
> this gives:
> .:/home/cecil/Python/PythonLibrary
>
> Then I start ipython3 and get/do the following:
> Python 3.4.1 (default, May 23 2014, 17:48:28) [GCC]
> Type "copyright", "credits" or "license" for more information.
>
> IPython 2.2.0 -- An enhanced Interactive Python. ? -> Introduction
> and overview of IPython's features. %quickref -> Quick reference.
> help -> Python's own help system. object? -> Details about 'object',
> use 'object??' for extra details.
>
> In [1]: import os
>
> In [2]: os.environ['PYTHONPATH']
> Out[2]: '.:/home/cecil/Python'
>
> And PYTHONPATH has a different value. That is why my module is not
> found.
>
> When I set PYTHONPATH to the correct value, everything works as
> expected: In [3]: os.environ['PYTHONPATH'] =
> '.:/home/cecil/Python/PythonLibrary/'
>
> In [4]: !python2 postOnTwitter.py --used Citation has 50 saved
> messages of 126: [6, 55, 43, 82, 28, 116, 2, 50, 100, 5, 0, 122, 75,
> 51, 121, 60, 114, 13, 102, 78, 31, 107, 73, 109, 54, 119, 72, 90,
> 89, 113, 118, 41, 11, 27, 48, 77, 19, 111, 62, 98, 110, 9, 10, 115,
> 63, 15, 53, 101, 94, 92]
>
> Tips has 30 saved messages of 94:
> [20, 37, 7, 59, 45, 49, 40, 87, 79, 78, 31, 14, 15, 25, 84, 18, 91,
> 53, 8, 35, 80, 92, 34, 42, 74, 69, 64, 22, 86, 62]
>
> That begs the question: what is the reason for the corruption of
> PYTHONPATH?
>
> I already answered it partly myself. When I change PYTHONPATH in
> bash and call ipython3 again, it has the same value as before, so it
> does not take its value from the calling bash as I would expect.

Well it was my fault again. Linux can run very long before you need to
restart it. I work with screen. Before PYTHONPATH was
.:/home/cecil/Python, but later on I changed it to
.:/home/cecil/Python/PythonLibrary. When starting a new shell, the
bash shell had the right value. But when starting ipython it got the
old value. Importing os is not even necessary:
    Python 3.4.1 (default, May 23 2014, 17:48:28) [GCC]
    Type "copyright", "credits" or "license" for more information.

    IPython 2.2.0 -- An enhanced Interactive Python.
    ?         -> Introduction and overview of IPython's features.
    %quickref -> Quick reference.
    help      -> Python's own help system.
    object?   -> Details about 'object', use 'object??' for extra details.

    In [1]: !python2 postOnTwitter.py --used
    python2: can't open file 'postOnTwitter.py': [Errno 2] No such file or directory

    In [2]: !python2 postOnTwitter.py --used
    Citation has 50 saved messages of 126:
    [6, 55, 43, 82, 28, 116, 2, 50, 100, 5, 0, 122, 75, 51, 121, 60, 114, 13, 102, 78, 31, 107, 73, 109, 54, 119, 72, 90, 89, 113, 118, 41, 11, 27, 48, 77, 19, 111, 62, 98, 110, 9, 10, 115, 63, 15, 53, 101, 94, 92]

    Tips has 30 saved messages of 94:
    [20, 37, 7, 59, 45, 49, 40, 87, 79, 78, 31, 14, 15, 25, 84, 18, 91, 53, 8, 35, 80, 92, 34, 42, 74, 69, 64, 22, 86, 62]


Feeling a bit silly at the moment. >:-(


By the way I use python2 because the program uses a library that does
not work with python3.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From marko at pacujo.net  Sat May 23 15:02:29 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 23 May 2015 22:02:29 +0300
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
 <mailman.284.1432403395.17265.python-list@python.org>
Message-ID: <87a8wvkth6.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> On Sun, May 24, 2015 at 2:53 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
>> Steven D'Aprano <steve at pearwood.info>:
>>> If you gave them veto power over all certificate authorities (since
>>> you need all four to agree, any of them can veto a CA),
>>
>> No, they wouldn't be able to veto a CA. At worst, they would be able
>> to refuse you a certificate. If they did that, they would risk being
>> dropped from the power pool.
>
> You start out by saying it's valid if vouched for by X, Y, Z., *and*
> A. That means that if it's vouched for by X, Y, and A, but not Z, then
> it's not valid. That gives Z the power to veto any certificate.
> Correspondingly each of the others.

CA = certificate authority != certificate


Marko


From Cecil at decebal.nl  Sat May 23 15:47:28 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sat, 23 May 2015 21:47:28 +0200
Subject: PYTHONPATH when calling from ipython
References: <874mn5qjc8.fsf@Equus.decebal.nl>
 <mailman.268.1432372378.17265.python-list@python.org>
 <87iobjpnt4.fsf@Equus.decebal.nl>
 <mailman.275.1432387517.17265.python-list@python.org>
 <87egm7petb.fsf@Equus.decebal.nl>
 <mailman.282.1432397364.17265.python-list@python.org>
 <87617jp5f4.fsf@Equus.decebal.nl>
 <mailman.285.1432404810.17265.python-list@python.org>
Message-ID: <87wpzznkj3.fsf@Equus.decebal.nl>

Op Saturday 23 May 2015 20:13 CEST schreef Mark Lawrence:

> On 23/05/2015 18:30, Cecil Westerhof wrote:
>>
>> I should have checked better. I think I found a bug that made it
>> look like PYTHONPATH does not work.
>>
>> In bash I give:
>> echo $PYTHONPATH
>> this gives:
>> .:/home/cecil/Python/PythonLibrary
>>
>> Then I start ipython3 and get/do the following:
>> Python 3.4.1 (default, May 23 2014, 17:48:28) [GCC]
>> Type "copyright", "credits" or "license" for more information.
>>
>> IPython 2.2.0 -- An enhanced Interactive Python.
>
> The latest iPython is 3.1.0.  Time to update?

Maybe. ;-)

Strange thing is that ipython (for python 2) uses 3.0.0. This is on
openSUSE 13.2. On Debian 8 both use 2.3.0.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From hamilton at nothere.com  Sat May 23 19:34:09 2015
From: hamilton at nothere.com (hamilton)
Date: Sat, 23 May 2015 17:34:09 -0600
Subject: Human Rights and Justice in Islam
In-Reply-To: <1050c913-df97-45c2-9fa8-041b6d17037a@googlegroups.com>
References: <1050c913-df97-45c2-9fa8-041b6d17037a@googlegroups.com>
Message-ID: <mjr2nl$8ke$1@dont-email.me>

On 5/23/2015 8:11 AM, bv4bv4bv4 at gmail.com wrote:
> Human Rights and Justice in Islam
>
>
> Description: A glimpse at the foundations of human rights laid by Islam.
> By islam-guide.com
>
> Islam provides many human rights for the individual.  The following are some of these human rights that Islam protects.
>
> The life and property of all citizens in an Islamic state

So if you are a Member of Islamic State you have rights, other wise you 
are an infidel and subject to death from the whim of Allah or whom ever 
thinks they are Allah.


Does that sound right ??



From benjamin at python.org  Sat May 23 19:50:04 2015
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 23 May 2015 19:50:04 -0400
Subject: [RELEASE] Python 2.7.10
Message-ID: <1432425004.1774211.276632913.5FA67C5E@webmail.messagingengine.com>

The next bugfix release of the Python 2.7.x series, Python 2.7.10, has
been released. The only interesting change since the release candidate
is a fix for a regression in cookie parsing.

Downloads are available at:
  https://www.python.org/downloads/release/python-2710/

Report bugs at:
  https://bugs.python.org

Enjoy your 2 digit versions,
Benjamin
(on behalf of 2.7.10's contributors)


From nad at acm.org  Sat May 23 20:14:34 2015
From: nad at acm.org (Ned Deily)
Date: Sat, 23 May 2015 17:14:34 -0700
Subject: need help with an accessibility prototype
References: <555F5960.9030501@harvee.org>
 <201505221950.t4MJo3pC010042@fido.openend.se>
 <201505222216.t4MMG0IN013302@fido.openend.se>
 F5BC.23311922055015@news.gmane.....
 <201505230925.t4N9PnZ8028675@fido.openend.se>
Message-ID: <nad-0B4442.17143423052015@news.gmane.org>

In article <201505230925.t4N9PnZ8028675 at fido.openend.se>,
 Laura Creighton <lac at openend.se> wrote:

> In a message of Fri, 22 May 2015 23:31:19 -0700, Ned Deily writes:
> >Tcl/Tk 8.4 is quite old and no longer maintained; 8.6.x is current, 
> >although 8.5.x is also still in use.
> >
> >http://www.tcl.tk/man/tcl8.6/TkCmd/keysyms.htm
> >
> > Ned Deily,
> > nad at acm.org
> 
> Thank you Ned.  Old bookmark from when I needed this the last time --
> sorry about that.  Do you know if you still need to use 8.5 if you are
> on Mac OS? i.e.  is this page current?
> https://www.python.org/download/mac/tcltk/

Yes, it is current.  Apple has not yet shipped a version of 8.6 with OS 
X, so the Apple-supplied Python in current OS X releases use 8.5; OS X 
10.5 and earlier systems shipped only with 8.4.  At the moment, the 
Pythons for OS X provided by python.org installers still use 8.5 for the 
10.6+ installers and 10.4 for the 10.5+ installers.  Some third-party 
distributors of Python and Tcl/Tk for OS X are now using 8.6, e.g. 
MacPorts.

-- 
 Ned Deily,
 nad at acm.org



From hayesstw at telkomsa.net  Sat May 23 21:04:36 2015
From: hayesstw at telkomsa.net (Steve Hayes)
Date: Sun, 24 May 2015 03:04:36 +0200
Subject: Extract email address from Java script in html source using python
References: <CAAXuHoeJ-YMQDB85qLDJ_o+9CrsfwLvm9wuOaRtbSj-i9kBaFA@mail.gmail.com>
 <mailman.267.1432371718.17265.python-list@python.org>
Message-ID: <cq82ma9d4pqufo7u11m52eh4ca2hgmi25a@4ax.com>

On Sat, 23 May 2015 19:01:55 +1000, Chris Angelico <rosuav at gmail.com>
wrote:

>On Sat, May 23, 2015 at 4:46 PM, savitha devi <savithad8 at gmail.com> wrote:
>> I am developing a web scraper code using HTMLParser. I need to extract
>> text/email address from java script with in the HTMLCode.I am beginner level
>> in python coding and totally lost here. Need some help on this. The java
>> script code is as below:
>>
>> <script type='text/javascript'>
>>  //<!--
>>  document.getElementById('cloak48218').innerHTML = '';
>>  var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
>>  var path = 'hr' + 'ef' + '=';
>>  var addy48218 = '&#105;nf&#111;' + '&#64;';
>>  addy48218 = addy48218 + 'tsv-n&#101;&#117;r&#105;&#101;d' + '&#46;' +
>> 'd&#101;';
>>  document.getElementById('cloak48218').innerHTML += '<a ' + path + '\'' +
>> prefix + ':' + addy48218 + '\'>' + addy48218+'<\/a>';
>>  //-->
>
>This is deliberately being done to prevent scripted usage. What
>exactly are you needing to do this for?

To sell addresses to spammers, of course. 


-- 
Terms and conditions apply. 

Steve Hayes
hayesmstw at hotmail.com


From zondervanz at gmail.com  Sat May 23 21:53:29 2015
From: zondervanz at gmail.com (Dr. John Q. Hacker)
Date: Sat, 23 May 2015 20:53:29 -0500
Subject: mix-in classes
Message-ID: <CADL-DviUKRgpk-hmK6kqYik0D6bWLXveRLjNWqZPAbximmHhSQ@mail.gmail.com>

The post on "different types of inheritence..." brought up a thought.

Let's say, I'm adding flexibility to a module by letting users change class
behaviors by adding different mix-in classes.

What should happen when there's a name collision on method names between
mix-ins?  Since they're mix-ins, it's not presumed that there is any parent
class to decide.  The proper thing would seem to call each method in the
order that they are written within the parent class definition.

I suppose one can create a method in the parent class, that runs the mixin
methods in the same order as in the inheritance list, but would there be a
better way for Python to enforce such a practice so as not to create class
anarchy?  (A problem for another topic.)

zipher
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150523/d584f3b9/attachment.html>

From torriem at gmail.com  Sat May 23 22:57:12 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sat, 23 May 2015 20:57:12 -0600
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <CAPTjJmp4P_ojbUpwgO_xtY9Do6X6p6oV25TNgKzrRnZ3sZ0xpw@mail.gmail.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
 <mailman.257.1432355416.17265.python-list@python.org>
 <slrnmm0o3c.864.jon+usenet@frosty.unequivocal.co.uk>
 <20150523063424.34308765@bigbox.christie.dr>
 <CAPTjJmp4P_ojbUpwgO_xtY9Do6X6p6oV25TNgKzrRnZ3sZ0xpw@mail.gmail.com>
Message-ID: <55613E08.70707@gmail.com>

On 05/23/2015 05:40 AM, Chris Angelico wrote:
> On Sat, May 23, 2015 at 9:34 PM, Tim Chase
> <python.list at tim.thechases.com> wrote:
>> A self-signed certificate may be of minimal worth the *first* time you
>> visit a site, but if you return to the site, that initial
>> certificate's signature can be used to confirm that you're talking to
>> the same site you talked to previously.  This is particularly
>> valuable on a laptop where you make initial contact over a (I
>> hesitate to say "more secure") less hostile connection through your
>> home ISP.  Then, when you're out at the library, coffee-shop, or some
>> hacker convention on their wifi, it's possible to determine whether
>> you're securely connecting to the *same* site, or whether an attempt
>> is being made to MitM because the cert changed.
> 
> You can get the exact same benefit (knowing when the cert changes)
> with an externally-signed cert too. How many people actually bother to
> check?

Except that you won't be notified automatically.  A MitM attack nowadays
most often uses a valid certificate signed by a recognized (though
untrustworthy) CA.  Thus with a self-signed cert that you've previously
accepted, you'll immediate know of the MitM attack.  The odds of this
happening inside China, for example, is very high.  Wasn't that long ago
bogus google certificates (still valid) were found in the wild.
Eventually Firefox and Chrome revoked the CA cert, but only after it was
found out.




From torriem at gmail.com  Sat May 23 23:00:56 2015
From: torriem at gmail.com (Michael Torrie)
Date: Sat, 23 May 2015 21:00:56 -0600
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <87382nmpkd.fsf@elektro.pacujo.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
Message-ID: <55613EE8.9030907@gmail.com>

On 05/23/2015 06:44 AM, Marko Rauhamaa wrote:
> Johannes Bauer <dfnsonfsduifb at gmx.de>:
> 
>> I dislike CAs as much as the next guy. But the problem of distributing
>> trust is just not easy to solve, a TTP is a way out. Do you have an
>> alternative that does not at the same time to providing a solution
>> also opens up obvious attack surface?
> 
> Here's an idea: an authentication is considered valid if it is vouched
> for by the United States, China, Russia *and* the European Union. Those
> governments are the only entities that would have the right to delegate
> their respective certification powers to private entities. The
> governments would also offer to certify anybody in the world free of
> charge.

Why trust governments?  Why not use peer-to-peer trust.  If I trust you
and you trust site X with a fingerprint of Y, then I should trust it
also.  Sadly though getting the unwashed masses educated enough to make
this work is impossible (like how PGP is pretty much dead).  Maybe it's
a harder problem than anyone can solve.


From random832 at fastmail.us  Sat May 23 23:27:09 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Sat, 23 May 2015 23:27:09 -0400
Subject: mix-in classes
In-Reply-To: <CADL-DviUKRgpk-hmK6kqYik0D6bWLXveRLjNWqZPAbximmHhSQ@mail.gmail.com>
References: <CADL-DviUKRgpk-hmK6kqYik0D6bWLXveRLjNWqZPAbximmHhSQ@mail.gmail.com>
Message-ID: <1432438029.804228.276693553.3D47573C@webmail.messagingengine.com>

On Sat, May 23, 2015, at 21:53, Dr. John Q. Hacker wrote:

> What should happen when there's a name collision on method names between
> mix-ins?  Since they're mix-ins, it's not presumed that there is any
> parent
> class to decide.  The proper thing would seem to call each method in the
> order that they are written within the parent class definition.

The way C#/.NET does its nearest equivalent to this is to have this be
determined by what "mix-in" is in scope at the call site. That is, for
example, the mix-ins for IEnumerable defined in System.Linq.Enumerable
is only called if the System.Linq namespace has been imported.

IIRC, it's an error for there to be two with identical signatures, which
isn't an issue since you can simply explicitly call the method as
Enumerable.Whatever(foo) instead of foo.Whatever().

One could imagine a similar mechanism for python... object.__getattr__
could, if nothing is found in the dictionary or by an overridden
__getattr__ method, find the call site (construct a stack trace),
iterate through an __mixins__ list in the calling module, search each
mixin, in order (since it's a list, let's take advantage of it having an
order since we haven't got the static type system to do a lot of
sophisticated overload resolution), for a matching name. Ideally we'd
also have a mechanism for mixin methods to be able to raise
NotImplemented (or TypeError?) and have it continue down the list, but
any mechanism I can think of for this has the disadvantage that if any
of the mixins contains a matching callable it has to return a proxy
rather than immediately raising an error, even if none of them actually
works.


From V at nguard.LH  Sun May 24 02:17:18 2015
From: V at nguard.LH (VanguardLH)
Date: Sun, 24 May 2015 01:17:18 -0500
Subject: Extract email address from Java script in html source using python
References: <CAAXuHoeJ-YMQDB85qLDJ_o+9CrsfwLvm9wuOaRtbSj-i9kBaFA@mail.gmail.com>
 <mailman.267.1432371718.17265.python-list@python.org>
 <cq82ma9d4pqufo7u11m52eh4ca2hgmi25a@4ax.com>
Message-ID: <csd8neFidb7U1@mid.individual.net>

Steve Hayes wrote:

> On Sat, 23 May 2015 19:01:55 +1000, Chris Angelico <rosuav at gmail.com>
> wrote:
> 
>>On Sat, May 23, 2015 at 4:46 PM, savitha devi <savithad8 at gmail.com> wrote:
>>> I am developing a web scraper code using HTMLParser. I need to extract
>>> text/email address from java script with in the HTMLCode.I am beginner level
>>> in python coding and totally lost here. Need some help on this. The java
>>> script code is as below:
>>>
>>> <script type='text/javascript'>
>>>  //<!--
>>>  document.getElementById('cloak48218').innerHTML = '';
>>>  var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
>>>  var path = 'hr' + 'ef' + '=';
>>>  var addy48218 = '&#105;nf&#111;' + '&#64;';
>>>  addy48218 = addy48218 + 'tsv-n&#101;&#117;r&#105;&#101;d' + '&#46;' +
>>> 'd&#101;';
>>>  document.getElementById('cloak48218').innerHTML += '<a ' + path + '\'' +
>>> prefix + ':' + addy48218 + '\'>' + addy48218+'<\/a>';
>>>  //-->
>>
>>This is deliberately being done to prevent scripted usage. What
>>exactly are you needing to do this for?
> 
> To sell addresses to spammers, of course.

The boob that uses this javascripted obfuscation (by slicing up the URL
across variables and using concatenation within a variable) hasn't a
clue that the javascript or user clicking on a URL will still have to
eventually go to the destination so it will still get blocked.  Duh!
Nothing is actually cloaked by the javascript (it's just another means
of building up the <A> tag) and the URL string (even if it used a
decimal value instead of IP-dotted) still has to connect to somewhere
and that gets detected and blocked.  Slicing up a URL across variables
and concantenation within a variable is a child's ploy to obfuscate.
Apparently savitha can't even distinguish between an e-mail address and
a URL string.


From ian.g.kelly at gmail.com  Sun May 24 03:06:05 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sun, 24 May 2015 01:06:05 -0600
Subject: Human Rights and Justice in Islam
In-Reply-To: <mjr2nl$8ke$1@dont-email.me>
References: <1050c913-df97-45c2-9fa8-041b6d17037a@googlegroups.com>
 <mjr2nl$8ke$1@dont-email.me>
Message-ID: <CALwzidkrH4uM9oPqAJfX0ZBTsgyicK8TBFjZsj62YAGTUOCh-Q@mail.gmail.com>

On Sat, May 23, 2015 at 5:34 PM, hamilton <hamilton at nothere.com> wrote:
> So if you are a Member of Islamic State you have rights, other wise you are
> an infidel and subject to death from the whim of Allah or whom ever thinks
> they are Allah.
>
>
> Does that sound right ??

Please don't reply to spammers. I guarantee you that they aren't going
to read it, so you're just creating more useless traffic for everybody
else who reads this list.


From ian.g.kelly at gmail.com  Sun May 24 03:22:23 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sun, 24 May 2015 01:22:23 -0600
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <55613E08.70707@gmail.com>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com>
 <mjomke$saf$1@ger.gmane.org> <555FF482.8020007@gmail.com>
 <CALwzidkxFMpWc2miioN_L2mCPE6WFuHC_=HQX3vAmLL_KqqDRw@mail.gmail.com>
 <mailman.257.1432355416.17265.python-list@python.org>
 <slrnmm0o3c.864.jon+usenet@frosty.unequivocal.co.uk>
 <20150523063424.34308765@bigbox.christie.dr>
 <CAPTjJmp4P_ojbUpwgO_xtY9Do6X6p6oV25TNgKzrRnZ3sZ0xpw@mail.gmail.com>
 <55613E08.70707@gmail.com>
Message-ID: <CALwzidkBkTg9tDwwO12ifWk061jy6vUyTAnMcjUGEneeAJ_uYw@mail.gmail.com>

On Sat, May 23, 2015 at 8:57 PM, Michael Torrie <torriem at gmail.com> wrote:
> On 05/23/2015 05:40 AM, Chris Angelico wrote:
>> On Sat, May 23, 2015 at 9:34 PM, Tim Chase
>> <python.list at tim.thechases.com> wrote:
>>> A self-signed certificate may be of minimal worth the *first* time you
>>> visit a site, but if you return to the site, that initial
>>> certificate's signature can be used to confirm that you're talking to
>>> the same site you talked to previously.  This is particularly
>>> valuable on a laptop where you make initial contact over a (I
>>> hesitate to say "more secure") less hostile connection through your
>>> home ISP.  Then, when you're out at the library, coffee-shop, or some
>>> hacker convention on their wifi, it's possible to determine whether
>>> you're securely connecting to the *same* site, or whether an attempt
>>> is being made to MitM because the cert changed.
>>
>> You can get the exact same benefit (knowing when the cert changes)
>> with an externally-signed cert too. How many people actually bother to
>> check?
>
> Except that you won't be notified automatically.  A MitM attack nowadays
> most often uses a valid certificate signed by a recognized (though
> untrustworthy) CA.  Thus with a self-signed cert that you've previously
> accepted, you'll immediate know of the MitM attack.

I fail to see how this is the case. If a new certificate is suddenly
provided, why should the status of the *previous* certificate have
anything to do with whether the browser automatically notifies you? A
change from a self-signed certificate to a CA certificate likely just
means that the site has upgraded its certificate, not that a MitM
attack is underway.


From ian.g.kelly at gmail.com  Sun May 24 03:39:41 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sun, 24 May 2015 01:39:41 -0600
Subject: mix-in classes
In-Reply-To: <CADL-DviUKRgpk-hmK6kqYik0D6bWLXveRLjNWqZPAbximmHhSQ@mail.gmail.com>
References: <CADL-DviUKRgpk-hmK6kqYik0D6bWLXveRLjNWqZPAbximmHhSQ@mail.gmail.com>
Message-ID: <CALwzidkJUmJNTYpFfhuTFa+4BAx9NJW2J+3vqepaOaM=kR4p8w@mail.gmail.com>

On Sat, May 23, 2015 at 7:53 PM, Dr. John Q. Hacker
<zondervanz at gmail.com> wrote:
> The post on "different types of inheritence..." brought up a thought.
>
> Let's say, I'm adding flexibility to a module by letting users change class
> behaviors by adding different mix-in classes.
>
> What should happen when there's a name collision on method names between
> mix-ins?  Since they're mix-ins, it's not presumed that there is any parent
> class to decide.  The proper thing would seem to call each method in the
> order that they are written within the parent class definition.
>
> I suppose one can create a method in the parent class, that runs the mixin
> methods in the same order as in the inheritance list, but would there be a
> better way for Python to enforce such a practice so as not to create class
> anarchy?  (A problem for another topic.)

Usually with mixins, one just wants to call a method of a specific
mixin; a name collision is likely a symptom of poor design, and it
would be unusual to want to call *all* mixin methods with the same
name. If you really want to do that for a particular method though, is
there some reason why super() won't suffice?


From marko at pacujo.net  Sun May 24 04:23:48 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sun, 24 May 2015 11:23:48 +0300
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <mailman.3.1432436461.5151.python-list@python.org>
Message-ID: <87twv2jsdn.fsf@elektro.pacujo.net>

Michael Torrie <torriem at gmail.com>:

> Why trust governments?

They have the means and are doing analogous things already wrt property
titles, passports, taxation, voting etc.

> Why not use peer-to-peer trust. If I trust you and you trust site X
> with a fingerprint of Y, then I should trust it also.

That's a huge leap of faith. Trust is not transitive.

I do trust the government far more than the word of mouth.


Marko


From steve at pearwood.info  Sun May 24 05:59:56 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 24 May 2015 19:59:56 +1000
Subject: Human Rights and Justice in Islam
References: <1050c913-df97-45c2-9fa8-041b6d17037a@googlegroups.com>
 <mjr2nl$8ke$1@dont-email.me>
Message-ID: <5561a11d$0$12999$c3e8da3$5496439d@news.astraweb.com>

On Sun, 24 May 2015 09:34 am, hamilton wrote:
[quoted bullshit from a spammer]
[tried to argue with said spammer]

Please don't reply to fly-by-spammers. Even if the spammer was interested in
honest debate -- and he is not (fortunately!) -- the last thing we want on
a Python newsgroup is for it to be hijacked about arguments about religion.


-- 
Steven



From steve at pearwood.info  Sun May 24 06:26:18 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 24 May 2015 20:26:18 +1000
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
Message-ID: <5561a74b$0$12977$c3e8da3$5496439d@news.astraweb.com>

On Sun, 24 May 2015 02:53 am, Marko Rauhamaa wrote:

> Steven D'Aprano <steve at pearwood.info>:
> 
>> On Sat, 23 May 2015 10:44 pm, Marko Rauhamaa wrote:
>>> Here's an idea: an authentication is considered valid if it is
>>> vouched for by the United States, China, Russia *and* the European
>>> Union. Those governments are the only entities that would have the
>>> right to delegate their respective certification powers to private
>>> entities.
>>
>> An interesting mix of:
>>
>> - one explicitly non-democratic one-party state;
>> - one nominally democratic but de facto autocratic state;
>> - one nominally democratic but de facto two-party corporatocracy;
>> - one supranational union of states;
> 
> Yes, the same principles that make UN do a lot of good in the world
> despite those shortcomings.
> 
>> If you gave them veto power over all certificate authorities (since
>> you need all four to agree, any of them can veto a CA),
> 
> No, they wouldn't be able to veto a CA. At worst, they would be able to
> refuse you a certificate. If they did that, they would risk being
> dropped from the power pool.

That's not what you said. You said, and I quote:

"an authentication is considered valid if it is vouched for by the United
States, China, Russia *and* the European Union."

[Emphasis in the original.]

So if (let's say) the US, China and Russia all agree that a Certs-R-Us are a
legitimate CA, but the EU disagrees for some reason. Then certificates
issued by Certs-R-Us will *not* be accepted as valid. Hence the EU has veto
power over CAs, and by extension, certificates. And likewise any of the
others: it only takes one refusal for the certificate to be invalid.

If the certificate is *not* invalidated, then people can trust certificates
regardless of whether they are vouched for by all four states (counting the
EU as a state for simplicity) or not. If I can choose to trust Certs-R-Us
despite the failure of the EU to vouch for them, then I can equally trust
*any* CA, whether they are vouched for by all four states or by none of
them. Which brings us right back to the present system.

(And by the way, I'd be more inclined to trust a CA that was vouched for by,
say, the Norwegian government than one vouched for by the Russian
government.)

And what's this "dropped from the power pool" business? You never mentioned
a mechanism for removing a state from the privileged group. Who has
authority to do that?

If it's too hard to change the four-state solution ("What, we have to
completely redesign the entire Internet, again?") then they will never be
removed no matter how they abuse their privilege. If it is too easy ("just
edit /etc/ca-approvals"), then we'll have chaos where nobody agrees on who
can authorise CAs. Somewhere in the middle there's a point where the four
states will never refuse a CA, no matter how dodgy, lest they get kicked
out. In which case we haven't really solved the problem we're trying to
solve, just moved it around a bit.


>>> The governments would also offer to certify anybody in the world free
>>> of charge.
>>
>> Why would they do that?
> 
> They would have something to gain and something to lose:
> 
>  1. They would gain protection for their citizens and companies against
>     foreign MitM attacks.
> 
>  2. They would lose the power to perform MitM attacks on their own
>     citizens.
> 
> Unfortunately, the governments of the world fear their own citizens more
> than each other, so they would likely not go with the kind of plan I
> presented.

Charles Stross has some interesting ideas on that. The economic and
political elites are clamping down on dissent as a preemptive
counter-revolution:

http://www.antipope.org/charlie/blog-static/2013/07/who-ordered-that.html
 
> At the moment any sovereign government and sizeable criminal outfit 

Or medium sized corporation. Oh wait, I see you already mentioned criminal
outfits.


> can cook up valid certificates for any website in the world. That's
> because each CA is trusted completely.

I'm not sure that it is much of a benefit to swap from a free-market
reputation based system to a four-party oligopoly




-- 
Steven



From Cecil at decebal.nl  Sun May 24 06:52:44 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 24 May 2015 12:52:44 +0200
Subject: Setting prompt in ipython
Message-ID: <87siamnt6r.fsf@Equus.decebal.nl>

In bash I have:
    \[\e[1m\][\D{%a, %d %b %T} \u@\h:\w]\n$ \[\e[0m\]
which gives me the following prompt:
    [Sun, 24 May 12:23:45 cecil at Equus:~]
    $

I want to have a similar prompt in ipython. At the moment I have come
to:
    %config PromptManager.in_template = r'[\T \u@\h:\Y1]\n\#> '
which gives:
    [12:51:32 cecil at Equus:startup]
    14> 


But how do I get weekday, day-number and short month into my prompt?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From steve at pearwood.info  Sun May 24 07:11:59 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 24 May 2015 21:11:59 +1000
Subject: mix-in classes
References: <mailman.1.1432432413.5151.python-list@python.org>
Message-ID: <5561b200$0$13002$c3e8da3$5496439d@news.astraweb.com>

On Sun, 24 May 2015 11:53 am, Dr. John Q. Hacker wrote:

> The post on "different types of inheritence..." brought up a thought.
> 
> Let's say, I'm adding flexibility to a module by letting users change
> class behaviors by adding different mix-in classes.
> 
> What should happen when there's a name collision on method names between
> mix-ins?  Since they're mix-ins, it's not presumed that there is any
> parent class to decide.  The proper thing would seem to call each method
> in the order that they are written within the parent class definition.

That's one possibility. Another is to have precedence rules, e.g. "first
mixin wins". If a mixin wishes to support additional mixins with the same
method, in a cooperative fashion, they can call super().

But, frankly, what you describe is more likely to be a weakness of multiple
inheritance and mixins, one which should be avoided. One attempt to avoid
this problem is with traits, an alternative to mixins which explicitly
deals with the problem of mixin conflicts.

http://www.artima.com/weblogs/viewpost.jsp?thread=246488

(Unfortunately, the language used to describe mixins, traits, multiple
inheritance and related concepts is not always consistent. Ruby mixins
don't use inheritance; Scala traits are more like Python mixins than Squeak
traits; and Python mixins are, of course, merely a convention layered over
multiple inheritance.)

> I suppose one can create a method in the parent class, that runs the mixin
> methods in the same order as in the inheritance list, but would there be a
> better way for Python to enforce such a practice so as not to create class
> anarchy?  (A problem for another topic.)

Try strait:

https://pypi.python.org/pypi/strait


-- 
Steven



From rosuav at gmail.com  Sun May 24 07:30:28 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 24 May 2015 21:30:28 +1000
Subject: Camelot a good tool for me
In-Reply-To: <20150522224229.GA62222@cskk.homeip.net>
References: <87fv6ol93p.fsf@nautilus.nautilus>
 <20150522224229.GA62222@cskk.homeip.net>
Message-ID: <CAPTjJmpCuhqTwkxfCW48e8We=9Hp7UG3EeOiynH1nNy_LjnsZw@mail.gmail.com>

On Sat, May 23, 2015 at 8:42 AM, Cameron Simpson <cs at zip.com.au> wrote:
> I like SQLAlchemy because:
>
>  - it quotes for me, avoiding an infinity of pain and injection risk
>
>  - it presents results as nice objects with attributes named after columns
>
>  - it lets me write SQL queries as nice parameterised Python syntax instead
> of clunky SQL
>
>  - it automatically hooks into various backends, which means I can write
> many  tests or prototypes using, for example, in-memory SQLite dummy
> databases  while still speaking to MySQL or PostgreSQL in the real world.
> (Obviously you  need to run some tests against the real thing, but looking
> for SQL logic  errors is readily tested against throwaway SQLite or the
> like.)

Interestingly, the Python DB API actually gives you three out of four of that.

It quotes for you:
cur.execute("insert into Students (name, year) values (%s, %s)",
("Robert'); DROP TABLE Students;--", 2012))

It automatically hooks into various backends:

import psycopg2
conn = psycopg2.connect("dbname=whatever user=me password=secret")

import mysqldb
conn = mysqldb.connect(user="me", password="secret", db="whatever")

import sqlite3
conn = sqlite3.connect("whatever.db")

After that, you can mostly use the same code everywhere. (The
differences will be because of inherent differences in the backends,
not issues with the API.)

You don't automatically get nice objects with attributes named after
columns, but neither will SQLAlchemy unless you're doing the
equivalent of "select * from". Example:

>>> session.query(Student).all()
[<__main__.Student object at 0x7fc2de3fc290>, <__main__.Student object
at 0x7fc2de3fc350>, <__main__.Student object at 0x7fc2de3fc3d0>]

That's a list of Student objects, because I asked for the entire table.

>>> session.query(Student.name, Student.year).all()
[(u'Fred', 2015), (u'Joe', 2015), (u"Robert'); DROP TABLE Students;--", 2012)]

That's a list of tuples, because I asked for just two columns. So what
you're really saying is that SQLAlchemy subtly suggests that you
should grab the entire table any time you need any information out of
it. This is bad for a couple of reasons: firstly, it defeats certain
optimizations (for instance, if you ask for an indexed column, the
database might not need to read from the base table at all), and
secondly, it opens up dangerous possibilities of excessive data
transfer. You start pickling these objects, or JSONifying them, or
whatever, and sending them to untrusted destinations, and voila,
you're giving away all your columns instead of just the few that
they're allowed to see. Yes, this isn't really a part of the
SQLAlchemy interface, but it's part of the same philosophy of "just
give me the object and let me do what I like with it". I've seen
commercial systems that have had this exact flaw, looking very much
like the programmers used too many of these fancy shortcuts and not
enough actual thinking about what they're doing.

So, yes, in the specific case where you want to take the entire table,
SQLAlchemy gives you something a little prettier. If you're
parsimonious, the two are the same.

The only one that PEP 249 doesn't give is your second point, that you
can use Python syntax to craft queries. Sure. That's SQLAlchemy's
schtick, and it's fine. But the other three of your points -
particularly the serious one about SQL injection prevention - are all
handled elsewhere too.

ChrisA


From lac at openend.se  Sun May 24 07:54:08 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 24 May 2015 13:54:08 +0200
Subject: Setting prompt in ipython
In-Reply-To: Message from Cecil Westerhof <Cecil@decebal.nl>
 of "Sun, 24 May 2015 12:52:44 +0200." <87siamnt6r.fsf@Equus.decebal.nl>
References: <87siamnt6r.fsf@Equus.decebal.nl>
Message-ID: <201505241154.t4OBs8cO000927@fido.openend.se>

This file supposedly lists what you can customise your ipython prompt
to show.  I have not tried this myself.

http://www.math.uiuc.edu/~gfrancis/illimath/windows/aszgard_mini/movpy-2.0.0-py2.4.4/manuals/IPython/node7.html

Laura




From python.list at tim.thechases.com  Sun May 24 08:05:23 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Sun, 24 May 2015 07:05:23 -0500
Subject: Camelot a good tool for me
In-Reply-To: <201505221303.t4MD301H032748@fido.openend.se>
References: <87zj4xoxfd.fsf@Equus.decebal.nl>
 <201505221303.t4MD301H032748@fido.openend.se>
Message-ID: <20150524070523.38d748c0@bigbox.christie.dr>

On 2015-05-22 15:03, Laura Creighton wrote:
> I don't know anything about Camelot.

Am I the only one who is disappointed that nobody has claimed
"Camelot...it's only a model"? :-)

-tkc






From Cecil at decebal.nl  Sun May 24 08:11:39 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 24 May 2015 14:11:39 +0200
Subject: Setting prompt in ipython
References: <87siamnt6r.fsf@Equus.decebal.nl>
 <mailman.11.1432468462.5151.python-list@python.org>
Message-ID: <87lhgenpj8.fsf@Equus.decebal.nl>

Op Sunday 24 May 2015 13:54 CEST schreef Laura Creighton:

> This file supposedly lists what you can customise your ipython
> prompt to show. I have not tried this myself.
>
> http://www.math.uiuc.edu/~gfrancis/illimath/windows/aszgard_mini/movpy-2.0.0-py2.4.4/manuals/IPython/node7.html

I will look into it.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From fomcl at yahoo.com  Sun May 24 08:50:03 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Sun, 24 May 2015 05:50:03 -0700
Subject: programmatically change windows regional settings?
Message-ID: <1432471803.77701.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>


Hi,

In Windows I can change the regional settings manually in the control panel. But how do I do this programmatically? I tried setting LANG but this does not work in Windows. 

Kernel32's SetLocaleInfo sounds promising, but "This setting only affects the user override portion of the locale settings; it does not set the system defaults." https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx

Another route might be _winreg (but that's probably more brittle and, more importantly, the registry sucks)

My goal is to easily run my unittests in a number of locales (actually a platform x locale x python version matrix)

Thanks!

Albert-Jan


From fomcl at yahoo.com  Sun May 24 08:50:03 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Sun, 24 May 2015 05:50:03 -0700
Subject: programmatically change windows regional settings?
Message-ID: <1432471803.77701.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>


Hi,

In Windows I can change the regional settings manually in the control panel. But how do I do this programmatically? I tried setting LANG but this does not work in Windows. 

Kernel32's SetLocaleInfo sounds promising, but "This setting only affects the user override portion of the locale settings; it does not set the system defaults." https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx

Another route might be _winreg (but that's probably more brittle and, more importantly, the registry sucks)

My goal is to easily run my unittests in a number of locales (actually a platform x locale x python version matrix)

Thanks!

Albert-Jan


From Cecil at decebal.nl  Sun May 24 09:07:34 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 24 May 2015 15:07:34 +0200
Subject: Setting prompt in ipython
References: <87siamnt6r.fsf@Equus.decebal.nl>
 <mailman.11.1432468462.5151.python-list@python.org>
 <87lhgenpj8.fsf@Equus.decebal.nl>
Message-ID: <87h9r2nmy1.fsf@Equus.decebal.nl>

Op Sunday 24 May 2015 14:11 CEST schreef Cecil Westerhof:

> Op Sunday 24 May 2015 13:54 CEST schreef Laura Creighton:
>
>> This file supposedly lists what you can customise your ipython
>> prompt to show. I have not tried this myself.
>>
>> http://www.math.uiuc.edu/~gfrancis/illimath/windows/aszgard_mini/movpy-2.0.0-py2.4.4/manuals/IPython/node7.html
>
> I will look into it.

I use the following in ~/.ipython/profile_default/startup/00-init.ipy:
    import time

    %config PromptManager.in_template = r'[{time.strftime("%a, %d %b %T")} \u@\h:\Y1]\nIn [\#]: '

Now I should contemplate what other things would be useful there. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun May 24 09:07:34 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 24 May 2015 15:07:34 +0200
Subject: Setting prompt in ipython
References: <87siamnt6r.fsf@Equus.decebal.nl>
 <mailman.11.1432468462.5151.python-list@python.org>
 <87lhgenpj8.fsf@Equus.decebal.nl>
Message-ID: <87h9r2nmy1.fsf@Equus.decebal.nl>

Op Sunday 24 May 2015 14:11 CEST schreef Cecil Westerhof:

> Op Sunday 24 May 2015 13:54 CEST schreef Laura Creighton:
>
>> This file supposedly lists what you can customise your ipython
>> prompt to show. I have not tried this myself.
>>
>> http://www.math.uiuc.edu/~gfrancis/illimath/windows/aszgard_mini/movpy-2.0.0-py2.4.4/manuals/IPython/node7.html
>
> I will look into it.

I use the following in ~/.ipython/profile_default/startup/00-init.ipy:
    import time

    %config PromptManager.in_template = r'[{time.strftime("%a, %d %b %T")} \u@\h:\Y1]\nIn [\#]: '

Now I should contemplate what other things would be useful there. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From breamoreboy at yahoo.co.uk  Sun May 24 09:07:37 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 24 May 2015 14:07:37 +0100
Subject: programmatically change windows regional settings?
In-Reply-To: <1432471803.77701.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>
References: <1432471803.77701.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>
Message-ID: <mjsif3$nvb$1@ger.gmane.org>

On 24/05/2015 13:50, Albert-Jan Roskam via Python-list wrote:
>
> Hi,
>
> In Windows I can change the regional settings manually in the control panel. But how do I do this programmatically? I tried setting LANG but this does not work in Windows.
>
> Kernel32's SetLocaleInfo sounds promising, but "This setting only affects the user override portion of the locale settings; it does not set the system defaults." https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx
>
> Another route might be _winreg (but that's probably more brittle and, more importantly, the registry sucks)
>
> My goal is to easily run my unittests in a number of locales (actually a platform x locale x python version matrix)
>
> Thanks!
>
> Albert-Jan
>

You're probably better off asking this on 
https://mail.python.org/mailman/listinfo/python-win32 which is also 
available as gmane.comp.python.windows, although you might get lucky 
here.  If you do a rather more specific statement than "this does not 
work in Windows" would be helpful.  Your Python and Windows version(s) 
might possibly assist as well :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Sun May 24 09:07:37 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 24 May 2015 14:07:37 +0100
Subject: programmatically change windows regional settings?
In-Reply-To: <1432471803.77701.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>
References: <1432471803.77701.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>
Message-ID: <mjsif3$nvb$1@ger.gmane.org>

On 24/05/2015 13:50, Albert-Jan Roskam via Python-list wrote:
>
> Hi,
>
> In Windows I can change the regional settings manually in the control panel. But how do I do this programmatically? I tried setting LANG but this does not work in Windows.
>
> Kernel32's SetLocaleInfo sounds promising, but "This setting only affects the user override portion of the locale settings; it does not set the system defaults." https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx
>
> Another route might be _winreg (but that's probably more brittle and, more importantly, the registry sucks)
>
> My goal is to easily run my unittests in a number of locales (actually a platform x locale x python version matrix)
>
> Thanks!
>
> Albert-Jan
>

You're probably better off asking this on 
https://mail.python.org/mailman/listinfo/python-win32 which is also 
available as gmane.comp.python.windows, although you might get lucky 
here.  If you do a rather more specific statement than "this does not 
work in Windows" would be helpful.  Your Python and Windows version(s) 
might possibly assist as well :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From nh.jones01 at gmail.com  Sun May 24 09:11:28 2015
From: nh.jones01 at gmail.com (Norah Jones)
Date: Sun, 24 May 2015 13:11:28 +0000
Subject: How to access Apples iCal-Server via Python?
Message-ID: <abcd1234abc123ab12a0000088133000020000001052@gmail.com>

Hi, 

I'm trying to access Apples iCal-Server on a Mac OS X Snow Leopard Server via Python. The server is up and running and working with it via the iCal-Application is just fine. Now I need to access this server via Python to use it as backend for resource planning.

So how can I read the events within a time range from a user's calendar using python?

Thanks,
Norah Jones




From nh.jones01 at gmail.com  Sun May 24 09:11:28 2015
From: nh.jones01 at gmail.com (Norah Jones)
Date: Sun, 24 May 2015 13:11:28 +0000
Subject: How to access Apples iCal-Server via Python?
Message-ID: <abcd1234abc123ab12a0000088133000020000001052@gmail.com>

Hi, 

I'm trying to access Apples iCal-Server on a Mac OS X Snow Leopard Server via Python. The server is up and running and working with it via the iCal-Application is just fine. Now I need to access this server via Python to use it as backend for resource planning.

So how can I read the events within a time range from a user's calendar using python?

Thanks,
Norah Jones




From lele at metapensiero.it  Sun May 24 09:15:18 2015
From: lele at metapensiero.it (Lele Gaifax)
Date: Sun, 24 May 2015 15:15:18 +0200
Subject: Camelot a good tool for me
References: <87zj4xoxfd.fsf@Equus.decebal.nl> <mjn804$4pe$1@dont-email.me>
 <87siaopxjs.fsf@Equus.decebal.nl>
 <201505221357.t4MDvgIV001904@fido.openend.se>
 <87k2w0le41.fsf@nautilus.nautilus>
 <CAPTjJmpnYwhOWdBz-=Zv8x+BO0G3e7OesPDvzd=nsX2a8Gsc-w@mail.gmail.com>
 <87fv6ol93p.fsf@nautilus.nautilus>
 <CAPTjJmoofyT1dWwoDBJWXtgs99cLiQQE-dvA1P7XJQvYxg5-sg@mail.gmail.com>
Message-ID: <87d21q2k2h.fsf@nautilus.nautilus>

Chris Angelico <rosuav at gmail.com> writes:

> On Sat, May 23, 2015 at 5:12 AM, Lele Gaifax <lele at metapensiero.it> wrote:
>> You are conflating two different layers, core and ORM. ORM relationships can
>> be declared either on the parent or on the child, it's up to your taste.
>
> Not sure why that's distinguishable. If I have two tables like this:
>
> Users:
>     id sequential primary key
>     name text
>
> Tasks
>     id sequential primary key
>     owner integer
>     assignee integer
>
> Both the owner and the assignee refer to the Users table; the owner is
> a mandatory connection (every task was created by someone, who
> initially owns it), and the assignee is an optional connection (a
> newly-created task isn't assigned to anyone). With me so far? Okay.
>
> Now, if I were to represent these tables in SQLAlchemy, obviously I
> need to have foreign key relationships encoded in SQLAlchemy. But if
> I'm to enforce these relationships on the underlying database, it's
> equally obvious that I need foreign key constraints. I would expect
> that a relationship encoded in SQLAlchemy should cause the creation of
> a constraint in the database. They're fundamentally the same thing.

No. A "relationship" is an ORM thingie, and can happily exist without an
underlaying constraint in the database.

> My point about backwards is that my tables here are declared in a
> strict order: parent table, then child table. In the child table, a
> constraint is created by saying "references Users", and the Users
> table already exists. At no point is there ever a forward reference.
> Code would look like this:
>
> create table Users (id serial primary key, name text not null default '');
> create table Tasks (id serial primary key, owner integer not null
> references Users, assignee integer references Users);
>
> But with SQLAlchemy, you have a tag in the Users table's definition
> saying that it has a relationship with Tasks, as well as a foreign key
> in Tasks stating the connection to Users. That violates "Define Before
> Use", which isn't a strict policy, but it does feel a little bit
> "dirty".

As said, that's not how I'd code it, as I'd probably implement those classes
as

class User(Base):
     id = Column(...)
     name = Column(...)

class Task(Base):
     id = Column(...)
     owner_id = Column(...)
     assignee_id = Column(...)

     owned_by = relationship('User', primaryjoin='User.id==Task.owner_id',
                             backref='own_tasks')
     assigned_to = relationship('User', primaryjoin='User.id==Task.assigned_id',
                             backref='assigned_tasks')

>>> When magic works, it's great; but when anything goes wrong, it's harder to
>>> see what happened.
>>
>> The same can be said of almost any upper layer in a software stack.
>
> Precisely. All magic has to justify itself. Some can, easily. Some
> can't. A lot is in the middle, where it's part of the tradeoffs.
>
>>> Also, when does a transaction begin and end?
>>
>> When I need transactions (that is, when I'm changing the database) I'm very
>> picky and use explicit begins, commits and rollbacks, so I don't recall
>> experiencing that doubt.
>
> That's all very well when you write your own code. Now try picking up
> someone else's code. Or, for a mid-way concern, try explaining to a
> junior developer how to make sure his transactions are right. With
> psycopg2, it's easy enough to do this:
>
> with conn, conn.cursor() as cur:
>     cur.execute("....")
>     cur.execute("....")

You're kidding, of course: on SA side you imagine a complex code written by
somebody else, while on the other side a plain sequence of statements. You can
write almost the same code either with SA (which at it's low level has a plain
DBAPI connection):

  http://docs.sqlalchemy.org/en/rel_1_0/core/connections.html#using-transactions

> When the with block exits, the transaction is either committed (if all
> went well) or rolled back (if an exception was raised). It's very
> simple, easy to do, and easy to audit ("all SQL queries must be inside
> a with block that grants a cursor", and possibly "cursor-granting with
> blocks must not be nested").

As you can see, there is almost no difference when using the equivalent SA
idiom.

>
>>> If you session.commit() in the middle of iterating over a query, will it
>>> break the query? What if you roll back? Can you see, instantly, in your
>>> code?
>>
>> Why would you do that? Are you closing your files while you iterate them,
>> without leaving the loop in some way at the same time?
>
> There's advice out there on the internet that says that committing
> periodically in the middle of a big job makes your code run faster.

No, I think you mean "flushing" a session, not committing. And again, that's
usually recommended when using the ORM layer, not at the SQL core we are
talking about here.

> It's from the PHP + MySQL school of thought, where the assumption is
> that finishing is the most important thing, finishing quickly is a
> close second, and guaranteeing correctness isn't even on the radar.
> Now try coping with code that was written under that model.

That's not how I code database-based applications, neither when using plain
DBAPI, nor with SA. I cannot even imagine using such a wierd approach.

> Fortunately, I haven't actually seen anything quite like this in
> SQLAlchemy. The worst I saw was a case where someone was iterating
> over a query and performed another query, which did indeed break the
> fetching of results. But the more magic you have, the less obvious
> that is.

I cannot understand your example: it's not so uncommon the need to perform a
query for each row of a previous one, and it's surely well supported at every
level, in SA.

>> I often have to deal with multiple DB engines at the same time, and being
>> able to "write" my queries with an abstract syntax is very valuable for me.
>
> Hmm, I'm not sure there's all that much that this helps with. Thanks
> to a consistent PEP 249 API, changing database engines is often just a
> matter of changing one import and a connection construction line
> (which you'd have to do anyway, given that the credentials will
> change). What else is there for SQLAlchemy to paper over? The
> differences remain; the common ground is already common.

There are a lot of subtle differences in how SQL is implemented by the various
engines out there. Things like "names quoting", "pagination", "data types",
...

Until you do not need a nice way to write the same thing against different
engines you won't appreciate how nice it is being able to do that :)

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.



From lele at metapensiero.it  Sun May 24 09:15:18 2015
From: lele at metapensiero.it (Lele Gaifax)
Date: Sun, 24 May 2015 15:15:18 +0200
Subject: Camelot a good tool for me
References: <87zj4xoxfd.fsf@Equus.decebal.nl> <mjn804$4pe$1@dont-email.me>
 <87siaopxjs.fsf@Equus.decebal.nl>
 <201505221357.t4MDvgIV001904@fido.openend.se>
 <87k2w0le41.fsf@nautilus.nautilus>
 <CAPTjJmpnYwhOWdBz-=Zv8x+BO0G3e7OesPDvzd=nsX2a8Gsc-w@mail.gmail.com>
 <87fv6ol93p.fsf@nautilus.nautilus>
 <CAPTjJmoofyT1dWwoDBJWXtgs99cLiQQE-dvA1P7XJQvYxg5-sg@mail.gmail.com>
Message-ID: <87d21q2k2h.fsf@nautilus.nautilus>

Chris Angelico <rosuav at gmail.com> writes:

> On Sat, May 23, 2015 at 5:12 AM, Lele Gaifax <lele at metapensiero.it> wrote:
>> You are conflating two different layers, core and ORM. ORM relationships can
>> be declared either on the parent or on the child, it's up to your taste.
>
> Not sure why that's distinguishable. If I have two tables like this:
>
> Users:
>     id sequential primary key
>     name text
>
> Tasks
>     id sequential primary key
>     owner integer
>     assignee integer
>
> Both the owner and the assignee refer to the Users table; the owner is
> a mandatory connection (every task was created by someone, who
> initially owns it), and the assignee is an optional connection (a
> newly-created task isn't assigned to anyone). With me so far? Okay.
>
> Now, if I were to represent these tables in SQLAlchemy, obviously I
> need to have foreign key relationships encoded in SQLAlchemy. But if
> I'm to enforce these relationships on the underlying database, it's
> equally obvious that I need foreign key constraints. I would expect
> that a relationship encoded in SQLAlchemy should cause the creation of
> a constraint in the database. They're fundamentally the same thing.

No. A "relationship" is an ORM thingie, and can happily exist without an
underlaying constraint in the database.

> My point about backwards is that my tables here are declared in a
> strict order: parent table, then child table. In the child table, a
> constraint is created by saying "references Users", and the Users
> table already exists. At no point is there ever a forward reference.
> Code would look like this:
>
> create table Users (id serial primary key, name text not null default '');
> create table Tasks (id serial primary key, owner integer not null
> references Users, assignee integer references Users);
>
> But with SQLAlchemy, you have a tag in the Users table's definition
> saying that it has a relationship with Tasks, as well as a foreign key
> in Tasks stating the connection to Users. That violates "Define Before
> Use", which isn't a strict policy, but it does feel a little bit
> "dirty".

As said, that's not how I'd code it, as I'd probably implement those classes
as

class User(Base):
     id = Column(...)
     name = Column(...)

class Task(Base):
     id = Column(...)
     owner_id = Column(...)
     assignee_id = Column(...)

     owned_by = relationship('User', primaryjoin='User.id==Task.owner_id',
                             backref='own_tasks')
     assigned_to = relationship('User', primaryjoin='User.id==Task.assigned_id',
                             backref='assigned_tasks')

>>> When magic works, it's great; but when anything goes wrong, it's harder to
>>> see what happened.
>>
>> The same can be said of almost any upper layer in a software stack.
>
> Precisely. All magic has to justify itself. Some can, easily. Some
> can't. A lot is in the middle, where it's part of the tradeoffs.
>
>>> Also, when does a transaction begin and end?
>>
>> When I need transactions (that is, when I'm changing the database) I'm very
>> picky and use explicit begins, commits and rollbacks, so I don't recall
>> experiencing that doubt.
>
> That's all very well when you write your own code. Now try picking up
> someone else's code. Or, for a mid-way concern, try explaining to a
> junior developer how to make sure his transactions are right. With
> psycopg2, it's easy enough to do this:
>
> with conn, conn.cursor() as cur:
>     cur.execute("....")
>     cur.execute("....")

You're kidding, of course: on SA side you imagine a complex code written by
somebody else, while on the other side a plain sequence of statements. You can
write almost the same code either with SA (which at it's low level has a plain
DBAPI connection):

  http://docs.sqlalchemy.org/en/rel_1_0/core/connections.html#using-transactions

> When the with block exits, the transaction is either committed (if all
> went well) or rolled back (if an exception was raised). It's very
> simple, easy to do, and easy to audit ("all SQL queries must be inside
> a with block that grants a cursor", and possibly "cursor-granting with
> blocks must not be nested").

As you can see, there is almost no difference when using the equivalent SA
idiom.

>
>>> If you session.commit() in the middle of iterating over a query, will it
>>> break the query? What if you roll back? Can you see, instantly, in your
>>> code?
>>
>> Why would you do that? Are you closing your files while you iterate them,
>> without leaving the loop in some way at the same time?
>
> There's advice out there on the internet that says that committing
> periodically in the middle of a big job makes your code run faster.

No, I think you mean "flushing" a session, not committing. And again, that's
usually recommended when using the ORM layer, not at the SQL core we are
talking about here.

> It's from the PHP + MySQL school of thought, where the assumption is
> that finishing is the most important thing, finishing quickly is a
> close second, and guaranteeing correctness isn't even on the radar.
> Now try coping with code that was written under that model.

That's not how I code database-based applications, neither when using plain
DBAPI, nor with SA. I cannot even imagine using such a wierd approach.

> Fortunately, I haven't actually seen anything quite like this in
> SQLAlchemy. The worst I saw was a case where someone was iterating
> over a query and performed another query, which did indeed break the
> fetching of results. But the more magic you have, the less obvious
> that is.

I cannot understand your example: it's not so uncommon the need to perform a
query for each row of a previous one, and it's surely well supported at every
level, in SA.

>> I often have to deal with multiple DB engines at the same time, and being
>> able to "write" my queries with an abstract syntax is very valuable for me.
>
> Hmm, I'm not sure there's all that much that this helps with. Thanks
> to a consistent PEP 249 API, changing database engines is often just a
> matter of changing one import and a connection construction line
> (which you'd have to do anyway, given that the credentials will
> change). What else is there for SQLAlchemy to paper over? The
> differences remain; the common ground is already common.

There are a lot of subtle differences in how SQL is implemented by the various
engines out there. Things like "names quoting", "pagination", "data types",
...

Until you do not need a nice way to write the same thing against different
engines you won't appreciate how nice it is being able to do that :)

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.



From lele at metapensiero.it  Sun May 24 09:28:07 2015
From: lele at metapensiero.it (Lele Gaifax)
Date: Sun, 24 May 2015 15:28:07 +0200
Subject: Camelot a good tool for me
References: <87fv6ol93p.fsf@nautilus.nautilus>
 <20150522224229.GA62222@cskk.homeip.net>
 <CAPTjJmpCuhqTwkxfCW48e8We=9Hp7UG3EeOiynH1nNy_LjnsZw@mail.gmail.com>
Message-ID: <878uce2jh4.fsf@nautilus.nautilus>

Chris Angelico <rosuav at gmail.com> writes:

> You don't automatically get nice objects with attributes named after
> columns, but neither will SQLAlchemy unless you're doing the
> equivalent of "select * from". Example:
>
>>>> session.query(Student).all()
> [<__main__.Student object at 0x7fc2de3fc290>, <__main__.Student object
> at 0x7fc2de3fc350>, <__main__.Student object at 0x7fc2de3fc3d0>]
>
> That's a list of Student objects, because I asked for the entire table.
>
>>>> session.query(Student.name, Student.year).all()
> [(u'Fred', 2015), (u'Joe', 2015), (u"Robert'); DROP TABLE Students;--", 2012)]
>
> That's a list of tuples, because I asked for just two columns.

Wrong:

    >>> res = session.query(Student.name, Student.year).all()
    >>> type(res)
    <class 'list'>
    >>> type(res[0])
    <class 'sqlalchemy.util._collections.result'>
    >>> type(res[0].name)
    <class 'str'>
    >>> type(res[0][0])
    <class 'str'>
    >>> res[0].name == res[0][0]
    True

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.



From lele at metapensiero.it  Sun May 24 09:28:07 2015
From: lele at metapensiero.it (Lele Gaifax)
Date: Sun, 24 May 2015 15:28:07 +0200
Subject: Camelot a good tool for me
References: <87fv6ol93p.fsf@nautilus.nautilus>
 <20150522224229.GA62222@cskk.homeip.net>
 <CAPTjJmpCuhqTwkxfCW48e8We=9Hp7UG3EeOiynH1nNy_LjnsZw@mail.gmail.com>
Message-ID: <878uce2jh4.fsf@nautilus.nautilus>

Chris Angelico <rosuav at gmail.com> writes:

> You don't automatically get nice objects with attributes named after
> columns, but neither will SQLAlchemy unless you're doing the
> equivalent of "select * from". Example:
>
>>>> session.query(Student).all()
> [<__main__.Student object at 0x7fc2de3fc290>, <__main__.Student object
> at 0x7fc2de3fc350>, <__main__.Student object at 0x7fc2de3fc3d0>]
>
> That's a list of Student objects, because I asked for the entire table.
>
>>>> session.query(Student.name, Student.year).all()
> [(u'Fred', 2015), (u'Joe', 2015), (u"Robert'); DROP TABLE Students;--", 2012)]
>
> That's a list of tuples, because I asked for just two columns.

Wrong:

    >>> res = session.query(Student.name, Student.year).all()
    >>> type(res)
    <class 'list'>
    >>> type(res[0])
    <class 'sqlalchemy.util._collections.result'>
    >>> type(res[0].name)
    <class 'str'>
    >>> type(res[0][0])
    <class 'str'>
    >>> res[0].name == res[0][0]
    True

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.



From hamilton at nothere.com  Sun May 24 09:29:15 2015
From: hamilton at nothere.com (hamilton)
Date: Sun, 24 May 2015 07:29:15 -0600
Subject: Human Rights and Justice in Islam
In-Reply-To: <5561a11d$0$12999$c3e8da3$5496439d@news.astraweb.com>
References: <1050c913-df97-45c2-9fa8-041b6d17037a@googlegroups.com>
 <mjr2nl$8ke$1@dont-email.me>
 <5561a11d$0$12999$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mjsjl5$pij$1@dont-email.me>

On 5/24/2015 3:59 AM, Steven D'Aprano wrote:
> On Sun, 24 May 2015 09:34 am, hamilton wrote:
> [quoted bullshit from a spammer]
> [tried to argue with said spammer]
>
> Please don't reply to fly-by-spammers. Even if the spammer was interested in
> honest debate -- and he is not (fortunately!) -- the last thing we want on
> a Python newsgroup is for it to be hijacked about arguments about religion.
>
>

Sorry, it just hit me ...... :-(



From hamilton at nothere.com  Sun May 24 09:29:15 2015
From: hamilton at nothere.com (hamilton)
Date: Sun, 24 May 2015 07:29:15 -0600
Subject: Human Rights and Justice in Islam
In-Reply-To: <5561a11d$0$12999$c3e8da3$5496439d@news.astraweb.com>
References: <1050c913-df97-45c2-9fa8-041b6d17037a@googlegroups.com>
 <mjr2nl$8ke$1@dont-email.me>
 <5561a11d$0$12999$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mjsjl5$pij$1@dont-email.me>

On 5/24/2015 3:59 AM, Steven D'Aprano wrote:
> On Sun, 24 May 2015 09:34 am, hamilton wrote:
> [quoted bullshit from a spammer]
> [tried to argue with said spammer]
>
> Please don't reply to fly-by-spammers. Even if the spammer was interested in
> honest debate -- and he is not (fortunately!) -- the last thing we want on
> a Python newsgroup is for it to be hijacked about arguments about religion.
>
>

Sorry, it just hit me ...... :-(



From breamoreboy at yahoo.co.uk  Sun May 24 09:56:01 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 24 May 2015 14:56:01 +0100
Subject: Setting prompt in ipython
In-Reply-To: <87h9r2nmy1.fsf@Equus.decebal.nl>
References: <87siamnt6r.fsf@Equus.decebal.nl>
 <mailman.11.1432468462.5151.python-list@python.org>
 <87lhgenpj8.fsf@Equus.decebal.nl> <87h9r2nmy1.fsf@Equus.decebal.nl>
Message-ID: <mjsl9r$1mi$1@ger.gmane.org>

On 24/05/2015 14:07, Cecil Westerhof wrote:
> Op Sunday 24 May 2015 14:11 CEST schreef Cecil Westerhof:
>
>> Op Sunday 24 May 2015 13:54 CEST schreef Laura Creighton:
>>
>>> This file supposedly lists what you can customise your ipython
>>> prompt to show. I have not tried this myself.
>>>
>>> http://www.math.uiuc.edu/~gfrancis/illimath/windows/aszgard_mini/movpy-2.0.0-py2.4.4/manuals/IPython/node7.html
>>
>> I will look into it.
>
> I use the following in ~/.ipython/profile_default/startup/00-init.ipy:
>      import time
>
>      %config PromptManager.in_template = r'[{time.strftime("%a, %d %b %T")} \u@\h:\Y1]\nIn [\#]: '
>
> Now I should contemplate what other things would be useful there. ;-)
>

One of the major warts that (i)Python has yet to address via a PEP is 
that it fails dismally when it comes to making the coffee and sandwiches 
to keep you going whilst using it.  Perhaps you'd like to look into this 
issue?  I'm not aware of anything on the bug tracker about it.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Sun May 24 09:56:01 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 24 May 2015 14:56:01 +0100
Subject: Setting prompt in ipython
In-Reply-To: <87h9r2nmy1.fsf@Equus.decebal.nl>
References: <87siamnt6r.fsf@Equus.decebal.nl>
 <mailman.11.1432468462.5151.python-list@python.org>
 <87lhgenpj8.fsf@Equus.decebal.nl> <87h9r2nmy1.fsf@Equus.decebal.nl>
Message-ID: <mjsl9r$1mi$1@ger.gmane.org>

On 24/05/2015 14:07, Cecil Westerhof wrote:
> Op Sunday 24 May 2015 14:11 CEST schreef Cecil Westerhof:
>
>> Op Sunday 24 May 2015 13:54 CEST schreef Laura Creighton:
>>
>>> This file supposedly lists what you can customise your ipython
>>> prompt to show. I have not tried this myself.
>>>
>>> http://www.math.uiuc.edu/~gfrancis/illimath/windows/aszgard_mini/movpy-2.0.0-py2.4.4/manuals/IPython/node7.html
>>
>> I will look into it.
>
> I use the following in ~/.ipython/profile_default/startup/00-init.ipy:
>      import time
>
>      %config PromptManager.in_template = r'[{time.strftime("%a, %d %b %T")} \u@\h:\Y1]\nIn [\#]: '
>
> Now I should contemplate what other things would be useful there. ;-)
>

One of the major warts that (i)Python has yet to address via a PEP is 
that it fails dismally when it comes to making the coffee and sandwiches 
to keep you going whilst using it.  Perhaps you'd like to look into this 
issue?  I'm not aware of anything on the bug tracker about it.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Sun May 24 10:00:03 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 25 May 2015 00:00:03 +1000
Subject: Camelot a good tool for me
In-Reply-To: <878uce2jh4.fsf@nautilus.nautilus>
References: <87fv6ol93p.fsf@nautilus.nautilus>
 <20150522224229.GA62222@cskk.homeip.net>
 <CAPTjJmpCuhqTwkxfCW48e8We=9Hp7UG3EeOiynH1nNy_LjnsZw@mail.gmail.com>
 <878uce2jh4.fsf@nautilus.nautilus>
Message-ID: <CAPTjJmp6w1u4p_2Afpbfz+jVtQhth-kF9S61O4Bs-iyb9bUV9Q@mail.gmail.com>

On Sun, May 24, 2015 at 11:28 PM, Lele Gaifax <lele at metapensiero.it> wrote:
> Chris Angelico <rosuav at gmail.com> writes:
>
>> You don't automatically get nice objects with attributes named after
>> columns, but neither will SQLAlchemy unless you're doing the
>> equivalent of "select * from". Example:
>>
>>>>> session.query(Student).all()
>> [<__main__.Student object at 0x7fc2de3fc290>, <__main__.Student object
>> at 0x7fc2de3fc350>, <__main__.Student object at 0x7fc2de3fc3d0>]
>>
>> That's a list of Student objects, because I asked for the entire table.
>>
>>>>> session.query(Student.name, Student.year).all()
>> [(u'Fred', 2015), (u'Joe', 2015), (u"Robert'); DROP TABLE Students;--", 2012)]
>>
>> That's a list of tuples, because I asked for just two columns.
>
> Wrong:
>
>     >>> res = session.query(Student.name, Student.year).all()
>     >>> type(res)
>     <class 'list'>
>     >>> type(res[0])
>     <class 'sqlalchemy.util._collections.result'>
>     >>> type(res[0].name)
>     <class 'str'>
>     >>> type(res[0][0])
>     <class 'str'>
>     >>> res[0].name == res[0][0]
>     True

Huh. Okay, was not aware of that. It's some sort of namedtuple-like
thing that acts as a tuple but has attributes too. Very nice. I
withdraw the criticism.

My other criticisms were mainly on the basis that bad code is easier
to write when you have more magic; rebuttals of the form "but nobody
would be dumb enough to write code like that" are valid only to the
extent that poor programmers are more likely to be using PHP than
Python, so we're seeing a better calibre of coder here. But it's still
very much possible to see - and have to maintain - some pretty
horrendous code. Someone chooses to iterate over the query rather than
calling .all() because "it's quicker that way" (it may or may not be,
given that iterating over the query is lazy but all() is eager), and
then calls session.commit() every hundred rows because "it's quicker
that way". I've seen both of the above, although not together and not
in SQLAlchemy. The latter is quite common; check out these advice
pages, which seem to consider committing to be something you do for
performance reasons rather than to enforce transactional integrity:

http://www.ajaxline.com/32-tips-to-speed-up-your-mysql-queries
http://stackoverflow.com/questions/14675147/why-does-transaction-commit-improve-performance-so-much-with-php-mysql-innodb
-- question asked "Would you recommend COMMIT'ing every 1,000 or so
INSERTS's..."

That thinking is out there, and if you don't keep your transactions
clear and clean, you WILL have problems.

ChrisA


From rosuav at gmail.com  Sun May 24 10:00:03 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 25 May 2015 00:00:03 +1000
Subject: Camelot a good tool for me
In-Reply-To: <878uce2jh4.fsf@nautilus.nautilus>
References: <87fv6ol93p.fsf@nautilus.nautilus>
 <20150522224229.GA62222@cskk.homeip.net>
 <CAPTjJmpCuhqTwkxfCW48e8We=9Hp7UG3EeOiynH1nNy_LjnsZw@mail.gmail.com>
 <878uce2jh4.fsf@nautilus.nautilus>
Message-ID: <CAPTjJmp6w1u4p_2Afpbfz+jVtQhth-kF9S61O4Bs-iyb9bUV9Q@mail.gmail.com>

On Sun, May 24, 2015 at 11:28 PM, Lele Gaifax <lele at metapensiero.it> wrote:
> Chris Angelico <rosuav at gmail.com> writes:
>
>> You don't automatically get nice objects with attributes named after
>> columns, but neither will SQLAlchemy unless you're doing the
>> equivalent of "select * from". Example:
>>
>>>>> session.query(Student).all()
>> [<__main__.Student object at 0x7fc2de3fc290>, <__main__.Student object
>> at 0x7fc2de3fc350>, <__main__.Student object at 0x7fc2de3fc3d0>]
>>
>> That's a list of Student objects, because I asked for the entire table.
>>
>>>>> session.query(Student.name, Student.year).all()
>> [(u'Fred', 2015), (u'Joe', 2015), (u"Robert'); DROP TABLE Students;--", 2012)]
>>
>> That's a list of tuples, because I asked for just two columns.
>
> Wrong:
>
>     >>> res = session.query(Student.name, Student.year).all()
>     >>> type(res)
>     <class 'list'>
>     >>> type(res[0])
>     <class 'sqlalchemy.util._collections.result'>
>     >>> type(res[0].name)
>     <class 'str'>
>     >>> type(res[0][0])
>     <class 'str'>
>     >>> res[0].name == res[0][0]
>     True

Huh. Okay, was not aware of that. It's some sort of namedtuple-like
thing that acts as a tuple but has attributes too. Very nice. I
withdraw the criticism.

My other criticisms were mainly on the basis that bad code is easier
to write when you have more magic; rebuttals of the form "but nobody
would be dumb enough to write code like that" are valid only to the
extent that poor programmers are more likely to be using PHP than
Python, so we're seeing a better calibre of coder here. But it's still
very much possible to see - and have to maintain - some pretty
horrendous code. Someone chooses to iterate over the query rather than
calling .all() because "it's quicker that way" (it may or may not be,
given that iterating over the query is lazy but all() is eager), and
then calls session.commit() every hundred rows because "it's quicker
that way". I've seen both of the above, although not together and not
in SQLAlchemy. The latter is quite common; check out these advice
pages, which seem to consider committing to be something you do for
performance reasons rather than to enforce transactional integrity:

http://www.ajaxline.com/32-tips-to-speed-up-your-mysql-queries
http://stackoverflow.com/questions/14675147/why-does-transaction-commit-improve-performance-so-much-with-php-mysql-innodb
-- question asked "Would you recommend COMMIT'ing every 1,000 or so
INSERTS's..."

That thinking is out there, and if you don't keep your transactions
clear and clean, you WILL have problems.

ChrisA


From rosuav at gmail.com  Sun May 24 10:02:41 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 25 May 2015 00:02:41 +1000
Subject: Setting prompt in ipython
In-Reply-To: <mjsl9r$1mi$1@ger.gmane.org>
References: <87siamnt6r.fsf@Equus.decebal.nl>
 <mailman.11.1432468462.5151.python-list@python.org>
 <87lhgenpj8.fsf@Equus.decebal.nl> <87h9r2nmy1.fsf@Equus.decebal.nl>
 <mjsl9r$1mi$1@ger.gmane.org>
Message-ID: <CAPTjJmrQ62efGXvrq5EWDx5siTaeOJQAzqw=qHf4QvY7MOK17Q@mail.gmail.com>

On Sun, May 24, 2015 at 11:56 PM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> One of the major warts that (i)Python has yet to address via a PEP is that
> it fails dismally when it comes to making the coffee and sandwiches to keep
> you going whilst using it.  Perhaps you'd like to look into this issue?  I'm
> not aware of anything on the bug tracker about it.

Oh, automated coffee management doesn't need a PEP. Just create a
PyCoffee module, upload it to PyPI, and see how many cups get
downloaded.

ChrisA


From rosuav at gmail.com  Sun May 24 10:02:41 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 25 May 2015 00:02:41 +1000
Subject: Setting prompt in ipython
In-Reply-To: <mjsl9r$1mi$1@ger.gmane.org>
References: <87siamnt6r.fsf@Equus.decebal.nl>
 <mailman.11.1432468462.5151.python-list@python.org>
 <87lhgenpj8.fsf@Equus.decebal.nl> <87h9r2nmy1.fsf@Equus.decebal.nl>
 <mjsl9r$1mi$1@ger.gmane.org>
Message-ID: <CAPTjJmrQ62efGXvrq5EWDx5siTaeOJQAzqw=qHf4QvY7MOK17Q@mail.gmail.com>

On Sun, May 24, 2015 at 11:56 PM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> One of the major warts that (i)Python has yet to address via a PEP is that
> it fails dismally when it comes to making the coffee and sandwiches to keep
> you going whilst using it.  Perhaps you'd like to look into this issue?  I'm
> not aware of anything on the bug tracker about it.

Oh, automated coffee management doesn't need a PEP. Just create a
PyCoffee module, upload it to PyPI, and see how many cups get
downloaded.

ChrisA


From fomcl at yahoo.com  Sun May 24 10:16:20 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Sun, 24 May 2015 07:16:20 -0700
Subject: programmatically change windows regional settings?
Message-ID: <1432476980.53449.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>


-----------------------------
On Sun, May 24, 2015 3:07 PM CEST Mark Lawrence wrote:

>On 24/05/2015 13:50, Albert-Jan Roskam via Python-list wrote:
>>
>> Hi,
>>
>> In Windows I can change the regional settings manually in the control panel. But how do I do this programmatically? I tried setting LANG but this does not work in Windows.
>>
>> Kernel32's SetLocaleInfo sounds promising, but "This setting only affects the user override portion of the locale settings; it does not set the system defaults." https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx
>>
>> Another route might be _winreg (but that's probably more brittle and, more importantly, the registry sucks)
>>
>> My goal is to easily run my unittests in a number of locales (actually a platform x locale x python version matrix)
>>
>> Thanks!
>>
>> Albert-Jan
>>
>
>You're probably better off asking this on 
>https://mail.python.org/mailman/listinfo/python-win32 which is also 
>available as gmane.comp.python.windows, although you might get lucky 
>here.  

Ok, I'll check that out, thanks. Hope that list is about more than win32com

If you do a rather more specific statement than "this does not 
>work in Windows" would be helpful. 

One can set LANG in Windows (of course), but Windows does not appear to ever use it. So it's pointless.
setx LANG France.French.1252 && python -c "from locale import *; setlocale(LC_ALL, ''); print(getlocale())" 

... Does not print a French locale on my non-French system 

>Your Python and Windows version(s) 
>might possibly assist as well :)


Windows Server 2012 R2 (x64), and preferably also Win 7 x64. Python 2.7, 3.3, 3.4, and ideally also pypy.


>-- 
>My fellow Pythonistas, ask not what our language can do for you, ask
>what you can do for our language.
>
>Mark Lawrence
>
>-- 
>https://mail.python.org/mailman/listinfo/python-list



From fomcl at yahoo.com  Sun May 24 10:16:20 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Sun, 24 May 2015 07:16:20 -0700
Subject: programmatically change windows regional settings?
Message-ID: <1432476980.53449.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>


-----------------------------
On Sun, May 24, 2015 3:07 PM CEST Mark Lawrence wrote:

>On 24/05/2015 13:50, Albert-Jan Roskam via Python-list wrote:
>>
>> Hi,
>>
>> In Windows I can change the regional settings manually in the control panel. But how do I do this programmatically? I tried setting LANG but this does not work in Windows.
>>
>> Kernel32's SetLocaleInfo sounds promising, but "This setting only affects the user override portion of the locale settings; it does not set the system defaults." https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx
>>
>> Another route might be _winreg (but that's probably more brittle and, more importantly, the registry sucks)
>>
>> My goal is to easily run my unittests in a number of locales (actually a platform x locale x python version matrix)
>>
>> Thanks!
>>
>> Albert-Jan
>>
>
>You're probably better off asking this on 
>https://mail.python.org/mailman/listinfo/python-win32 which is also 
>available as gmane.comp.python.windows, although you might get lucky 
>here.  

Ok, I'll check that out, thanks. Hope that list is about more than win32com

If you do a rather more specific statement than "this does not 
>work in Windows" would be helpful. 

One can set LANG in Windows (of course), but Windows does not appear to ever use it. So it's pointless.
setx LANG France.French.1252 && python -c "from locale import *; setlocale(LC_ALL, ''); print(getlocale())" 

... Does not print a French locale on my non-French system 

>Your Python and Windows version(s) 
>might possibly assist as well :)


Windows Server 2012 R2 (x64), and preferably also Win 7 x64. Python 2.7, 3.3, 3.4, and ideally also pypy.


>-- 
>My fellow Pythonistas, ask not what our language can do for you, ask
>what you can do for our language.
>
>Mark Lawrence
>
>-- 
>https://mail.python.org/mailman/listinfo/python-list



From lac at openend.se  Sun May 24 10:45:26 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 24 May 2015 16:45:26 +0200
Subject: programmatically change windows regional settings?
In-Reply-To: Message from Mark Lawrence <breamoreboy@yahoo.co.uk>
 of "Sun, 24 May 2015 14:07:37 +0100." <mjsif3$nvb$1@ger.gmane.org>
References: <1432471803.77701.BPMail_high_carrier@web163802.mail.gq1.yahoo.com><mjsif3$nvb$1@ger.gmane.org>
Message-ID: <201505241445.t4OEjQ8D004829@fido.openend.se>

In a message of Sun, 24 May 2015 14:07:37 +0100, Mark Lawrence writes:
>On 24/05/2015 13:50, Albert-Jan Roskam via Python-list wrote:
>>
>> Hi,
>>
>> In Windows I can change the regional settings manually in the control panel. But how do I do this programmatically? I tried setting LANG but this does not work in Windows.
>>
>> Kernel32's SetLocaleInfo sounds promising, but "This setting only affects the user override portion of the locale settings; it does not set the system defaults." https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx
>>
>> Another route might be _winreg (but that's probably more brittle and, more importantly, the registry sucks)
>>
>> My goal is to easily run my unittests in a number of locales (actually a platform x locale x python version matrix)
>>
>> Thanks!
>>
>> Albert-Jan

>You're probably better off asking this on 
>https://mail.python.org/mailman/listinfo/python-win32 which is also 
>available as gmane.comp.python.windows, although you might get lucky 
>here.  If you do a rather more specific statement than "this does not 
>work in Windows" would be helpful.  Your Python and Windows version(s) 
>might possibly assist as well :)
>
>-- 
>My fellow Pythonistas, ask not what our language can do for you, ask
>what you can do for our language.
>
>Mark Lawrence

I actually think he will get the best advice from the testing-in-python
mailing list.

http://lists.idyll.org/listinfo/testing-in-python

Laura



From lac at openend.se  Sun May 24 10:45:26 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 24 May 2015 16:45:26 +0200
Subject: programmatically change windows regional settings?
In-Reply-To: Message from Mark Lawrence <breamoreboy@yahoo.co.uk>
 of "Sun, 24 May 2015 14:07:37 +0100." <mjsif3$nvb$1@ger.gmane.org>
References: <1432471803.77701.BPMail_high_carrier@web163802.mail.gq1.yahoo.com><mjsif3$nvb$1@ger.gmane.org>
Message-ID: <201505241445.t4OEjQ8D004829@fido.openend.se>

In a message of Sun, 24 May 2015 14:07:37 +0100, Mark Lawrence writes:
>On 24/05/2015 13:50, Albert-Jan Roskam via Python-list wrote:
>>
>> Hi,
>>
>> In Windows I can change the regional settings manually in the control panel. But how do I do this programmatically? I tried setting LANG but this does not work in Windows.
>>
>> Kernel32's SetLocaleInfo sounds promising, but "This setting only affects the user override portion of the locale settings; it does not set the system defaults." https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx
>>
>> Another route might be _winreg (but that's probably more brittle and, more importantly, the registry sucks)
>>
>> My goal is to easily run my unittests in a number of locales (actually a platform x locale x python version matrix)
>>
>> Thanks!
>>
>> Albert-Jan

>You're probably better off asking this on 
>https://mail.python.org/mailman/listinfo/python-win32 which is also 
>available as gmane.comp.python.windows, although you might get lucky 
>here.  If you do a rather more specific statement than "this does not 
>work in Windows" would be helpful.  Your Python and Windows version(s) 
>might possibly assist as well :)
>
>-- 
>My fellow Pythonistas, ask not what our language can do for you, ask
>what you can do for our language.
>
>Mark Lawrence

I actually think he will get the best advice from the testing-in-python
mailing list.

http://lists.idyll.org/listinfo/testing-in-python

Laura



From fomcl at yahoo.com  Sun May 24 11:04:30 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Sun, 24 May 2015 08:04:30 -0700
Subject: programmatically change windows regional settings?
Message-ID: <1432479870.69383.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>


-----------------------------
On Sun, May 24, 2015 4:45 PM CEST Laura Creighton wrote:

>In a message of Sun, 24 May 2015 14:07:37 +0100, Mark Lawrence writes:
>>On 24/05/2015 13:50, Albert-Jan Roskam via Python-list wrote:
>>
>> Hi,
>>
>> In Windows I can change the regional settings manually in the control panel. But how do I do this programmatically? I tried setting LANG but this does not work in Windows.
>>
>> Kernel32's SetLocaleInfo sounds promising, but "This setting only affects the user override portion of the locale settings; it does not set the system defaults." https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx
>>
>> Another route might be _winreg (but that's probably more brittle and, more importantly, the registry sucks)
>>
>> My goal is to easily run my unittests in a number of locales (actually a platform x locale x python version matrix)
>>
>> Thanks!
>>
>> Albert-Jan
>
>>You're probably better off asking this on 
>>https://mail.python.org/mailman/listinfo/python-win32 which is also 
>>available as gmane.comp.python.windows, although you might get lucky 
>>here.  If you do a rather more specific statement than "this does not 
>>work in Windows" would be helpful.  Your Python and Windows version(s) 
>>might possibly assist as well :)
>>
>>-- 
>>My fellow Pythonistas, ask not what our language can do for you, ask
>>what you can do for our language.
>>
>>Mark Lawrence
>
>I actually think he will get the best advice from the testing-in-python
>mailing list.


:-) That's indeed what I've done initially, but my message does not get posted. But maybe that list is moderated and I'm too impatient. But TIP is indeed the place to be for Tox-related questions.


From fomcl at yahoo.com  Sun May 24 11:04:30 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Sun, 24 May 2015 08:04:30 -0700
Subject: programmatically change windows regional settings?
Message-ID: <1432479870.69383.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>


-----------------------------
On Sun, May 24, 2015 4:45 PM CEST Laura Creighton wrote:

>In a message of Sun, 24 May 2015 14:07:37 +0100, Mark Lawrence writes:
>>On 24/05/2015 13:50, Albert-Jan Roskam via Python-list wrote:
>>
>> Hi,
>>
>> In Windows I can change the regional settings manually in the control panel. But how do I do this programmatically? I tried setting LANG but this does not work in Windows.
>>
>> Kernel32's SetLocaleInfo sounds promising, but "This setting only affects the user override portion of the locale settings; it does not set the system defaults." https://msdn.microsoft.com/en-us/library/ee491893(v=winembedded.60).aspx
>>
>> Another route might be _winreg (but that's probably more brittle and, more importantly, the registry sucks)
>>
>> My goal is to easily run my unittests in a number of locales (actually a platform x locale x python version matrix)
>>
>> Thanks!
>>
>> Albert-Jan
>
>>You're probably better off asking this on 
>>https://mail.python.org/mailman/listinfo/python-win32 which is also 
>>available as gmane.comp.python.windows, although you might get lucky 
>>here.  If you do a rather more specific statement than "this does not 
>>work in Windows" would be helpful.  Your Python and Windows version(s) 
>>might possibly assist as well :)
>>
>>-- 
>>My fellow Pythonistas, ask not what our language can do for you, ask
>>what you can do for our language.
>>
>>Mark Lawrence
>
>I actually think he will get the best advice from the testing-in-python
>mailing list.


:-) That's indeed what I've done initially, but my message does not get posted. But maybe that list is moderated and I'm too impatient. But TIP is indeed the place to be for Tox-related questions.


From breamoreboy at yahoo.co.uk  Sun May 24 11:23:05 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 24 May 2015 16:23:05 +0100
Subject: programmatically change windows regional settings?
In-Reply-To: <1432476980.53449.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>
References: <1432476980.53449.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>
Message-ID: <mjsqd3$b9v$1@ger.gmane.org>

On 24/05/2015 15:16, Albert-Jan Roskam via Python-list wrote:
>
> -----------------------------
> On Sun, May 24, 2015 3:07 PM CEST Mark Lawrence wrote:
>>
>> You're probably better off asking this on
>> https://mail.python.org/mailman/listinfo/python-win32 which is also
>> available as gmane.comp.python.windows, although you might get lucky
>> here.
>
> Ok, I'll check that out, thanks. Hope that list is about more than win32com
>

 From the above link "All issues related to programming Python on 
Windows. win32 extensions, COM, you name it."

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Sun May 24 11:23:05 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 24 May 2015 16:23:05 +0100
Subject: programmatically change windows regional settings?
In-Reply-To: <1432476980.53449.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>
References: <1432476980.53449.BPMail_high_carrier@web163802.mail.gq1.yahoo.com>
Message-ID: <mjsqd3$b9v$1@ger.gmane.org>

On 24/05/2015 15:16, Albert-Jan Roskam via Python-list wrote:
>
> -----------------------------
> On Sun, May 24, 2015 3:07 PM CEST Mark Lawrence wrote:
>>
>> You're probably better off asking this on
>> https://mail.python.org/mailman/listinfo/python-win32 which is also
>> available as gmane.comp.python.windows, although you might get lucky
>> here.
>
> Ok, I'll check that out, thanks. Hope that list is about more than win32com
>

 From the above link "All issues related to programming Python on 
Windows. win32 extensions, COM, you name it."

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From marko at pacujo.net  Sun May 24 11:26:44 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sun, 24 May 2015 18:26:44 +0300
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
 <5561a74b$0$12977$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87fv6mj8sr.fsf@elektro.pacujo.net>

Steven D'Aprano <steve at pearwood.info>:

> On Sun, 24 May 2015 02:53 am, Marko Rauhamaa wrote:
> "an authentication is considered valid if it is vouched for by the United
> States, China, Russia *and* the European Union."
>
> [Emphasis in the original.]
>
> So if (let's say) the US, China and Russia all agree that a Certs-R-Us are a
> legitimate CA,

I never proposed those countries should agree on a legitimate CA. Each
country would have their distinct, respective sets of CAs. A website
would be considered legitimate only if it possessed certificates from
all of the four domains.

> but the EU disagrees for some reason. Then certificates issued by
> Certs-R-Us will *not* be accepted as valid. Hence the EU has veto
> power over CAs, and by extension, certificates. And likewise any of
> the others: it only takes one refusal for the certificate to be
> invalid.

For the scheme to work, the countries would agree never to refuse to
certify a legitimate entity.

> (And by the way, I'd be more inclined to trust a CA that was vouched
> for by, say, the Norwegian government than one vouched for by the
> Russian government.)

You'd be asked to trust a server if it were vouched for by four CAs from
four independent power blocks. It would be in the power blocks'
interests to do their certification work properly because sloppiness
would only benefit their "adversaries."

> And what's this "dropped from the power pool" business? You never
> mentioned a mechanism for removing a state from the privileged group.
> Who has authority to do that?

A mechanism like that would have to be fleshed out in detail. There
could be many variations. But the main point is that no single
government or CA should have the means to spoof a peer.

> I'm not sure that it is much of a benefit to swap from a free-market
> reputation based system to a four-party oligopoly

The CA system is not based on reputation. Anybody in the world has an
equal right to become a root CA. You only need to convince the
OS/browser vendors:

   https://bugzilla.mozilla.org/show_bug.cgi?id=647959


The oligopoly would fail if the US, Russia, China and EU conspired
against a particular peer. Can you see that happening?


Marko


From marko at pacujo.net  Sun May 24 11:26:44 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sun, 24 May 2015 18:26:44 +0300
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
 <5561a74b$0$12977$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87fv6mj8sr.fsf@elektro.pacujo.net>

Steven D'Aprano <steve at pearwood.info>:

> On Sun, 24 May 2015 02:53 am, Marko Rauhamaa wrote:
> "an authentication is considered valid if it is vouched for by the United
> States, China, Russia *and* the European Union."
>
> [Emphasis in the original.]
>
> So if (let's say) the US, China and Russia all agree that a Certs-R-Us are a
> legitimate CA,

I never proposed those countries should agree on a legitimate CA. Each
country would have their distinct, respective sets of CAs. A website
would be considered legitimate only if it possessed certificates from
all of the four domains.

> but the EU disagrees for some reason. Then certificates issued by
> Certs-R-Us will *not* be accepted as valid. Hence the EU has veto
> power over CAs, and by extension, certificates. And likewise any of
> the others: it only takes one refusal for the certificate to be
> invalid.

For the scheme to work, the countries would agree never to refuse to
certify a legitimate entity.

> (And by the way, I'd be more inclined to trust a CA that was vouched
> for by, say, the Norwegian government than one vouched for by the
> Russian government.)

You'd be asked to trust a server if it were vouched for by four CAs from
four independent power blocks. It would be in the power blocks'
interests to do their certification work properly because sloppiness
would only benefit their "adversaries."

> And what's this "dropped from the power pool" business? You never
> mentioned a mechanism for removing a state from the privileged group.
> Who has authority to do that?

A mechanism like that would have to be fleshed out in detail. There
could be many variations. But the main point is that no single
government or CA should have the means to spoof a peer.

> I'm not sure that it is much of a benefit to swap from a free-market
> reputation based system to a four-party oligopoly

The CA system is not based on reputation. Anybody in the world has an
equal right to become a root CA. You only need to convince the
OS/browser vendors:

   https://bugzilla.mozilla.org/show_bug.cgi?id=647959


The oligopoly would fail if the US, Russia, China and EU conspired
against a particular peer. Can you see that happening?


Marko


From rosuav at gmail.com  Sun May 24 11:35:37 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 25 May 2015 01:35:37 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <87fv6mj8sr.fsf@elektro.pacujo.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
 <5561a74b$0$12977$c3e8da3$5496439d@news.astraweb.com>
 <87fv6mj8sr.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmoQaBexwnLg0C6LmoXyZYe13sUeUsJS+yH8JmZhFiNwzg@mail.gmail.com>

On Mon, May 25, 2015 at 1:26 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Steven D'Aprano <steve at pearwood.info>:
>
>> On Sun, 24 May 2015 02:53 am, Marko Rauhamaa wrote:
>> "an authentication is considered valid if it is vouched for by the United
>> States, China, Russia *and* the European Union."
>>
>> [Emphasis in the original.]
>>
>> So if (let's say) the US, China and Russia all agree that a Certs-R-Us are a
>> legitimate CA,
>
> I never proposed those countries should agree on a legitimate CA. Each
> country would have their distinct, respective sets of CAs. A website
> would be considered legitimate only if it possessed certificates from
> all of the four domains.

You've added extra levels of indirection, but it comes to the same
thing. You're requiring that everyone who wants to conduct business on
the internet (taking credit card numbers etc) has to go through four
separate authentication processes, and a failure in any one of them
means the site is not considered legit.

> For the scheme to work, the countries would agree never to refuse to
> certify a legitimate entity.

Right. And "legitimate" is defined as "not refused by any of the four
countries". All they have to do is decide that something's not
legitimate, and bam, they're off air. ANY ONE of your four has this
power of veto.

ChrisA


From rosuav at gmail.com  Sun May 24 11:35:37 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 25 May 2015 01:35:37 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <87fv6mj8sr.fsf@elektro.pacujo.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
 <5561a74b$0$12977$c3e8da3$5496439d@news.astraweb.com>
 <87fv6mj8sr.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmoQaBexwnLg0C6LmoXyZYe13sUeUsJS+yH8JmZhFiNwzg@mail.gmail.com>

On Mon, May 25, 2015 at 1:26 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Steven D'Aprano <steve at pearwood.info>:
>
>> On Sun, 24 May 2015 02:53 am, Marko Rauhamaa wrote:
>> "an authentication is considered valid if it is vouched for by the United
>> States, China, Russia *and* the European Union."
>>
>> [Emphasis in the original.]
>>
>> So if (let's say) the US, China and Russia all agree that a Certs-R-Us are a
>> legitimate CA,
>
> I never proposed those countries should agree on a legitimate CA. Each
> country would have their distinct, respective sets of CAs. A website
> would be considered legitimate only if it possessed certificates from
> all of the four domains.

You've added extra levels of indirection, but it comes to the same
thing. You're requiring that everyone who wants to conduct business on
the internet (taking credit card numbers etc) has to go through four
separate authentication processes, and a failure in any one of them
means the site is not considered legit.

> For the scheme to work, the countries would agree never to refuse to
> certify a legitimate entity.

Right. And "legitimate" is defined as "not refused by any of the four
countries". All they have to do is decide that something's not
legitimate, and bam, they're off air. ANY ONE of your four has this
power of veto.

ChrisA


From anthra.norell at bluewin.ch  Sun May 24 13:48:27 2015
From: anthra.norell at bluewin.ch (Friedrich Rentsch)
Date: Sun, 24 May 2015 19:48:27 +0200
Subject: Extract email address from Java script in html source using python
In-Reply-To: <CAAXuHod4NH74+VK1EY78-vCOyiOA3879qp1uPqKOFit7qVE5sQ@mail.gmail.com>
References: <CAAXuHoeJ-YMQDB85qLDJ_o+9CrsfwLvm9wuOaRtbSj-i9kBaFA@mail.gmail.com>
 <CAPTjJmqBDu0nB2u_mf7KMpFzxxJFUqB7o-7dJiGgu-xyOL2uzg@mail.gmail.com>
 <CAAXuHod4NH74+VK1EY78-vCOyiOA3879qp1uPqKOFit7qVE5sQ@mail.gmail.com>
Message-ID: <55620EEB.9000601@bluewin.ch>



On 05/23/2015 04:15 PM, savitha devi wrote:
> What I exactly want is the java script is in the html code. I am trying for
> a regular expression to find the email address embedded with in the java
> script.
>
> On Sat, May 23, 2015 at 2:31 PM, Chris Angelico <rosuav at gmail.com> wrote:
>
>> On Sat, May 23, 2015 at 4:46 PM, savitha devi <savithad8 at gmail.com> wrote:
>>> I am developing a web scraper code using HTMLParser. I need to extract
>>> text/email address from java script with in the HTMLCode.I am beginner
>> level
>>> in python coding and totally lost here. Need some help on this. The java
>>> script code is as below:
>>>
>>> <script type='text/javascript'>
>>>   //<!--
>>>   document.getElementById('cloak48218').innerHTML = '';
>>>   var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
>>>   var path = 'hr' + 'ef' + '=';
>>>   var addy48218 = '&#105;nf&#111;' + '&#64;';
>>>   addy48218 = addy48218 + 'tsv-n&#101;&#117;r&#105;&#101;d' + '&#46;' +
>>> 'd&#101;';
>>>   document.getElementById('cloak48218').innerHTML += '<a ' + path + '\'' +
>>> prefix + ':' + addy48218 + '\'>' + addy48218+'<\/a>';
>>>   //-->
>> This is deliberately being done to prevent scripted usage. What
>> exactly are you needing to do this for?
>>
>> You're basically going to have to execute the entire block of
>> JavaScript code, and then decode the entities to get to what you want.
>> Doing it manually is pretty easy; doing it automatically will
>> virtually require a language interpreter.
>>
>> ChrisA
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>

This is just about nuts and bolts, not about the ethics of presumed 
intentions.

Hope it helps one way or other

Frederic


------------------------------------------------------------------------------- 


sample = '''//<!--
  document.getElementById('cloak48218').innerHTML = '';
  var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
  var path = 'hr' + 'ef' + '=';
  var addy48218 = '&#105;nf&#111;' + '&#64;';
  addy48218 = addy48218 + 'tsv-n&#101;&#117;r&#105;&#101;d' + '&#46;' +
'd&#101;';
  document.getElementById('cloak48218').innerHTML += '<a ' + path + '\'' +
prefix + ':' + addy48218 + ''>' + addy48218+'<\/a>';
  //-->'''

 >>> import SE  # Download from PyPi at https://pypi.python.org/pypi/SE

 >>> def make_se_translator ():

     # Make SE substitutions
     subs_list = []

     # Make &# code substitutions
     for i in range (256):
         subs_list.append ('&#%d;=%c' % (i, chr(i)))

     # Delete Java stuff
     subs_list.append (' "document.getElementById(\'cloak48218\').=" ')
     subs_list.append (' "var =" "\n=" //<!--= //-->= ')

     # Java syntax? Tweaks needed to get the sample working
     subs_list.append (' "+ \'\'\'=" \'\'>\'=\'>\' <\/=</ ')

     # Add more as needed trial and error style
     # subs_list.append ( . . . format: ' old=new "delete this=" '

     # Make text
     subs = '\n'.join (subs_list)

     # Make SE translator
     translator = SE.SE (subs)

     # return translator, subs   # print subs, if you want to see what 
they look like
     return translator


 >>> translator = make_se_translator ()

 >>> translation = translator (sample)

 >>> print translation   # See
  innerHTML = ''; prefix = 'ma' + 'il' + 'to'; path = 'hr' + 'ef' + '='; 
addy48218 = 'info' + '@'; addy48218 = addy48218 + 'tsv-neuried' + '.' 
+'de'; innerHTML += '<a ' + path  +prefix + ':' + addy48218 + '>' + 
addy48218+'</a>';

 >>> exec (translation.lstrip ())

 >>> print innerHTML
<a href=mailto:info at tsv-neuried.de>info at tsv-neuried.de</a>



From anthra.norell at bluewin.ch  Sun May 24 13:48:27 2015
From: anthra.norell at bluewin.ch (Friedrich Rentsch)
Date: Sun, 24 May 2015 19:48:27 +0200
Subject: Extract email address from Java script in html source using python
In-Reply-To: <CAAXuHod4NH74+VK1EY78-vCOyiOA3879qp1uPqKOFit7qVE5sQ@mail.gmail.com>
References: <CAAXuHoeJ-YMQDB85qLDJ_o+9CrsfwLvm9wuOaRtbSj-i9kBaFA@mail.gmail.com>
 <CAPTjJmqBDu0nB2u_mf7KMpFzxxJFUqB7o-7dJiGgu-xyOL2uzg@mail.gmail.com>
 <CAAXuHod4NH74+VK1EY78-vCOyiOA3879qp1uPqKOFit7qVE5sQ@mail.gmail.com>
Message-ID: <55620EEB.9000601@bluewin.ch>



On 05/23/2015 04:15 PM, savitha devi wrote:
> What I exactly want is the java script is in the html code. I am trying for
> a regular expression to find the email address embedded with in the java
> script.
>
> On Sat, May 23, 2015 at 2:31 PM, Chris Angelico <rosuav at gmail.com> wrote:
>
>> On Sat, May 23, 2015 at 4:46 PM, savitha devi <savithad8 at gmail.com> wrote:
>>> I am developing a web scraper code using HTMLParser. I need to extract
>>> text/email address from java script with in the HTMLCode.I am beginner
>> level
>>> in python coding and totally lost here. Need some help on this. The java
>>> script code is as below:
>>>
>>> <script type='text/javascript'>
>>>   //<!--
>>>   document.getElementById('cloak48218').innerHTML = '';
>>>   var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
>>>   var path = 'hr' + 'ef' + '=';
>>>   var addy48218 = '&#105;nf&#111;' + '&#64;';
>>>   addy48218 = addy48218 + 'tsv-n&#101;&#117;r&#105;&#101;d' + '&#46;' +
>>> 'd&#101;';
>>>   document.getElementById('cloak48218').innerHTML += '<a ' + path + '\'' +
>>> prefix + ':' + addy48218 + '\'>' + addy48218+'<\/a>';
>>>   //-->
>> This is deliberately being done to prevent scripted usage. What
>> exactly are you needing to do this for?
>>
>> You're basically going to have to execute the entire block of
>> JavaScript code, and then decode the entities to get to what you want.
>> Doing it manually is pretty easy; doing it automatically will
>> virtually require a language interpreter.
>>
>> ChrisA
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>

This is just about nuts and bolts, not about the ethics of presumed 
intentions.

Hope it helps one way or other

Frederic


------------------------------------------------------------------------------- 


sample = '''//<!--
  document.getElementById('cloak48218').innerHTML = '';
  var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
  var path = 'hr' + 'ef' + '=';
  var addy48218 = '&#105;nf&#111;' + '&#64;';
  addy48218 = addy48218 + 'tsv-n&#101;&#117;r&#105;&#101;d' + '&#46;' +
'd&#101;';
  document.getElementById('cloak48218').innerHTML += '<a ' + path + '\'' +
prefix + ':' + addy48218 + ''>' + addy48218+'<\/a>';
  //-->'''

 >>> import SE  # Download from PyPi at https://pypi.python.org/pypi/SE

 >>> def make_se_translator ():

     # Make SE substitutions
     subs_list = []

     # Make &# code substitutions
     for i in range (256):
         subs_list.append ('&#%d;=%c' % (i, chr(i)))

     # Delete Java stuff
     subs_list.append (' "document.getElementById(\'cloak48218\').=" ')
     subs_list.append (' "var =" "\n=" //<!--= //-->= ')

     # Java syntax? Tweaks needed to get the sample working
     subs_list.append (' "+ \'\'\'=" \'\'>\'=\'>\' <\/=</ ')

     # Add more as needed trial and error style
     # subs_list.append ( . . . format: ' old=new "delete this=" '

     # Make text
     subs = '\n'.join (subs_list)

     # Make SE translator
     translator = SE.SE (subs)

     # return translator, subs   # print subs, if you want to see what 
they look like
     return translator


 >>> translator = make_se_translator ()

 >>> translation = translator (sample)

 >>> print translation   # See
  innerHTML = ''; prefix = 'ma' + 'il' + 'to'; path = 'hr' + 'ef' + '='; 
addy48218 = 'info' + '@'; addy48218 = addy48218 + 'tsv-neuried' + '.' 
+'de'; innerHTML += '<a ' + path  +prefix + ':' + addy48218 + '>' + 
addy48218+'</a>';

 >>> exec (translation.lstrip ())

 >>> print innerHTML
<a href=mailto:info at tsv-neuried.de>info at tsv-neuried.de</a>



From dfnsonfsduifb at gmx.de  Sun May 24 14:29:31 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sun, 24 May 2015 20:29:31 +0200
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <87oalbkyvj.fsf@elektro.pacujo.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>	<mailman.222.1432309028.17265.python-list@python.org>	<555FA253.3020304@tundraware.com>
 <mjomke$saf$1@ger.gmane.org>	<mailman.252.1432352224.17265.python-list@python.org>	<mjp88e$e1$1@news.albasani.net>
 <556062B5.7050800@tundraware.com>	<mjpv34$g1a$1@news.albasani.net>
 <87oalbkyvj.fsf@elektro.pacujo.net>
Message-ID: <mjt5ab$gl1$1@news.albasani.net>

On 23.05.2015 19:05, Marko Rauhamaa wrote:
> Johannes Bauer <dfnsonfsduifb at gmx.de>:
> 
>> I think the major flaw of the X.509 certificate PKI we have today is
>> that there's no namespacing whatsoever. This is a major problem, as
>> the Government of Untrustworthia may give out certifictes for
>> google.de if they wish to do so.
> 
> But you're fine with the Government of Germany, I take it? Or any
> accredited German CA?

Of course not. But namespacing *enables* separation of trusted entities
where we currently have none whatsoever.

>> Sounds like it's trivial to implement, I wonder why it's not in place.
>> It must have some huge drawback that I can't think of right now.
> 
> How would your scheme address .com, .net, .org etc?

I don't see any problem, why do you see one?

The thing was that I was just giving an example of how nesting could
work. If those are domain names or nested OIDs or any other form of
unique identifier does not matter at all. de, org, fudis, it's all the same.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From dfnsonfsduifb at gmx.de  Sun May 24 14:29:31 2015
From: dfnsonfsduifb at gmx.de (Johannes Bauer)
Date: Sun, 24 May 2015 20:29:31 +0200
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <87oalbkyvj.fsf@elektro.pacujo.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>	<mailman.222.1432309028.17265.python-list@python.org>	<555FA253.3020304@tundraware.com>
 <mjomke$saf$1@ger.gmane.org>	<mailman.252.1432352224.17265.python-list@python.org>	<mjp88e$e1$1@news.albasani.net>
 <556062B5.7050800@tundraware.com>	<mjpv34$g1a$1@news.albasani.net>
 <87oalbkyvj.fsf@elektro.pacujo.net>
Message-ID: <mjt5ab$gl1$1@news.albasani.net>

On 23.05.2015 19:05, Marko Rauhamaa wrote:
> Johannes Bauer <dfnsonfsduifb at gmx.de>:
> 
>> I think the major flaw of the X.509 certificate PKI we have today is
>> that there's no namespacing whatsoever. This is a major problem, as
>> the Government of Untrustworthia may give out certifictes for
>> google.de if they wish to do so.
> 
> But you're fine with the Government of Germany, I take it? Or any
> accredited German CA?

Of course not. But namespacing *enables* separation of trusted entities
where we currently have none whatsoever.

>> Sounds like it's trivial to implement, I wonder why it's not in place.
>> It must have some huge drawback that I can't think of right now.
> 
> How would your scheme address .com, .net, .org etc?

I don't see any problem, why do you see one?

The thing was that I was just giving an example of how nesting could
work. If those are domain names or nested OIDs or any other form of
unique identifier does not matter at all. de, org, fudis, it's all the same.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht ?ffentlich!
Ah, der neueste und bis heute genialste Streich unsere gro?en
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos ?ber R?diger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>


From larry at hastings.org  Sun May 24 19:39:00 2015
From: larry at hastings.org (Larry Hastings)
Date: Sun, 24 May 2015 16:39:00 -0700
Subject: [RELEASED] Python 3.5.0b1 is now available
Message-ID: <55626114.6070003@hastings.org>



On behalf of the Python development community and the Python 3.5 release 
team, I'm pleased to announce the availability of Python 3.5.0b1.  
Python 3.5 has now entered "feature freeze". By default new features may 
no longer be added to Python 3.5. (However, there are a handful of 
features that weren't quite ready for Python 3.5.0 beta 1; these were 
granted exceptions to the freeze, and are scheduled to be added before 
beta 2.)

This is a preview release, and its use is not recommended for production 
settings.

Three important notes for Windows users about Python 3.5.0b1:

  * If you have previously installed Python 3.5.0a1, you may need to
    manually uninstall it before installing Python 3.5.0b1 (issue23612).
  * If installing Python 3.5.0b1 as a non-privileged user, you may need
    to escalate to administrator privileges to install an update to your
    C runtime libraries.
  * There is now a third type of Windows build for Python 3.5.  In
    addition to the conventional installer and the web-based installer,
    Python 3.5 now has an embeddable release designed to be deployed as
    part of a larger application's installer for apps using or extending
    Python.  During the 3.5 alpha releases, this was an executable
    installer; as of 3.5.0 beta 1 the embeddable build of Python is now
    shipped in a zip file.


You can find Python 3.5.0b1 here:

    https://www.python.org/downloads/release/python-350b1/

Happy hacking,


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150524/f9a13047/attachment.html>

From larry at hastings.org  Sun May 24 19:39:00 2015
From: larry at hastings.org (Larry Hastings)
Date: Sun, 24 May 2015 16:39:00 -0700
Subject: [RELEASED] Python 3.5.0b1 is now available
Message-ID: <55626114.6070003@hastings.org>



On behalf of the Python development community and the Python 3.5 release 
team, I'm pleased to announce the availability of Python 3.5.0b1.  
Python 3.5 has now entered "feature freeze". By default new features may 
no longer be added to Python 3.5. (However, there are a handful of 
features that weren't quite ready for Python 3.5.0 beta 1; these were 
granted exceptions to the freeze, and are scheduled to be added before 
beta 2.)

This is a preview release, and its use is not recommended for production 
settings.

Three important notes for Windows users about Python 3.5.0b1:

  * If you have previously installed Python 3.5.0a1, you may need to
    manually uninstall it before installing Python 3.5.0b1 (issue23612).
  * If installing Python 3.5.0b1 as a non-privileged user, you may need
    to escalate to administrator privileges to install an update to your
    C runtime libraries.
  * There is now a third type of Windows build for Python 3.5.  In
    addition to the conventional installer and the web-based installer,
    Python 3.5 now has an embeddable release designed to be deployed as
    part of a larger application's installer for apps using or extending
    Python.  During the 3.5 alpha releases, this was an executable
    installer; as of 3.5.0 beta 1 the embeddable build of Python is now
    shipped in a zip file.


You can find Python 3.5.0b1 here:

    https://www.python.org/downloads/release/python-350b1/

Happy hacking,


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150524/f9a13047/attachment-0001.html>

From marko at pacujo.net  Mon May 25 02:57:28 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 25 May 2015 09:57:28 +0300
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
 <5561a74b$0$12977$c3e8da3$5496439d@news.astraweb.com>
 <87fv6mj8sr.fsf@elektro.pacujo.net>
 <mailman.30.1432533745.5151.python-list@python.org>
Message-ID: <87mw0tb0vb.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> You've added extra levels of indirection, but it comes to the same
> thing. You're requiring that everyone who wants to conduct business on
> the internet (taking credit card numbers etc) has to go through four
> separate authentication processes, and a failure in any one of them
> means the site is not considered legit.

Yes. What you get is some much higher-level security than at the moment.

You could do with passport-level security with a single national
government but then you would be prone to attacks (or blunders) from
that government.

> Right. And "legitimate" is defined as "not refused by any of the four
> countries". All they have to do is decide that something's not
> legitimate, and bam, they're off air. ANY ONE of your four has this
> power of veto.

Certificates can be revoked, kinda, yes. Or more to the point,
roadblocks could be put in the way of certifying some applicants.
However, if that started happening, the OS and browser makers would
simply drop the obnoxious government from the prestigious club. It
really wouldn't be in their interests to start acting up.


Marko


From marko at pacujo.net  Mon May 25 02:57:28 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 25 May 2015 09:57:28 +0300
Subject: Ah Python, you have spoiled me for all other languages
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
 <5561a74b$0$12977$c3e8da3$5496439d@news.astraweb.com>
 <87fv6mj8sr.fsf@elektro.pacujo.net>
 <mailman.30.1432533745.5151.python-list@python.org>
Message-ID: <87mw0tb0vb.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> You've added extra levels of indirection, but it comes to the same
> thing. You're requiring that everyone who wants to conduct business on
> the internet (taking credit card numbers etc) has to go through four
> separate authentication processes, and a failure in any one of them
> means the site is not considered legit.

Yes. What you get is some much higher-level security than at the moment.

You could do with passport-level security with a single national
government but then you would be prone to attacks (or blunders) from
that government.

> Right. And "legitimate" is defined as "not refused by any of the four
> countries". All they have to do is decide that something's not
> legitimate, and bam, they're off air. ANY ONE of your four has this
> power of veto.

Certificates can be revoked, kinda, yes. Or more to the point,
roadblocks could be put in the way of certifying some applicants.
However, if that started happening, the OS and browser makers would
simply drop the obnoxious government from the prestigious club. It
really wouldn't be in their interests to start acting up.


Marko


From wxjmfauth at gmail.com  Mon May 25 05:20:10 2015
From: wxjmfauth at gmail.com (wxjmfauth at gmail.com)
Date: Mon, 25 May 2015 02:20:10 -0700 (PDT)
Subject: [RELEASED] Python 3.5.0b1 is now available
In-Reply-To: <mailman.27.1432510751.5151.python-list@python.org>
References: <mailman.27.1432510751.5151.python-list@python.org>
Message-ID: <993f1217-1b59-4de3-a129-30d27670db91@googlegroups.com>

Le lundi 25 mai 2015 01:39:22 UTC+2, Larry Hastings a ?crit?:
> On behalf of the Python development community and the Python
>           3.5 release team, I'm pleased to announce the availability of
>           Python 3.5.0b1.? Python 3.5 has now entered "feature freeze".?
>           By default new features may no longer be added to Python 3.5.?
>           (However, there are a handful of features that weren't quite
>           ready for Python 3.5.0 beta 1; these were granted exceptions
>           to the freeze, and are scheduled to be added before beta 2.)
> 
>           
> 
>           This is a preview release, and its use is not recommended for
>           production settings.
> 
>           
> 
>           Three important notes for Windows users about Python 3.5.0b1:
> 
>           
>             If you have previously installed Python 3.5.0a1, you may
>               need to manually uninstall it before installing Python
>               3.5.0b1 (issue23612).
>             If installing Python 3.5.0b1 as a non-privileged user,
>               you may need to escalate to administrator privileges to
>               install an update to your C runtime libraries.
>             There is now a third type of Windows build for Python
>               3.5.? In addition to the conventional installer and the
>               web-based installer, Python 3.5 now has an embeddable
>               release designed to be deployed as part of a larger
>               application's installer for apps using or extending
>               Python.? During the 3.5 alpha releases, this was an
>               executable installer; as of 3.5.0 beta 1 the embeddable
>               build of Python is now shipped in a zip file.
> 
>             
>           
>           
> 
>           You can find Python 3.5.0b1 here:
> 
>           https://www.python.org/downloads/release/python-350b1/
> 
>           
>           Happy hacking,
> 
>           

Windows 7 pro

Installation of "python-3.5.0b1.exe" as administrator: no luck.

Explaining what's going on behind the scene or explaining why the installer
is simply silently failing and does no do its job is beyond my knowledge
and my understanding.



From wxjmfauth at gmail.com  Mon May 25 05:20:10 2015
From: wxjmfauth at gmail.com (wxjmfauth at gmail.com)
Date: Mon, 25 May 2015 02:20:10 -0700 (PDT)
Subject: [RELEASED] Python 3.5.0b1 is now available
In-Reply-To: <mailman.27.1432510751.5151.python-list@python.org>
References: <mailman.27.1432510751.5151.python-list@python.org>
Message-ID: <993f1217-1b59-4de3-a129-30d27670db91@googlegroups.com>

Le lundi 25 mai 2015 01:39:22 UTC+2, Larry Hastings a ?crit?:
> On behalf of the Python development community and the Python
>           3.5 release team, I'm pleased to announce the availability of
>           Python 3.5.0b1.? Python 3.5 has now entered "feature freeze".?
>           By default new features may no longer be added to Python 3.5.?
>           (However, there are a handful of features that weren't quite
>           ready for Python 3.5.0 beta 1; these were granted exceptions
>           to the freeze, and are scheduled to be added before beta 2.)
> 
>           
> 
>           This is a preview release, and its use is not recommended for
>           production settings.
> 
>           
> 
>           Three important notes for Windows users about Python 3.5.0b1:
> 
>           
>             If you have previously installed Python 3.5.0a1, you may
>               need to manually uninstall it before installing Python
>               3.5.0b1 (issue23612).
>             If installing Python 3.5.0b1 as a non-privileged user,
>               you may need to escalate to administrator privileges to
>               install an update to your C runtime libraries.
>             There is now a third type of Windows build for Python
>               3.5.? In addition to the conventional installer and the
>               web-based installer, Python 3.5 now has an embeddable
>               release designed to be deployed as part of a larger
>               application's installer for apps using or extending
>               Python.? During the 3.5 alpha releases, this was an
>               executable installer; as of 3.5.0 beta 1 the embeddable
>               build of Python is now shipped in a zip file.
> 
>             
>           
>           
> 
>           You can find Python 3.5.0b1 here:
> 
>           https://www.python.org/downloads/release/python-350b1/
> 
>           
>           Happy hacking,
> 
>           

Windows 7 pro

Installation of "python-3.5.0b1.exe" as administrator: no luck.

Explaining what's going on behind the scene or explaining why the installer
is simply silently failing and does no do its job is beyond my knowledge
and my understanding.



From donal.duane at ericsson.com  Mon May 25 05:33:06 2015
From: donal.duane at ericsson.com (Donal Duane)
Date: Mon, 25 May 2015 09:33:06 +0000
Subject: Query on Python  3.2 and supported OpenSSL Versions
Message-ID: <867F5C4940403B47B2474432369BAF790FFBD992@ESESSMB309.ericsson.se>


Hi Python Users,

I was hoping you might be able to assist me with a query:

2 Questions:


1.       Could Python 3.2, when compiled against OpenSSL 1.0.0j, be affected by the poodle bug? https://www.openssl.org/~bodo/ssl-poodle.pdf



2.       If yes - are the following OpenSSL versions  approved for use with Python 3.2:

OpenSSL 0.9.8zf
OpenSSL 1.0.0r
OpenSSL 1.0.1m
OpenSSL 1.0.2a

Regards,
D?nal


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150525/582b8aa5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 70 bytes
Desc: image001.gif
URL: <http://mail.python.org/pipermail/python-list/attachments/20150525/582b8aa5/attachment.gif>

From donal.duane at ericsson.com  Mon May 25 05:33:06 2015
From: donal.duane at ericsson.com (Donal Duane)
Date: Mon, 25 May 2015 09:33:06 +0000
Subject: Query on Python  3.2 and supported OpenSSL Versions
Message-ID: <867F5C4940403B47B2474432369BAF790FFBD992@ESESSMB309.ericsson.se>


Hi Python Users,

I was hoping you might be able to assist me with a query:

2 Questions:


1.       Could Python 3.2, when compiled against OpenSSL 1.0.0j, be affected by the poodle bug? https://www.openssl.org/~bodo/ssl-poodle.pdf



2.       If yes - are the following OpenSSL versions  approved for use with Python 3.2:

OpenSSL 0.9.8zf
OpenSSL 1.0.0r
OpenSSL 1.0.1m
OpenSSL 1.0.2a

Regards,
D?nal


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150525/582b8aa5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 70 bytes
Desc: image001.gif
URL: <http://mail.python.org/pipermail/python-list/attachments/20150525/582b8aa5/attachment-0001.gif>

From lac at openend.se  Mon May 25 05:39:24 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 25 May 2015 11:39:24 +0200
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: Message from Marko Rauhamaa <marko@pacujo.net>
 of "Mon, 25 May 2015 09:57:28 +0300." <87mw0tb0vb.fsf@elektro.pacujo.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
 <5561a74b$0$12977$c3e8da3$5496439d@news.astraweb.com>
 <87fv6mj8sr.fsf@elektro.pacujo.net>
 <mailman.30.1432533745.5151.python-list@python.org><87mw0tb0vb.fsf@elektro.pacujo.net>
Message-ID: <201505250939.t4P9dO50031555@fido.openend.se>

In a message of Mon, 25 May 2015 09:57:28 +0300, Marko Rauhamaa writes:
>Certificates can be revoked, kinda, yes. Or more to the point,
>roadblocks could be put in the way of certifying some applicants.
>However, if that started happening, the OS and browser makers would
>simply drop the obnoxious government from the prestigious club. It
>really wouldn't be in their interests to start acting up.
'
Unfortunately, you cannot count on governments giving two hoots about
what you and I perceive as their interests.  Josef Stalin would just
go off and kill OS and browser makers in his country until he got
things the way he wanted it.  A US government more subservient to big
business could declare, for reasons of patriotism, national security,
and the ever popular 'to stop child pornography' that we all have to
use Windows software and run Microsoft browsers.

These are really bad ideas, even from the point of view of the
governments involved -- it would not accomplish anything like it was
supposed to accomplish (unless spreading fear and terror is the real
goal, in which case it might work great for that).  But having really
bad ideas and being willing to act on them is one of the
characteristics of the sort of obnoxious government we would like to
drop from the club, so no joy there.

What people need to understand is that unless you want to stamp out
freedom altogether, there will be crime.  You cannot have one without
the other, because all crime is, is 'somebody doing something that
some other people have said they are not allowed to do'.  You can have
quite a bit of crime without destroying the basic decent kindness that
most people have for each other.  Everybody says 'tisk tisk' and goes
on helping each other and being decent to each other.  But systematic
oppression, especially state-advocated opression done by the state
cuts to the heart of things and kills such goodwill and spontaneous
kindness dead.

Laura


From lac at openend.se  Mon May 25 05:39:24 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 25 May 2015 11:39:24 +0200
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: Message from Marko Rauhamaa <marko@pacujo.net>
 of "Mon, 25 May 2015 09:57:28 +0300." <87mw0tb0vb.fsf@elektro.pacujo.net>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
 <5561a74b$0$12977$c3e8da3$5496439d@news.astraweb.com>
 <87fv6mj8sr.fsf@elektro.pacujo.net>
 <mailman.30.1432533745.5151.python-list@python.org><87mw0tb0vb.fsf@elektro.pacujo.net>
Message-ID: <201505250939.t4P9dO50031555@fido.openend.se>

In a message of Mon, 25 May 2015 09:57:28 +0300, Marko Rauhamaa writes:
>Certificates can be revoked, kinda, yes. Or more to the point,
>roadblocks could be put in the way of certifying some applicants.
>However, if that started happening, the OS and browser makers would
>simply drop the obnoxious government from the prestigious club. It
>really wouldn't be in their interests to start acting up.
'
Unfortunately, you cannot count on governments giving two hoots about
what you and I perceive as their interests.  Josef Stalin would just
go off and kill OS and browser makers in his country until he got
things the way he wanted it.  A US government more subservient to big
business could declare, for reasons of patriotism, national security,
and the ever popular 'to stop child pornography' that we all have to
use Windows software and run Microsoft browsers.

These are really bad ideas, even from the point of view of the
governments involved -- it would not accomplish anything like it was
supposed to accomplish (unless spreading fear and terror is the real
goal, in which case it might work great for that).  But having really
bad ideas and being willing to act on them is one of the
characteristics of the sort of obnoxious government we would like to
drop from the club, so no joy there.

What people need to understand is that unless you want to stamp out
freedom altogether, there will be crime.  You cannot have one without
the other, because all crime is, is 'somebody doing something that
some other people have said they are not allowed to do'.  You can have
quite a bit of crime without destroying the basic decent kindness that
most people have for each other.  Everybody says 'tisk tisk' and goes
on helping each other and being decent to each other.  But systematic
oppression, especially state-advocated opression done by the state
cuts to the heart of things and kills such goodwill and spontaneous
kindness dead.

Laura


From lac at openend.se  Mon May 25 06:31:15 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 25 May 2015 12:31:15 +0200
Subject: Query on Python  3.2 and supported OpenSSL Versions (fwd)
Message-ID: <201505251031.t4PAVFIs032755@fido.openend.se>

This showed up on Python list.

------- Forwarded Message

Return-Path: <python-list-bounces+lac=openend.se at python.org>
Received: from mail.python.org (mail.python.org [82.94.164.166])
From: Donal Duane <donal.duane at ericsson.com>
To: "python-list at python.org" <python-list at python.org>
Subject: Query on Python  3.2 and supported OpenSSL Versions
Cc: Alex Ying <alex.ying at ericsson.com>,
        Anthony McMahon <anthony.mcmahon at ericsson.com>


Hi Python Users,

I was hoping you might be able to assist me with a query:

2 Questions:


1.       Could Python 3.2, when compiled against OpenSSL 1.0.0j, be affecte=
d by the poodle bug? https://www.openssl.org/~bodo/ssl-poodle.pdf



2.       If yes - are the following OpenSSL versions  approved for use with=
 Python 3.2:

OpenSSL 0.9.8zf
OpenSSL 1.0.0r
OpenSSL 1.0.1m
OpenSSL 1.0.2a

Regards,
D?nal

------- End of Forwarded Message


From lac at openend.se  Mon May 25 06:31:15 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 25 May 2015 12:31:15 +0200
Subject: Query on Python  3.2 and supported OpenSSL Versions (fwd)
Message-ID: <201505251031.t4PAVFIs032755@fido.openend.se>

This showed up on Python list.

------- Forwarded Message

Return-Path: <python-list-bounces+lac=openend.se at python.org>
Received: from mail.python.org (mail.python.org [82.94.164.166])
From: Donal Duane <donal.duane at ericsson.com>
To: "python-list at python.org" <python-list at python.org>
Subject: Query on Python  3.2 and supported OpenSSL Versions
Cc: Alex Ying <alex.ying at ericsson.com>,
        Anthony McMahon <anthony.mcmahon at ericsson.com>


Hi Python Users,

I was hoping you might be able to assist me with a query:

2 Questions:


1.       Could Python 3.2, when compiled against OpenSSL 1.0.0j, be affecte=
d by the poodle bug? https://www.openssl.org/~bodo/ssl-poodle.pdf



2.       If yes - are the following OpenSSL versions  approved for use with=
 Python 3.2:

OpenSSL 0.9.8zf
OpenSSL 1.0.0r
OpenSSL 1.0.1m
OpenSSL 1.0.2a

Regards,
D?nal

------- End of Forwarded Message


From simon+python at bleah.co.uk  Mon May 25 07:07:01 2015
From: simon+python at bleah.co.uk (Simon Ward)
Date: Mon, 25 May 2015 12:07:01 +0100
Subject: Query on Python  3.2 and supported OpenSSL Versions
In-Reply-To: <867F5C4940403B47B2474432369BAF790FFBD992@ESESSMB309.ericsson.se>
References: <867F5C4940403B47B2474432369BAF790FFBD992@ESESSMB309.ericsson.se>
Message-ID: <20150525110701.GA30841@squawkencluck.cosgrove.lan>

On Mon, May 25, 2015 at 09:33:06AM +0000, Donal Duane wrote:
> 
> Hi Python Users,
> 
> I was hoping you might be able to assist me with a query:
> 
> 2 Questions:
> 
> 
> 1.       Could Python 3.2, when compiled against OpenSSL 1.0.0j, be
> affected by the poodle bug?
> https://www.openssl.org/~bodo/ssl-poodle.pdf

Yes, POODLE is two things:

 1. Due to client?side fallback mechanisms outside of SSL and TLS, it
    is possible to force a client to connect with a different version of
    the protocol, i.e. a protocol downgrade.

 2. Due to the way padding is implemented in SSL 3.0 there is a padding
    oracle attack on CBC mode ciphers. This affects all implementations
    of SSL 3.0, and some of TLS 1.0 (but not OpenSSL).

All versions of OpenSSL (that support SSL 3.0) are affected by (2)
because it is inherent in the protocol. The fix is not to allow SSL, and
enable the latest versions of TLS.

Later versions of OpenSSL (0.9.8zc, 1.0.0o, 1.0.1j) added a workaround,
the TLS_FALLBACK_SCSV mechanism, which allows clients to specify when
they have falled back to an earlier version, preventing downgrade
attacks (1). It only works if the both the client and server support the
mechanism, and if SSL 3.0 is used it is still affected by the padding
oracle attack (2).

> 2.       If yes - are the following OpenSSL versions  approved for use
> with Python 3.2:

I can?t answer for approved but I have builds of Python than use OpenSSL
0.9.8 and 1.0.1 and it should build against any version in these
branches. I also have pyOpenSSL built against these branches.

Simon
-- 
A complex system that works is invariably found to have evolved from a
simple system that works.?John Gall


From simon+python at bleah.co.uk  Mon May 25 07:07:01 2015
From: simon+python at bleah.co.uk (Simon Ward)
Date: Mon, 25 May 2015 12:07:01 +0100
Subject: Query on Python  3.2 and supported OpenSSL Versions
In-Reply-To: <867F5C4940403B47B2474432369BAF790FFBD992@ESESSMB309.ericsson.se>
References: <867F5C4940403B47B2474432369BAF790FFBD992@ESESSMB309.ericsson.se>
Message-ID: <20150525110701.GA30841@squawkencluck.cosgrove.lan>

On Mon, May 25, 2015 at 09:33:06AM +0000, Donal Duane wrote:
> 
> Hi Python Users,
> 
> I was hoping you might be able to assist me with a query:
> 
> 2 Questions:
> 
> 
> 1.       Could Python 3.2, when compiled against OpenSSL 1.0.0j, be
> affected by the poodle bug?
> https://www.openssl.org/~bodo/ssl-poodle.pdf

Yes, POODLE is two things:

 1. Due to client?side fallback mechanisms outside of SSL and TLS, it
    is possible to force a client to connect with a different version of
    the protocol, i.e. a protocol downgrade.

 2. Due to the way padding is implemented in SSL 3.0 there is a padding
    oracle attack on CBC mode ciphers. This affects all implementations
    of SSL 3.0, and some of TLS 1.0 (but not OpenSSL).

All versions of OpenSSL (that support SSL 3.0) are affected by (2)
because it is inherent in the protocol. The fix is not to allow SSL, and
enable the latest versions of TLS.

Later versions of OpenSSL (0.9.8zc, 1.0.0o, 1.0.1j) added a workaround,
the TLS_FALLBACK_SCSV mechanism, which allows clients to specify when
they have falled back to an earlier version, preventing downgrade
attacks (1). It only works if the both the client and server support the
mechanism, and if SSL 3.0 is used it is still affected by the padding
oracle attack (2).

> 2.       If yes - are the following OpenSSL versions  approved for use
> with Python 3.2:

I can?t answer for approved but I have builds of Python than use OpenSSL
0.9.8 and 1.0.1 and it should build against any version in these
branches. I also have pyOpenSSL built against these branches.

Simon
-- 
A complex system that works is invariably found to have evolved from a
simple system that works.?John Gall


From rosuav at gmail.com  Mon May 25 07:09:03 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 25 May 2015 21:09:03 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <201505250939.t4P9dO50031555@fido.openend.se>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
 <5561a74b$0$12977$c3e8da3$5496439d@news.astraweb.com>
 <87fv6mj8sr.fsf@elektro.pacujo.net>
 <mailman.30.1432533745.5151.python-list@python.org>
 <marko@pacujo.net> <87mw0tb0vb.fsf@elektro.pacujo.net>
 <201505250939.t4P9dO50031555@fido.openend.se>
Message-ID: <CAPTjJmrng+maivUi2GFPdWOWtOgq5J-eyWPmcsnTpabwdEumGg@mail.gmail.com>

On Mon, May 25, 2015 at 7:39 PM, Laura Creighton <lac at openend.se> wrote:
> What people need to understand is that unless you want to stamp out
> freedom altogether, there will be crime.

Or stamp out legislation altogether and have complete anarchy. There's
no such thing as crime among animals, because there's no law beyond
"survive".

The solution isn't to try to eliminate crime, but to cope with it.
Same with our own errors: accept and acknowledge that you WILL make
mistakes, and cope with that. In a spiritual sense, that might define
your religion; in a programming sense, that's exactly why we have
source control, so we can find out what happened (and why) and fix
problems once we find them. Would you use a program that got launched
as version 1 and never changed? Would you trust it on the basis that
it clearly has no bugs, because nobody's ever needed to fix any? I
certainly wouldn't.

Some things work well centralized, because differences are worse than
slight benefits one way or another. In any given country, we usually
all drive on the same side of the road. But a lot of things work
better *de*centralized, so that if one person makes a mistake, other
people can do things differently, and hindsight evaluation lets us
choose which one to encourage. PyPI is decentralized; the Python
standard library is centralized. The guardians of the latter are
rightly slow to choose from multiple alternatives, preferring to let
the decentralized collective mind of the former figure out which is
the clear best - if there even is one.

The best form of security is probably the GPG web of trust, being
fundamentally decentralized and based on personal reputation. Imagine
if, once you register a domain, you go talk to someone about getting a
GPG key signed for it - or, better still, sign the server's key
yourself, if you have a decent WoT for your own key (which I don't).
It wouldn't be hard to use self-signed SSL certificates, sign those
certs with a GPG key, and then let people download and install certs
for anyone they consider trustworthy. In fact, this seems so obvious
that I'm sure it's already been done. Trouble is, GPG isn't nearly
well enough known for mass use... but it is going to be a lot more
reliable than anything that depends on four countries' governments [1]
agreeing.

ChrisA

[1] Yes, technically the United States of Europe is not a country. But
just how structurally different is it from the United States of
America?


From rosuav at gmail.com  Mon May 25 07:09:03 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 25 May 2015 21:09:03 +1000
Subject: Ah Python, you have spoiled me for all other languages
In-Reply-To: <201505250939.t4P9dO50031555@fido.openend.se>
References: <555f440a$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <mailman.222.1432309028.17265.python-list@python.org>
 <555FA253.3020304@tundraware.com> <mjomke$saf$1@ger.gmane.org>
 <mailman.252.1432352224.17265.python-list@python.org>
 <mjp88e$e1$1@news.albasani.net> <87382nmpkd.fsf@elektro.pacujo.net>
 <556087ee$0$13004$c3e8da3$5496439d@news.astraweb.com>
 <87siankzfk.fsf@elektro.pacujo.net>
 <5561a74b$0$12977$c3e8da3$5496439d@news.astraweb.com>
 <87fv6mj8sr.fsf@elektro.pacujo.net>
 <mailman.30.1432533745.5151.python-list@python.org>
 <marko@pacujo.net> <87mw0tb0vb.fsf@elektro.pacujo.net>
 <201505250939.t4P9dO50031555@fido.openend.se>
Message-ID: <CAPTjJmrng+maivUi2GFPdWOWtOgq5J-eyWPmcsnTpabwdEumGg@mail.gmail.com>

On Mon, May 25, 2015 at 7:39 PM, Laura Creighton <lac at openend.se> wrote:
> What people need to understand is that unless you want to stamp out
> freedom altogether, there will be crime.

Or stamp out legislation altogether and have complete anarchy. There's
no such thing as crime among animals, because there's no law beyond
"survive".

The solution isn't to try to eliminate crime, but to cope with it.
Same with our own errors: accept and acknowledge that you WILL make
mistakes, and cope with that. In a spiritual sense, that might define
your religion; in a programming sense, that's exactly why we have
source control, so we can find out what happened (and why) and fix
problems once we find them. Would you use a program that got launched
as version 1 and never changed? Would you trust it on the basis that
it clearly has no bugs, because nobody's ever needed to fix any? I
certainly wouldn't.

Some things work well centralized, because differences are worse than
slight benefits one way or another. In any given country, we usually
all drive on the same side of the road. But a lot of things work
better *de*centralized, so that if one person makes a mistake, other
people can do things differently, and hindsight evaluation lets us
choose which one to encourage. PyPI is decentralized; the Python
standard library is centralized. The guardians of the latter are
rightly slow to choose from multiple alternatives, preferring to let
the decentralized collective mind of the former figure out which is
the clear best - if there even is one.

The best form of security is probably the GPG web of trust, being
fundamentally decentralized and based on personal reputation. Imagine
if, once you register a domain, you go talk to someone about getting a
GPG key signed for it - or, better still, sign the server's key
yourself, if you have a decent WoT for your own key (which I don't).
It wouldn't be hard to use self-signed SSL certificates, sign those
certs with a GPG key, and then let people download and install certs
for anyone they consider trustworthy. In fact, this seems so obvious
that I'm sure it's already been done. Trouble is, GPG isn't nearly
well enough known for mass use... but it is going to be a lot more
reliable than anything that depends on four countries' governments [1]
agreeing.

ChrisA

[1] Yes, technically the United States of Europe is not a country. But
just how structurally different is it from the United States of
America?


From rosuav at gmail.com  Mon May 25 07:16:04 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 25 May 2015 21:16:04 +1000
Subject: Query on Python 3.2 and supported OpenSSL Versions
In-Reply-To: <867F5C4940403B47B2474432369BAF790FFBD992@ESESSMB309.ericsson.se>
References: <867F5C4940403B47B2474432369BAF790FFBD992@ESESSMB309.ericsson.se>
Message-ID: <CAPTjJmp=uyzdP2ahX4dSmq68k3VHT_e3dv_hurvzpXyThAHxZA@mail.gmail.com>

On Mon, May 25, 2015 at 7:33 PM, Donal Duane <donal.duane at ericsson.com> wrote:
>
> Hi Python Users,
>
> I was hoping you might be able to assist me with a query:
>
> 2 Questions:
>
> 1.       Could Python 3.2, when compiled against OpenSSL 1.0.0j, be affected by the poodle bug? https://www.openssl.org/~bodo/ssl-poodle.pdf
>

Just out of interest, is there a reason you can't use a newer Python
version? Since you talk about compiling Python, I assume you mean
CPython; and given that you're considering using a different OpenSSL,
you're quite probably already planning to compile from source. If
that's true, I would recommend building the latest - either a source
tarball of CPython 3.4, or the very latest beta of 3.5; personally,
I'm quite happy with the beta, but you might prefer to use a properly
released and numbered version.

If you can't recompile [C]Python, can you explain more about what
you're planning to do to change OpenSSL version, and why it needs to
be 3.2? That might help with the second part of your question.

ChrisA


From rosuav at gmail.com  Mon May 25 07:16:04 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 25 May 2015 21:16:04 +1000
Subject: Query on Python 3.2 and supported OpenSSL Versions
In-Reply-To: <867F5C4940403B47B2474432369BAF790FFBD992@ESESSMB309.ericsson.se>
References: <867F5C4940403B47B2474432369BAF790FFBD992@ESESSMB309.ericsson.se>
Message-ID: <CAPTjJmp=uyzdP2ahX4dSmq68k3VHT_e3dv_hurvzpXyThAHxZA@mail.gmail.com>

On Mon, May 25, 2015 at 7:33 PM, Donal Duane <donal.duane at ericsson.com> wrote:
>
> Hi Python Users,
>
> I was hoping you might be able to assist me with a query:
>
> 2 Questions:
>
> 1.       Could Python 3.2, when compiled against OpenSSL 1.0.0j, be affected by the poodle bug? https://www.openssl.org/~bodo/ssl-poodle.pdf
>

Just out of interest, is there a reason you can't use a newer Python
version? Since you talk about compiling Python, I assume you mean
CPython; and given that you're considering using a different OpenSSL,
you're quite probably already planning to compile from source. If
that's true, I would recommend building the latest - either a source
tarball of CPython 3.4, or the very latest beta of 3.5; personally,
I'm quite happy with the beta, but you might prefer to use a properly
released and numbered version.

If you can't recompile [C]Python, can you explain more about what
you're planning to do to change OpenSSL version, and why it needs to
be 3.2? That might help with the second part of your question.

ChrisA


From garyr at fidalgo.net  Mon May 25 11:19:52 2015
From: garyr at fidalgo.net (garyr)
Date: Mon, 25 May 2015 08:19:52 -0700
Subject: cl.exe missing
Message-ID: <mjvejo$rmu$1@speranza.aioe.org>

I posted this on the Anaconda NG but haven't gotten an answer.

I recently installed Python 2.7 using Miniconda. I'm now trying to build a
Python extension module. My setup.py file is:

from distutils.core import setup, Extension
    module1 = Extension('pyssound',
    sources=['ssound.cpp', 'pyssound.cpp'])
    setup(name="pyssound",
         version=1.0,
         ext_modules = [module1])

Initially I got the vcvarsall.bat missing error and I found the fix for
that:
    set MSSDK=1
    set DISTUTILS_USE_SDK=1

Now when I run setup I get:

python setup.py build
running build
running build_ext
building 'pyssound' extension
cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG "-IC:\Documents and
Settings\Owner\M
iniconda\include" "-IC:\Documents and Settings\Owner\Miniconda\PC"
/Tpssound.cpp
 /Fobuild\temp.win32-2.7\Release\ssound.obj
error: command 'cl.exe' failed: No such file or directory

cl.exe is the Visual Studio VS2008 compiler which is the one used to compile
Python 2.7. A search for VS2008 turns up this site:
http://www.microsoft.com/en-us/download/details.aspx?id=10986 which mentions
vs2010, 2012 and 2013 but not 2008.

Where do I find VS2008?






From garyr at fidalgo.net  Mon May 25 11:19:52 2015
From: garyr at fidalgo.net (garyr)
Date: Mon, 25 May 2015 08:19:52 -0700
Subject: cl.exe missing
Message-ID: <mjvejo$rmu$1@speranza.aioe.org>

I posted this on the Anaconda NG but haven't gotten an answer.

I recently installed Python 2.7 using Miniconda. I'm now trying to build a
Python extension module. My setup.py file is:

from distutils.core import setup, Extension
    module1 = Extension('pyssound',
    sources=['ssound.cpp', 'pyssound.cpp'])
    setup(name="pyssound",
         version=1.0,
         ext_modules = [module1])

Initially I got the vcvarsall.bat missing error and I found the fix for
that:
    set MSSDK=1
    set DISTUTILS_USE_SDK=1

Now when I run setup I get:

python setup.py build
running build
running build_ext
building 'pyssound' extension
cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG "-IC:\Documents and
Settings\Owner\M
iniconda\include" "-IC:\Documents and Settings\Owner\Miniconda\PC"
/Tpssound.cpp
 /Fobuild\temp.win32-2.7\Release\ssound.obj
error: command 'cl.exe' failed: No such file or directory

cl.exe is the Visual Studio VS2008 compiler which is the one used to compile
Python 2.7. A search for VS2008 turns up this site:
http://www.microsoft.com/en-us/download/details.aspx?id=10986 which mentions
vs2010, 2012 and 2013 but not 2008.

Where do I find VS2008?






From mail at timgolden.me.uk  Mon May 25 11:51:03 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Mon, 25 May 2015 16:51:03 +0100
Subject: cl.exe missing
In-Reply-To: <mjvejo$rmu$1@speranza.aioe.org>
References: <mjvejo$rmu$1@speranza.aioe.org>
Message-ID: <556344E7.2090204@timgolden.me.uk>

On 25/05/2015 16:19, garyr wrote:
> I posted this on the Anaconda NG but haven't gotten an answer.
>
> I recently installed Python 2.7 using Miniconda. I'm now trying to build a
> Python extension module. My setup.py file is:
>
> from distutils.core import setup, Extension
>      module1 = Extension('pyssound',
>      sources=['ssound.cpp', 'pyssound.cpp'])
>      setup(name="pyssound",
>           version=1.0,
>           ext_modules = [module1])
>
> Initially I got the vcvarsall.bat missing error and I found the fix for
> that:
>      set MSSDK=1
>      set DISTUTILS_USE_SDK=1
>
> Now when I run setup I get:
>
> python setup.py build
> running build
> running build_ext
> building 'pyssound' extension
> cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG "-IC:\Documents and
> Settings\Owner\M
> iniconda\include" "-IC:\Documents and Settings\Owner\Miniconda\PC"
> /Tpssound.cpp
>   /Fobuild\temp.win32-2.7\Release\ssound.obj
> error: command 'cl.exe' failed: No such file or directory
>
> cl.exe is the Visual Studio VS2008 compiler which is the one used to compile
> Python 2.7. A search for VS2008 turns up this site:
> http://www.microsoft.com/en-us/download/details.aspx?id=10986 which mentions
> vs2010, 2012 and 2013 but not 2008.
>
> Where do I find VS2008?

Try this:

   https://www.microsoft.com/en-gb/download/details.aspx?id=44266

TJG


From mail at timgolden.me.uk  Mon May 25 11:51:03 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Mon, 25 May 2015 16:51:03 +0100
Subject: cl.exe missing
In-Reply-To: <mjvejo$rmu$1@speranza.aioe.org>
References: <mjvejo$rmu$1@speranza.aioe.org>
Message-ID: <556344E7.2090204@timgolden.me.uk>

On 25/05/2015 16:19, garyr wrote:
> I posted this on the Anaconda NG but haven't gotten an answer.
>
> I recently installed Python 2.7 using Miniconda. I'm now trying to build a
> Python extension module. My setup.py file is:
>
> from distutils.core import setup, Extension
>      module1 = Extension('pyssound',
>      sources=['ssound.cpp', 'pyssound.cpp'])
>      setup(name="pyssound",
>           version=1.0,
>           ext_modules = [module1])
>
> Initially I got the vcvarsall.bat missing error and I found the fix for
> that:
>      set MSSDK=1
>      set DISTUTILS_USE_SDK=1
>
> Now when I run setup I get:
>
> python setup.py build
> running build
> running build_ext
> building 'pyssound' extension
> cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG "-IC:\Documents and
> Settings\Owner\M
> iniconda\include" "-IC:\Documents and Settings\Owner\Miniconda\PC"
> /Tpssound.cpp
>   /Fobuild\temp.win32-2.7\Release\ssound.obj
> error: command 'cl.exe' failed: No such file or directory
>
> cl.exe is the Visual Studio VS2008 compiler which is the one used to compile
> Python 2.7. A search for VS2008 turns up this site:
> http://www.microsoft.com/en-us/download/details.aspx?id=10986 which mentions
> vs2010, 2012 and 2013 but not 2008.
>
> Where do I find VS2008?

Try this:

   https://www.microsoft.com/en-gb/download/details.aspx?id=44266

TJG


From denismfmcmahon at gmail.com  Mon May 25 14:39:52 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Mon, 25 May 2015 18:39:52 +0000 (UTC)
Subject: Extract email address from Java script in html source using python
References: <mailman.265.1432369272.17265.python-list@python.org>
Message-ID: <mjvq9o$9i3$11@dont-email.me>

On Sat, 23 May 2015 12:16:06 +0530, savitha devi wrote:

> I am developing a web scraper code using HTMLParser. I need to extract
> text/email address from java script with in the HTMLCode.I am beginner
> level in python coding and totally lost here. Need some help on this.

(a) Try a less ambitious learning project.
(b) Start reading the relevant documentation.

Pick one and go with it.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From denismfmcmahon at gmail.com  Mon May 25 14:39:52 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Mon, 25 May 2015 18:39:52 +0000 (UTC)
Subject: Extract email address from Java script in html source using python
References: <mailman.265.1432369272.17265.python-list@python.org>
Message-ID: <mjvq9o$9i3$11@dont-email.me>

On Sat, 23 May 2015 12:16:06 +0530, savitha devi wrote:

> I am developing a web scraper code using HTMLParser. I need to extract
> text/email address from java script with in the HTMLCode.I am beginner
> level in python coding and totally lost here. Need some help on this.

(a) Try a less ambitious learning project.
(b) Start reading the relevant documentation.

Pick one and go with it.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From alanoe at linux.vnet.ibm.com  Mon May 25 14:42:45 2015
From: alanoe at linux.vnet.ibm.com (Alan Evangelista)
Date: Mon, 25 May 2015 15:42:45 -0300
Subject: msgfmt.py and pygettext.py are LGPL or LGPL-compatible?
Message-ID: <55636D25.4000702@linux.vnet.ibm.com>

https://docs.python.org/2/library/gettext.html suggests that I use 
msgfmt.py and pygettext.py, available
at Python Subversion ( 
http://svn.python.org/view/python/trunk/Tools/i18n/). What license those 
executable
scripts use? Are they LGPL? I want to convert these executables to 
Python modules and use them in my
applications, but I fear the viral effect of LGPL over my code. Could 
someone clarify if I can do it
without legal concerns?

If this is not the correct mailing list to discuss this topic, pls point 
me to the correct one.
I have found Python-legal-sig at python.org, but it seems quite inactive.


Regards,
Alan Evangelista



From alanoe at linux.vnet.ibm.com  Mon May 25 14:42:45 2015
From: alanoe at linux.vnet.ibm.com (Alan Evangelista)
Date: Mon, 25 May 2015 15:42:45 -0300
Subject: msgfmt.py and pygettext.py are LGPL or LGPL-compatible?
Message-ID: <55636D25.4000702@linux.vnet.ibm.com>

https://docs.python.org/2/library/gettext.html suggests that I use 
msgfmt.py and pygettext.py, available
at Python Subversion ( 
http://svn.python.org/view/python/trunk/Tools/i18n/). What license those 
executable
scripts use? Are they LGPL? I want to convert these executables to 
Python modules and use them in my
applications, but I fear the viral effect of LGPL over my code. Could 
someone clarify if I can do it
without legal concerns?

If this is not the correct mailing list to discuss this topic, pls point 
me to the correct one.
I have found Python-legal-sig at python.org, but it seems quite inactive.


Regards,
Alan Evangelista



From zljubisicmob at gmail.com  Mon May 25 15:10:47 2015
From: zljubisicmob at gmail.com (zljubisicmob at gmail.com)
Date: Mon, 25 May 2015 12:10:47 -0700 (PDT)
Subject: Why dropbox.client.DropboxClient.put_file needs file_obj as a
 parameter?
Message-ID: <93ce55bf-3b0d-42ce-b514-2326f45ca2bb@googlegroups.com>

Hi

as I can see dropbox.client.DropboxClient.put_file has four parameters:

"
    full_path
        The full path to upload the file to, including the file name. If the destination folder does not yet exist, it will be created.
    file_obj
        A file-like object to upload. If you would like, you can pass a string as file_obj.
    overwrite
        Whether to overwrite an existing file at the given path. (Default False.) If overwrite is False and a file already exists there, Dropbox will rename the upload to make sure it doesn't overwrite anything. You need to check the metadata returned for the new name. This field should only be True if your intent is to potentially clobber changes to a file that you don't know about.
    parent_rev
        Optional rev field from the 'parent' of this upload. If your intent is to update the file at the given path, you should pass the parent_rev parameter set to the rev value from the most recent metadata you have of the existing file at that path. If the server has a more recent version of the file at the specified path, it will automatically rename your uploaded file, spinning off a conflict. Using this parameter effectively causes the overwrite parameter to be ignored. The file will always be overwritten if you send the most recent parent_rev, and it will never be overwritten if you send a less recent one.
"

What is the purpose of file_obj parameter?

f = open('working-draft.txt', 'rb')
response = client.put_file('/magnum-opus.txt', f)
print "uploaded:", response

In the example above there are two files working-draft.txt and magnum-opus.txt.
If I want to upload magnum-opus.txt, why do I need working-draft.txt?

Furthermore, if I want to update the file, how to find out parent_rev parameter?

As you can see I am a little bit confused. :)

Regards.


From zljubisicmob at gmail.com  Mon May 25 15:10:47 2015
From: zljubisicmob at gmail.com (zljubisicmob at gmail.com)
Date: Mon, 25 May 2015 12:10:47 -0700 (PDT)
Subject: Why dropbox.client.DropboxClient.put_file needs file_obj as a
 parameter?
Message-ID: <93ce55bf-3b0d-42ce-b514-2326f45ca2bb@googlegroups.com>

Hi

as I can see dropbox.client.DropboxClient.put_file has four parameters:

"
    full_path
        The full path to upload the file to, including the file name. If the destination folder does not yet exist, it will be created.
    file_obj
        A file-like object to upload. If you would like, you can pass a string as file_obj.
    overwrite
        Whether to overwrite an existing file at the given path. (Default False.) If overwrite is False and a file already exists there, Dropbox will rename the upload to make sure it doesn't overwrite anything. You need to check the metadata returned for the new name. This field should only be True if your intent is to potentially clobber changes to a file that you don't know about.
    parent_rev
        Optional rev field from the 'parent' of this upload. If your intent is to update the file at the given path, you should pass the parent_rev parameter set to the rev value from the most recent metadata you have of the existing file at that path. If the server has a more recent version of the file at the specified path, it will automatically rename your uploaded file, spinning off a conflict. Using this parameter effectively causes the overwrite parameter to be ignored. The file will always be overwritten if you send the most recent parent_rev, and it will never be overwritten if you send a less recent one.
"

What is the purpose of file_obj parameter?

f = open('working-draft.txt', 'rb')
response = client.put_file('/magnum-opus.txt', f)
print "uploaded:", response

In the example above there are two files working-draft.txt and magnum-opus.txt.
If I want to upload magnum-opus.txt, why do I need working-draft.txt?

Furthermore, if I want to update the file, how to find out parent_rev parameter?

As you can see I am a little bit confused. :)

Regards.


From ravas at outlook.com  Mon May 25 15:21:46 2015
From: ravas at outlook.com (ravas)
Date: Mon, 25 May 2015 12:21:46 -0700 (PDT)
Subject: a more precise distance algorithm
Message-ID: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>

I read an interesting comment:
"""
The coolest thing I've ever discovered about Pythagorean's Theorem is an alternate way to calculate it. If you write a program that uses the distance form c = sqrt(a^2 + b^2) you will suffer from the lose of half of your available precision because the square root operation is last. A more accurate calculation is c = a * sqrt(1 + b^2 / a^2). If a is less than b, you should swap them and of course handle the special case of a = 0.
"""

Is this valid? Does it apply to python?
Any other thoughts? :D

My imagining:

def distance(A, B):
    """
    A & B are objects with x and y attributes
    :return: the distance between A and B
    """
    dx = B.x - A.x
    dy = B.y - A.y
    a = min(dx, dy)
    b = max(dx, dy)
    if a == 0:
        return b
    elif b == 0:
        return a
    else:
        return a * sqrt(1 + (b / a)**2)


From ravas at outlook.com  Mon May 25 15:21:46 2015
From: ravas at outlook.com (ravas)
Date: Mon, 25 May 2015 12:21:46 -0700 (PDT)
Subject: a more precise distance algorithm
Message-ID: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>

I read an interesting comment:
"""
The coolest thing I've ever discovered about Pythagorean's Theorem is an alternate way to calculate it. If you write a program that uses the distance form c = sqrt(a^2 + b^2) you will suffer from the lose of half of your available precision because the square root operation is last. A more accurate calculation is c = a * sqrt(1 + b^2 / a^2). If a is less than b, you should swap them and of course handle the special case of a = 0.
"""

Is this valid? Does it apply to python?
Any other thoughts? :D

My imagining:

def distance(A, B):
    """
    A & B are objects with x and y attributes
    :return: the distance between A and B
    """
    dx = B.x - A.x
    dy = B.y - A.y
    a = min(dx, dy)
    b = max(dx, dy)
    if a == 0:
        return b
    elif b == 0:
        return a
    else:
        return a * sqrt(1 + (b / a)**2)


From skip.montanaro at gmail.com  Mon May 25 16:02:48 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 25 May 2015 15:02:48 -0500
Subject: Why dropbox.client.DropboxClient.put_file needs file_obj as a
 parameter?
In-Reply-To: <93ce55bf-3b0d-42ce-b514-2326f45ca2bb@googlegroups.com>
References: <93ce55bf-3b0d-42ce-b514-2326f45ca2bb@googlegroups.com>
Message-ID: <CANc-5UxzO82PkdyaM-k-Z_i9=e1dW4+Vm2yt-A7PmsWQ8kHWqQ@mail.gmail.com>

Looks like the file_obj is what you are reading locally, and
magnum_opus.txt is the destination name on Dropbox.

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150525/c3cd0087/attachment.html>

From skip.montanaro at gmail.com  Mon May 25 16:02:48 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 25 May 2015 15:02:48 -0500
Subject: Why dropbox.client.DropboxClient.put_file needs file_obj as a
 parameter?
In-Reply-To: <93ce55bf-3b0d-42ce-b514-2326f45ca2bb@googlegroups.com>
References: <93ce55bf-3b0d-42ce-b514-2326f45ca2bb@googlegroups.com>
Message-ID: <CANc-5UxzO82PkdyaM-k-Z_i9=e1dW4+Vm2yt-A7PmsWQ8kHWqQ@mail.gmail.com>

Looks like the file_obj is what you are reading locally, and
magnum_opus.txt is the destination name on Dropbox.

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150525/c3cd0087/attachment-0001.html>

From felix at epepm.cupet.cu  Mon May 25 16:06:06 2015
From: felix at epepm.cupet.cu (felix)
Date: Mon, 25 May 2015 16:06:06 -0400
Subject: a more precise distance algorithm
In-Reply-To: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
Message-ID: <556380AE.6010609@epepm.cupet.cu>

El 25/05/15 15:21, ravas escribi?:
> I read an interesting comment:
> """
> The coolest thing I've ever discovered about Pythagorean's Theorem is an alternate way to calculate it. If you write a program that uses the distance form c = sqrt(a^2 + b^2) you will suffer from the lose of half of your available precision because the square root operation is last. A more accurate calculation is c = a * sqrt(1 + b^2 / a^2). If a is less than b, you should swap them and of course handle the special case of a = 0.
> """
>
> Is this valid? Does it apply to python?
> Any other thoughts? :D
>
> My imagining:
>
> def distance(A, B):
>      """
>      A & B are objects with x and y attributes
>      :return: the distance between A and B
>      """
>      dx = B.x - A.x
>      dy = B.y - A.y
>      a = min(dx, dy)
>      b = max(dx, dy)
>      if a == 0:
>          return b
>      elif b == 0:
>          return a
>      else:
>          return a * sqrt(1 + (b / a)**2)
I don't know if precision lose fits here but the second way you gave to 
calculate c is just Math. Nothing extraordinary here.

c = a * sqrt(1 + b^2 / a^2)
c = sqrt(a^2(1 + b^2 / a^2)) applying the inverse function to introduce 
a inside the square root
c = sqrt(a^2 + a^2*b^2/a^2) then just simplify
c = sqrt(a^2 + b^2)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150525/b7c91065/attachment.html>

From felix at epepm.cupet.cu  Mon May 25 16:06:06 2015
From: felix at epepm.cupet.cu (felix)
Date: Mon, 25 May 2015 16:06:06 -0400
Subject: a more precise distance algorithm
In-Reply-To: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
Message-ID: <556380AE.6010609@epepm.cupet.cu>

El 25/05/15 15:21, ravas escribi?:
> I read an interesting comment:
> """
> The coolest thing I've ever discovered about Pythagorean's Theorem is an alternate way to calculate it. If you write a program that uses the distance form c = sqrt(a^2 + b^2) you will suffer from the lose of half of your available precision because the square root operation is last. A more accurate calculation is c = a * sqrt(1 + b^2 / a^2). If a is less than b, you should swap them and of course handle the special case of a = 0.
> """
>
> Is this valid? Does it apply to python?
> Any other thoughts? :D
>
> My imagining:
>
> def distance(A, B):
>      """
>      A & B are objects with x and y attributes
>      :return: the distance between A and B
>      """
>      dx = B.x - A.x
>      dy = B.y - A.y
>      a = min(dx, dy)
>      b = max(dx, dy)
>      if a == 0:
>          return b
>      elif b == 0:
>          return a
>      else:
>          return a * sqrt(1 + (b / a)**2)
I don't know if precision lose fits here but the second way you gave to 
calculate c is just Math. Nothing extraordinary here.

c = a * sqrt(1 + b^2 / a^2)
c = sqrt(a^2(1 + b^2 / a^2)) applying the inverse function to introduce 
a inside the square root
c = sqrt(a^2 + a^2*b^2/a^2) then just simplify
c = sqrt(a^2 + b^2)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150525/b7c91065/attachment-0001.html>

From gary.herron at islandtraining.com  Mon May 25 16:20:07 2015
From: gary.herron at islandtraining.com (Gary Herron)
Date: Mon, 25 May 2015 13:20:07 -0700
Subject: a more precise distance algorithm
In-Reply-To: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
Message-ID: <556383F7.5070800@islandtraining.com>

On 05/25/2015 12:21 PM, ravas wrote:
> I read an interesting comment:
> """
> The coolest thing I've ever discovered about Pythagorean's Theorem is an alternate way to calculate it. If you write a program that uses the distance form c = sqrt(a^2 + b^2) you will suffer from the lose of half of your available precision because the square root operation is last. A more accurate calculation is c = a * sqrt(1 + b^2 / a^2). If a is less than b, you should swap them and of course handle the special case of a = 0.
> """
>
> Is this valid?

> Does it apply to python?

This is a statement about floating point numeric calculations on a 
computer,.  As such, it does apply to Python which uses the underlying 
hardware for floating point calculations.

Validity is another matter.  Where did you find the quote?

Gary Herron


> Any other thoughts? :D
>
> My imagining:
>
> def distance(A, B):
>      """
>      A & B are objects with x and y attributes
>      :return: the distance between A and B
>      """
>      dx = B.x - A.x
>      dy = B.y - A.y
>      a = min(dx, dy)
>      b = max(dx, dy)
>      if a == 0:
>          return b
>      elif b == 0:
>          return a
>      else:
>          return a * sqrt(1 + (b / a)**2)



From gary.herron at islandtraining.com  Mon May 25 16:20:07 2015
From: gary.herron at islandtraining.com (Gary Herron)
Date: Mon, 25 May 2015 13:20:07 -0700
Subject: a more precise distance algorithm
In-Reply-To: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
Message-ID: <556383F7.5070800@islandtraining.com>

On 05/25/2015 12:21 PM, ravas wrote:
> I read an interesting comment:
> """
> The coolest thing I've ever discovered about Pythagorean's Theorem is an alternate way to calculate it. If you write a program that uses the distance form c = sqrt(a^2 + b^2) you will suffer from the lose of half of your available precision because the square root operation is last. A more accurate calculation is c = a * sqrt(1 + b^2 / a^2). If a is less than b, you should swap them and of course handle the special case of a = 0.
> """
>
> Is this valid?

> Does it apply to python?

This is a statement about floating point numeric calculations on a 
computer,.  As such, it does apply to Python which uses the underlying 
hardware for floating point calculations.

Validity is another matter.  Where did you find the quote?

Gary Herron


> Any other thoughts? :D
>
> My imagining:
>
> def distance(A, B):
>      """
>      A & B are objects with x and y attributes
>      :return: the distance between A and B
>      """
>      dx = B.x - A.x
>      dy = B.y - A.y
>      a = min(dx, dy)
>      b = max(dx, dy)
>      if a == 0:
>          return b
>      elif b == 0:
>          return a
>      else:
>          return a * sqrt(1 + (b / a)**2)



From auriocus at gmx.de  Mon May 25 16:27:14 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Mon, 25 May 2015 22:27:14 +0200
Subject: a more precise distance algorithm
In-Reply-To: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
Message-ID: <mk00gr$kku$1@dont-email.me>

Am 25.05.15 um 21:21 schrieb ravas:
> I read an interesting comment:
> """
> The coolest thing I've ever discovered about Pythagorean's Theorem is an alternate way to calculate it. If you write a program that uses the distance form c = sqrt(a^2 + b^2) you will suffer from the lose of half of your available precision because the square root operation is last. A more accurate calculation is c = a * sqrt(1 + b^2 / a^2). If a is less than b, you should swap them and of course handle the special case of a = 0.
> """
>
> Is this valid?

Yes. Valid for floating point math, which can overflow and lose precision.

> Does it apply to python?

Yes. Python uses floating point math by default

> Any other thoughts? :D
>
> My imagining:
>
> def distance(A, B):

Wrong. Just use the built-in function Math.hypot() - it should handle 
these cases and also overflow, infinity etc. in the best possible way.

Apfelkiste:~ chris$ python
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import math
 >>> math.hypot(3,4)
5.0

	Christian


From auriocus at gmx.de  Mon May 25 16:27:14 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Mon, 25 May 2015 22:27:14 +0200
Subject: a more precise distance algorithm
In-Reply-To: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
Message-ID: <mk00gr$kku$1@dont-email.me>

Am 25.05.15 um 21:21 schrieb ravas:
> I read an interesting comment:
> """
> The coolest thing I've ever discovered about Pythagorean's Theorem is an alternate way to calculate it. If you write a program that uses the distance form c = sqrt(a^2 + b^2) you will suffer from the lose of half of your available precision because the square root operation is last. A more accurate calculation is c = a * sqrt(1 + b^2 / a^2). If a is less than b, you should swap them and of course handle the special case of a = 0.
> """
>
> Is this valid?

Yes. Valid for floating point math, which can overflow and lose precision.

> Does it apply to python?

Yes. Python uses floating point math by default

> Any other thoughts? :D
>
> My imagining:
>
> def distance(A, B):

Wrong. Just use the built-in function Math.hypot() - it should handle 
these cases and also overflow, infinity etc. in the best possible way.

Apfelkiste:~ chris$ python
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import math
 >>> math.hypot(3,4)
5.0

	Christian


From ravas at outlook.com  Mon May 25 17:03:09 2015
From: ravas at outlook.com (ravas)
Date: Mon, 25 May 2015 14:03:09 -0700 (PDT)
Subject: a more precise distance algorithm
In-Reply-To: <mk00gr$kku$1@dont-email.me>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
 <mk00gr$kku$1@dont-email.me>
Message-ID: <a17f7167-0159-4c1f-b594-c753a4d5da1c@googlegroups.com>

On Monday, May 25, 2015 at 1:27:24 PM UTC-7, Christian Gollwitzer wrote:
> Wrong. Just use the built-in function Math.hypot() - it should handle 
> these cases and also overflow, infinity etc. in the best possible way.
> 
> Apfelkiste:~ chris$ python
> Python 2.7.2 (default, Oct 11 2012, 20:14:37)
> [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import math
>  >>> math.hypot(3,4)
> 5.0
> 
> 	Christian

Thank you! :D I forgot about that one.
I wonder how the sympy Point.distance method compares...


From ravas at outlook.com  Mon May 25 17:05:50 2015
From: ravas at outlook.com (ravas)
Date: Mon, 25 May 2015 14:05:50 -0700 (PDT)
Subject: a more precise distance algorithm
In-Reply-To: <mailman.42.1432585603.5151.python-list@python.org>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
 <mailman.42.1432585603.5151.python-list@python.org>
Message-ID: <af8c8100-7aaa-450d-b2a1-95621b5a8126@googlegroups.com>

On Monday, May 25, 2015 at 1:27:43 PM UTC-7, Gary Herron wrote: 
> This is a statement about floating point numeric calculations on a 
> computer,.  As such, it does apply to Python which uses the underlying 
> hardware for floating point calculations.
> 
> Validity is another matter.  Where did you find the quote?

Thank you.

You can find the quote in the 4th comment at the bottom of:
http://betterexplained.com/articles/surprising-uses-of-the-pythagorean-theorem/


From zljubisicmob at gmail.com  Mon May 25 17:26:22 2015
From: zljubisicmob at gmail.com (Zoran Ljubisic (mob))
Date: Mon, 25 May 2015 23:26:22 +0200
Subject: Why dropbox.client.DropboxClient.put_file needs file_obj as a
 parameter?
In-Reply-To: <CANc-5UxzO82PkdyaM-k-Z_i9=e1dW4+Vm2yt-A7PmsWQ8kHWqQ@mail.gmail.com>
References: <93ce55bf-3b0d-42ce-b514-2326f45ca2bb@googlegroups.com>
 <CANc-5UxzO82PkdyaM-k-Z_i9=e1dW4+Vm2yt-A7PmsWQ8kHWqQ@mail.gmail.com>
Message-ID: <CALGHU=FSqma_UnBaKLkvS1N-jM_jOFjsMAyL0TLsS8MTh7Mmzg@mail.gmail.com>

Looks like you are right.
Thanks for clarification.

Regards.
On May 25, 2015 10:02 PM, "Skip Montanaro" <skip.montanaro at gmail.com> wrote:

> Looks like the file_obj is what you are reading locally, and
> magnum_opus.txt is the destination name on Dropbox.
>
> Skip
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150525/4ac29c52/attachment.html>

From rosuav at gmail.com  Mon May 25 19:13:41 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 26 May 2015 09:13:41 +1000
Subject: msgfmt.py and pygettext.py are LGPL or LGPL-compatible?
In-Reply-To: <55636D25.4000702@linux.vnet.ibm.com>
References: <55636D25.4000702@linux.vnet.ibm.com>
Message-ID: <CAPTjJmpLXuVScJPegBCzKri9yGVxCzspQ+fD_4O1-pn7BGzJLg@mail.gmail.com>

On Tue, May 26, 2015 at 4:42 AM, Alan Evangelista
<alanoe at linux.vnet.ibm.com> wrote:
> https://docs.python.org/2/library/gettext.html suggests that I use msgfmt.py
> and pygettext.py, available
> at Python Subversion ( http://svn.python.org/view/python/trunk/Tools/i18n/).
> What license those executable
> scripts use? Are they LGPL? I want to convert these executables to Python
> modules and use them in my
> applications, but I fear the viral effect of LGPL over my code. Could
> someone clarify if I can do it
> without legal concerns?

Where did you get the svn link from? If nothing else, you'll get a
newer version of the file by looking in Mercurial, as the file you
link to hasn't changed in some years.

Those files aren't binary executables, so you can just have a look at
them to see if there's a license comment. And AFAIK, you should be
able to keep them completely separate from the rest of your code. I
don't entirely follow what they're doing (only skimmed the page), but
my understanding is that you would use these tools as part of building
your code. It's like using gcc to compile a C program - gcc itself is
GPL (not even LGPL), but that doesn't affect your code at all.

But I'm no lawyer, just a programmer.

ChrisA


From alanoe at linux.vnet.ibm.com  Mon May 25 20:37:36 2015
From: alanoe at linux.vnet.ibm.com (Alan Evangelista)
Date: Mon, 25 May 2015 21:37:36 -0300
Subject: msgfmt.py and pygettext.py are LGPL or LGPL-compatible?
In-Reply-To: <CAPTjJmpLXuVScJPegBCzKri9yGVxCzspQ+fD_4O1-pn7BGzJLg@mail.gmail.com>
References: <55636D25.4000702@linux.vnet.ibm.com>
 <CAPTjJmpLXuVScJPegBCzKri9yGVxCzspQ+fD_4O1-pn7BGzJLg@mail.gmail.com>
Message-ID: <5563C050.6080309@linux.vnet.ibm.com>

On 05/25/2015 08:13 PM, Chris Angelico wrote:
> On Tue, May 26, 2015 at 4:42 AM, Alan Evangelista
> <alanoe at linux.vnet.ibm.com> wrote:
>> https://docs.python.org/2/library/gettext.html suggests that I use msgfmt.py
>> and pygettext.py, available
>> at Python Subversion ( http://svn.python.org/view/python/trunk/Tools/i18n/).
>> What license those executable
>> scripts use? Are they LGPL? I want to convert these executables to Python
>> modules and use them in my
>> applications, but I fear the viral effect of LGPL over my code. Could
>> someone clarify if I can do it
>> without legal concerns?
> Where did you get the svn link from? If nothing else, you'll get a
> newer version of the file by looking in Mercurial, as the file you
> link to hasn't changed in some years.

https://docs.python.org/2/library/gettext.html#id4 mentions that 
msgfmt.py is in Tools/i18n
directory, but does not mention the source code repository URL, so I had 
to look by myself.
I googled "msgfmt.py download" and the SVN URL came up (btw it has the 
dir structure
mentioned in Python doc).

I could not find msgfmt.py and pygettext.py quickly looking at the 
Mercurial repositories list at
https://hg.python.org/ . Anyway, the 2 files in SVN work fine for me.


> Those files aren't binary executables, so you can just have a look at
> them to see if there's a license comment.

No license in the source files, just authorship. I am not sure that this 
is enough to assert
they have no license, though.


> And AFAIK, you should be
> able to keep them completely separate from the rest of your code.

I can use the Python executables directly, but I'd like to convert them 
to Python modules
and eliminate the part of the code I do not use, hence my legal question.


Regards,
Alan Evangelista



From garyr at fidalgo.net  Mon May 25 21:47:41 2015
From: garyr at fidalgo.net (garyr)
Date: Mon, 25 May 2015 18:47:41 -0700
Subject: cl.exe missing
References: <mjvejo$rmu$1@speranza.aioe.org>
 <mailman.38.1432569064.5151.python-list@python.org>
Message-ID: <mk0jcg$ldc$1@speranza.aioe.org>


>> Where do I find VS2008?
>
> Try this:
>
>   https://www.microsoft.com/en-gb/download/details.aspx?id=44266
>
> TJG

Yes, that's it. Many thanks.





From steve at pearwood.info  Mon May 25 22:17:30 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 26 May 2015 12:17:30 +1000
Subject: Documentaion of dunder methods
Message-ID: <5563d7bb$0$12982$c3e8da3$5496439d@news.astraweb.com>

PEP 8 states that developers should never invent their own dunder methods:

    __double_leading_and_trailing_underscore__ : 
        "magic" objects or attributes that live in user-controlled
        namespaces. E.g. __init__ , __import__ or __file__ . Never 
        invent such names; only use them as documented. 

https://www.python.org/dev/peps/pep-0008/#naming-conventions


In other words, dunder methods are reserved for use by the core developers
for the use of the Python interpreter.

Apart from PEP 8, is this documented anywhere in the official documentation?
If so, I have been unable to find it.



-- 
Steven



From davidfstr at gmail.com  Mon May 25 22:24:20 2015
From: davidfstr at gmail.com (davidfstr at gmail.com)
Date: Mon, 25 May 2015 19:24:20 -0700 (PDT)
Subject: Creating a reliable sandboxed Python environment
Message-ID: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>

I am writing a web service that accepts Python programs as input, runs the provided program with some profiling hooks, and returns various information about the program's runtime behavior. To do this in a safe manner, I need to be able to create a sandbox that restricts what the submitted Python program can do on the web server.

Almost all discussion about Python sandboxes I have seen on the internet involves selectively blacklisting functionality that gives access to system resources, such as trying to hide the "open" builtin to restrict access to file I/O. All such approaches are doomed to fail because you can always find a way around a blacklist.

For my particular sandbox, I wish to allow *only* the following kinds of actions (in a whitelist):
* reading from stdin & writing to stdout;
* reading from files, within a set of whitelisted directories;
* pure Python computation.

In particular all other operations available through system calls are banned. This includes, but is not limited to:
* writing to files;
* manipulating network sockets;
* communicating with other processes.

I believe it is not possible to limit such operations at the Python level. The best you could do is try replacing all the standard library modules, but that is again just a blacklist - it won't prevent a determined attacker from doing things like constructing their own 'code' object and executing it.

It might be necessary to isolate the Python process at the operating system level.
* A chroot jail on Linux & OS X can limit access to the filesystem. Again this is just a blacklist.
* No obvious way to block socket creation. Again this would be just a blacklist.
* No obvious way to detect unapproved system calls and block them.

In the limit, I could dynamically spin up a virtual machine and execute the Python program in the machine. However that's extremely expensive in computational time.

Has anyone on this list attempted to sandbox Python programs in a serious fashion? I'd be interested to hear your approach.

- David


From rustompmody at gmail.com  Mon May 25 22:36:50 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 25 May 2015 19:36:50 -0700 (PDT)
Subject: Documentaion of dunder methods
In-Reply-To: <5563d7bb$0$12982$c3e8da3$5496439d@news.astraweb.com>
References: <5563d7bb$0$12982$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <abbde97b-2f04-456b-8975-3aa8bcdba67e@googlegroups.com>

On Tuesday, May 26, 2015 at 7:47:41 AM UTC+5:30, Steven D'Aprano wrote:
> PEP 8 states that developers should never invent their own dunder methods:
> 
>     __double_leading_and_trailing_underscore__ : 
>         "magic" objects or attributes that live in user-controlled
>         namespaces. E.g. __init__ , __import__ or __file__ . Never 
>         invent such names; only use them as documented. 
> 
> https://www.python.org/dev/peps/pep-0008/#naming-conventions
> 
> 
> In other words, dunder methods are reserved for use by the core developers
> for the use of the Python interpreter.
> 
> Apart from PEP 8, is this documented anywhere in the official documentation?
> If so, I have been unable to find it.

Thanks for bringing this up.
dunder methods are a super-powerful and very pythonesque feature rendered 
somewhat useless by bad documentation.
Also I happen to disagree with (the literal rendering of your:

> In other words, dunder methods are reserved for use by the core developers
> for the use of the Python interpreter.

Of course defining YOUR OWN dunder method, disregarding the intended protocol
for that dunder method's usage, should not be done.
However if you interpret your statement as "Just dont go near them" it seems
like they are some kind of best-avoided leaky-abstraction of the (C)python 
implementation. And I guess you dont really want to assert that!!

What I'd like to see in the docs is something that 'zips' the operator table 
with the corresponding dunder method.  Something like:
| + | __add__ __radd__ |
| []| __getitem__ |
etc

Yeah I realize the most critical dunders dont 'have' an associated operator --
__init__.  But even there I'd put something like
|Class constructor?| __init__

? Yeah I know about the 'initializer-vs-constructor' debates. Not really taking
a side on that one


From rosuav at gmail.com  Mon May 25 22:44:34 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 26 May 2015 12:44:34 +1000
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
Message-ID: <CAPTjJmoRT2AOrjfZSg29QEcD++UnSk+d90op4Zxj=5bEuiyYBw@mail.gmail.com>

On Tue, May 26, 2015 at 12:24 PM,  <davidfstr at gmail.com> wrote:
> I believe it is not possible to limit such operations at the Python level. The best you could do is try replacing all the standard library modules, but that is again just a blacklist - it won't prevent a determined attacker from doing things like constructing their own 'code' object and executing it.
>
> It might be necessary to isolate the Python process at the operating system level.
> * A chroot jail on Linux & OS X can limit access to the filesystem. Again this is just a blacklist.
> * No obvious way to block socket creation. Again this would be just a blacklist.
> * No obvious way to detect unapproved system calls and block them.
>
> In the limit, I could dynamically spin up a virtual machine and execute the Python program in the machine. However that's extremely expensive in computational time.
>
> Has anyone on this list attempted to sandbox Python programs in a serious fashion? I'd be interested to hear your approach.

Yes, I had a project along similar lines to yours, a few years back.
We wanted to let our end users customize our service using a Python
script. Our conclusions were:

1) As you say, it is fundamentally not possible to make this work at
the Python level.
2) It's extremely difficult to do at any other level, too.
3) Python is a great language, despite my then-boss's dislike of it.
4) Lua isn't as great a language, but it's much easier to sandbox.
5) Unicode is important, even if my then-boss took a lot of convincing
on that one. (Was a big point in Python's favour, and against Lua.)
6) Efficient transfer of complex structured data across a process
boundary is difficult.
7) Letting end users script your system safely is a fundamentally hard problem.

We ended up abandoning Python altogether and using ECMAScript (with
Google's V8 interpreter) as our scripting language, and even then, we
had to do all sorts of things to make it safe. (And I wouldn't bet my
life on it being safe even now. Not even sure I'd bet my data or
uptime on it being safe, either.)

My recommendation to you: If you absolutely have to run untrusted
Python code, don't concern yourself with *anything* that the Python
code can and can't do. You'll end up making gross and ugly hacks that
stop people from doing legitimate things, in an attempt to prevent
abuses. Instead, *just* guard yourself at the OS level - a chroot jail
to protect what matters, iptables rules to prevent anything going to
the outside world, run as a non-significant user with minimal
permissions, ulimit everything so they can't hurt you. Whatever it
takes, make it so that you could protect C code, because trust me,
it'll be less headaches than trying to sandbox anything at the Python
level. Or, worse, you won't get headaches, you'll just have a flawed
security model that eventually gets exploited.

There are a couple of alternatives. You could go for a really extreme
protection system and actually spin up a virtual machine, where
they're welcome to do whatever they like, and it'll run inside X
amount of memory and Y amount of CPU. Pretty costly (the overhead of a
full OS for every client), but it'll work. Or you could go to the
other extreme, and instead of actually permitting arbitrary Python
code, you instead allow a "Python-like syntax" wherein people can
manipulate the input. You'd need to then create some special hacks to
allow file I/O, so this probably wouldn't work for your scenario, but
imagine writing a sed-like program that accepts Python code. You could
do something like this:

for line in input:
    print(evaluate_user_code(line), file=output)

where evaluate_user_code() is a protected evaluator, like
ast.literal_eval() but additionally allowing access to one name
"line", which obviously would be the line in question.

But for your case, I think that'd require too many hacks to be useful.

ChrisA


From steve at pearwood.info  Mon May 25 23:11:14 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 26 May 2015 13:11:14 +1000
Subject: a more precise distance algorithm
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
Message-ID: <5563e453$0$12990$c3e8da3$5496439d@news.astraweb.com>

On Tue, 26 May 2015 05:21 am, ravas wrote:

> I read an interesting comment:
> """
> The coolest thing I've ever discovered about Pythagorean's Theorem is an
> alternate way to calculate it. If you write a program that uses the
> distance form c = sqrt(a^2 + b^2) you will suffer from the lose of half of
> your available precision because the square root operation is last. A more
> accurate calculation is c = a * sqrt(1 + b^2 / a^2). If a is less than b,
> you should swap them and of course handle the special case of a = 0. """


Let's compare three methods. 


import math
import random

def naive(a, b):
    return math.sqrt(a**2 + b**2)

def alternate(a, b):
    a, b = min(a, b), max(a, b)
    if a == 0:  return b
    if b == 0:  return a
    return a * math.sqrt(1 + b**2 / a**2)

counter = 0
print("Type Ctrl-C to exit")
while True:
    counter += 1
    a = random.uniform(0, 1000)
    b = random.uniform(0, 1000)
    d1 = naive(a, b) 
    d2 = alternate(a, b)
    d3 = math.hypot(a, b)
    if not (d1 == d2 == d3):
        print("mismatch after %d trials" % counter)
        print("naive:", d1)
        print("alternate:", d2)
        print("hypot:", d3)
        break



When I run that, I get:

mismatch after 3 trials
naive: 1075.6464259886257
alternate: 1075.6464259886257
hypot: 1075.6464259886254

A second run gives:

mismatch after 3 trials
naive: 767.3916150255787
alternate: 767.3916150255789
hypot: 767.3916150255787


which shows that:

(1) It's not hard to find mismatches;
(2) It's not obvious which of the three methods is more accurate.



-- 
Steven



From steve at pearwood.info  Mon May 25 23:17:59 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 26 May 2015 13:17:59 +1000
Subject: Documentaion of dunder methods
References: <5563d7bb$0$12982$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5563e5e8$0$12983$c3e8da3$5496439d@news.astraweb.com>

On Tue, 26 May 2015 12:17 pm, Steven D'Aprano wrote:

> In other words, dunder methods are reserved for use by the core developers
> for the use of the Python interpreter.

Er, that's easy to misinterpret. Let me try rewording:

You should not invent new dunder methods.

And if possible, you should not call such dunder methods directly. Instead,
use the official protocol (e.g. next(x), not x.__next__()) whenever
possible. The operator module contains many helpers for that.

But of course you may *write* dunder methods.


-- 
Steven



From rustompmody at gmail.com  Mon May 25 23:30:07 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Mon, 25 May 2015 20:30:07 -0700 (PDT)
Subject: Documentaion of dunder methods
In-Reply-To: <5563e5e8$0$12983$c3e8da3$5496439d@news.astraweb.com>
References: <5563d7bb$0$12982$c3e8da3$5496439d@news.astraweb.com>
 <5563e5e8$0$12983$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <0c3aeb41-aaf9-4453-9ccc-b2c8891f1394@googlegroups.com>

On Tuesday, May 26, 2015 at 8:48:11 AM UTC+5:30, Steven D'Aprano wrote:
> On Tue, 26 May 2015 12:17 pm, Steven D'Aprano wrote:
> 
> > In other words, dunder methods are reserved for use by the core developers
> > for the use of the Python interpreter.
> 
> Er, that's easy to misinterpret. Let me try rewording:
> 
> You should not invent new dunder methods.
> 
> And if possible, you should not call such dunder methods directly. Instead,
> use the official protocol (e.g. next(x), not x.__next__()) whenever
> possible. The operator module contains many helpers for that.
> 
> But of course you may *write* dunder methods.

I guess you and I know what we want to convey and to proscribe.
However the above doesn't pass muster as documentation
The most glaring lack in the docs is some reification of the acceptable 
'protocols'.
Yeah people like to wax about duck-typing but that translates into semi-formal
protocols and they are just not documented 


From ravas at outlook.com  Tue May 26 00:13:17 2015
From: ravas at outlook.com (ravas)
Date: Mon, 25 May 2015 21:13:17 -0700 (PDT)
Subject: a more precise distance algorithm
In-Reply-To: <5563e453$0$12990$c3e8da3$5496439d@news.astraweb.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
 <5563e453$0$12990$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <f7bd33f0-f6e0-48cd-a8ea-bdd05fcced2a@googlegroups.com>

On Monday, May 25, 2015 at 8:11:25 PM UTC-7, Steven D'Aprano wrote:
> Let's compare three methods. 
> ...
> which shows that:
> 
> (1) It's not hard to find mismatches;
> (2) It's not obvious which of the three methods is more accurate.

Thank you; that is very helpful!

I'm curious: what about the sqrt() function being last is detrimental?
>From a point of ignorance it seems like we are just producing errors sooner, 
and then multiplying them, with this alternative method.



From ian.g.kelly at gmail.com  Tue May 26 00:34:51 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 25 May 2015 22:34:51 -0600
Subject: Documentaion of dunder methods
In-Reply-To: <5563d7bb$0$12982$c3e8da3$5496439d@news.astraweb.com>
References: <5563d7bb$0$12982$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CALwzid=STBakZGrRUeJMcy09C9icys6wDVuN7z3eAHeNaqOq=Q@mail.gmail.com>

On Mon, May 25, 2015 at 8:17 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> PEP 8 states that developers should never invent their own dunder methods:
>
>     __double_leading_and_trailing_underscore__ :
>         "magic" objects or attributes that live in user-controlled
>         namespaces. E.g. __init__ , __import__ or __file__ . Never
>         invent such names; only use them as documented.
>
> https://www.python.org/dev/peps/pep-0008/#naming-conventions
>
>
> In other words, dunder methods are reserved for use by the core developers
> for the use of the Python interpreter.
>
> Apart from PEP 8, is this documented anywhere in the official documentation?
> If so, I have been unable to find it.

https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers


From ian.g.kelly at gmail.com  Tue May 26 00:42:51 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 25 May 2015 22:42:51 -0600
Subject: a more precise distance algorithm
In-Reply-To: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
Message-ID: <CALwzid=geKZ2o49ZROCE5U6J+BtiH_tUeku20d17Hqaz9A-Nvg@mail.gmail.com>

On Mon, May 25, 2015 at 1:21 PM, ravas <ravas at outlook.com> wrote:
> I read an interesting comment:
> """
> The coolest thing I've ever discovered about Pythagorean's Theorem is an alternate way to calculate it. If you write a program that uses the distance form c = sqrt(a^2 + b^2) you will suffer from the lose of half of your available precision because the square root operation is last. A more accurate calculation is c = a * sqrt(1 + b^2 / a^2). If a is less than b, you should swap them and of course handle the special case of a = 0.
> """
>
> Is this valid? Does it apply to python?
> Any other thoughts? :D
>
> My imagining:
>
> def distance(A, B):
>     """
>     A & B are objects with x and y attributes
>     :return: the distance between A and B
>     """
>     dx = B.x - A.x
>     dy = B.y - A.y
>     a = min(dx, dy)
>     b = max(dx, dy)
>     if a == 0:
>         return b
>     elif b == 0:
>         return a

This branch is incorrect because a could be negative.

You don't need this anyway; the a == 0 branch is only there because of
the division by a in the else branch.

>     else:
>         return a * sqrt(1 + (b / a)**2)

Same issue; if a is negative then the result will have the wrong sign.


From ravas at outlook.com  Tue May 26 00:59:01 2015
From: ravas at outlook.com (ravas)
Date: Mon, 25 May 2015 21:59:01 -0700 (PDT)
Subject: a more precise distance algorithm
In-Reply-To: <mailman.48.1432615414.5151.python-list@python.org>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
 <mailman.48.1432615414.5151.python-list@python.org>
Message-ID: <f8f63666-9d02-468e-a40f-f966cce855f6@googlegroups.com>

Oh ya... true >_<

Thanks :D

On Monday, May 25, 2015 at 9:43:47 PM UTC-7, Ian wrote:
> > def distance(A, B):
> >     """
> >     A & B are objects with x and y attributes
> >     :return: the distance between A and B
> >     """
> >     dx = B.x - A.x
> >     dy = B.y - A.y
> >     a = min(dx, dy)
> >     b = max(dx, dy)
> >     if a == 0:
> >         return b
> >     elif b == 0:
> >         return a
> 
> This branch is incorrect because a could be negative.
> 
> You don't need this anyway; the a == 0 branch is only there because of
> the division by a in the else branch.
> 
> >     else:
> >         return a * sqrt(1 + (b / a)**2)
> 
> Same issue; if a is negative then the result will have the wrong sign.



From gherron at digipen.edu  Tue May 26 01:09:53 2015
From: gherron at digipen.edu (Gary Herron)
Date: Mon, 25 May 2015 22:09:53 -0700
Subject: a more precise distance algorithm
In-Reply-To: <f7bd33f0-f6e0-48cd-a8ea-bdd05fcced2a@googlegroups.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
 <5563e453$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <f7bd33f0-f6e0-48cd-a8ea-bdd05fcced2a@googlegroups.com>
Message-ID: <55640021.60200@digipen.edu>

On 05/25/2015 09:13 PM, ravas wrote:
> On Monday, May 25, 2015 at 8:11:25 PM UTC-7, Steven D'Aprano wrote:
>> Let's compare three methods.
>> ...
>> which shows that:
>>
>> (1) It's not hard to find mismatches;
>> (2) It's not obvious which of the three methods is more accurate.
> Thank you; that is very helpful!
>
> I'm curious: what about the sqrt() function being last is detrimental?
>  From a point of ignorance it seems like we are just producing errors sooner,
> and then multiplying them, with this alternative method.

It's probably not the square root that's causing the inaccuracies. In 
many other cases, and probably here also, it's the summing of two 
numbers that have vastly different values that loses precision.  A 
demonstration:

 >>> big = 100000000.0
 >>> small = 0.000000001
 >>> (big+small)-big # Should produce a value =small, but gives an exact 
zero instead.
0.0

The squaring of the two values in x*x+y*y just makes the addition even 
more error prone since the squares make large values even larger and 
small values even smaller.


Gary Herron.

-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418



From auriocus at gmx.de  Tue May 26 01:33:46 2015
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Tue, 26 May 2015 07:33:46 +0200
Subject: a more precise distance algorithm
In-Reply-To: <5563e453$0$12990$c3e8da3$5496439d@news.astraweb.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
 <5563e453$0$12990$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mk10hj$oa8$1@dont-email.me>

Am 26.05.15 um 05:11 schrieb Steven D'Aprano:
> mismatch after 3 trials
> naive: 767.3916150255787
> alternate: 767.3916150255789
> hypot: 767.3916150255787
>
>
> which shows that:
>
> (1) It's not hard to find mismatches;
> (2) It's not obvious which of the three methods is more accurate.


The main problem is not necessarily precision. A square root is a very 
precise operation in floating point math, the relative precision 
*increases* by sqrt. The big problem is overflow. Take e.g. a=3*10^160, 
b=4*10^160, then the exact result is c=5*10^160. But:

 >>> a=3e160
 >>> b=4e160
 >>> math.sqrt(a**2+b**2)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
OverflowError: (34, 'Result too large')
 >>> math.hypot(a,b)
5e+160


	Christian




From ravas at outlook.com  Tue May 26 01:49:51 2015
From: ravas at outlook.com (ravas)
Date: Mon, 25 May 2015 22:49:51 -0700 (PDT)
Subject: a more precise distance algorithm
In-Reply-To: <mailman.49.1432617306.5151.python-list@python.org>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
 <5563e453$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <f7bd33f0-f6e0-48cd-a8ea-bdd05fcced2a@googlegroups.com>
 <mailman.49.1432617306.5151.python-list@python.org>
Message-ID: <773bfec3-061b-4443-8b93-53e88b92bc46@googlegroups.com>

On Monday, May 25, 2015 at 10:16:02 PM UTC-7, Gary Herron wrote:
> It's probably not the square root that's causing the inaccuracies. In 
> many other cases, and probably here also, it's the summing of two 
> numbers that have vastly different values that loses precision.  A 
> demonstration:
> 
>  >>> big = 100000000.0
>  >>> small = 0.000000001
>  >>> (big+small)-big # Should produce a value =small, but gives an exact 
> zero instead.
> 0.0
> 
> The squaring of the two values in x*x+y*y just makes the addition even 
> more error prone since the squares make large values even larger and 
> small values even smaller.

I appreciate your help.


From no.email at nospam.invalid  Tue May 26 02:17:38 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Mon, 25 May 2015 23:17:38 -0700
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
Message-ID: <87twuzyi9p.fsf@jester.gateway.sonic.net>

davidfstr at gmail.com writes:
> Has anyone on this list attempted to sandbox Python programs in a
> serious fashion? I'd be interested to hear your approach.

There is something like that for C++ and it is quite complicated:

https://github.com/Eelis/geordi

I expect that for Python you'd have to do most of the same stuff.


From wxjmfauth at gmail.com  Tue May 26 02:34:38 2015
From: wxjmfauth at gmail.com (wxjmfauth at gmail.com)
Date: Mon, 25 May 2015 23:34:38 -0700 (PDT)
Subject: [RELEASED] Python 3.5.0b1 is now available
In-Reply-To: <993f1217-1b59-4de3-a129-30d27670db91@googlegroups.com>
References: <mailman.27.1432510751.5151.python-list@python.org>
 <993f1217-1b59-4de3-a129-30d27670db91@googlegroups.com>
Message-ID: <820f6f0c-80d7-4f8a-9df4-d734093cd55c@googlegroups.com>

Le lundi 25 mai 2015 11:20:27 UTC+2, wxjm... at gmail.com a ?crit?:
> Le lundi 25 mai 2015 01:39:22 UTC+2, Larry Hastings a ?crit?:
> > On behalf of the Python development community and the Python
> >           3.5 release team, I'm pleased to announce the availability of
> >           Python 3.5.0b1.? Python 3.5 has now entered "feature freeze".?
> >           By default new features may no longer be added to Python 3.5.?
> >           (However, there are a handful of features that weren't quite
> >           ready for Python 3.5.0 beta 1; these were granted exceptions
> >           to the freeze, and are scheduled to be added before beta 2.)
> > 
> >           
> > 
> >           This is a preview release, and its use is not recommended for
> >           production settings.
> > 
> >           
> > 
> >           Three important notes for Windows users about Python 3.5.0b1:
> > 
> >           
> >             If you have previously installed Python 3.5.0a1, you may
> >               need to manually uninstall it before installing Python
> >               3.5.0b1 (issue23612).
> >             If installing Python 3.5.0b1 as a non-privileged user,
> >               you may need to escalate to administrator privileges to
> >               install an update to your C runtime libraries.
> >             There is now a third type of Windows build for Python
> >               3.5.? In addition to the conventional installer and the
> >               web-based installer, Python 3.5 now has an embeddable
> >               release designed to be deployed as part of a larger
> >               application's installer for apps using or extending
> >               Python.? During the 3.5 alpha releases, this was an
> >               executable installer; as of 3.5.0 beta 1 the embeddable
> >               build of Python is now shipped in a zip file.
> > 
> >             
> >           
> >           
> > 
> >           You can find Python 3.5.0b1 here:
> > 
> >           https://www.python.org/downloads/release/python-350b1/
> > 
> >           
> >           Happy hacking,
> > 
> >           
> 
> Windows 7 pro
> 
> Installation of "python-3.5.0b1.exe" as administrator: no luck.
> 
> Explaining what's going on behind the scene or explaining why the installer
> is simply silently failing and does no do its job is beyond my knowledge
> and my understanding.

====

I hope the problem will surface.
For me (us), this is just blackbox.
Spending almost a whole day to try to install a piece
of software, ...


From steve+comp.lang.python at pearwood.info  Tue May 26 03:10:30 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 26 May 2015 17:10:30 +1000
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
Message-ID: <55641c67$0$12894$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 26 May 2015 12:24, davidfstr at gmail.com wrote:

> I am writing a web service that accepts Python programs as input, runs the
> provided program with some profiling hooks, and returns various
> information about the program's runtime behavior. To do this in a safe
> manner, I need to be able to create a sandbox that restricts what the
> submitted Python program can do on the web server.
> 
> Almost all discussion about Python sandboxes I have seen on the internet
> involves selectively blacklisting functionality that gives access to
> system resources, such as trying to hide the "open" builtin to restrict
> access to file I/O. All such approaches are doomed to fail because you can
> always find a way around a blacklist.

It's not so much that you can find your way around a blacklist, but that a 
blacklist only bans things which you have thought of. Perhaps the attacker 
has thought of something else.

Ideally, a sandbox will whitelist functions which you know are safe, with a 
"default deny" policy. That requires building your own parser which only 
allows code that passes your whitelist. Even then, the problem is that 
perhaps there is an attack vector you didn't think of: something you thought 
was safe, actually is not.

Have you read Tav's admirable but failed attempt to sandbox file IO?

http://tav.espians.com/a-challenge-to-break-python-security.html

http://tav.espians.com/paving-the-way-to-securing-the-python-
interpreter.html

http://tav.espians.com/update-on-securing-the-python-interpreter.html

Also google for "Capabilities Python" or CapPython.

My sense is that the only way to safely sandbox Python is to create your own 
Python implementation designed with security in mind. You can't get there 
starting from CPython. Maybe Jython?


> For my particular sandbox, I wish to allow *only* the following kinds of
> actions (in a whitelist): * reading from stdin & writing to stdout;
> * reading from files, within a set of whitelisted directories;
> * pure Python computation.

Pure Python computation can be used to DOS your machine, e.g. 
(100**100)**100 will, I think, do it. (I'm not about to try it.)



-- 
Steve



From steve+comp.lang.python at pearwood.info  Tue May 26 03:12:04 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Tue, 26 May 2015 17:12:04 +1000
Subject: Documentaion of dunder methods
References: <5563d7bb$0$12982$c3e8da3$5496439d@news.astraweb.com>
 <mailman.47.1432614940.5151.python-list@python.org>
Message-ID: <55641cc4$0$12894$c3e8da3$5496439d@news.astraweb.com>

On Tuesday 26 May 2015 14:34, Ian Kelly wrote:

>> Apart from PEP 8, is this documented anywhere in the official
>> documentation? If so, I have been unable to find it.
> 
> https://docs.python.org/3/reference/lexical_analysis.html#reserved-
classes-of-identifiers


That's the bunny!

Thanks for that.


-- 
Steve



From breamoreboy at yahoo.co.uk  Tue May 26 03:24:38 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 26 May 2015 08:24:38 +0100
Subject: Documentaion of dunder methods
In-Reply-To: <0c3aeb41-aaf9-4453-9ccc-b2c8891f1394@googlegroups.com>
References: <5563d7bb$0$12982$c3e8da3$5496439d@news.astraweb.com>
 <5563e5e8$0$12983$c3e8da3$5496439d@news.astraweb.com>
 <0c3aeb41-aaf9-4453-9ccc-b2c8891f1394@googlegroups.com>
Message-ID: <mk1741$teh$1@ger.gmane.org>

On 26/05/2015 04:30, Rustom Mody wrote:
> On Tuesday, May 26, 2015 at 8:48:11 AM UTC+5:30, Steven D'Aprano wrote:
>> On Tue, 26 May 2015 12:17 pm, Steven D'Aprano wrote:
>>
>>> In other words, dunder methods are reserved for use by the core developers
>>> for the use of the Python interpreter.
>>
>> Er, that's easy to misinterpret. Let me try rewording:
>>
>> You should not invent new dunder methods.
>>
>> And if possible, you should not call such dunder methods directly. Instead,
>> use the official protocol (e.g. next(x), not x.__next__()) whenever
>> possible. The operator module contains many helpers for that.
>>
>> But of course you may *write* dunder methods.
>
> I guess you and I know what we want to convey and to proscribe.
> However the above doesn't pass muster as documentation
> The most glaring lack in the docs is some reification of the acceptable
> 'protocols'.
> Yeah people like to wax about duck-typing but that translates into semi-formal
> protocols and they are just not documented
>

All you need do is raise an issue on the bug tracker with an attached patch.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From lac at openend.se  Tue May 26 03:53:56 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 26 May 2015 09:53:56 +0200
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: Message from "Steven D'Aprano"
 <steve+comp.lang.python@pearwood.info> of "Tue,
 26 May 2015 17:10:30 +1000."
 <55641c67$0$12894$c3e8da3$5496439d@news.astraweb.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com><55641c67$0$12894$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <201505260753.t4Q7rucM005855@fido.openend.se>

In a message of Tue, 26 May 2015 17:10:30 +1000, "Steven D'Aprano" writes:
>My sense is that the only way to safely sandbox Python is to create your own 
>Python implementation designed with security in mind. You can't get there 
>starting from CPython. Maybe Jython?

You get there starting with PyPy.
see: http://pypy.readthedocs.org/en/latest/sandbox.html

Note: this is not very heavily used.  You may find bugs we don't
know about yet.

Laura



From lac at openend.se  Tue May 26 04:02:03 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 26 May 2015 10:02:03 +0200
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: Message from Laura Creighton <lac@openend.se> of "Tue,
 26 May 2015 09:53:56 +0200." <201505260753.t4Q7rucM005855@fido.openend.se>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com><55641c67$0$12894$c3e8da3$5496439d@news.astraweb.com><201505260753.t4Q7rucM005855@fido.openend.se>
Message-ID: <201505260802.t4Q823xf006234@fido.openend.se>

In a message of Tue, 26 May 2015 09:53:56 +0200, Laura Creighton writes:
>In a message of Tue, 26 May 2015 17:10:30 +1000, "Steven D'Aprano" writes:
>>My sense is that the only way to safely sandbox Python is to create your own 
>>Python implementation designed with security in mind. You can't get there 
>>starting from CPython. Maybe Jython?
>
>You get there starting with PyPy.
>see: http://pypy.readthedocs.org/en/latest/sandbox.html
>
>Note: this is not very heavily used.  You may find bugs we don't
>know about yet.
>
>Laura

Also, the place to discuss this is pypy-dev at python.org or the #pypy
channel on freenode.

Laura



From ned at nedbatchelder.com  Tue May 26 06:21:41 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 26 May 2015 03:21:41 -0700 (PDT)
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
Message-ID: <62c19f45-5b25-4049-83ab-895af36c55c8@googlegroups.com>

On Monday, May 25, 2015 at 10:24:32 PM UTC-4, davi... at gmail.com wrote:
> I am writing a web service that accepts Python programs as input, runs the provided program with some profiling hooks, and returns various information about the program's runtime behavior. To do this in a safe manner, I need to be able to create a sandbox that restricts what the submitted Python program can do on the web server.
> 
> Almost all discussion about Python sandboxes I have seen on the internet involves selectively blacklisting functionality that gives access to system resources, such as trying to hide the "open" builtin to restrict access to file I/O. All such approaches are doomed to fail because you can always find a way around a blacklist.
> 
> For my particular sandbox, I wish to allow *only* the following kinds of actions (in a whitelist):
> * reading from stdin & writing to stdout;
> * reading from files, within a set of whitelisted directories;
> * pure Python computation.
> 
> In particular all other operations available through system calls are banned. This includes, but is not limited to:
> * writing to files;
> * manipulating network sockets;
> * communicating with other processes.
> 
> I believe it is not possible to limit such operations at the Python level. The best you could do is try replacing all the standard library modules, but that is again just a blacklist - it won't prevent a determined attacker from doing things like constructing their own 'code' object and executing it.
> 
> It might be necessary to isolate the Python process at the operating system level.
> * A chroot jail on Linux & OS X can limit access to the filesystem. Again this is just a blacklist.
> * No obvious way to block socket creation. Again this would be just a blacklist.
> * No obvious way to detect unapproved system calls and block them.
> 
> In the limit, I could dynamically spin up a virtual machine and execute the Python program in the machine. However that's extremely expensive in computational time.
> 
> Has anyone on this list attempted to sandbox Python programs in a serious fashion? I'd be interested to hear your approach.
> 
> - David

When we needed this at edX, we wrote CodeJail (https://github.com/edx/codejail).
It's a wrapper around AppArmor to provide OS-level protection of code
execution in subprocesses.  It has Python-specific features, but because it
is based on AppArmor, can sandbox any process, so long as it's configured
properly.

--Ned.


From marco.nawijn at colosso.nl  Tue May 26 08:01:38 2015
From: marco.nawijn at colosso.nl (marco.nawijn at colosso.nl)
Date: Tue, 26 May 2015 05:01:38 -0700 (PDT)
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
Message-ID: <66e5667e-c368-4ce3-86e1-0987365af5eb@googlegroups.com>

On Tuesday, May 26, 2015 at 4:24:32 AM UTC+2, davi... at gmail.com wrote:
> I am writing a web service that accepts Python programs as input, runs the provided program with some profiling hooks, and returns various information about the program's runtime behavior. To do this in a safe manner, I need to be able to create a sandbox that restricts what the submitted Python program can do on the web server.
> 
> Almost all discussion about Python sandboxes I have seen on the internet involves selectively blacklisting functionality that gives access to system resources, such as trying to hide the "open" builtin to restrict access to file I/O. All such approaches are doomed to fail because you can always find a way around a blacklist.
> 
> For my particular sandbox, I wish to allow *only* the following kinds of actions (in a whitelist):
> * reading from stdin & writing to stdout;
> * reading from files, within a set of whitelisted directories;
> * pure Python computation.
> 
> In particular all other operations available through system calls are banned. This includes, but is not limited to:
> * writing to files;
> * manipulating network sockets;
> * communicating with other processes.
> 
> I believe it is not possible to limit such operations at the Python level. The best you could do is try replacing all the standard library modules, but that is again just a blacklist - it won't prevent a determined attacker from doing things like constructing their own 'code' object and executing it.
> 
> It might be necessary to isolate the Python process at the operating system level.
> * A chroot jail on Linux & OS X can limit access to the filesystem. Again this is just a blacklist.
> * No obvious way to block socket creation. Again this would be just a blacklist.
> * No obvious way to detect unapproved system calls and block them.
> 
> In the limit, I could dynamically spin up a virtual machine and execute the Python program in the machine. However that's extremely expensive in computational time.
> 
> Has anyone on this list attempted to sandbox Python programs in a serious fashion? I'd be interested to hear your approach.
> 
> - David

What about launching the Python process in a Docker container?
Spinning up a new container is pretty quick and it might provide
you with enough isolation. Probably not a perfect solution, but
I do believe that it would be easier than trying to sandbox Python
itself.

Marco


From siv.devops at gmail.com  Tue May 26 08:57:19 2015
From: siv.devops at gmail.com (pra devOPS)
Date: Tue, 26 May 2015 05:57:19 -0700
Subject: Json Comaprision
In-Reply-To: <midfe5$ma7$1@reader1.panix.com>
References: <mailman.164.1430902487.12865.python-list@python.org>
 <midfe5$ma7$1@reader1.panix.com>
Message-ID: <CANvYX9WQrWwN3AnSu2xnO+g363L1_UHrtdpjOcNZGLdRBV8gAQ@mail.gmail.com>

Hey all thanks for the answers,

I am done wiht the same .

Sorry fore delayed replied

On Wed, May 6, 2015 at 9:27 AM, John Gordon <gordon at panix.com> wrote:

> In <mailman.164.1430902487.12865.python-list at python.org> pra devOPS <
> siv.devops at gmail.com> writes:
>
> > I wanted to compare two json files ignoring few of the keys in the json
> > files.
>
> > Can anybody suggest me few things?
>
> Load each json file into a python object, delete the keys you don't care
> about, and compare the two objects.
>
> --
> John Gordon         Imagine what it must be like for a real medical doctor
> to
> gordon at panix.com    watch 'House', or a real serial killer to watch
> 'Dexter'.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150526/c3a8af0f/attachment.html>

From markos at c2o.pro.br  Tue May 26 09:24:30 2015
From: markos at c2o.pro.br (Markos)
Date: Tue, 26 May 2015 10:24:30 -0300
Subject: OpenCV with Python (cv or cv2)
Message-ID: <5564740E.3090206@c2o.pro.br>

Hi,

I want to use OpenCV with Python.

I installed version 2.4.9 
(http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.4.9/opencv-2.4.9.zip/) 
in debian Squeeze running Python 2.6.6. Using the tutorial: 
http://indranilsinharoy.com/2012/11/01/installing-opencv-on-linux/

(I tried to install versions 2.4.11 and 2.4.10 (in Squeeze with python 
2.2.6) but I couldn't.)

I lost some time trying to load the module cv2:

> >> Import cv2.cv the cv
Traceback (most recent call last):
File "<stdin>", line 1, in <module >>
ImportError: No module named cv2.cv

After a while I realized I could use the cv module. (import cv)

But still I do not understand the differences between the modules cv and 
cv2.

Do you suggest any good tutorial about the differences between cv and cv2?

I will have to make many changes in python programs using cv to reuse 
the code later with cv2?

Thanks,
Markos


From random832 at fastmail.us  Tue May 26 09:40:53 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 26 May 2015 09:40:53 -0400
Subject: a more precise distance algorithm
In-Reply-To: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
Message-ID: <1432647653.1149540.278304225.25D9F2E4@webmail.messagingengine.com>

On Mon, May 25, 2015, at 15:21, ravas wrote:
> Is this valid? Does it apply to python?
> Any other thoughts? :D

The math.hypot function uses the C library's function which should deal
with such concerns internally. There is a fallback version in case the C
library does not have this function, in Python/pymath.c - which,
incidentally, does use your algorithm.


From random832 at fastmail.us  Tue May 26 09:51:57 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 26 May 2015 09:51:57 -0400
Subject: a more precise distance algorithm
In-Reply-To: <1432647653.1149540.278304225.25D9F2E4@webmail.messagingengine.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
 <1432647653.1149540.278304225.25D9F2E4@webmail.messagingengine.com>
Message-ID: <1432648317.1154051.278328873.25A47810@webmail.messagingengine.com>

On Tue, May 26, 2015, at 09:40, random832 at fastmail.us wrote:
> On Mon, May 25, 2015, at 15:21, ravas wrote:
> > Is this valid? Does it apply to python?
> > Any other thoughts? :D
> 
> The math.hypot function uses the C library's function which should deal
> with such concerns internally. There is a fallback version in case the C
> library does not have this function, in Python/pymath.c - which,
> incidentally, does use your algorithm.

Well, I should say, not _precisely_ your algorithm.

The "0 special case" mentioned in the text you read was for both values
being zero, not just one. The biggest flaw in your function, though, was
the failure to take the absolute values of the differences. This defeats
the point of swapping them (which I assume is to get the magnitudes in
the order needed for best precision), and makes it possible for your
function to return a negative value when the other is zero.

Here's the equivalent python code for the hypot function in pymath.c,
and for your distance function.

from math import sqrt

def hypot(x, y):
    x = abs(x)
    y = abs(y)
    if x < y:
        x, y = y, x
    if x == 0:  # both are 0 due to the swap
        return 0.0
    else:
        return x*sqrt(1.0 + (y/x)**2)

def distance(A, B):
    return hypot(A.x-B.x, A.y-B.y)

What I wonder is if there's a best way to do it for three dimensions. I
mean, you could simply do hypot(hypot(dx, dy), dz), but should you
choose the largest, smallest, or middle value to be the odd one out?


From lac at openend.se  Tue May 26 10:21:11 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 26 May 2015 16:21:11 +0200
Subject: OpenCV with Python (cv or cv2)
In-Reply-To: Message from Markos <markos@c2o.pro.br>
 of "Tue, 26 May 2015 10:24:30 -0300." <5564740E.3090206@c2o.pro.br>
References: <5564740E.3090206@c2o.pro.br>
Message-ID: <201505261421.t4QELBUd015477@fido.openend.se>

In a message of Tue, 26 May 2015 10:24:30 -0300, Markos writes:
>Hi,
>
>I want to use OpenCV with Python.
>
>I installed version 2.4.9 
>(http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.4.9/opencv-2.4.9.zip/) 
>in debian Squeeze running Python 2.6.6. Using the tutorial: 
>http://indranilsinharoy.com/2012/11/01/installing-opencv-on-linux/
>
>(I tried to install versions 2.4.11 and 2.4.10 (in Squeeze with python 
>2.2.6) but I couldn't.)

You mean python 2.6.6, right?
and this package: https://packages.debian.org/source/squeeze/opencv
which is 2.1.0-3 is too old for you?

What do you mean that you
tried to install versions 2.4.11 and 2.4.10 but couldn't?  What
error messages did you get?

>I lost some time trying to load the module cv2:
>
>> >> Import cv2.cv the cv

s/the/as/ ?
 
>Traceback (most recent call last):
>File "<stdin>", line 1, in <module >>
>ImportError: No module named cv2.cv
>
>After a while I realized I could use the cv module. (import cv)
>
>But still I do not understand the differences between the modules cv and 
>cv2.
>
>Do you suggest any good tutorial about the differences between cv and cv2?
>
>I will have to make many changes in python programs using cv to reuse 
>the code later with cv2?
>
>Thanks,
>Markos
>-- 
>https://mail.python.org/mailman/listinfo/python-list

post the output you got from cmake

It will probably complain that it is missing some headers, so couldn't
make libcv2.so

If you haven't installed python-dev (not just python) and numpy I
guarantee it will not find needed headers; if you have but it is
still not being made we will have to look harder at what is going wrong.

If it made a libcv.so for you, did it put it in a place you can see
with your PYTHONPATH?

Laura




From dreamingforward at gmail.com  Tue May 26 10:57:16 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 26 May 2015 07:57:16 -0700 (PDT)
Subject: different types of inheritence...
In-Reply-To: <544f53cb-e404-41a4-aa2e-0db27c845546@googlegroups.com>
References: <544f53cb-e404-41a4-aa2e-0db27c845546@googlegroups.com>
Message-ID: <c6b24dce-c0da-4753-9775-b38816b579c8@googlegroups.com>

> Apart from object composition or mix-in style, I want to illustrate something regarding the "arrow" of inheritance.
> 
> class super_dict(dict):
> 
>     def __init__(self, init={}, default_value=0, collision_function=None):
>        *expands what dict can do*
> 
>     def get_default(self):  #stupid method to illustrate a point
>        return self._default_value
> 
> class specialized_dict(dict):
> 
>     def update(self, other):
>         *change the behavior of how updates work*
> 
>     def setdefault(self, key, value):
>         if key=sentinel:
>             self[key]=0
>         else:
>             self[key]=value

Okay, perhaps I wasn't being clear.... 

THESE OBJECTS HAVE THE SAME CLASS *SYNTAX*, BUT COMPLETELY DIFFERENT *SEMANTICS*.

Comprende?  I'm not trying to be cryptic here.  This is a bit of OOP theory to be discussed.

Mark





From PointedEars at web.de  Tue May 26 11:46:43 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Tue, 26 May 2015 17:46:43 +0200
Subject: OpenCV with Python (cv or cv2)
References: <mailman.55.1432648308.5151.python-list@python.org>
Message-ID: <1510511.19yO2C5bcj@PointedEars.de>

Markos wrote:
^^^^^^
Please append your last name.

> I want to use OpenCV with Python.
> 
> I installed version 2.4.9
> (http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.4.9/opencv-2.4.9.zip/)
> in debian Squeeze running Python 2.6.6. Using the tutorial:
> http://indranilsinharoy.com/2012/11/01/installing-opencv-on-linux/

Do not do that.  Install the distribution packages instead.

> (I tried to install versions 2.4.11 and 2.4.10 (in Squeeze with python
> 2.2.6) but I couldn't.)

I have stretch/sid with python-opencv=2.4.9+dsfg-1.1 installed.

squeeze has python-opencv=2.1.0-3+squeeze1.  squeeze is *oldoldstable* now.  
You want to upgrade NOW.

<https://packages.debian.org/search?suite=squeeze&searchon=names&keywords=python-opencv>

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From al.basili at gmail.com  Tue May 26 11:48:18 2015
From: al.basili at gmail.com (alb)
Date: 26 May 2015 15:48:18 GMT
Subject: SyntaxError on progress module
Message-ID: <csjitvF6cptU1@mid.individual.net>

Hi everyone,

I've installed the 'progress' module (ver 1.2) and I have the following 
error when used:

  File "/home/debian/repos/2418_IASI-NG/Documents/Tools/tex_tool/venv/local/lib/python3.2/site-packages/progress/bar.py", line 48
    empty_fill = u'?'
                      ^
SyntaxError: invalid syntax

I believe I have some problems with unicode handling but it's just a 
rough guess.

I'm running in a virtual environment with python3.2 with the following 
configuration:

(venv)debian at debian:tex_tool$ pip list
pip (7.0.1)
progress (1.2)
pypandoc (0.9.7)
setuptools (15.0)

Any suggestions/comments/pointer is appreciated.

Al

-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?


From breamoreboy at yahoo.co.uk  Tue May 26 11:59:01 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 26 May 2015 16:59:01 +0100
Subject: SyntaxError on progress module
In-Reply-To: <csjitvF6cptU1@mid.individual.net>
References: <csjitvF6cptU1@mid.individual.net>
Message-ID: <mk2586$d30$1@ger.gmane.org>

On 26/05/2015 16:48, alb wrote:
> Hi everyone,
>
> I've installed the 'progress' module (ver 1.2) and I have the following
> error when used:
>
>    File "/home/debian/repos/2418_IASI-NG/Documents/Tools/tex_tool/venv/local/lib/python3.2/site-packages/progress/bar.py", line 48
>      empty_fill = u'?'
>                        ^
> SyntaxError: invalid syntax
>
> I believe I have some problems with unicode handling but it's just a
> rough guess.
>
> I'm running in a virtual environment with python3.2 with the following
> configuration:
>
> (venv)debian at debian:tex_tool$ pip list
> pip (7.0.1)
> progress (1.2)
> pypandoc (0.9.7)
> setuptools (15.0)
>
> Any suggestions/comments/pointer is appreciated.
>
> Al
>

Python 3.0 removed the 'u' for unicode in front of strings but due to 
popular demand to ease porting it was reinstated in 3.3.  Strip it away 
and you should be fine to go.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Tue May 26 12:27:01 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 27 May 2015 02:27:01 +1000
Subject: SyntaxError on progress module
In-Reply-To: <mk2586$d30$1@ger.gmane.org>
References: <csjitvF6cptU1@mid.individual.net>
	<mk2586$d30$1@ger.gmane.org>
Message-ID: <CAPTjJmrrEyccHaX9zDi3Fe5i7wSeo8t6mh1745-jAG4SsyYpRQ@mail.gmail.com>

On Wed, May 27, 2015 at 1:59 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> On 26/05/2015 16:48, alb wrote:
>>
>> Hi everyone,
>>
>> I've installed the 'progress' module (ver 1.2) and I have the following
>> error when used:
>>
>>    File
>> "/home/debian/repos/2418_IASI-NG/Documents/Tools/tex_tool/venv/local/lib/python3.2/site-packages/progress/bar.py",
>> line 48
>>      empty_fill = u'?'
>>                        ^
>> SyntaxError: invalid syntax
>>
>> I'm running in a virtual environment with python3.2
>
> Python 3.0 removed the 'u' for unicode in front of strings but due to
> popular demand to ease porting it was reinstated in 3.3.  Strip it away and
> you should be fine to go.

Or upgrade to 3.3 or better; is there anything holding you on 3.2?
Building CPython from source is pretty easy on Debian, and of course
upgrading to Jessie will correspondingly upgrade you to a more recent
Python (3.4, to be precise).

ChrisA


From dreamingforward at gmail.com  Tue May 26 12:37:29 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 26 May 2015 09:37:29 -0700 (PDT)
Subject: should "self" be changed?
Message-ID: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>

Would it be prudent to rid the long-standing "argument" (pun unintended) about self and the ulterior spellings of it, by changing it into a symbol rather than a name?  

Something like:

class MyClass(object):

    def __init__(@):
        @.dummy = None

OR, even better, getting *rid of it* in the parameter list, so it stops confusing people about how many parameters a method needs, and transform it into a true *operator*.

class MyClass(object):

    def __init__(): #takes no arguments!
        @.dummy = None  #the @ invokes the class object's dictionary

That would seem to be a nice solution to the problem, really.  It doesn't become PERLish because you've made it into a genuine operator -- "self" was always a non-variable that looked like a variable and hence created an itch that couldn't be scratched.

Anyone else have any thoughts?

--mark


From lac at openend.se  Tue May 26 12:57:19 2015
From: lac at openend.se (Laura Creighton)
Date: Tue, 26 May 2015 18:57:19 +0200
Subject: should "self" be changed?
In-Reply-To: Message from zipher <dreamingforward@gmail.com> of "Tue,
 26 May 2015 09:37:29 -0700."
 <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
Message-ID: <201505261657.t4QGvJYW019538@fido.openend.se>

In a message of Tue, 26 May 2015 09:37:29 -0700, zipher writes:
>Would it be prudent to rid the long-standing "argument" (pun unintended) about self and the ulterior spellings of it, by changing it into a symbol rather than a name?  
>
>Something like:
>
>class MyClass(object):
>
>    def __init__(@):
>        @.dummy = None
>
>OR, even better, getting *rid of it* in the parameter list, so it stops confusing people about how many parameters a method needs, and transform it into a true *operator*.
>
>class MyClass(object):
>
>    def __init__(): #takes no arguments!
>        @.dummy = None  #the @ invokes the class object's dictionary
>
>That would seem to be a nice solution to the problem, really.  It doesn't become PERLish because you've made it into a genuine operator -- "self" was always a non-variable that looked like a variable and hence created an itch that couldn't be scratched.
>
>Anyone else have any thoughts?

Guido did.  :)
http://neopythonic.blogspot.se/2008/10/why-explicit-self-has-to-stay.html

Laura


From laurent.pointal at free.fr  Tue May 26 12:59:47 2015
From: laurent.pointal at free.fr (Laurent Pointal)
Date: Tue, 26 May 2015 18:59:47 +0200
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
Message-ID: <5564a683$0$3027$426a74cc@news.free.fr>

zipher wrote:

> Would it be prudent to rid the long-standing "argument" (pun unintended)
> about self and the ulterior spellings of it, by changing it into a symbol
> rather than a name?
> 
> Something like:
> 
> class MyClass(object):
> 
>     def __init__(@):
>         @.dummy = None

Just seeing the Python3.5 announce with @ used as matrix multiplication 
operator ;-)

> OR, even better, getting *rid of it* in the parameter list, so it stops
> confusing people about how many parameters a method needs, and transform
> it into a true *operator*.

Self is not an operator, its the target object as first parameter. And with 
Python you can call some methods directly using theclass.themethod(objet, 
param1, param2).

> class MyClass(object):
> 
>     def __init__(): #takes no arguments!
>         @.dummy = None  #the @ invokes the class object's dictionary
> 
> That would seem to be a nice solution to the problem, really.  It doesn't
> become PERLish because you've made it into a genuine operator -- "self"
> was always a non-variable that looked like a variable and hence created an
> itch that couldn't be scratched.

?Explicit is better than implicit?

I think that googling for that idea you will find other people already 
proposing it (I've seen propositions to directly remove part before the dot 
like this: .dummy = None). Just read it.

> Anyone else have any thoughts?

IMHO Zero chance that it be adopted.

A+
Laurent.



From rustompmody at gmail.com  Tue May 26 13:02:58 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Tue, 26 May 2015 10:02:58 -0700 (PDT)
Subject: OpenCV with Python (cv or cv2)
In-Reply-To: <1510511.19yO2C5bcj@PointedEars.de>
References: <mailman.55.1432648308.5151.python-list@python.org>
 <1510511.19yO2C5bcj@PointedEars.de>
Message-ID: <294d27bb-1d61-4e66-a357-61bab4056a74@googlegroups.com>

On Tuesday, May 26, 2015 at 9:19:35 PM UTC+5:30, Thomas 'PointedEars' Lahn wrote:
> Markos wrote:
> ^^^^^^
> Please append your last name.

No more required than an affidavit proving PointedEars is your legal name


From breamoreboy at yahoo.co.uk  Tue May 26 13:28:11 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 26 May 2015 18:28:11 +0100
Subject: should "self" be changed?
In-Reply-To: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
Message-ID: <mk2afb$ak3$1@ger.gmane.org>

On 26/05/2015 17:37, zipher wrote:
> Would it be prudent to rid the long-standing "argument" (pun unintended) about self and the ulterior spellings of it, by changing it into a symbol rather than a name?
>
> Something like:
>
> class MyClass(object):
>
>      def __init__(@):
>          @.dummy = None
>
> OR, even better, getting *rid of it* in the parameter list, so it stops confusing people about how many parameters a method needs, and transform it into a true *operator*.
>
> class MyClass(object):
>
>      def __init__(): #takes no arguments!
>          @.dummy = None  #the @ invokes the class object's dictionary
>
> That would seem to be a nice solution to the problem, really.  It doesn't become PERLish because you've made it into a genuine operator -- "self" was always a non-variable that looked like a variable and hence created an itch that couldn't be scratched.
>
> Anyone else have any thoughts?
>
> --mark
>

Yes, how about you taking a permanent holiday rather than bother this 
list with more of your nonsense?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From rosuav at gmail.com  Tue May 26 13:31:41 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 27 May 2015 03:31:41 +1000
Subject: should "self" be changed?
In-Reply-To: <mk2afb$ak3$1@ger.gmane.org>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mk2afb$ak3$1@ger.gmane.org>
Message-ID: <CAPTjJmrbeiyfD-Su9iAR3jsQ9-jKmviDOmxBQGb0vcKexnWOOw@mail.gmail.com>

On Wed, May 27, 2015 at 3:28 AM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> Yes, how about you taking a permanent holiday rather than bother this list
> with more of your nonsense?

No need to be nasty about it. The suggestion is a plausible one, it
just happens to not fit Python's philosophy. The best response is to
quote Guido's statement on the subject... which is exactly what Laura
already did. :)

ChrisA


From random832 at fastmail.us  Tue May 26 14:11:42 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Tue, 26 May 2015 14:11:42 -0400
Subject: should "self" be changed?
In-Reply-To: <201505261657.t4QGvJYW019538@fido.openend.se>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <201505261657.t4QGvJYW019538@fido.openend.se>
Message-ID: <1432663902.1224464.278568529.1F1A4275@webmail.messagingengine.com>

On Tue, May 26, 2015, at 12:57, Laura Creighton wrote:
> Guido did.  :)
> http://neopythonic.blogspot.se/2008/10/why-explicit-self-has-to-stay.html

It's worth noting that the "dynamically modify a class" argument (and to
some extent the decorator argument) misses Javascript's solution - _any_
function may refer to "this" (which is not in the argument list), which
will be the global scope object (the browser window for browser-hosted
javascript - presumably the current module for a hypothetical equivalent
python feature, though it might be more prudent to simply make it None.)
if the function is called without an object reference. Of course,
Javascript also lacks bound methods, which makes it much more likely to
happen by accident.

I can't really think of anything that you can do with decorators,
either, in the current model, that you _couldn't_ do in a JS-alike
function call model... but I doubt it would be possible to implement
backwards-compatibly.

In principle, if you added a class keyword (hey, technically, isn't
there one already?) you wouldn't need decorators at all for the
staticmethod/classmethod/instance method case.


From marko at pacujo.net  Tue May 26 15:46:57 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 26 May 2015 22:46:57 +0300
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
Message-ID: <877frvf7f2.fsf@elektro.pacujo.net>

zipher <dreamingforward at gmail.com>:

> Would it be prudent to rid the long-standing "argument" (pun
> unintended) about self and the ulterior spellings of it, by changing
> it into a symbol rather than a name?
>
> Something like:
>
> class MyClass(object):
>
>     def __init__(@):
>         @.dummy = None
>
> OR, even better, getting *rid of it* in the parameter list, so it
> stops confusing people about how many parameters a method needs, and
> transform it into a true *operator*.

Python's practice works. However, a small problem is presented by nested
classes:

    class Connection:
        def __init__(self):
            class Idle:
                def signal_start(self):
                    # how to refer to the outer self
                    :
            :

Solutions include:

    class Connection:
        def __init__(self):
            class Idle:
                def signal_start(_):
                    self.set_state(Ready)
            :


    class Connection:
        def __init__(self):
            conn = self
            class Idle:
                def signal_start(self):
                    conn.set_state(Ready)
            :


I have used the latter method recently.


Marko


From ned at nedbatchelder.com  Tue May 26 16:04:14 2015
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 26 May 2015 13:04:14 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <877frvf7f2.fsf@elektro.pacujo.net>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
Message-ID: <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>

On Tuesday, May 26, 2015 at 3:47:20 PM UTC-4, Marko Rauhamaa wrote:
> zipher <dreamingforward at gmail.com>:
> 
> > Would it be prudent to rid the long-standing "argument" (pun
> > unintended) about self and the ulterior spellings of it, by changing
> > it into a symbol rather than a name?
> >
> > Something like:
> >
> > class MyClass(object):
> >
> >     def __init__(@):
> >         @.dummy = None
> >
> > OR, even better, getting *rid of it* in the parameter list, so it
> > stops confusing people about how many parameters a method needs, and
> > transform it into a true *operator*.
> 
> Python's practice works. However, a small problem is presented by nested
> classes:
> 
>     class Connection:
>         def __init__(self):
>             class Idle:
>                 def signal_start(self):
>                     # how to refer to the outer self
>                     :
>             :
> 
> Solutions include:
> 
> ...
>
>     class Connection:
>         def __init__(self):
>             conn = self
>             class Idle:
>                 def signal_start(self):
>                     conn.set_state(Ready)
>             :
> 
> 
> I have used the latter method recently.

I would find it much clearer to not use a nested class
at all, and instead to pass the object into the constructor:

    class Idle:
        def __init__(self, conn):
            self.conn = conn
        def signal_start(self):
            self.conn.set_state(Ready)

    class Connection:
        def __init__(self):
            something(Idle(self))

--Ned.


From garabik-news-2005-05 at kassiopeia.juls.savba.sk  Tue May 26 16:26:22 2015
From: garabik-news-2005-05 at kassiopeia.juls.savba.sk (garabik-news-2005-05 at kassiopeia.juls.savba.sk)
Date: Tue, 26 May 2015 20:26:22 +0000 (UTC)
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
Message-ID: <mk2kte$opk$2@speranza.aioe.org>

zipher <dreamingforward at gmail.com> wrote:
> Would it be prudent to rid the long-standing "argument" (pun
> unintended) about self and the ulterior spellings of it, by changing
> it into a symbol rather than a name?  
> 
> Something like:
> 
> class MyClass(object):
> 
>     def __init__(@):
>         @.dummy = None

Believe or not, python3 (via Guido's time machine) already anticipated 
this suggestion and you can indeed use a symbol instead of 'self':

class MyClass(object):

   def __init__(?):
       ?.dummy = None

-- 
 -----------------------------------------------------------
| Radovan Garab?k http://kassiopeia.juls.savba.sk/~garabik/ |
| __..--^^^--..__    garabik @ kassiopeia.juls.savba.sk     |
 -----------------------------------------------------------
Antivirus alert: file .signature infected by signature virus.
Hi! I'm a signature virus! Copy me into your signature file to help me spread!


From marko at pacujo.net  Tue May 26 16:36:37 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Tue, 26 May 2015 23:36:37 +0300
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
Message-ID: <87r3q3dqju.fsf@elektro.pacujo.net>

Ned Batchelder <ned at nedbatchelder.com>:

> I would find it much clearer to not use a nested class at all, and
> instead to pass the object into the constructor:

Nested classes are excellent and expressing the state pattern <URL:
http://en.wikipedia.org/wiki/State_pattern>.


Marko


From breamoreboy at yahoo.co.uk  Tue May 26 16:45:36 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 26 May 2015 21:45:36 +0100
Subject: should "self" be changed?
In-Reply-To: <mk2kte$opk$2@speranza.aioe.org>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mk2kte$opk$2@speranza.aioe.org>
Message-ID: <mk2m1h$pti$1@ger.gmane.org>

On 26/05/2015 21:26, garabik-news-2005-05 at kassiopeia.juls.savba.sk wrote:
> zipher <dreamingforward at gmail.com> wrote:
>> Would it be prudent to rid the long-standing "argument" (pun
>> unintended) about self and the ulterior spellings of it, by changing
>> it into a symbol rather than a name?
>>
>> Something like:
>>
>> class MyClass(object):
>>
>>      def __init__(@):
>>          @.dummy = None
>
> Believe or not, python3 (via Guido's time machine) already anticipated
> this suggestion and you can indeed use a symbol instead of 'self':
>
> class MyClass(object):
>
>     def __init__(?):
>         ?.dummy = None
>

Apart from breaking all the tools that rely on "self" being spelt "self" 
this looks like an excellent idea.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From torriem at gmail.com  Tue May 26 16:52:56 2015
From: torriem at gmail.com (Michael Torrie)
Date: Tue, 26 May 2015 14:52:56 -0600
Subject: different types of inheritence...
In-Reply-To: <c6b24dce-c0da-4753-9775-b38816b579c8@googlegroups.com>
References: <544f53cb-e404-41a4-aa2e-0db27c845546@googlegroups.com>
 <c6b24dce-c0da-4753-9775-b38816b579c8@googlegroups.com>
Message-ID: <5564DD28.6010406@gmail.com>

On 05/26/2015 08:57 AM, zipher wrote:
> Comprende?  I'm not trying to be cryptic here.  This is a bit of OOP
> theory to be discussed.

No, sorry.  Maybe an actual example (with use case) would spur discussion.


From python.list at tim.thechases.com  Tue May 26 17:10:42 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Tue, 26 May 2015 16:10:42 -0500
Subject: should "self" be changed?
In-Reply-To: <mk2m1h$pti$1@ger.gmane.org>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mk2kte$opk$2@speranza.aioe.org> <mk2m1h$pti$1@ger.gmane.org>
Message-ID: <20150526161042.51d86097@bigbox.christie.dr>

On 2015-05-26 21:45, Mark Lawrence wrote:
>> class MyClass(object):
>>     def __init__(?):
>>         ?.dummy = None
> 
> Apart from breaking all the tools that rely on "self" being spelt
> "self" this looks like an excellent idea.

Though to be fair, they *are* broken tools if they rely on "self"
since there's nothing in the Python specs that require that.  It's
just a convention[1].

-tkc


[1]
https://docs.python.org/2/tutorial/classes.html#random-remarks
"""
Often, the first argument of a method is called self. This is nothing
more than a convention: the name self has absolutely no special
meaning to Python. Note, however, that by not following the
convention your code may be less readable to other Python
programmers, and it is also conceivable that a class browser program
might be written that relies upon such a convention.
"""
It does give fair warning, but I'd consider that a warning to authors
of "class browser program[s]" as much as to developers.





From ian.g.kelly at gmail.com  Tue May 26 17:12:26 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 26 May 2015 15:12:26 -0600
Subject: different types of inheritence...
In-Reply-To: <5564DD28.6010406@gmail.com>
References: <544f53cb-e404-41a4-aa2e-0db27c845546@googlegroups.com>
 <c6b24dce-c0da-4753-9775-b38816b579c8@googlegroups.com>
 <5564DD28.6010406@gmail.com>
Message-ID: <CALwzidkGLLaqhJ9DpY=LJJPRKRzKcgzjGWnCvQa4NxrEq4KH9Q@mail.gmail.com>

On Tue, May 26, 2015 at 2:52 PM, Michael Torrie <torriem at gmail.com> wrote:
> On 05/26/2015 08:57 AM, zipher wrote:
>> Comprende?  I'm not trying to be cryptic here.  This is a bit of OOP
>> theory to be discussed.
>
> No, sorry.  Maybe an actual example (with use case) would spur discussion.

Better yet, ignore the troll.


From vito.detullio at gmail.com  Tue May 26 17:17:03 2015
From: vito.detullio at gmail.com (Vito De Tullio)
Date: Tue, 26 May 2015 23:17:03 +0200
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mk2kte$opk$2@speranza.aioe.org> <mk2m1h$pti$1@ger.gmane.org>
Message-ID: <mk2nrv$elh$1@ger.gmane.org>

Mark Lawrence wrote:

>>     def __init__(?):
>>         ?.dummy = None

> Apart from breaking all the tools that rely on "self" being spelt "self"
> this looks like an excellent idea.

too bad for the tools: using the name "self" is just a convention, not a 
rule.

-- 
By ZeD



From PointedEars at web.de  Tue May 26 18:17:08 2015
From: PointedEars at web.de (Thomas 'PointedEars' Lahn)
Date: Wed, 27 May 2015 00:17:08 +0200
Subject: OpenCV with Python (cv or cv2)
References: <mailman.55.1432648308.5151.python-list@python.org>
 <1510511.19yO2C5bcj@PointedEars.de>
 <294d27bb-1d61-4e66-a357-61bab4056a74@googlegroups.com>
Message-ID: <29851508.nIeS5LJI7T@PointedEars.de>

Rustom Mody wrote:

> On Tuesday, May 26, 2015 at 9:19:35 PM UTC+5:30, Thomas 'PointedEars' Lahn
> wrote:
>> Markos wrote:
>> ^^^^^^
>> Please append your last name.
> 
> No more required than an affidavit proving PointedEars is your legal name

In addition to this utter nonsense, do you have anything constructive to say 
about the rest of my posting?  If not, troll elsewhere.

-- 
PointedEars

Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.


From rriehle at itu.edu  Tue May 26 20:43:31 2015
From: rriehle at itu.edu (richard_riehle)
Date: Tue, 26 May 2015 19:43:31 -0500
Subject: Array of Functions
References: <42b534f0-aeef-411f-80c5-db5d20c80b55@googlegroups.com>
Message-ID: <TbKdnVQBR8OujvjInZ2dnUU7-LOdnZ2d@giganews.com>

I realized that I mentioned earlier that I found a solution to my original question, but that I never posted an example of the solution.   So, here is a simplified example for anyone who is interested.

def fArray(fselect, fparm  = 1):
	def A1(p = fparm):
		if p == 1:
			print("printing A1[1]")
		else:
			print("printing A1[other]")
	def A2(p = fparm):
		if p == 1:
			print("printing A2[1]")
		else:
			print("printing A2[other]")
	A = [A1, A2]
	A[fselect]()

In this example, I enclosed two functions within another function, and then put those two functions in a list.  Then, with appropriate parameters, I called one of the functions in the list, associated the formal parameter with the function in a call to the array, and presto, it performs the function.

The more advanced problem I wanted to solve, a two dimensional array of functions, once this example is understood, becomes trivial to implement.  In fact, the more interesting problem I wanted to solve involved a dictionary of functions in a two-dimensional array, and that too was easy to do in Python.  

When I compare what this would require in C, C++, Java, or most other languages, I find Python to be really easy for solving this kind of problem.

Next, I plan to develop the solution using decorators and assertions to empower it with a greater level of portability and to make the functions more generic.

I hope someone finds this interesting.

Richard Riehle, PhD




From lac at openend.se  Tue May 26 21:05:30 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 27 May 2015 03:05:30 +0200
Subject: Array of Functions
In-Reply-To: Message from richard_riehle <rriehle@itu.edu> of "Tue,
 26 May 2015 19:43:31 -0500." <TbKdnVQBR8OujvjInZ2dnUU7-LOdnZ2d@giganews.com>
References: <42b534f0-aeef-411f-80c5-db5d20c80b55@googlegroups.com><TbKdnVQBR8OujvjInZ2dnUU7-LOdnZ2d@giganews.com>
Message-ID: <201505270105.t4R15U3I030865@fido.openend.se>

In a message of Tue, 26 May 2015 19:43:31 -0500, richard_riehle writes:
>I realized that I mentioned earlier that I found a solution to my original question, but that I never posted an example of the solution.   So, here is a simplified example for anyone who is interested.
>
>def fArray(fselect, fparm  = 1):
>	def A1(p = fparm):
>		if p == 1:
>			print("printing A1[1]")
>		else:
>			print("printing A1[other]")
>	def A2(p = fparm):
>		if p == 1:
>			print("printing A2[1]")
>		else:
>			print("printing A2[other]")
>	A = [A1, A2]
>	A[fselect]()
>
>In this example, I enclosed two functions within another function, and then put those two functions in a list.  Then, with appropriate parameters, I called one of the functions in the list, associated the formal parameter with the function in a call to the array, and presto, it performs the function.
>
>The more advanced problem I wanted to solve, a two dimensional array of functions, once this example is understood, becomes trivial to implement.  In fact, the more interesting problem I wanted to solve involved a dictionary of functions in a two-dimensional array, and that too was easy to do in Python.  
>
>When I compare what this would require in C, C++, Java, or most other languages, I find Python to be really easy for solving this kind of problem.
>
>Next, I plan to develop the solution using decorators and assertions to empower it with a greater level of portability and to make the functions more generic.
>
>I hope someone finds this interesting.
>
>Richard Riehle, PhD

I am happy you found a solution, but I fear that adding decorators and
assertions will not do what you are asking for.  I missed the
original question, but it looks to me as if what you were looking
for is a nice python dictionary with a tuple as an index.  Since
tuples are immutable, they can be used as indices, while arrays
cannot.

Or, despite finding it interesting, have I misunderstood what
you are doing altogether?

Laura


From gary.herron at islandtraining.com  Tue May 26 21:18:04 2015
From: gary.herron at islandtraining.com (Gary Herron)
Date: Tue, 26 May 2015 18:18:04 -0700
Subject: Array of Functions
In-Reply-To: <TbKdnVQBR8OujvjInZ2dnUU7-LOdnZ2d@giganews.com>
References: <42b534f0-aeef-411f-80c5-db5d20c80b55@googlegroups.com>
 <TbKdnVQBR8OujvjInZ2dnUU7-LOdnZ2d@giganews.com>
Message-ID: <55651B4C.4020409@islandtraining.com>

On 05/26/2015 05:43 PM, richard_riehle wrote:
> I realized that I mentioned earlier that I found a solution to my original question, but that I never posted an example of the solution.   So, here is a simplified example for anyone who is interested.
>
> def fArray(fselect, fparm  = 1):
> 	def A1(p = fparm):
> 		if p == 1:
> 			print("printing A1[1]")
> 		else:
> 			print("printing A1[other]")
> 	def A2(p = fparm):
> 		if p == 1:
> 			print("printing A2[1]")
> 		else:
> 			print("printing A2[other]")
> 	A = [A1, A2]
> 	A[fselect]()

Nested functions are often confusing and unclear, and there is really no 
need for them here.   This does the same thing:


def A1(p):
     print("printing",  "A1[1]" if p==1 else "A1[other]"))

def A2(p):
     print("printing",  "A2[1]" if p==1 else "A2[other]"))

def fArray(fselect, fparm=1):
     A = [A1,A2]
     A[fselect](fparm)


> In this example, I enclosed two functions within another function, and then put those two functions in a list.  Then, with appropriate parameters, I called one of the functions in the list, associated the formal parameter with the function in a call to the array, and presto, it performs the function.
>
> The more advanced problem I wanted to solve, a two dimensional array of functions, once this example is understood, becomes trivial to implement.  In fact, the more interesting problem I wanted to solve involved a dictionary of functions in a two-dimensional array, and that too was easy to do in Python.
>
> When I compare what this would require in C, C++, Java, or most other languages, I find Python to be really easy for solving this kind of problem.
>
> Next, I plan to develop the solution using decorators and assertions to empower it with a greater level of portability and to make the functions more generic.
>
> I hope someone finds this interesting.
>
> Richard Riehle, PhD
>
>



From fabiofz at gmail.com  Tue May 26 21:48:53 2015
From: fabiofz at gmail.com (Fabio Zadrozny)
Date: Tue, 26 May 2015 22:48:53 -0300
Subject: PyDev 4.1.0 Released
Message-ID: <CANXBEFoytHFVBN-spnQSdV+3GoqeRt8KVD58y1aAhKmg2xe6wQ@mail.gmail.com>

What is PyDev?
---------------------------

PyDev is an open-source Python IDE on top of Eclipse for Python, Jython and
IronPython development.

It comes with goodies such as code completion, syntax highlighting, syntax
analysis, code analysis, refactor, debug, interactive console, etc.

Details on PyDev: http://pydev.org
Details on its development: http://pydev.blogspot.com


What is LiClipse?
---------------------------

LiClipse is a PyDev standalone with goodies such as support for Multiple
cursors, theming, TextMate bundles and a number of other languages such as
Django Templates, Jinja2, Kivy Language, Mako Templates, Html, Javascript,
etc.

It's also a commercial counterpart which helps supporting the development
of PyDev.

Details on LiClipse: http://www.liclipse.com/


Release Highlights:
-------------------------------

* **Code Completion**

* Improved unpacking of compound types on more situations (PyDev-573).

* **Debugger**

* PyDev remote debugging no longer blocks running program to completion
(PyDev-574).
* When there are too many referrers to some object, results are trimmed.

* **Python 3 grammar**

* Accepting **@** as matrix multiplication operator.
* **async** and **await** are properly parsed.
* Fixed issue parsing 'list remainder' construct (PyDev-568).
* **Others**

* Fixed issue showing editor title name when more than one dot was present
in the filename.
* Support automatic folding elements when opening a file -- must be enabled
in PyDev > Editor > Code Folding (patch by Andreas Pakulat).
* Fixed issue on search page.
* Included css to set default editor colors for PyDev for in Eclipse dark
theme.
* Tab-stops on comments added and enabled by default (patch by jheiv).
* Fixed StackOverflowError on code-completion (PyDev-570)


Cheers,

--
Fabio Zadrozny
------------------------------------------------------
Software Developer

LiClipse
http://www.liclipse.com

PyDev - Python Development Environment for Eclipse
http://pydev.org
http://pydev.blogspot.com

PyVmMonitor - Python Profiler
http://www.pyvmmonitor.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150526/2c280640/attachment.html>

From dreamingforward at gmail.com  Tue May 26 22:48:12 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 26 May 2015 19:48:12 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <mailman.61.1432661300.5151.python-list@python.org>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.61.1432661300.5151.python-list@python.org>
Message-ID: <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>

On Tuesday, May 26, 2015 at 12:28:31 PM UTC-5, Mark Lawrence wrote:
> On 26/05/2015 17:37, zipher wrote:
> > Would it be prudent to rid the long-standing "argument" (pun unintended) about self and the ulterior spellings of it, by changing it into a symbol rather than a name?
> 
> Yes, how about you taking a permanent holiday rather than bother this 
> list with more of your nonsense?
> 
> Mark Lawrence

Mr. Lawrence, could you calm down with your sadism on the list.  Our session last time ended with a lot of semen on your floor instead of in your mouth.

Your bosom buddy,

Mark Janssen


From dreamingforward at gmail.com  Tue May 26 23:01:36 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 26 May 2015 20:01:36 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <mailman.60.1432659453.5151.python-list@python.org>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.60.1432659453.5151.python-list@python.org>
Message-ID: <d9a95956-4542-4dc0-95de-1158aca6bb23@googlegroups.com>

On Tuesday, May 26, 2015 at 11:57:44 AM UTC-5, Laura Creighton wrote:
> In a message of Tue, 26 May 2015 09:37:29 -0700, zipher writes:
> >Would it be prudent to rid the long-standing "argument" (pun unintended) about self and the ulterior spellings of it, by changing it into a symbol rather than a name?  

> >class MyClass(object):
> >
> >    def __init__(@):
> >        @.dummy = None
> >
> >OR, even better, getting *rid of it* in the parameter list, so it stops confusing people about how many parameters a method needs, and transform it into a true *operator*.
> >
> >class MyClass(object):
> >
> >    def __init__(): #takes no arguments!
> >        @.dummy = None  #the @ invokes the class object's dictionary
> >
> >That would seem to be a nice solution to the problem, really.  It doesn't become PERLish because you've made it into a genuine operator -- "self" was always a non-variable that looked like a variable and hence created an itch that couldn't be scratched.
>
> Guido did.  :)
> http://neopythonic.blogspot.se/2008/10/why-explicit-self-has-to-stay.html

Sweet link.  I see now that my confusion surrounds the mistaken notion that Python is lexing python source into *abstract syntax trees* (wikipedia describes nicely), so that code inside a method knows what class it's in.

But GvR hasn't defined his language to the extent where the notion of "object" even exists.  He's only set aside a keyword called "class".  So of course he has to treat the method code, practically, like a [C] function.  But that limits the language, not to define what an object is inside the lexical structure:

object ::= "class" identifier "(" inheritence_list ")" ":"
inheritence_list ::= [[et cetera]]

If he did this, then code inside the class could already know what class they're in and all the objections in Laura's link would be moot.

Mark J



From dreamingforward at gmail.com  Tue May 26 23:15:25 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 26 May 2015 20:15:25 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <mailman.60.1432659453.5151.python-list@python.org>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.60.1432659453.5151.python-list@python.org>
Message-ID: <a24dc44a-bfe2-433e-97e9-bc1a72cc70b0@googlegroups.com>

On Tuesday, May 26, 2015 at 11:57:44 AM UTC-5, Laura Creighton wrote:
> In a message of Tue, 26 May 2015 09:37:29 -0700, zipher writes:
> >Would it be prudent to rid the long-standing "argument" (pun unintended) about self and the ulterior spellings of it, by changing it into a symbol rather than a name?  
> >
> >Something like:
> >
> >class MyClass(object):
> >
> >    def __init__(@):
> >        @.dummy = None
> >
> >OR, even better, getting *rid of it* in the parameter list, so it stops confusing people about how many parameters a method needs, and transform it into a true *operator*.
> >
> >class MyClass(object):
> >
> >    def __init__(): #takes no arguments!
> >        @.dummy = None  #the @ invokes the class object's dictionary
> >
> >That would seem to be a nice solution to the problem, really.  It doesn't become PERLish because you've made it into a genuine operator -- "self" was always a non-variable that looked like a variable and hence created an itch that couldn't be scratched.
> >
> >Anyone else have any thoughts?
> 
> Guido did.  :)
> http://neopythonic.blogspot.se/2008/10/why-explicit-self-has-to-stay.html

Nice link, thanks.

I see the problem.  I was under the false impression that Python's lexer built an *abstract syntax tree* (see wikipedia for a nice description) like C does so that lexical items like functions and objects are defined.  As it stands now, Python doesn't even seem to know what an *expression* is.

Guido hasn't defined his language so that an object is defined lexically, so he's cheating a little by requiring "self" to be passed in.  If Python were to be defined more completely, all his points in reference to Bruce Eckel's suggestion would be moot.  A method would automatically (not auto*magically*) know what class they are in.

Mark


From dreamingforward at gmail.com  Tue May 26 23:17:31 2015
From: dreamingforward at gmail.com (zipher)
Date: Tue, 26 May 2015 20:17:31 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.61.1432661300.5151.python-list@python.org>
 <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>
Message-ID: <241377fe-2cac-46f8-bbe1-698de608cef4@googlegroups.com>

On Tuesday, May 26, 2015 at 9:48:25 PM UTC-5, zipher wrote:
> On Tuesday, May 26, 2015 at 12:28:31 PM UTC-5, Mark Lawrence wrote:
> > On 26/05/2015 17:37, zipher wrote:
> > > Would it be prudent to rid the long-standing "argument" (pun unintended) about self and the ulterior spellings of it, by changing it into a symbol rather than a name?
> > 
> > Yes, how about you taking a permanent holiday rather than bother this 
> > list with more of your nonsense?
> > 
> > Mark Lawrence
> 
> Mr. Lawrence, could you calm down with your sadism on the list.  Our session last time ended with a lot of semen on your floor instead of in your mouth.
> 
> Your bosom buddy,
> 
> Mark Janssen

Arrgh.  Sorry, that was meant privately...

-m


From ben+python at benfinney.id.au  Wed May 27 00:39:51 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 27 May 2015 14:39:51 +1000
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.61.1432661300.5151.python-list@python.org>
 <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>
 <241377fe-2cac-46f8-bbe1-698de608cef4@googlegroups.com>
Message-ID: <85r3q2ws4o.fsf@benfinney.id.au>

zipher <dreamingforward at gmail.com> writes:

> Arrgh.  Sorry, that was meant privately...

I'm glad we saw it publicly, so that we get more of an idea how you
treat people.

That kind of homophobic slur is inappropriate from anyone in this
community. Kindly cut it out altogether.

-- 
 \       ?Always do right. This will gratify some people, and astonish |
  `\                                            the rest.? ?Mark Twain |
_o__)                                                                  |
Ben Finney



From mr.smittye at gmail.com  Wed May 27 00:54:27 2015
From: mr.smittye at gmail.com (IronManMark20)
Date: Tue, 26 May 2015 21:54:27 -0700 (PDT)
Subject: OpenCV with Python (cv or cv2)
In-Reply-To: <mailman.55.1432648308.5151.python-list@python.org>
References: <mailman.55.1432648308.5151.python-list@python.org>
Message-ID: <b2e15c41-b1bf-44a4-9c2b-cf844f4bb580@googlegroups.com>

On Tuesday, May 26, 2015 at 6:52:00 AM UTC-7, Markos wrote:
> Hi,
> 
> I want to use OpenCV with Python.
> 
> I installed version 2.4.9 
> (http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.4.9/opencv-2.4.9.zip/) 
> in debian Squeeze running Python 2.6.6. Using the tutorial: 
> http://indranilsinharoy.com/2012/11/01/installing-opencv-on-linux/
> 
> (I tried to install versions 2.4.11 and 2.4.10 (in Squeeze with python 
> 2.2.6) but I couldn't.)
> 
> I lost some time trying to load the module cv2:
> 
> > >> Import cv2.cv the cv
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module >>
> ImportError: No module named cv2.cv
> 
> After a while I realized I could use the cv module. (import cv)
> 
> But still I do not understand the differences between the modules cv and 
> cv2.
> 
> Do you suggest any good tutorial about the differences between cv and cv2?
> 
> I will have to make many changes in python programs using cv to reuse 
> the code later with cv2?
> 
> Thanks,
> Markos

To actually answer your questions, cv2 is the newest and best opencv binding. It uses numpy arrays, instead of what cv uses. This means it is much easier to deal with. I will also point you to https://stackoverflow.com/questions/10417108/what-is-different-between-all-these-opencv-python-interfaces


From steve+comp.lang.python at pearwood.info  Wed May 27 01:17:51 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 27 May 2015 15:17:51 +1000
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
Message-ID: <55655380$0$2796$c3e8da3$76491128@news.astraweb.com>

On Wednesday 27 May 2015 02:37, zipher wrote:

> Would it be prudent to rid the long-standing "argument" (pun unintended)
> about self and the ulterior spellings of it, by changing it into a symbol
> rather than a name?

No.


> Something like:
> 
> class MyClass(object):
> 
>     def __init__(@):
>         @.dummy = None
> 
> OR, even better, getting *rid of it* in the parameter list, so it stops
> confusing people about how many parameters a method needs, and transform
> it into a true *operator*.

I think you misspelled a word. There's no "be", "tt" or "er" in "worse" :-)

All joking aside, I cannot imagine why you think that the first argument to 
a method should be considered an operator. It's an argument, a named 
variable, not an operator.


> class MyClass(object):
> 
>     def __init__(): #takes no arguments!

But that is completely wrong. __init__ DOES take a single argument -- the 
instance. Methods, it can be either bound to the instance or unbound:

py> str.upper("hello world")  # unbound method, self provided explicitly
'HELLO WORLD'
py> "hello world".upper()  # bound method, self provided automatically
'HELLO WORLD'

I suggest you study these two examples carefully.



>         @.dummy = None  #the @ invokes the class object's dictionary

It certainly shouldn't do that. It should invoke the full attribute lookup 
machinery, which does far more than invoke the class __dict__. In fact, if 
it invoked the class __dict__, that would *completely* break the object 
oriented paradigm. Writing to the class __dict__ means that all instances 
share the same value.


> That would seem to be a nice solution to the problem, really.  It doesn't
> become PERLish because you've made it into a genuine operator -- "self"
> was always a non-variable that looked like a variable and hence created an
> itch that couldn't be scratched.

That is completely wrong. self is, and always has been, a real variable. 
That's why you can call it anything you like -- self is just the convention, 
nothing more.



-- 
Steven



From steve+comp.lang.python at pearwood.info  Wed May 27 01:20:41 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 27 May 2015 15:20:41 +1000
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
Message-ID: <5565542a$0$2796$c3e8da3$76491128@news.astraweb.com>

On Wednesday 27 May 2015 05:46, Marko Rauhamaa wrote:

> Python's practice works. However, a small problem is presented by nested
> classes:
> 
>     class Connection:
>         def __init__(self):
>             class Idle:
>                 def signal_start(self):
>                     # how to refer to the outer self
>                     :
>             :


The best solution is not to use nested classes, but if you really must:


class Connection:
    def __init__(self):
        class Idle:
            def signal_start(this):
                # this is the Idle instance;
                # self is the Connection instance



-- 
Steve



From steve+comp.lang.python at pearwood.info  Wed May 27 01:23:37 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 27 May 2015 15:23:37 +1000
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mk2kte$opk$2@speranza.aioe.org>
 <mailman.64.1432673145.5151.python-list@python.org>
Message-ID: <556554d9$0$2796$c3e8da3$76491128@news.astraweb.com>

On Wednesday 27 May 2015 06:45, Mark Lawrence wrote:

> Apart from breaking all the tools that rely on "self" being spelt "self"
> this looks like an excellent idea.

Tools which rely on self being spelled "self" are already broken. It's a 
convention, nothing more, and there are various good reasons for breaking 
the convention:

- metaclasses
- classmethods
- custom descriptors
- nested classes



-- 
Steve



From steve+comp.lang.python at pearwood.info  Wed May 27 01:47:48 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Wed, 27 May 2015 15:47:48 +1000
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.61.1432661300.5151.python-list@python.org>
 <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>
 <241377fe-2cac-46f8-bbe1-698de608cef4@googlegroups.com>
 <mailman.72.1432701603.5151.python-list@python.org>
Message-ID: <55655a86$0$12908$c3e8da3$5496439d@news.astraweb.com>

On Wednesday 27 May 2015 14:39, Ben Finney wrote:

> zipher <dreamingforward at gmail.com> writes:
> 
>> Arrgh.  Sorry, that was meant privately...
> 
> I'm glad we saw it publicly, so that we get more of an idea how you
> treat people.
> 
> That kind of homophobic slur is inappropriate from anyone in this
> community. Kindly cut it out altogether.

I look forward to the day when people would read the earlier insult and be 
perplexed as to why it is a slur at all. In the same way as "your mother 
wears army boots" has become a joke slur, not taken seriously.

So long as it does nobody any harm, and all participants have consented, 
what one adult chooses to take into their mouth is nobody else's business 
and certainly no reason to look down on them.



-- 
Steve



From rosuav at gmail.com  Wed May 27 02:32:48 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 27 May 2015 16:32:48 +1000
Subject: should "self" be changed?
In-Reply-To: <556554d9$0$2796$c3e8da3$76491128@news.astraweb.com>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mk2kte$opk$2@speranza.aioe.org>
 <mailman.64.1432673145.5151.python-list@python.org>
 <556554d9$0$2796$c3e8da3$76491128@news.astraweb.com>
Message-ID: <CAPTjJmpZ3Tc1TBa5TokC+P8YLRs2jxAykPJxO7fUKwb9tvpiGQ@mail.gmail.com>

On Wed, May 27, 2015 at 3:23 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Wednesday 27 May 2015 06:45, Mark Lawrence wrote:
>
>> Apart from breaking all the tools that rely on "self" being spelt "self"
>> this looks like an excellent idea.
>
> Tools which rely on self being spelled "self" are already broken. It's a
> convention, nothing more, and there are various good reasons for breaking
> the convention:
>
> - metaclasses
> - classmethods
> - custom descriptors
> - nested classes

If it truly relies on it, yes. But if I see a function that has "self"
as its first parameter, I'm going to read it as a (probable) method
rather than a stand-alone function, and it wouldn't surprise me if
introspection and/or source code parsing made the same assumption.
It's as strong a convention as "don't touch the ones that begin with
an underscore", which is broken by the namedtuple due to namespacing
requirements, but otherwise is fairly dependable (eg if tab completion
skipped the underscore attributes, it'd be a bit less useful on
namedtuples, but probably more practical overall). Using some other
name in place of "self" should definitely remain *possible*, but not
commonly done.

ChrisA


From radler63 at gmail.com  Wed May 27 03:04:20 2015
From: radler63 at gmail.com (radler63 at gmail.com)
Date: Wed, 27 May 2015 00:04:20 -0700 (PDT)
Subject: FYI: Micro Python running on kickstarter pyBoard project,
 now shipping
In-Reply-To: <a98e8e1c-fbcd-4597-9f83-fd15a78eda89@googlegroups.com>
References: <25b4884f-a3f9-436d-8d7d-f49f83c3e544@googlegroups.com>
 <ddddc8b1-e540-45a8-90a2-83399749262e@googlegroups.com>
 <a98e8e1c-fbcd-4597-9f83-fd15a78eda89@googlegroups.com>
Message-ID: <c0727664-fee0-4638-a0a6-680f4521a717@googlegroups.com>

I guess the RPi has no ADC and has a fixed system architecture, not willing to include a co-processor running as watchdog or ADC, which is mandatory for industrial control. However if you run RESTful embedded controllers as limited functionality end devices, the RPi may act as main controller, if you manage a high MTBF.
regardsG.

Am Samstag, 8. November 2014 20:41:41 UTC+1 schrieb John Pinner:
> On Thursday, 23 October 2014 22:12:10 UTC+1, sohca... at gmail.com  wrote:
> > On Thursday, October 23, 2014 10:07:26 AM UTC-7, jkn wrote:
> > > Hi all
> > >     I haven't heard in mentioned here, but since I saw one of the boards today thought I'd pass on the news:
> > > 
> > > The Kickstarter 'MicroPython' project, which has a tiny 'pyboard' (only a couple of sq.inches in size) with a processor running 'a complete re-write of the Python (version 3.4) programming language so that it fits and runs on a microcontroller' is now shipping.
> > > 
> > >     https://micropython.org/
> > > 
> > > Looks nice; I have no connection but will be getting one myself to play with...
> > > 
> > >     Cheers
> > >     J^n
> > 
> > 
> > Is there any particular reason to get one of these when I can get a Raspberry Pi which is faster, has more memory, and a bundle of other features?
> > 
> > I mean, the idea seems cool and all, but I'm trying to understand why I would want to spend the ~$45 on something like that when a ~$35 Raspberry Pi will do everything and more, and do it faster.
> 
> They are quite different devices:
> 
> * The Raspberry Pi is a low-power general purpose computer designed specifically for education purposes. It just so happens that it's ideal for geek experimentation as well...
> 
> * MicroPython is an optimised version of Python 3 running on a micro-controller board, designed specifically for controlling 'things' (eg robots). Doing what it is designed for, it will run far faster and use far less power than a Pi, but cannot do any of the general computing things a Pi can do, for example it has no means of editing programs for MicroPython, you have to do this on, say, your PC and download them to the MicroPython board. It won't do *any* of the other things you can do with a Pi - watch videos, browse the net, etc etc, but what it can do it will do faster and better.
> 
> If you want a low-power, cheap, computer to play with and learn from, get a Pi.
> 
> If you want a nifty micro-controller you can program in Python, buy a MicroPython board.
> 
> John
> --



From al.basili at gmail.com  Wed May 27 03:24:21 2015
From: al.basili at gmail.com (alb)
Date: 27 May 2015 07:24:21 GMT
Subject: SyntaxError on progress module
References: <csjitvF6cptU1@mid.individual.net> <mk2586$d30$1@ger.gmane.org>
 <mailman.59.1432657630.5151.python-list@python.org>
Message-ID: <csl9p3FjduaU1@mid.individual.net>

Hi Chris,

Chris Angelico <rosuav at gmail.com> wrote:
[]
>> Python 3.0 removed the 'u' for unicode in front of strings but due to
>> popular demand to ease porting it was reinstated in 3.3.  Strip it away and
>> you should be fine to go.
> 
> Or upgrade to 3.3 or better; is there anything holding you on 3.2?
> Building CPython from source is pretty easy on Debian, and of course
> upgrading to Jessie will correspondingly upgrade you to a more recent
> Python (3.4, to be precise).

I moved recently from squeeze to wheezy in my production environment, 
I've nothing really holding me back on my current configuration...

I'll give it a try in the coming days.
Thanks for the prompt answer,

Al

-- 
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?


From al.basili at gmail.com  Wed May 27 03:30:31 2015
From: al.basili at gmail.com (alb)
Date: 27 May 2015 07:30:31 GMT
Subject: SyntaxError on progress module
References: <csjitvF6cptU1@mid.individual.net>
 <mailman.58.1432655950.5151.python-list@python.org>
Message-ID: <csla4lFjduaU2@mid.individual.net>

Hi Mark,
Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
[]
>>    File "/home/debian/repos/2418_IASI-NG/Documents/Tools/tex_tool/venv/local/lib/python3.2/site-packages/progress/bar.py", line 48
>>      empty_fill = u'?'
>>                        ^
>> SyntaxError: invalid syntax
>>
[]
> 
> Python 3.0 removed the 'u' for unicode in front of strings but due to 
> popular demand to ease porting it was reinstated in 3.3.  Strip it away 
> and you should be fine to go.

I'm not particularly comfortable in fiddling with the library source, 
even if the change seems really minor, therefore I guess that I'll 
upgrade to a more recent version of Python/Debian. 

But here I have another question, as a python novice is there really any 
reason for me to use any particular version of Python?

Should I start directly with the newest? What about 2.7?

Al


From marko at pacujo.net  Wed May 27 03:39:08 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 27 May 2015 10:39:08 +0300
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mk2kte$opk$2@speranza.aioe.org>
 <mailman.64.1432673145.5151.python-list@python.org>
 <556554d9$0$2796$c3e8da3$76491128@news.astraweb.com>
 <mailman.73.1432708376.5151.python-list@python.org>
Message-ID: <874mmybhb7.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> Using some other name in place of "self" should definitely remain
> *possible*, but not commonly done.

You are effectively making the argument that Python has made a mistake
by not giving "self" a special, language-level status.


Marko


From dpalao.python at gmail.com  Wed May 27 03:47:06 2015
From: dpalao.python at gmail.com (David Palao)
Date: Wed, 27 May 2015 09:47:06 +0200
Subject: SyntaxError on progress module
In-Reply-To: <csla4lFjduaU2@mid.individual.net>
References: <csjitvF6cptU1@mid.individual.net>
 <mailman.58.1432655950.5151.python-list@python.org>
 <csla4lFjduaU2@mid.individual.net>
Message-ID: <CAKUKWzmR9WVVYVpGegnmUXut-Z5En6FZ1Bed+Pk_ZHSTtWAohg@mail.gmail.com>

2015-05-27 9:30 GMT+02:00 alb <al.basili at gmail.com>:
> Hi Mark,
> Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> []
>>>    File "/home/debian/repos/2418_IASI-NG/Documents/Tools/tex_tool/venv/local/lib/python3.2/site-packages/progress/bar.py", line 48
>>>      empty_fill = u'?'
>>>                        ^
>>> SyntaxError: invalid syntax
>>>
> []
>>
>> Python 3.0 removed the 'u' for unicode in front of strings but due to
>> popular demand to ease porting it was reinstated in 3.3.  Strip it away
>> and you should be fine to go.
>
> I'm not particularly comfortable in fiddling with the library source,
> even if the change seems really minor, therefore I guess that I'll
> upgrade to a more recent version of Python/Debian.
>
> But here I have another question, as a python novice is there really any
> reason for me to use any particular version of Python?
>
> Should I start directly with the newest? What about 2.7?
>
> Al
> --
> https://mail.python.org/mailman/listinfo/python-list

Hi,
I would recommend to start with a 3.x Python. Perhaps 3.3 or 3.4? If
the version provided by your OS is older, you can always use a virtual
environment. Ask if you don't know about it.

Best


From rosuav at gmail.com  Wed May 27 04:20:27 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 27 May 2015 18:20:27 +1000
Subject: should "self" be changed?
In-Reply-To: <874mmybhb7.fsf@elektro.pacujo.net>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mk2kte$opk$2@speranza.aioe.org>
 <mailman.64.1432673145.5151.python-list@python.org>
 <556554d9$0$2796$c3e8da3$76491128@news.astraweb.com>
 <mailman.73.1432708376.5151.python-list@python.org>
 <874mmybhb7.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmqh9sHrFk2Wg6OdkdsxVizzazGtkUsxkvJHMm=Pp5Zddw@mail.gmail.com>

On Wed, May 27, 2015 at 5:39 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Chris Angelico <rosuav at gmail.com>:
>
>> Using some other name in place of "self" should definitely remain
>> *possible*, but not commonly done.
>
> You are effectively making the argument that Python has made a mistake
> by not giving "self" a special, language-level status.

Uhh... no I'm not. I'm saying it should be possible to use some other
name instead of "self", which means that Python got it right by not
making it special.

ChrisA


From rosuav at gmail.com  Wed May 27 04:22:55 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 27 May 2015 18:22:55 +1000
Subject: SyntaxError on progress module
In-Reply-To: <csla4lFjduaU2@mid.individual.net>
References: <csjitvF6cptU1@mid.individual.net>
 <mailman.58.1432655950.5151.python-list@python.org>
 <csla4lFjduaU2@mid.individual.net>
Message-ID: <CAPTjJmoHzNeJc5tNsaZGVC6k_sJN4ytasBpT-pPFkFtDfb91mQ@mail.gmail.com>

On Wed, May 27, 2015 at 5:30 PM, alb <al.basili at gmail.com> wrote:
> But here I have another question, as a python novice is there really any
> reason for me to use any particular version of Python?
>
> Should I start directly with the newest? What about 2.7?
>

Start with the newest that's conveniently available. With Debian
Jessie, Python 3.4.2 is a simple apt-get away, so that's pretty
convenient.

ChrisA


From Cecil at decebal.nl  Wed May 27 04:42:25 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 27 May 2015 10:42:25 +0200
Subject: SyntaxError on progress module
References: <csjitvF6cptU1@mid.individual.net>
 <mailman.58.1432655950.5151.python-list@python.org>
 <csla4lFjduaU2@mid.individual.net>
Message-ID: <87y4kammxa.fsf@Equus.decebal.nl>

Op Wednesday 27 May 2015 09:30 CEST schreef alb:

> But here I have another question, as a python novice is there really
> any reason for me to use any particular version of Python?
>
> Should I start directly with the newest? What about 2.7?

In principal you should use the ?latest? 3. The only problem is that a
lot of libraries are not converted to 3 yet. If you need one of those,
then you have ?no choice? and have to use 2.7. But I would recommend
to use ?from __future__' to make the 2.7 code as much as possible 3
compliant.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From techtonik at gmail.com  Wed May 27 07:15:06 2015
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 27 May 2015 14:15:06 +0300
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
In-Reply-To: <CAPkN8xKTXJu2nhvocG8KuyO1XkJVfK_WsmY6dM=hWsVyg+BVyA@mail.gmail.com>
References: <CAPkN8xKTXJu2nhvocG8KuyO1XkJVfK_WsmY6dM=hWsVyg+BVyA@mail.gmail.com>
Message-ID: <CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com>

Hi.

This was labelled offtopic in python-ideas, so I edited and forwarded
it here. Please CC as I am not subscribed.


In short. I need is a bulletproof way to convert from anything to
unicode. This requires some kind of escaping to go forward and back.
Some helper function like u2b() (unicode to binary) and b2u() (that
also removes escaping). So far I can't find any code that does just
that.


Background story. I need to print SCons graph. SCons is a build tool,
so it has a graph of nodes - what depends on what. I have no idea
what a node object could be. I know only that it can have human
readable representation. Sometimes node is a filename in some
encoding that is not utf-8, and without knowing the encoding,
converting it to unicode is not possible without loosing the information
about that filename.

So, here is what Python proposes:

https://docs.python.org/2.7/library/functions.html?highlight=unicode#unicode

unicode() type constructor that doesn't allow you to do conversion
without losing the data. It offers only two basic strategies - crash or
corrupt:

1. ignore  - meaning skip and corrupt the data
2. replace  - just corrupt the data
3. strict - just crash

Python design leaves the decision how to implement safe
interoperability to you, and that's basically the reason why Python 3
fails. Without a safe approach (get my binary data back frum that
unicode) people just can't wrap their heads around that.

Python design assumes that people know the encoding of data they
are processing, but that's not true in many cases. The data may also
be just broken or invalid. So, the real world coding assumptions are:

1. external data encoding is unknown or varies
2. external data has binary chunks that are invalid for
conversion to unicode

In real world UnicodeDecode crashes is not an option for deal with
unknown or broken and invalid input (such as when I need to print
human representation of Node to the screen). In many (most?)
situations lossless garbage is more welcome than crash or dataloss
and that should be a default behaviour.


The solution is to have filter preprocess the binary string to escape all
non-unicode symbols so that the following lossless transformation
becomes possible:

   binary -> escaped utf-8 string -> unicode -> binary

I want to know if that's real? I need to accomplish that with
Python 2.x, but the use case is probably valid for Python 3 as well.

This stuff is critical to port SCons to Python 3.x and I expect for other
similar tools that have to deal with unknown ascii-binary strings too.

-- 
anatoly t.


From ben+python at benfinney.id.au  Wed May 27 07:29:39 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Wed, 27 May 2015 21:29:39 +1000
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.61.1432661300.5151.python-list@python.org>
 <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>
 <241377fe-2cac-46f8-bbe1-698de608cef4@googlegroups.com>
 <mailman.72.1432701603.5151.python-list@python.org>
 <55655a86$0$12908$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <85mw0qw95o.fsf@benfinney.id.au>

Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:

> On Wednesday 27 May 2015 14:39, Ben Finney wrote:
>
> > That kind of homophobic slur is inappropriate from anyone in this
> > community. Kindly cut it out altogether.
>
> I look forward to the day when people would read the earlier insult
> and be perplexed as to why it is a slur at all. In the same way as
> "your mother wears army boots" has become a joke slur, not taken
> seriously.

Yes, let's all work toward an end to the use of gender, sexuality,
ethnicity, and other inborn traits as the punchline of insults or jokes.

Until that happy day, let's work to improve the lot of those who are
made the butt of such slurs. Part of that work must be to call foul when
someone invokes an entire class of people as an insult.

-- 
 \         ?Pinky, are you pondering what I'm pondering?? ?I think so, |
  `\        Brain, but why would anyone want to see Snow White and the |
_o__)                           Seven Samurai?? ?_Pinky and The Brain_ |
Ben Finney



From corrado.garrone at roche.com  Wed May 27 07:30:06 2015
From: corrado.garrone at roche.com (Garrone, Corrado)
Date: Wed, 27 May 2015 13:30:06 +0200
Subject: Accessing DataSocket Server with Python
Message-ID: <CAO2fJ-+UjZNqo2yQRRkuSsEJwFAGOnbB9JPCCWX7zX7jt_5POg@mail.gmail.com>

Dear Python Team,

currently I am working on a research project for my bachelor degree. A
LabVIEW application is used for current and power measurements, whereas the
measured data are sent to DataSocket Server, a technology by National
Instruments used for data exchange between computers and applications.
DataSocket is based on TCP/IP and thus requesting data from DataSocket
should be similar to an internet request.
I know with the socket library in Python it is possible with to establish
sockets, send internet requests and communicate between clients and
servers.
Is there a possibility to access NI DataSocket and get measurement data
with Python on the same computer where Python is installed and the codes
are executed? Can you maybe send me an example code where such a connection
with DataSocket is established?

If you got any queries, please do not hesitate to contact me.

Thank you very much for your efforts.

Kind regards,

 *Corrado Garrone*
DH-Student Fachrichtung Elektrotechnik / Co-op Student B.Eng. Electrical
Engineering

Roche Diagnostics GmbH
DFGHMV8Y6164
Sandhofer Strasse 116
68305 Mannheim / Germany

Phone: apprentice
mailto:corrado.garrone at roche.com <corrado.garrone at roche.com>

 *Roche Diagnostics GmbH*
Sandhofer Stra?e 116; D?68305 Mannheim; Telefon +49?621?759?0;
Telefax +49?621?759?2890
Sitz der Gesellschaft: Mannheim - Registergericht: AG Mannheim HRB 3962 -
Gesch?ftsf?hrung: Dr. Ursula Redeker, Sprecherin; Edgar Vieth -
Aufsichtsratsvorsitzender: Dr. Severin Schwan

 *Confidentiality Note*
This message is intended only for the use of the named recipient(s) and may
contain confidential and/or privileged information. If you are not the
intended recipient, please contact the sender and delete the message. Any
unauthorized use of the information contained in this message is prohibited.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150527/34a12c5f/attachment.html>

From lac at openend.se  Wed May 27 07:35:34 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 27 May 2015 13:35:34 +0200
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
In-Reply-To: Message from anatoly techtonik <techtonik@gmail.com> of "Wed,
 27 May 2015 14:15:06 +0300."
 <CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com>
References: <CAPkN8xKTXJu2nhvocG8KuyO1XkJVfK_WsmY6dM=hWsVyg+BVyA@mail.gmail.com><CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com>
Message-ID: <201505271135.t4RBZY6F013836@fido.openend.se>

I am missing something.  Why do you need unicode at all?  Why can you
not just keep your binary data as binary data?

I feel like I must be missing something obvious here ...

Laura



From techtonik at gmail.com  Wed May 27 07:47:33 2015
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 27 May 2015 14:47:33 +0300
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
In-Reply-To: <201505271135.t4RBZY6F013836@fido.openend.se>
References: <CAPkN8xKTXJu2nhvocG8KuyO1XkJVfK_WsmY6dM=hWsVyg+BVyA@mail.gmail.com>
 <techtonik@gmail.com>
 <CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com>
 <201505271135.t4RBZY6F013836@fido.openend.se>
Message-ID: <CAPkN8xJGnut_V0BmEah1ravN3cXESqx7c6U=mOAS0coD3fcPjA@mail.gmail.com>

On Wed, May 27, 2015 at 2:35 PM, Laura Creighton <lac at openend.se> wrote:
> I am missing something.  Why do you need unicode at all?  Why can you
> not just keep your binary data as binary data?

Good question. From the SCons code I see that we need unicode, because
we switched to io.StringIO which is advertised as the future (and Python 3
way of doing things, because Python 3 doesn't have non-unicode StringIO).

A really deep and exhaustive answer.
advertisement (first link on StringIO vs io.StringIO):
http://stackoverflow.com/questions/3410309/what-is-the-difference-between-stringio-and-io-stringio-in-python2-7
peaceful details
https://bitbucket.org/scons/scons/commits/05d5af305a5d
gory consequences
https://bitbucket.org/scons/scons/pull-request/235/fix-tree-all-print-when-build-tree

> I feel like I must be missing something obvious here ...

Not that obvious as it appears.
-- 
anatoly t.


From techtonik at gmail.com  Wed May 27 07:52:40 2015
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 27 May 2015 14:52:40 +0300
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
In-Reply-To: <CAPkN8xJGnut_V0BmEah1ravN3cXESqx7c6U=mOAS0coD3fcPjA@mail.gmail.com>
References: <CAPkN8xKTXJu2nhvocG8KuyO1XkJVfK_WsmY6dM=hWsVyg+BVyA@mail.gmail.com>
 <techtonik@gmail.com>
 <CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com>
 <201505271135.t4RBZY6F013836@fido.openend.se>
 <CAPkN8xJGnut_V0BmEah1ravN3cXESqx7c6U=mOAS0coD3fcPjA@mail.gmail.com>
Message-ID: <CAPkN8xJTA_cfWuXD8+MUqKB6yTMp=vzUHk0sQsNz3acyJL-LXg@mail.gmail.com>

On Wed, May 27, 2015 at 2:47 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> On Wed, May 27, 2015 at 2:35 PM, Laura Creighton <lac at openend.se> wrote:
>> I am missing something.  Why do you need unicode at all?  Why can you
>> not just keep your binary data as binary data?
>
> Good question. From the SCons code I see that we need unicode, because
> we switched to io.StringIO which is advertised as the future (and Python 3
> way of doing things, because Python 3 doesn't have non-unicode StringIO).
>
> A really deep and exhaustive answer.
> advertisement (first link on StringIO vs io.StringIO):
> http://stackoverflow.com/questions/3410309/what-is-the-difference-between-stringio-and-io-stringio-in-python2-7
> peaceful details
> https://bitbucket.org/scons/scons/commits/05d5af305a5d
> gory consequences
> https://bitbucket.org/scons/scons/pull-request/235/fix-tree-all-print-when-build-tree
>
>> I feel like I must be missing something obvious here ...
>
> Not that obvious as it appears.

And the short answer is that we need unicode because we are printing this
information to the stdout, and stdout is opened in text mode at least on
Windows, and without explicit conversion, Python will try to decode stuff
as being `ascii` and fail anyway.
-- 
anatoly t.


From rosuav at gmail.com  Wed May 27 07:59:03 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 27 May 2015 21:59:03 +1000
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
In-Reply-To: <CAPkN8xJTA_cfWuXD8+MUqKB6yTMp=vzUHk0sQsNz3acyJL-LXg@mail.gmail.com>
References: <CAPkN8xKTXJu2nhvocG8KuyO1XkJVfK_WsmY6dM=hWsVyg+BVyA@mail.gmail.com>
 <techtonik@gmail.com>
 <CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com>
 <201505271135.t4RBZY6F013836@fido.openend.se>
 <CAPkN8xJGnut_V0BmEah1ravN3cXESqx7c6U=mOAS0coD3fcPjA@mail.gmail.com>
 <CAPkN8xJTA_cfWuXD8+MUqKB6yTMp=vzUHk0sQsNz3acyJL-LXg@mail.gmail.com>
Message-ID: <CAPTjJmoM=0ZGEJjxwhQRAEuqHfgGLUYgSkYwEuUp7QdaT-GGqQ@mail.gmail.com>

On Wed, May 27, 2015 at 9:52 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> And the short answer is that we need unicode because we are printing this
> information to the stdout, and stdout is opened in text mode at least on
> Windows, and without explicit conversion, Python will try to decode stuff
> as being `ascii` and fail anyway.

So you're working with text. That means you HAVE to decode it somehow;
you fundamentally cannot print bytes to the console. Lossless
concealment of arbitrary bytes won't help you. If you can't adequately
decode everything, either backslash-escape the rest, or use a
replacement character; you can't print out those bytes.

And no, I will not cc you. Subscribe to the list if you're going to
ask a question.

ChrisA


From random832 at fastmail.us  Wed May 27 08:40:38 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 27 May 2015 08:40:38 -0400
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
In-Reply-To: <CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com>
References: <CAPkN8xKTXJu2nhvocG8KuyO1XkJVfK_WsmY6dM=hWsVyg+BVyA@mail.gmail.com>
 <CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com>
Message-ID: <1432730438.1548242.279303617.0A0F1738@webmail.messagingengine.com>

On Wed, May 27, 2015, at 07:15, anatoly techtonik wrote:
> The solution is to have filter preprocess the binary string to escape all
> non-unicode symbols so that the following lossless transformation
> becomes possible:
> 
>    binary -> escaped utf-8 string -> unicode -> binary
> 
> I want to know if that's real? I need to accomplish that with
> Python 2.x, but the use case is probably valid for Python 3 as well.

In Python 3, you could *in principle* use surrogateescape (this would be
more of a binary -> escaped unicode workflow), but see below. It is
worth noting that when you *read* posix filenames in unicode form (e.g.
listdir with a unicode argument), they are decoded with surrogateescape,
and can be returned to bytes format with
fn.encode(sys.getfilesystemencoding(), errors='surrogateescape').
However keep in mind that on *windows*, the native filename format is a
sequence of 16-bit WCHAR values, not a sequence of bytes.

> This stuff is critical to port SCons to Python 3.x and I expect for other
> similar tools that have to deal with unknown ascii-binary strings too.

Even if your filename *is* valid UTF-8 (or whatever other encoding), it
might contain invisible control characters that make it difficult to
read. You'd probably be better off simply working directly with the
binary representation, iterating over it and replacing all
non-*ascii*-printable bytes with an escaped representation. As it
happens, the repr() function should work well for doing exactly this.
(note: repr on a *unicode* string in python 3 will pass non-ascii
characters, but ideally you're working with byte strings.)

There's no real need to go beyond this unless you're working in a
problem domain where filenames are likely to legitimately include
non-ascii characters (e.g. user documents of non-technical users who use
languages other than English).


From dreamingforward at gmail.com  Wed May 27 08:40:51 2015
From: dreamingforward at gmail.com (zipher)
Date: Wed, 27 May 2015 05:40:51 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <mailman.79.1432726201.5151.python-list@python.org>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.61.1432661300.5151.python-list@python.org>
 <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>
 <241377fe-2cac-46f8-bbe1-698de608cef4@googlegroups.com>
 <mailman.72.1432701603.5151.python-list@python.org>
 <55655a86$0$12908$c3e8da3$5496439d@news.astraweb.com>
 <mailman.79.1432726201.5151.python-list@python.org>
Message-ID: <d4c77505-be5f-4434-a4b3-3d3fe083add6@googlegroups.com>

On Wednesday, May 27, 2015 at 6:30:16 AM UTC-5, Ben Finney wrote:
> Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
> 
> > On Wednesday 27 May 2015 14:39, Ben Finney wrote:
> >
> > > That kind of homophobic slur is inappropriate from anyone in this
> > > community. Kindly cut it out altogether.
> >
> > I look forward to the day when people would read the earlier insult
> > and be perplexed as to why it is a slur at all. In the same way as
> > "your mother wears army boots" has become a joke slur, not taken
> > seriously.
> 
> Yes, let's all work toward an end to the use of gender, sexuality,
> ethnicity, and other inborn traits as the punchline of insults or jokes.

Oh God, you people are being idiots.  It's poop. And shall we all so look forward to the day, when people who eat poop are also welcome into the circle of humanity? 

Everyday, you let atrocity happen, and you're fighting for oppressed feltchers?

If so, you dumbasses don't deserve much of a future.

Mark


From steve at pearwood.info  Wed May 27 08:47:46 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 27 May 2015 22:47:46 +1000
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
References: <CAPkN8xKTXJu2nhvocG8KuyO1XkJVfK_WsmY6dM=hWsVyg+BVyA@mail.gmail.com>
 <mailman.78.1432725335.5151.python-list@python.org>
Message-ID: <5565bcf3$0$12978$c3e8da3$5496439d@news.astraweb.com>

On Wed, 27 May 2015 09:15 pm, anatoly techtonik wrote:

> Hi.
> 
> This was labelled offtopic in python-ideas, so I edited and forwarded
> it here. Please CC as I am not subscribed.
> 
> 
> In short. I need is a bulletproof way to convert from anything to
> unicode. This requires some kind of escaping to go forward and back.

Why do you need to go back? Just keep the node, and use that.


> Some helper function like u2b() (unicode to binary) and b2u() (that
> also removes escaping). So far I can't find any code that does just
> that.


def bytes2unicode(bytes):
    # Converts bytes to Unicode, allowing garbage (moji-bake).
    return bytes.decode('latin1')

def unicode2bytes(unicode):
    # Convert unicode containing garbage (moji-bake) to bytes.
    return unicode.encode('latin1')


It correctly does the round trip from any sequence of bytes to unicode and
back to bytes, losslessly:


py> import random
py> node = bytes([random.randrange(0, 256) for _ in range(100000)])
py> uni = bytes2unicode(node)
py> b = unicode2bytes(uni)
py> b == node
True


But take careful note that you can't start with Unicode and still expect to
round-trip losslessly. Many perfectly readable Unicode strings do *not*
convert to bytes:

py> unicode2bytes(u'??')  # two Cyrillic letters
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in unicode2bytes
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-1:
ordinal not in range(256)


That means that if you take a correctly encoded string, it will round-trip,
but it will also display as garbage:

py> s = u'??'
py> node = s.encode('utf-8')
py> print(node)  # Correctly encoded UTF-8
b'\xd0\x94\xd0\x99'
py> node == unicode2bytes(bytes2unicode(node))  # round trips okay
True
py> print(repr(bytes2unicode(node)))  # but prints as crap
'?\x94?\x99'



> Background story. I need to print SCons graph. SCons is a build tool,
> so it has a graph of nodes - what depends on what. I have no idea
> what a node object could be. I know only that it can have human
> readable representation. Sometimes node is a filename in some
> encoding that is not utf-8, and without knowing the encoding,
> converting it to unicode is not possible without loosing the information
> about that filename.

py> filename = "My Russian ?? name"  # Unicode
py> b = filename.encode('koi8-r')  # Oops, not UTF-8!
py> b.decode("utf-8")  # Fails
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 11:
invalid continuation byte
py> b.decode("utf-8", errors="replace")  # lossy, but works
'My Russian ?? name'
py> s = b.decode("utf-8", errors="surrogateescape")  # magic!
py> s
'My Russian \udce4\udcea name'


It round-trips as well:

py> s.encode("utf-8", errors="surrogateescape") == b
True


Converting this back to Python 2.7 is left as an exercise for the reader.



-- 
Steven



From random832 at fastmail.us  Wed May 27 08:49:01 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 27 May 2015 08:49:01 -0400
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
In-Reply-To: <CAPkN8xJGnut_V0BmEah1ravN3cXESqx7c6U=mOAS0coD3fcPjA@mail.gmail.com>
References: <CAPkN8xKTXJu2nhvocG8KuyO1XkJVfK_WsmY6dM=hWsVyg+BVyA@mail.gmail.com>
 <techtonik@gmail.com>
 <CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com>
 <201505271135.t4RBZY6F013836@fido.openend.se>
 <CAPkN8xJGnut_V0BmEah1ravN3cXESqx7c6U=mOAS0coD3fcPjA@mail.gmail.com>
Message-ID: <1432730941.1551012.279318505.2897B115@webmail.messagingengine.com>

On Wed, May 27, 2015, at 07:47, anatoly techtonik wrote:
> because Python 3 doesn't have non-unicode StringIO

That's actually not true - the non-unicode equivalent is BytesIO.
However, it's probably not actually what you want, if the point is to
display the filenames to the user.


From lac at openend.se  Wed May 27 08:57:01 2015
From: lac at openend.se (Laura Creighton)
Date: Wed, 27 May 2015 14:57:01 +0200
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
 (fwd)
Message-ID: <201505271257.t4RCv1R2015793@fido.openend.se>

Chris Angelico apparantly has a problem with cc'd people who aren't
on the list.  python-list is very quiet these days, so if you
subscribe it won't be drinking from the firehose.  And you can
always turn off delivery when you are done.  Or you can just
go read the archives: https://mail.python.org/pipermail/python-list/2015-May/thread.html

Laura

------- Forwarded Message

Return-Path: <python-list-bounces+lac=openend.se at python.org>
Received: from mail.python.org (mail.python.org [82.94.164.166])
	by theraft.openend.se (8.14.4/8.14.4/Debian-4) with ESMTP id t4RC09ap02From: Chris Angelico <rosuav at gmail.com>
Cc: "python-list at python.org" <python-list at python.org>


On Wed, May 27, 2015 at 9:52 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> And the short answer is that we need unicode because we are printing this
> information to the stdout, and stdout is opened in text mode at least on
> Windows, and without explicit conversion, Python will try to decode stuff
> as being `ascii` and fail anyway.

So you're working with text. That means you HAVE to decode it somehow;
you fundamentally cannot print bytes to the console. Lossless
concealment of arbitrary bytes won't help you. If you can't adequately
decode everything, either backslash-escape the rest, or use a
replacement character; you can't print out those bytes.

And no, I will not cc you. Subscribe to the list if you're going to
ask a question.

ChrisA
- -- 
https://mail.python.org/mailman/listinfo/python-list

------- End of Forwarded Message


From toddrjen at gmail.com  Wed May 27 09:00:04 2015
From: toddrjen at gmail.com (Todd)
Date: Wed, 27 May 2015 15:00:04 +0200
Subject: should "self" be changed?
In-Reply-To: <d4c77505-be5f-4434-a4b3-3d3fe083add6@googlegroups.com>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.61.1432661300.5151.python-list@python.org>
 <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>
 <241377fe-2cac-46f8-bbe1-698de608cef4@googlegroups.com>
 <mailman.72.1432701603.5151.python-list@python.org>
 <55655a86$0$12908$c3e8da3$5496439d@news.astraweb.com>
 <mailman.79.1432726201.5151.python-list@python.org>
 <d4c77505-be5f-4434-a4b3-3d3fe083add6@googlegroups.com>
Message-ID: <CAFpSVpKpweuz8Z5dCC9iKHVQ=v9XBcY0uRrN9HZHpHEvuzf46g@mail.gmail.com>

On Wed, May 27, 2015 at 2:40 PM, zipher <dreamingforward at gmail.com> wrote:

> On Wednesday, May 27, 2015 at 6:30:16 AM UTC-5, Ben Finney wrote:
> > Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
> >
> > > On Wednesday 27 May 2015 14:39, Ben Finney wrote:
> > >
> > > > That kind of homophobic slur is inappropriate from anyone in this
> > > > community. Kindly cut it out altogether.
> > >
> > > I look forward to the day when people would read the earlier insult
> > > and be perplexed as to why it is a slur at all. In the same way as
> > > "your mother wears army boots" has become a joke slur, not taken
> > > seriously.
> >
> > Yes, let's all work toward an end to the use of gender, sexuality,
> > ethnicity, and other inborn traits as the punchline of insults or jokes.
>
> Oh God, you people are being idiots.  It's poop. And shall we all so look
> forward to the day, when people who eat poop are also welcome into the
> circle of humanity?
>
> Everyday, you let atrocity happen, and you're fighting for oppressed
> feltchers?
>
> If so, you dumbasses don't deserve much of a future.
>

If your goal is to get people to stop calling you a troll, you are going
about it the wrong way.  If it isn't, why are you even here?  Please
remember the first rule of holes: if you find yourself in a hole, stop
digging.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150527/79300e38/attachment.html>

From robin at reportlab.com  Wed May 27 09:02:35 2015
From: robin at reportlab.com (Robin Becker)
Date: Wed, 27 May 2015 14:02:35 +0100
Subject: a more precise distance algorithm
In-Reply-To: <1432648317.1154051.278328873.25A47810@webmail.messagingengine.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
 <1432647653.1149540.278304225.25D9F2E4@webmail.messagingengine.com>
 <1432648317.1154051.278328873.25A47810@webmail.messagingengine.com>
Message-ID: <5565C06B.1000201@chamonix.reportlab.co.uk>

A minor point is that if you just need to compare distances you don't need to 
compute the hypotenuse, its square will do so no subtractions etc etc.
-- 
Robin Becker



From breamoreboy at yahoo.co.uk  Wed May 27 09:44:23 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 27 May 2015 14:44:23 +0100
Subject: SyntaxError on progress module
In-Reply-To: <87y4kammxa.fsf@Equus.decebal.nl>
References: <csjitvF6cptU1@mid.individual.net>
 <mailman.58.1432655950.5151.python-list@python.org>
 <csla4lFjduaU2@mid.individual.net> <87y4kammxa.fsf@Equus.decebal.nl>
Message-ID: <mk4hnp$ajh$1@ger.gmane.org>

On 27/05/2015 09:42, Cecil Westerhof wrote:
> Op Wednesday 27 May 2015 09:30 CEST schreef alb:
>
>> But here I have another question, as a python novice is there really
>> any reason for me to use any particular version of Python?
>>
>> Should I start directly with the newest? What about 2.7?
>
> In principal you should use the ?latest? 3. The only problem is that a
> lot of libraries are not converted to 3 yet. If you need one of those,
> then you have ?no choice? and have to use 2.7. But I would recommend
> to use ?from __future__' to make the 2.7 code as much as possible 3
> compliant.
>

Please define "a lot" whilst bearing in mind green against red here 
https://python3wos.appspot.com/

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From Cecil at decebal.nl  Wed May 27 10:11:23 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 27 May 2015 16:11:23 +0200
Subject: SyntaxError on progress module
References: <csjitvF6cptU1@mid.individual.net>
 <mailman.58.1432655950.5151.python-list@python.org>
 <csla4lFjduaU2@mid.individual.net> <87y4kammxa.fsf@Equus.decebal.nl>
 <mailman.90.1432734279.5151.python-list@python.org>
Message-ID: <87siaim7p0.fsf@Equus.decebal.nl>

Op Wednesday 27 May 2015 15:44 CEST schreef Mark Lawrence:

> On 27/05/2015 09:42, Cecil Westerhof wrote:
>> Op Wednesday 27 May 2015 09:30 CEST schreef alb:
>>
>>> But here I have another question, as a python novice is there
>>> really any reason for me to use any particular version of Python?
>>>
>>> Should I start directly with the newest? What about 2.7?
>>
>> In principal you should use the ?latest? 3. The only problem is
>> that a lot of libraries are not converted to 3 yet. If you need one
>> of those, then you have ?no choice? and have to use 2.7. But I
>> would recommend to use ?from __future__' to make the 2.7 code as
>> much as possible 3 compliant.
>>
>
> Please define "a lot" whilst bearing in mind green against red here
> https://python3wos.appspot.com/

I just started using Python again and the first ?real? program I wrote
I had to write with Python 2 because the needed library (libturpial,
that is not listed on your link) works only with Python 2. A short
search about which of the two to use gives similar answers to mine.
And as far as I can see in my neighbourhood Python 2 is almost
exclusively used because used libraries are only available in Python
2.

This is not a scientifically substantiated argument, but for me good
enough to use a lot.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From invalid at invalid.invalid  Wed May 27 10:19:47 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Wed, 27 May 2015 14:19:47 +0000 (UTC)
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.61.1432661300.5151.python-list@python.org>
 <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>
 <241377fe-2cac-46f8-bbe1-698de608cef4@googlegroups.com>
 <mailman.72.1432701603.5151.python-list@python.org>
 <55655a86$0$12908$c3e8da3$5496439d@news.astraweb.com>
 <mailman.79.1432726201.5151.python-list@python.org>
 <d4c77505-be5f-4434-a4b3-3d3fe083add6@googlegroups.com>
 <mailman.88.1432731626.5151.python-list@python.org>
Message-ID: <mk4jq3$kp8$1@reader1.panix.com>

On 2015-05-27, Todd <toddrjen at gmail.com> wrote:
> On Wed, May 27, 2015 at 2:40 PM, zipher <dreamingforward at gmail.com> wrote:
>
>> [some stupid crap]
>
> If your goal is to get people to stop calling you a troll, you are
> going about it the wrong way.  If it isn't, why are you even here?
> Please remember the first rule of holes: if you find yourself in a
> hole, stop digging.

And thanks to everybody who keeps replying to zipher's posts so that
those of use who's newsreaders are configured to filter him out get to
see them anyway.

-- 
Grant Edwards               grant.b.edwards        Yow! I need to discuss
                                  at               BUY-BACK PROVISIONS
                              gmail.com            with at least six studio
                                                   SLEAZEBALLS!!


From breamoreboy at yahoo.co.uk  Wed May 27 10:51:52 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 27 May 2015 15:51:52 +0100
Subject: SyntaxError on progress module
In-Reply-To: <87siaim7p0.fsf@Equus.decebal.nl>
References: <csjitvF6cptU1@mid.individual.net>
 <mailman.58.1432655950.5151.python-list@python.org>
 <csla4lFjduaU2@mid.individual.net> <87y4kammxa.fsf@Equus.decebal.nl>
 <mailman.90.1432734279.5151.python-list@python.org>
 <87siaim7p0.fsf@Equus.decebal.nl>
Message-ID: <mk4lma$gvr$1@ger.gmane.org>

On 27/05/2015 15:11, Cecil Westerhof wrote:
> Op Wednesday 27 May 2015 15:44 CEST schreef Mark Lawrence:
>
>> On 27/05/2015 09:42, Cecil Westerhof wrote:
>>> Op Wednesday 27 May 2015 09:30 CEST schreef alb:
>>>
>>>> But here I have another question, as a python novice is there
>>>> really any reason for me to use any particular version of Python?
>>>>
>>>> Should I start directly with the newest? What about 2.7?
>>>
>>> In principal you should use the ?latest? 3. The only problem is
>>> that a lot of libraries are not converted to 3 yet. If you need one
>>> of those, then you have ?no choice? and have to use 2.7. But I
>>> would recommend to use ?from __future__' to make the 2.7 code as
>>> much as possible 3 compliant.
>>>
>>
>> Please define "a lot" whilst bearing in mind green against red here
>> https://python3wos.appspot.com/
>
> I just started using Python again and the first ?real? program I wrote
> I had to write with Python 2 because the needed library (libturpial,
> that is not listed on your link) works only with Python 2. A short
> search about which of the two to use gives similar answers to mine.
> And as far as I can see in my neighbourhood Python 2 is almost
> exclusively used because used libraries are only available in Python
> 2.
>
> This is not a scientifically substantiated argument, but for me good
> enough to use a lot.
>

Have you actaully tried running libturpial with Python 3 or have you 
simply taken somebody or something's word for it?  I've taken code in 
the past that was "only Python 2", run it thought the 2to3 fixer and job 
done.  Perhaps you could do the same.  Perhaps you've already tried. 
Again, you're the only person who actually knows.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From Cecil at decebal.nl  Wed May 27 11:18:48 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 27 May 2015 17:18:48 +0200
Subject: SyntaxError on progress module
References: <csjitvF6cptU1@mid.individual.net>
 <mailman.58.1432655950.5151.python-list@python.org>
 <csla4lFjduaU2@mid.individual.net> <87y4kammxa.fsf@Equus.decebal.nl>
 <mailman.90.1432734279.5151.python-list@python.org>
 <87siaim7p0.fsf@Equus.decebal.nl>
 <mailman.91.1432738329.5151.python-list@python.org>
Message-ID: <87oal6m4kn.fsf@Equus.decebal.nl>

Op Wednesday 27 May 2015 16:51 CEST schreef Mark Lawrence:

> On 27/05/2015 15:11, Cecil Westerhof wrote:
>> Op Wednesday 27 May 2015 15:44 CEST schreef Mark Lawrence:
>>
>>> On 27/05/2015 09:42, Cecil Westerhof wrote:
>>>> Op Wednesday 27 May 2015 09:30 CEST schreef alb:
>>>>
>>>>> But here I have another question, as a python novice is there
>>>>> really any reason for me to use any particular version of
>>>>> Python?
>>>>>
>>>>> Should I start directly with the newest? What about 2.7?
>>>>
>>>> In principal you should use the ?latest? 3. The only problem is
>>>> that a lot of libraries are not converted to 3 yet. If you need
>>>> one of those, then you have ?no choice? and have to use 2.7. But
>>>> I would recommend to use ?from __future__' to make the 2.7 code
>>>> as much as possible 3 compliant.
>>>>
>>>
>>> Please define "a lot" whilst bearing in mind green against red
>>> here https://python3wos.appspot.com/
>>
>> I just started using Python again and the first ?real? program I
>> wrote I had to write with Python 2 because the needed library
>> (libturpial, that is not listed on your link) works only with
>> Python 2. A short search about which of the two to use gives
>> similar answers to mine. And as far as I can see in my
>> neighbourhood Python 2 is almost exclusively used because used
>> libraries are only available in Python 2.
>>
>> This is not a scientifically substantiated argument, but for me
>> good enough to use a lot.
>>
>
> Have you actaully tried running libturpial with Python 3 or have you
> simply taken somebody or something's word for it? I've taken code in
> the past that was "only Python 2", run it thought the 2to3 fixer and
> job done. Perhaps you could do the same. Perhaps you've already
> tried. Again, you're the only person who actually knows.

Of-course I tried: that is why I used ?had to?. The library itself and
libraries it depends on are only existing in a 2 version (at the
moment). I write code that should work in 2 and 3 both as long as 2 is
still a significant part. I call programs with python3 (even while it
is 10 to 20 percent slower) and only when that is not possible I use
Python 2. (Except to test if code also works with Python 2.)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From bblais at gmail.com  Wed May 27 14:00:24 2015
From: bblais at gmail.com (Brian Blais)
Date: Wed, 27 May 2015 14:00:24 -0400
Subject: a more precise distance algorithm
In-Reply-To: <5563e453$0$12990$c3e8da3$5496439d@news.astraweb.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
 <5563e453$0$12990$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CADzuzGNggKfgBiq3RMtWk3GBF3ynbkjm2brdXj4cuUQwQ5yQkg@mail.gmail.com>

On Mon, May 25, 2015 at 11:11 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>
> Let's compare three methods.
>
> def naive(a, b):
>     return math.sqrt(a**2 + b**2)
>
> def alternate(a, b):
>     a, b = min(a, b), max(a, b)
>     if a == 0:  return b
>     if b == 0:  return a
>     return a * math.sqrt(1 + b**2 / a**2)


>     d1 = naive(a, b)
>     d2 = alternate(a, b)
>     d3 = math.hypot(a, b)
>

> which shows that:
>
> (1) It's not hard to find mismatches;
> (2) It's not obvious which of the three methods is more accurate.
>

Bottom line: they all suck.  :)

I ran the program you posted, and, like you, got the following two examples:

for fun in [naive, alternate, math.hypot]:
    print '%.20f' % fun(222.44802484683657,680.255801504161)

715.70320611153294976248
715.70320611153283607564
715.70320611153283607564

and

for fun in [naive, alternate, math.hypot]:
    print '%.20f' % fun(376.47153302262484,943.1877995550265)

1015.54617837194291496417
1015.54617837194280127733
1015.54617837194291496417

but when comparing to Wolfram Alpha, which calculates these out many
more decimal places, we have for the two cases:

715.7032061115328768204988784125331443593766145937358347357252...
715.70320611153294976248
715.70320611153283607564
715.70320611153283607564

1015.546178371942943007625196455666280385821355370154991424749...
1015.54617837194291496417
1015.54617837194280127733
1015.54617837194291496417

where all of the methods deviate at the 13/14 decimal place.



bb


-- 
-----------------

             bblais at gmail.com
             http://web.bryant.edu/~bblais


From dreamingforward at gmail.com  Wed May 27 14:12:07 2015
From: dreamingforward at gmail.com (zipher)
Date: Wed, 27 May 2015 11:12:07 -0700 (PDT)
Subject: different types of inheritence...
In-Reply-To: <mailman.65.1432673588.5151.python-list@python.org>
References: <544f53cb-e404-41a4-aa2e-0db27c845546@googlegroups.com>
 <c6b24dce-c0da-4753-9775-b38816b579c8@googlegroups.com>
 <mailman.65.1432673588.5151.python-list@python.org>
Message-ID: <86e8f088-9681-4b69-8f44-3022d22d8204@googlegroups.com>

On Tuesday, May 26, 2015 at 3:53:25 PM UTC-5, Michael Torrie wrote:
> On 05/26/2015 08:57 AM, zipher wrote:
> > Comprende?  I'm not trying to be cryptic here.  This is a bit of OOP
> > theory to be discussed.
> 
> No, sorry.  Maybe an actual example (with use case) would spur discussion.

In the first example, super_dict changes the behavior of <dict> *for the user*, expanding the API, but keeping all the prior method behaviors the same.  

In the second example, specialized_dict changes the behavior in the machine -- the API stays the same so the user of the code just gets new benefits from whatever the specialized_dict is improving internally *without changing his/her code*.  It's a drop-in replacement.

In other words, the second class changes the INTERNALS of <dict>.  While the first class changes the *externals* of <dict>.  Yet they both use the same class definition, but semantically are completely different.  It's TWO different definition of IS-A relationship.  Two definitions of the word "is".

m


From gvanem at yahoo.no  Wed May 27 14:13:06 2015
From: gvanem at yahoo.no (Gisle Vanem)
Date: Wed, 27 May 2015 20:13:06 +0200
Subject: Python 3.5 crashes on wrong %PYTHONHOME
Message-ID: <55660932.6010706@yahoo.no>

I just installed the 32-bit Python 3.5b4 via the Web-installer.
First, I was a bit annoyed by the fact it installed under
   'f:\ProgramFiler-x86\Python35' instead of
   'f:\ProgramFiler\Python35' as I customided for. But I guess
this is a WOW64 thing. I'm on Win-8.1 (64-bit).

But then I noticed the whole thing crashes when calling
python35!Py_FatalError. On a simple 'python -v' command.

Presumably because my env-var:
  PYTHONHOME=f:\Programfiler\Python27

(and not 'PYTHONHOME=f:\Programfiler\Python35').

Reading the message on this list, gave me the impression
Python 2.7 and Python 3.5 can co-exist with no problem.
Doesn't look the case so far. Is there another 'PYTHONxx'
env-var to override 'PYTHONHOME' in this case? Or what should
I do to keep on using both 27 and 35?

For reference, the call-stack:

   ucrtbase!abort+0x4b
   python35!Py_FatalError+0xbc
   python35!Py_InitializeEx_Private+0x45b
   python35!Py_Main+0x72c
   python+0x11df
   KERNEL32!BaseThreadInitThunk+0x24
   ntdll!__RtlUserThreadStart+0x2f
   ntdll!_RtlUserThreadStart+0x1b

And some of the modules shown in WinDbg:

   ModLoad: 1c470000 1c47c000   F:\ProgramFiler\Python35\python.exe
   ModLoad: 77720000 7788e000   C:\Windows\SYSTEM32\ntdll.dll
   ModLoad: 77500000 77640000   C:\Windows\SYSTEM32\KERNEL32.DLL
   ModLoad: 76e20000 76ef7000   C:\Windows\SYSTEM32\KERNELBASE.dll
   ModLoad: 6cd50000 6d057000   F:\ProgramFiler\Python35\python35.dll
   ModLoad: 6e7e0000 6e7f5000   C:\Windows\SYSTEM32\VCRUNTIME140.dll

(the new universal CRT from MS)


-- 
--gv


From newinferno at gmail.com  Wed May 27 14:55:52 2015
From: newinferno at gmail.com (Uladzimir Kryvian)
Date: Wed, 27 May 2015 21:55:52 +0300
Subject: Embedded Python. Debug Version and _ctypes
Message-ID: <CAFYy4bhC9k35H7dUJmHweE9pFpTJYAX+ybi7WVSDobij-PAMkg@mail.gmail.com>

Hi!
I'm trying to use embedding of Python in my program.

Simple C-program, compiled in Debug, that uses py-script that just
imports "ctypes" gives me an error about "no module named "_ctypes".

How to compile python lib in Visual Studio statically with ctypes
support? Or how to use shared ctypes lib in debug mode?


From zachary.ware+pylist at gmail.com  Wed May 27 15:03:54 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Wed, 27 May 2015 14:03:54 -0500
Subject: Python 3.5 crashes on wrong %PYTHONHOME
In-Reply-To: <55660932.6010706@yahoo.no>
References: <55660932.6010706@yahoo.no>
Message-ID: <CAKJDb-NP-cFf+RXprO2fwnDZ+Q31Dp0mRJbOcXtn2jjBDUKfbg@mail.gmail.com>

On Wed, May 27, 2015 at 1:13 PM, Gisle Vanem <gvanem at yahoo.no> wrote:
> I just installed the 32-bit Python 3.5b4 via the Web-installer.
> First, I was a bit annoyed by the fact it installed under
>   'f:\ProgramFiler-x86\Python35' instead of
>   'f:\ProgramFiler\Python35' as I customided for. But I guess
> this is a WOW64 thing. I'm on Win-8.1 (64-bit).
>
> But then I noticed the whole thing crashes when calling
> python35!Py_FatalError. On a simple 'python -v' command.
>
> Presumably because my env-var:
>  PYTHONHOME=f:\Programfiler\Python27
>
> (and not 'PYTHONHOME=f:\Programfiler\Python35').
>
> Reading the message on this list, gave me the impression
> Python 2.7 and Python 3.5 can co-exist with no problem.
> Doesn't look the case so far. Is there another 'PYTHONxx'
> env-var to override 'PYTHONHOME' in this case? Or what should
> I do to keep on using both 27 and 35?

Is there any particular reason you're setting PYTHONHOME in the first
place?  It shouldn't be necessary unless you're doing something
abnormal.  I'd try unsetting it entirely and see if things just work.

-- 
Zach


From gvanem at yahoo.no  Wed May 27 15:30:06 2015
From: gvanem at yahoo.no (Gisle Vanem)
Date: Wed, 27 May 2015 21:30:06 +0200
Subject: Python 3.5 crashes on wrong %PYTHONHOME
In-Reply-To: <CAKJDb-NP-cFf+RXprO2fwnDZ+Q31Dp0mRJbOcXtn2jjBDUKfbg@mail.gmail.com>
References: <55660932.6010706@yahoo.no>
 <CAKJDb-NP-cFf+RXprO2fwnDZ+Q31Dp0mRJbOcXtn2jjBDUKfbg@mail.gmail.com>
Message-ID: <55661B3E.5080407@yahoo.no>

Zachary Ware wrote:

> Is there any particular reason you're setting PYTHONHOME in the first
> place?

I have several GNU Makefiles that picks up this variable
(to find Python.h, python*.lib etc.). But using Python 27
and 35 w/o having a %PYTHONHOME set, seems to work fine.
I have to revise those Makefiles.

-- 
--gv


From 114piyush at gmail.com  Wed May 27 16:01:17 2015
From: 114piyush at gmail.com (Piyush Verma)
Date: Thu, 28 May 2015 01:31:17 +0530
Subject: PUDB Traceback
Message-ID: <CAJV9OM-MYJuTjBE_5-98i=5QEtLDb_H764v=XkRdPFR-8MQQKg@mail.gmail.com>

Hi,

I'm facing traceback when using pudb python debugger to debug. Any help to
resolve it.

# python -m pudb file.py <input
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/pudb/__main__.py", line 3,
in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/pudb/run.py", line 30, in
main
    steal_output=options.steal_output)
  File "/usr/local/lib/python2.7/dist-packages/pudb/__init__.py", line 84,
in runscript
    dbg.interaction(None, sys.exc_info())
  File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line 338,
in interaction
    show_exc_dialog=show_exc_dialog)
  File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line
2020, in call_with_ui
    return f(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line
2212, in interaction
    self.show_exception_dialog(exc_tuple)
  File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line
1966, in show_exception_dialog
    ("Save traceback", "save"),
  File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line
1939, in dialog
    return self.event_loop(w)[0]
  File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line
2185, in event_loop
    keys = self.screen.get_input()
  File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py", line
324, in get_input
    keys, raw = self.parse_input(None, None, self.get_available_raw_input())
  File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py", line
426, in get_available_raw_input
    codes = self._get_gpm_codes() + self._get_keyboard_codes()
  File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py", line
500, in _get_keyboard_codes
    code = self._getch_nodelay()
  File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py", line
634, in _getch_nodelay
    return self._getch(0)
  File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py", line
544, in _getch
    return ord(os.read(self._term_input_file.fileno(), 1))
TypeError: ord() expected a character, but string of length 0 found

Regards,
~Piyush
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150528/f6e15998/attachment.html>

From breamoreboy at yahoo.co.uk  Wed May 27 16:03:48 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 27 May 2015 21:03:48 +0100
Subject: SyntaxError on progress module
In-Reply-To: <87oal6m4kn.fsf@Equus.decebal.nl>
References: <csjitvF6cptU1@mid.individual.net>
 <mailman.58.1432655950.5151.python-list@python.org>
 <csla4lFjduaU2@mid.individual.net> <87y4kammxa.fsf@Equus.decebal.nl>
 <mailman.90.1432734279.5151.python-list@python.org>
 <87siaim7p0.fsf@Equus.decebal.nl>
 <mailman.91.1432738329.5151.python-list@python.org>
 <87oal6m4kn.fsf@Equus.decebal.nl>
Message-ID: <mk57v6$m5m$1@ger.gmane.org>

On 27/05/2015 16:18, Cecil Westerhof wrote:
> Op Wednesday 27 May 2015 16:51 CEST schreef Mark Lawrence:
>
>> On 27/05/2015 15:11, Cecil Westerhof wrote:
>>> Op Wednesday 27 May 2015 15:44 CEST schreef Mark Lawrence:
>>>
>>>> On 27/05/2015 09:42, Cecil Westerhof wrote:
>>>>> Op Wednesday 27 May 2015 09:30 CEST schreef alb:
>>>>>
>>>>>> But here I have another question, as a python novice is there
>>>>>> really any reason for me to use any particular version of
>>>>>> Python?
>>>>>>
>>>>>> Should I start directly with the newest? What about 2.7?
>>>>>
>>>>> In principal you should use the ?latest? 3. The only problem is
>>>>> that a lot of libraries are not converted to 3 yet. If you need
>>>>> one of those, then you have ?no choice? and have to use 2.7. But
>>>>> I would recommend to use ?from __future__' to make the 2.7 code
>>>>> as much as possible 3 compliant.
>>>>>
>>>>
>>>> Please define "a lot" whilst bearing in mind green against red
>>>> here https://python3wos.appspot.com/
>>>
>>> I just started using Python again and the first ?real? program I
>>> wrote I had to write with Python 2 because the needed library
>>> (libturpial, that is not listed on your link) works only with
>>> Python 2. A short search about which of the two to use gives
>>> similar answers to mine. And as far as I can see in my
>>> neighbourhood Python 2 is almost exclusively used because used
>>> libraries are only available in Python 2.
>>>
>>> This is not a scientifically substantiated argument, but for me
>>> good enough to use a lot.
>>>
>>
>> Have you actaully tried running libturpial with Python 3 or have you
>> simply taken somebody or something's word for it? I've taken code in
>> the past that was "only Python 2", run it thought the 2to3 fixer and
>> job done. Perhaps you could do the same. Perhaps you've already
>> tried. Again, you're the only person who actually knows.
>
> Of-course I tried: that is why I used ?had to?. The library itself and
> libraries it depends on are only existing in a 2 version (at the
> moment). I write code that should work in 2 and 3 both as long as 2 is
> still a significant part. I call programs with python3 (even while it
> is 10 to 20 percent slower) and only when that is not possible I use
> Python 2. (Except to test if code also works with Python 2.)
>

Did you, noting that you didn't actually answer my question, and also 
noting that you do ask a lot of questions yourself?  So let's try again, 
did you or did you not run libturpial through 2to3?  If yes what went 
wrong that stopped you porting it to Python 3?

You make the statement regarding Python3 "even while it is 10 to 20 
percent slower". Where is your evidence to support this statement?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From breamoreboy at yahoo.co.uk  Wed May 27 16:10:09 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 27 May 2015 21:10:09 +0100
Subject: Python 3.5 crashes on wrong %PYTHONHOME
In-Reply-To: <55661B3E.5080407@yahoo.no>
References: <55660932.6010706@yahoo.no>
 <CAKJDb-NP-cFf+RXprO2fwnDZ+Q31Dp0mRJbOcXtn2jjBDUKfbg@mail.gmail.com>
 <55661B3E.5080407@yahoo.no>
Message-ID: <mk58b3$ru7$1@ger.gmane.org>

On 27/05/2015 20:30, Gisle Vanem wrote:
> Zachary Ware wrote:
>
>> Is there any particular reason you're setting PYTHONHOME in the first
>> place?
>
> I have several GNU Makefiles that picks up this variable
> (to find Python.h, python*.lib etc.). But using Python 27
> and 35 w/o having a %PYTHONHOME set, seems to work fine.
> I have to revise those Makefiles.
>

Can you please explain precisely what you're doing, as it's not clear 
(at least to me) why you're installing Python 3.5 with the web installer 
but then mentioning GNU makefiles.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From tjreedy at udel.edu  Wed May 27 17:26:47 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 27 May 2015 17:26:47 -0400
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
In-Reply-To: <CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com>
References: <CAPkN8xKTXJu2nhvocG8KuyO1XkJVfK_WsmY6dM=hWsVyg+BVyA@mail.gmail.com>
 <CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com>
Message-ID: <55663697.9040108@udel.edu>

On 5/27/2015 7:15 AM, anatoly techtonik wrote:
> Hi.
>
> This was labelled offtopic in python-ideas, so I edited and forwarded
> it here. Please CC as I am not subscribed.

I am not subcribed either, but I do not need or want CCs.
python-list is mirrored at news.gmane.org where you can access it as 
either newsgroup gmane.comp.python.idle or via their web interface. 
Many mail programs have embedded news readers, and there are standalone 
news readers.  I have used both Outlook Express and now Thunderbird.

I am CCing this this time, but it is a nuisance for me, so just this once.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Wed May 27 17:59:08 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 27 May 2015 17:59:08 -0400
Subject: should "self" be changed?
In-Reply-To: <CAPTjJmpZ3Tc1TBa5TokC+P8YLRs2jxAykPJxO7fUKwb9tvpiGQ@mail.gmail.com>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mk2kte$opk$2@speranza.aioe.org>
 <mailman.64.1432673145.5151.python-list@python.org>
 <556554d9$0$2796$c3e8da3$76491128@news.astraweb.com>
 <CAPTjJmpZ3Tc1TBa5TokC+P8YLRs2jxAykPJxO7fUKwb9tvpiGQ@mail.gmail.com>
Message-ID: <mk5env$5k5$1@ger.gmane.org>

On 5/27/2015 2:32 AM, Chris Angelico wrote:
> On Wed, May 27, 2015 at 3:23 PM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>> On Wednesday 27 May 2015 06:45, Mark Lawrence wrote:
>>
>>> Apart from breaking all the tools that rely on "self" being spelt "self"
>>> this looks like an excellent idea.
>>
>> Tools which rely on self being spelled "self" are already broken. It's a
>> convention, nothing more, and there are various good reasons for breaking
>> the convention:
>>
>> - metaclasses
>> - classmethods
>> - custom descriptors
>> - nested classes
>
> If it truly relies on it, yes. But if I see a function that has "self"
> as its first parameter, I'm going to read it as a (probable) method
> rather than a stand-alone function, and it wouldn't surprise me if
> introspection and/or source code parsing made the same assumption.

The Idlelib calltips function does not look at names.  It leaves the 
signature of functions as is and removes the first name, whatever it is, 
of bound methods, including bound class methods.  See 
idlelib.CallTips.get_argspec for details. To make sure there is no name 
assumption, test_calltips.py tests that the first name, not 'self', is 
the name removed.  One test method, Tc.t6, has function signature "(no, 
self)", which makes the bound method signature "(self)".


-- 
Terry Jan Reedy



From oscar.j.benjamin at gmail.com  Wed May 27 18:03:49 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 27 May 2015 23:03:49 +0100
Subject: a more precise distance algorithm
In-Reply-To: <CADzuzGNggKfgBiq3RMtWk3GBF3ynbkjm2brdXj4cuUQwQ5yQkg@mail.gmail.com>
References: <b2e66a94-7a89-4be9-bbf7-9434396cc178@googlegroups.com>
 <5563e453$0$12990$c3e8da3$5496439d@news.astraweb.com>
 <CADzuzGNggKfgBiq3RMtWk3GBF3ynbkjm2brdXj4cuUQwQ5yQkg@mail.gmail.com>
Message-ID: <CAHVvXxQQWqgAVuJc362yyWoO9bzTSz_PoBpkqiA4_jLKE554dg@mail.gmail.com>

On 27 May 2015 at 19:00, Brian Blais <bblais at gmail.com> wrote:
> On Mon, May 25, 2015 at 11:11 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>>
>> Let's compare three methods.
>>
>> def naive(a, b):
>>     return math.sqrt(a**2 + b**2)
>>
>> def alternate(a, b):
>>     a, b = min(a, b), max(a, b)
>>     if a == 0:  return b
>>     if b == 0:  return a
>>     return a * math.sqrt(1 + b**2 / a**2)
>
>
>>     d1 = naive(a, b)
>>     d2 = alternate(a, b)
>>     d3 = math.hypot(a, b)
>>
>
>> which shows that:
>>
>> (1) It's not hard to find mismatches;
>> (2) It's not obvious which of the three methods is more accurate.
>>
>
> Bottom line: they all suck.  :)
>
> I ran the program you posted, and, like you, got the following two examples:
>
> for fun in [naive, alternate, math.hypot]:
>     print '%.20f' % fun(222.44802484683657,680.255801504161)
>
> 715.70320611153294976248
> 715.70320611153283607564
> 715.70320611153283607564
>
> and
>
> for fun in [naive, alternate, math.hypot]:
>     print '%.20f' % fun(376.47153302262484,943.1877995550265)
>
> 1015.54617837194291496417
> 1015.54617837194280127733
> 1015.54617837194291496417
>
> but when comparing to Wolfram Alpha, which calculates these out many
> more decimal places, we have for the two cases:
>
> 715.7032061115328768204988784125331443593766145937358347357252...
> 715.70320611153294976248
> 715.70320611153283607564
> 715.70320611153283607564
>
> 1015.546178371942943007625196455666280385821355370154991424749...
> 1015.54617837194291496417
> 1015.54617837194280127733
> 1015.54617837194291496417
>
> where all of the methods deviate at the 13/14 decimal place.

So they have 12/13 correct digits after the decimal point. Including
the digits before the decimal point they all have 15/16 correct
decimal digits. This is exactly what you should expect when using
double precision floating point. Feel free to print as many digits as
you like but the extra ones won't add any more accuracy.

The difference between the answers you showed are just 1 ULP:
>>> a = 715.70320611153283607564
>>> b = 715.70320611153294976248
>>> a
715.7032061115328
>>> a + 5e-14
715.7032061115328
>>> a + 6e-14
715.703206111533
>>> a + 6e-14 == b
True

Since you don't show the source numbers we can't know whether the true
result lies between these two or not.


Oscar


From vincent at vincentdavis.net  Wed May 27 18:09:27 2015
From: vincent at vincentdavis.net (Vincent Davis)
Date: Wed, 27 May 2015 16:09:27 -0600
Subject: assertRaises() help
Message-ID: <CALyJZZV9DjUbyYRfGoaWzPG1FVnVx0+AsSOyud_ZOyE914A55Q@mail.gmail.com>

I am trying to add a test to pandas Int the first case I assert that I get
a NaT value, in the second I what to test that I get a value error.
def
test_day_not_in_month_coerce_true
() works
I am trying to duplicate them with coerce=
False which
will give a ValueError but I cant get the tests to work.


class TestDaysInMonth(tm.TestCase):
    def test_day_not_in_month_coerce_true(self):
        self.assertTrue(isnull(to_datetime('2015-02-29', coerce=True)))
        self.assertTrue(isnull(to_datetime('2015-02-29', format="%Y-%m-%d",
coerce=True)))
        self.assertTrue(isnull(to_datetime('2015-02-32', format="%Y-%m-%d",
coerce=True)))
        self.assertTrue(isnull(to_datetime('2015-04-31', format="%Y-%m-%d",
coerce=True)))
    def test_day_not_in_month_coerce_false(self):
        self.assertRaises(ValueError, to_datetime, '2015-02-29',
coerce=False)

what I get is

FAIL: test_day_not_in_month_coerce_false
(pandas.tests.test_tseries.TestDaysInMonth)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/vmd/GitHub/pandas_vmd/pandas/tests/test_tseries.py", line
747, in test_day_not_in_month_coerce_false
    self.assertRaises(ValueError, to_datetime, '2015-02-29', coerce=False)
  File "/Users/vmd/GitHub/pandas_vmd/pandas/util/testing.py", line 1576, in
assertRaises
    _callable(*args, **kwargs)
  File "/Users/vmd/GitHub/pandas_vmd/pandas/util/testing.py", line 1640, in
__exit__
    raise AssertionError("{0} not raised.".format(name))
AssertionError: ValueError not raised.

>From the docs maybe I should be using a "with" statement?.


Vincent Davis
720-301-3003
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150527/3609c958/attachment.html>

From karthik.sharma at gmail.com  Wed May 27 18:23:31 2015
From: karthik.sharma at gmail.com (Karthik Sharma)
Date: Wed, 27 May 2015 15:23:31 -0700 (PDT)
Subject: Decoding JSON file using python
Message-ID: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>

I have the JSON structure shown below and the python code shown below to manipulate the JSON structure.

    import json
    
    json_input = { 
        "msgType": "0",
        "tid": "1",
        "data": "[{\"Severity\":\"warn\",\"Subject\":\"Reporting \",\"Message\":\"tdetails:{\\\"Product\\\":\\\"Gecko\\\",\\\"CPUs\\\":8,\\\"Language\\\":\\\"en-GB\\\",\\\"isEvent\\\":\\\">
        "Timestamp": "1432703193431",
        "Host": "myserver.com",
        "Agent": "Experience",
        "AppName": "undefined",
        "AppInstance": "my_server",
        "Group": "UndefinedGroup"
    }
    
    
    data = json_input['data']
    tdetails = data['Message']
    print('json_input {} \n\ndata {} \n\n tdetails {}\n\n'.format(json_input,data,tdetails))  

I am getting the error.

    Traceback (most recent call last):
      File "test_json.py", line 19, in <module>
        tdetails = data['Message']
    TypeError: string indices must be integers, not str

The JSON structure is valid as shown by http://jsonlint.com/
   I want to be able to access the different fields inside `data` such as `severity`, `subject` and also fields inside `tdetails` such as `CPUs` and `Product`. How do I do this?


From cs at zip.com.au  Wed May 27 18:52:49 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 28 May 2015 08:52:49 +1000
Subject: Decoding JSON file using python
In-Reply-To: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>
References: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>
Message-ID: <20150527225248.GA48231@cskk.homeip.net>

On 27May2015 15:23, Karthik Sharma <karthik.sharma at gmail.com> wrote:
>I have the JSON structure shown below and the python code shown below to manipulate the JSON structure.
>
>    import json
>
>    json_input = {
>        "msgType": "0",
>        "tid": "1",
>        "data": "[{\"Severity\":\"warn\",\"Subject\":\"Reporting \",\"Message\":\"tdetails:{\\\"Product\\\":\\\"Gecko\\\",\\\"CPUs\\\":8,\\\"Language\\\":\\\"en-GB\\\",\\\"isEvent\\\":\\\">
>        "Timestamp": "1432703193431",
>        "Host": "myserver.com",
>        "Agent": "Experience",
>        "AppName": "undefined",
>        "AppInstance": "my_server",
>        "Group": "UndefinedGroup"
>    }
>
>
>    data = json_input['data']
>    tdetails = data['Message']
>    print('json_input {} \n\ndata {} \n\n tdetails {}\n\n'.format(json_input,data,tdetails))
>
>I am getting the error.
>
>    Traceback (most recent call last):
>      File "test_json.py", line 19, in <module>
>        tdetails = data['Message']
>    TypeError: string indices must be integers, not str

That will be because of this line:

  tdetails = data['Message']

because "data" is just the string from your dict.

>The JSON structure is valid as shown by http://jsonlint.com/
>   I want to be able to access the different fields inside `data` such as `severity`, `subject` and also fields inside `tdetails` such as `CPUs` and `Product`. How do I do this?

Then you need to decode "data". Example (untested):

  data_decoded = json.loads(data)

and then access:

  data_decoded['Message']

Cheers,
Cameron Simpson <cs at zip.com.au>

Here's a great .sig I wrote, so good it doesn't rhyme.
        Jon Benger <jbenger at agravaine.st.nepean.uws.edu.au>


From cs at zip.com.au  Wed May 27 18:55:54 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 28 May 2015 08:55:54 +1000
Subject: assertRaises() help
In-Reply-To: <CALyJZZV9DjUbyYRfGoaWzPG1FVnVx0+AsSOyud_ZOyE914A55Q@mail.gmail.com>
References: <CALyJZZV9DjUbyYRfGoaWzPG1FVnVx0+AsSOyud_ZOyE914A55Q@mail.gmail.com>
Message-ID: <20150527225554.GA55582@cskk.homeip.net>

On 27May2015 16:09, Vincent Davis <vincent at vincentdavis.net> wrote:
>I am trying to add a test to pandas Int the first case I assert that I get
>a NaT value, in the second I what to test that I get a value error.
>def
>test_day_not_in_month_coerce_true
>() works
>I am trying to duplicate them with coerce=
>False which
>will give a ValueError but I cant get the tests to work.
>
>
>class TestDaysInMonth(tm.TestCase):
>    def test_day_not_in_month_coerce_true(self):
>        self.assertTrue(isnull(to_datetime('2015-02-29', coerce=True)))
>        self.assertTrue(isnull(to_datetime('2015-02-29', format="%Y-%m-%d",
>coerce=True)))
>        self.assertTrue(isnull(to_datetime('2015-02-32', format="%Y-%m-%d",
>coerce=True)))
>        self.assertTrue(isnull(to_datetime('2015-04-31', format="%Y-%m-%d",
>coerce=True)))
>    def test_day_not_in_month_coerce_false(self):
>        self.assertRaises(ValueError, to_datetime, '2015-02-29',
>coerce=False)
>
>what I get is
>
>FAIL: test_day_not_in_month_coerce_false
>(pandas.tests.test_tseries.TestDaysInMonth)
>----------------------------------------------------------------------
[...]
>AssertionError: ValueError not raised.
>
>From the docs maybe I should be using a "with" statement?.

Unlikely.

First, test your test by hand running:

  to_datetime('2015-02-29', coerce=False)

_Does_ it raise ValueError?

Cheers,
Cameron Simpson <cs at zip.com.au>

Q: How many user support people does it take to change a light bulb?
A: We have an exact copy of the light bulb here and it seems to be
   working fine.  Can you tell me what kind of system you have?


From markos at c2o.pro.br  Wed May 27 18:56:03 2015
From: markos at c2o.pro.br (Markos)
Date: Wed, 27 May 2015 19:56:03 -0300
Subject: OpenCV with Python (cv or cv2)
In-Reply-To: <201505261421.t4QELBUd015477@fido.openend.se>
References: <5564740E.3090206@c2o.pro.br>
 <201505261421.t4QELBUd015477@fido.openend.se>
Message-ID: <55664B83.30809@c2o.pro.br>

Hi Laura,

On 26-05-2015 11:21, Laura Creighton wrote:
> In a message of Tue, 26 May 2015 10:24:30 -0300, Markos writes:
>    
>> Hi,
>>
>> I want to use OpenCV with Python.
>>
>> I installed version 2.4.9
>> (http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.4.9/opencv-2.4.9.zip/)
>> in debian Squeeze running Python 2.6.6. Using the tutorial:
>> http://indranilsinharoy.com/2012/11/01/installing-opencv-on-linux/
>>
>> (I tried to install versions 2.4.11 and 2.4.10 (in Squeeze with python
>> 2.2.6) but I couldn't.)
>>      
> You mean python 2.6.6, right?
> and this package: https://packages.debian.org/source/squeeze/opencv
> which is 2.1.0-3 is too old for you?
>
>    

I installed this package (python-opencv) but I thought this package 
would be only some API for python to use the opencv library and I 
imagined that I would still have to install the opencv library from source.

> What do you mean that you
> tried to install versions 2.4.11 and 2.4.10 but couldn't?  What
> error messages did you get?
>
>    
For example when trying to install the 2.4.11 version after running cmake:

cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D 
WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D 
INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON 
-D WITH_QT=ON -D WITH_OPENGL=ON ..

(Output of cmake at the end of message.)

and make -j2 to compile, but an error occurred after compiling 55%:

[ 55%] Building CXX object 
modules/legacy/CMakeFiles/opencv_legacy.dir/src/facedetection.cpp.o
Linking CXX shared library ../../lib/libopencv_legacy.so
[ 55%] Built target opencv_legacy
make: ** [all] Erro 2

>> I lost some time trying to load the module cv2:
>>
>>      
>>>>> Import cv2.cv the cv
>>>>>            
> s/the/as/ ?
>    

import cv2.cv as cv


>
>    
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in<module>>
>> ImportError: No module named cv2.cv
>>
>> After a while I realized I could use the cv module. (import cv)
>>
>> But still I do not understand the differences between the modules cv and
>> cv2.
>>
>> Do you suggest any good tutorial about the differences between cv and cv2?
>>
>> I will have to make many changes in python programs using cv to reuse
>> the code later with cv2?
>>
>> Thanks,
>> Markos
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>>      
> post the output you got from cmake
>    

At the end of message.

> It will probably complain that it is missing some headers, so couldn't
> make libcv2.so
>
> If you haven't installed python-dev (not just python) and numpy I
> guarantee it will not find needed headers; if you have but it is
> still not being made we will have to look harder at what is going wrong.
>
>    
I installed the following packages:

apt-get install cmake
apt-get install build-essential
apt-get install pkg-config
apt-get install libgtk2.0-dev
apt-get install python-opencv python-dev python-numpy
apt-get install libpng12-0 libpng12-dev libpng++-dev libpng3 libpnglite-dev libpngwriter0-dev libpngwriter0c2 zlib1g-dbg
zlib1g zlib1g-dev pngtools libjasper-dev libjasper-runtime libjasper1 libjpeg8 libjpeg8-dbg libjpeg62 libjpeg62-dev 

libjpeg-progs libtiff4-dev libtiff4 libtiffxx0c2 libtiff-tools ffmpeg libavcodec-dev libavcodec52 libavformat52 

libavformat-dev libswscale0 libswscale-dev openexr libopenexr6 libopenexr-dev

apt-get install libgstreamer0.10-0-dbg libgstreamer0.10-0 libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev 

libxine1-ffmpeg libxine-dev libxine1-bin libunicap2 libunicap2-dev libucil2 libucil2-dev libdc1394-22-dev 

libdc1394-22 libdc1394-utils libv4l-0 libv4l-dev
apt-get install libqt4-dev

> If it made a libcv.so for you, did it put it in a place you can see
> with your PYTHONPATH?
>
> Laura
>
>
>    
 >>> import sys
 >>> print sys.path
['', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', 
'/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old', 
'/usr/lib/python2.6/lib-dynload', 
'/usr/local/lib/python2.6/dist-packages', 
'/usr/lib/python2.6/dist-packages', 
'/usr/lib/python2.6/dist-packages/PIL', 
'/usr/lib/python2.6/dist-packages/gst-0.10', 
'/usr/lib/pymodules/python2.6', '/usr/lib/pymodules/python2.6/gtk-2.0', 
'/usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode']

import cv works fine but I didn't find libcv.so in my system.

I suppose that cv had already been installed by the package 
python-opencv, and the compilation of the opencv didn't install cv2.

Thanks for your attention,
Markos


Return of cmake
--     Linker flags (Release):
--     Linker flags (Debug):
--     Precompiled headers:         YES
-- 
--   OpenCV modules:
--     To be built:                 core flann imgproc highgui 
features2d calib3d ml video legacy objdetect photo gpu ocl nonfree 
contrib stitching superres ts videostab
--     Disabled:                    world
--     Disabled by dependency:      -
--     Unavailable:                 androidcamera dynamicuda java python viz
-- 
--   GUI:
--     QT 4.x:                      YES (ver 4.6.3 EDITION = OpenSource)
--     QT OpenGL support:           YES (/usr/lib/libQtOpenGL.so)
--     OpenGL support:              YES (/usr/lib/libGLU.so 
/usr/lib/libGL.so /usr/lib/libSM.so /usr/lib/libICE.so 
/usr/lib/libX11.so /usr/lib/libXext.so)
--     VTK support:                 NO
-- 
--   Media I/O:
--     ZLib:                        /usr/lib/libz.so (ver 1.2.3.4)
--     JPEG:                        /usr/lib/libjpeg.so (ver 62)
--     PNG:                         /usr/lib/libpng.so (ver 1.2.44)
--     TIFF:                        /usr/lib/libtiff.so (ver 42 - 3.9.4)
--     JPEG 2000:                   /usr/lib/libjasper.so (ver 1.900.1)
--     OpenEXR:                     /usr/lib/libImath.so 
/usr/lib/libIlmImf.so /usr/lib/libIex.so /usr/lib/libHalf.so 
/usr/lib/libIlmThread.so (ver 1.6.1)
-- 
--   Video I/O:
--     DC1394 1.x:                  NO
--     DC1394 2.x:                  YES (ver 2.1.2)
--     FFMPEG:                      YES
--       codec:                     YES (ver 52.123.0)
--       format:                    YES (ver 52.111.0)
--       util:                      YES (ver 50.43.0)
--       swscale:                   YES (ver 0.14.1)
--       gentoo-style:              YES
--     GStreamer:
--       base:                      YES (ver 0.10.30)
--       video:                     YES (ver 0.10.30)
--       app:                       YES (ver 0.10.30)
--       riff:                      YES (ver 0.10.30)
--       pbutils:                   YES (ver 0.10.30)
--     OpenNI:                      NO
--     OpenNI PrimeSensor Modules:  NO
--     PvAPI:                       NO
--     GigEVisionSDK:               NO
--     UniCap:                      NO
--     UniCap ucil:                 NO
--     V4L/V4L2:                    Using libv4l1 (ver 0.8.0) / libv4l2 
(ver 0.8.0)
--     XIMEA:                       NO
--     Xine:                        NO
-- 
--   Other third-party libraries:
--     Use IPP:                     NO
--     Use Eigen:                   NO
--     Use TBB:                     NO
--     Use OpenMP:                  NO
--     Use GCD                      NO
--     Use Concurrency              NO
--     Use C=:                      NO
--     Use Cuda:                    NO
--     Use OpenCL:                  YES
-- 
--   OpenCL:
--     Version:                     dynamic
--     Include path:                
~/Programacao/Python/OpenCV/opencv-2.4.11/3rdparty/include/opencl/1.2
--     Use AMD FFT:                 NO
--     Use AMD BLAS:                NO
-- 
--   Python:
--     Interpreter:                 /usr/bin/python2.6 (ver 2.6.6)
--     Libraries:                   NO
--     numpy:                       
/usr/lib/pymodules/python2.6/numpy/core/include (ver 1.4.1)
--     packages path:               lib/python2.6/dist-packages
-- 
--   Java:
--     ant:                         NO
--     JNI:                         /usr/lib/jvm/java-6-sun/include 
/usr/lib/jvm/java-6-sun/include/linux /usr/lib/jvm/java-6-sun/include
--     Java tests:                  NO
-- 
--   Documentation:
--     Build Documentation:         NO
--     Sphinx:                      NO
--     PdfLaTeX compiler:           /usr/bin/pdflatex
--     Doxygen:                     NO
-- 
--   Tests and samples:
--     Tests:                       YES
--     Performance tests:           YES
--     C/C++ Examples:              YES
-- 
--   Install path:                  /usr/local
-- 
--   cvconfig.h is in:              
~/Programacao/Python/OpenCV/opencv-2.4.11/build
-- -----------------------------------------------------------------
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: 
~/Programacao/Python/OpenCV/opencv-2.4.11/build



From random832 at fastmail.us  Wed May 27 19:06:50 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 27 May 2015 19:06:50 -0400
Subject: Decoding JSON file using python
In-Reply-To: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>
References: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>
Message-ID: <1432768010.3366327.279927809.42D90FE2@webmail.messagingengine.com>

On Wed, May 27, 2015, at 18:23, Karthik Sharma wrote:
> The JSON structure is valid as shown by http://jsonlint.com/
>    I want to be able to access the different fields inside `data` such as
>    `severity`, `subject` and also fields inside `tdetails` such as `CPUs`
>    and `Product`. How do I do this?

your data object is a string, it doesn't have anything inside it. if you
want it to be a json object you'll need to actually use the json parser
on it.


From python at mrabarnett.plus.com  Wed May 27 19:08:42 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 28 May 2015 00:08:42 +0100
Subject: Decoding JSON file using python
In-Reply-To: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>
References: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>
Message-ID: <55664E7A.6080807@mrabarnett.plus.com>

On 2015-05-27 23:23, Karthik Sharma wrote:
> I have the JSON structure shown below and the python code shown below to manipulate the JSON structure.
>
>      import json
>
>      json_input = {
>          "msgType": "0",
>          "tid": "1",
>          "data": "[{\"Severity\":\"warn\",\"Subject\":\"Reporting \",\"Message\":\"tdetails:{\\\"Product\\\":\\\"Gecko\\\",\\\"CPUs\\\":8,\\\"Language\\\":\\\"en-GB\\\",\\\"isEvent\\\":\\\">
>          "Timestamp": "1432703193431",
>          "Host": "myserver.com",
>          "Agent": "Experience",
>          "AppName": "undefined",
>          "AppInstance": "my_server",
>          "Group": "UndefinedGroup"
>      }
>
>
>      data = json_input['data']
>      tdetails = data['Message']
>      print('json_input {} \n\ndata {} \n\n tdetails {}\n\n'.format(json_input,data,tdetails))
>
> I am getting the error.
>
>      Traceback (most recent call last):
>        File "test_json.py", line 19, in <module>
>          tdetails = data['Message']
>      TypeError: string indices must be integers, not str
>
> The JSON structure is valid as shown by http://jsonlint.com/
>     I want to be able to access the different fields inside `data` such as `severity`, `subject` and also fields inside `tdetails` such as `CPUs` and `Product`. How do I do this?
>
json_input['Message'] is a string, not a dict.


From python at mrabarnett.plus.com  Wed May 27 19:11:11 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 28 May 2015 00:11:11 +0100
Subject: PUDB Traceback
In-Reply-To: <CAJV9OM-MYJuTjBE_5-98i=5QEtLDb_H764v=XkRdPFR-8MQQKg@mail.gmail.com>
References: <CAJV9OM-MYJuTjBE_5-98i=5QEtLDb_H764v=XkRdPFR-8MQQKg@mail.gmail.com>
Message-ID: <55664F0F.3030107@mrabarnett.plus.com>

On 2015-05-27 21:01, Piyush Verma wrote:
> Hi,
>
> I'm facing traceback when using pudb python debugger to debug. Any help
> to resolve it.
>
> # python -m pudb file.py <input
> Traceback (most recent call last):
>    File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
>      "__main__", fname, loader, pkg_name)
>    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
>      exec code in run_globals
>    File "/usr/local/lib/python2.7/dist-packages/pudb/__main__.py", line
> 3, in <module>
>      main()
>    File "/usr/local/lib/python2.7/dist-packages/pudb/run.py", line 30,
> in main
>      steal_output=options.steal_output)
>    File "/usr/local/lib/python2.7/dist-packages/pudb/__init__.py", line
> 84, in runscript
>      dbg.interaction(None, sys.exc_info())
>    File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line
> 338, in interaction
>      show_exc_dialog=show_exc_dialog)
>    File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line
> 2020, in call_with_ui
>      return f(*args, **kwargs)
>    File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line
> 2212, in interaction
>      self.show_exception_dialog(exc_tuple)
>    File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line
> 1966, in show_exception_dialog
>      ("Save traceback", "save"),
>    File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line
> 1939, in dialog
>      return self.event_loop(w)[0]
>    File "/usr/local/lib/python2.7/dist-packages/pudb/debugger.py", line
> 2185, in event_loop
>      keys = self.screen.get_input()
>    File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py",
> line 324, in get_input
>      keys, raw = self.parse_input(None, None,
> self.get_available_raw_input())
>    File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py",
> line 426, in get_available_raw_input
>      codes = self._get_gpm_codes() + self._get_keyboard_codes()
>    File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py",
> line 500, in _get_keyboard_codes
>      code = self._getch_nodelay()
>    File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py",
> line 634, in _getch_nodelay
>      return self._getch(0)
>    File "/usr/local/lib/python2.7/dist-packages/urwid/raw_display.py",
> line 544, in _getch
>      return ord(os.read(self._term_input_file.fileno(), 1))
> TypeError: ord() expected a character, but string of length 0 found
>
> Regards,
> ~Piyush
>
>
At a guess I'd say it has reached the end of the input, so os.read(...)
is returning an empty string.



From vincent at vincentdavis.net  Wed May 27 19:13:36 2015
From: vincent at vincentdavis.net (Vincent Davis)
Date: Wed, 27 May 2015 17:13:36 -0600
Subject: assertRaises() help
In-Reply-To: <20150527225554.GA55582@cskk.homeip.net>
References: <CALyJZZV9DjUbyYRfGoaWzPG1FVnVx0+AsSOyud_ZOyE914A55Q@mail.gmail.com>
 <20150527225554.GA55582@cskk.homeip.net>
Message-ID: <CALyJZZWAL8O5DH9bq+92hRVFGmsjfVdx=jpu9k0_231J_5rVBw@mail.gmail.com>

On Wed, May 27, 2015 at 4:55 PM, Cameron Simpson <cs at zip.com.au> wrote:

> First, test your test by hand running:
>
>  to_datetime('2015-02-29', coerce=False)
>
> _Does_ it raise ValueError?
>

?Well that was not expected.? Thanks


Vincent Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150527/a316c5f9/attachment.html>

From karthik.sharma at gmail.com  Wed May 27 19:51:57 2015
From: karthik.sharma at gmail.com (Karthik Sharma)
Date: Wed, 27 May 2015 16:51:57 -0700 (PDT)
Subject: Decoding JSON file using python
In-Reply-To: <mailman.108.1432768474.5151.python-list@python.org>
References: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>
 <mailman.108.1432768474.5151.python-list@python.org>
Message-ID: <75320b35-72b6-4aae-a47f-630bd7bef812@googlegroups.com>

I tried modifying the program as follows as per your suggestion.Doesn't seem to work.

import simplejson as json                                                                                                                                                                
import cjson

json_input = { "msgType": "0",
    "tid": "1",
    "data": "[{\"Severity\":\"warn\",\"Subject\":\"Reporting \",\"Message\":\"tdetails:{\\\"Product\\\":\\\"Gecko\\\",\\\"CPUs\\\":8,\\\"Language\\\":\\\"en-GB\\\",\\\"isEvent\\\":\\\">
    "Timestamp": "1432703193431",
    "Host": "myserver.com",
    "Agent": "Experience",
    "AppName": "undefined",
    "AppInstance": "my_server",
    "Group": "UndefinedGroup"
}

print('json input original  {} \n\n'.format(json_input))

data = json_input['data']

print('data {} \n\n'.format(data))

message = json.loads(data)

print('message {} \n\n'.format(message['Message']))

I get the following error.

Traceback (most recent call last):
  File "test_json.py", line 23, in <module>
    print('message {} \n\n'.format(message['Message']))
TypeError: list indices must be integers, not str
karthik.sharma at aukksharma2:~$ vim test_json.py 





On Thursday, 28 May 2015 11:14:44 UTC+12, Cameron Simpson  wrote:
> On 27May2015 15:23, Karthik Sharma <karthik.sharma at gmail.com> wrote:
> >I have the JSON structure shown below and the python code shown below to manipulate the JSON structure.
> >
> >    import json
> >
> >    json_input = {
> >        "msgType": "0",
> >        "tid": "1",
> >        "data": "[{\"Severity\":\"warn\",\"Subject\":\"Reporting \",\"Message\":\"tdetails:{\\\"Product\\\":\\\"Gecko\\\",\\\"CPUs\\\":8,\\\"Language\\\":\\\"en-GB\\\",\\\"isEvent\\\":\\\">
> >        "Timestamp": "1432703193431",
> >        "Host": "myserver.com",
> >        "Agent": "Experience",
> >        "AppName": "undefined",
> >        "AppInstance": "my_server",
> >        "Group": "UndefinedGroup"
> >    }
> >
> >
> >    data = json_input['data']
> >    tdetails = data['Message']
> >    print('json_input {} \n\ndata {} \n\n tdetails {}\n\n'.format(json_input,data,tdetails))
> >
> >I am getting the error.
> >
> >    Traceback (most recent call last):
> >      File "test_json.py", line 19, in <module>
> >        tdetails = data['Message']
> >    TypeError: string indices must be integers, not str
> 
> That will be because of this line:
> 
>   tdetails = data['Message']
> 
> because "data" is just the string from your dict.
> 
> >The JSON structure is valid as shown by http://jsonlint.com/
> >   I want to be able to access the different fields inside `data` such as `severity`, `subject` and also fields inside `tdetails` such as `CPUs` and `Product`. How do I do this?
> 
> Then you need to decode "data". Example (untested):
> 
>   data_decoded = json.loads(data)
> 
> and then access:
> 
>   data_decoded['Message']
> 
> Cheers,
> Cameron Simpson <cs at zip.com.au>
> 
> Here's a great .sig I wrote, so good it doesn't rhyme.
>         Jon Benger <jbenger at agravaine.st.nepean.uws.edu.au>



From cs at zip.com.au  Wed May 27 20:21:05 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 28 May 2015 10:21:05 +1000
Subject: assertRaises() help
In-Reply-To: <CALyJZZWAL8O5DH9bq+92hRVFGmsjfVdx=jpu9k0_231J_5rVBw@mail.gmail.com>
References: <CALyJZZWAL8O5DH9bq+92hRVFGmsjfVdx=jpu9k0_231J_5rVBw@mail.gmail.com>
Message-ID: <20150528002105.GA90148@cskk.homeip.net>

On 27May2015 17:13, Vincent Davis <vincent at vincentdavis.net> wrote:
>On Wed, May 27, 2015 at 4:55 PM, Cameron Simpson <cs at zip.com.au> wrote:
>> First, test your test by hand running:
>>  to_datetime('2015-02-29', coerce=False)
>> _Does_ it raise ValueError?
>
>?Well that was not expected.? Thanks

See? If you'd just believed your unit test... :-)

Welcome to test driven development!

Cheers,
Cameron Simpson <cs at zip.com.au>

Acceptance Testing: Dropping your mods straight into the production
                    environment to see if the users will accept them.


From steve at pearwood.info  Wed May 27 21:38:05 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 28 May 2015 11:38:05 +1000
Subject: assertRaises() help
References: <mailman.103.1432764591.5151.python-list@python.org>
Message-ID: <5566717e$0$13003$c3e8da3$5496439d@news.astraweb.com>

On Thu, 28 May 2015 08:09 am, Vincent Davis wrote:

> I am trying to add a test to pandas Int the first case I assert that I get
> a NaT value, in the second I what to test that I get a value error.

Your sentence has just enough typos to lead me to doubt I understand you.
What do you mean, add a test "to pandas Int the first case..." ?

What's a NaT value?


> def
> test_day_not_in_month_coerce_true
> () works

If it works, why is it relevant to your question?


> I am trying to duplicate them with coerce=
> False which
> will give a ValueError but I cant get the tests to work.
[...]

>     def test_day_not_in_month_coerce_false(self):
>         self.assertRaises(ValueError, to_datetime, '2015-02-29',
> coerce=False)


What leads you to believe that to_datetime('2015-02-29', coerce=False) will
raise ValueError?


> what I get is
> 
> FAIL: test_day_not_in_month_coerce_false
> (pandas.tests.test_tseries.TestDaysInMonth)

This failure clearly shows that to_datetime(...) does not raise ValueError.

I don't see anything obviously wrong in your use of the assertRaises method,
*assuming* that it is the assertRaises method from the standard library
unittest module. If it is some custom method written by you, or part of
pandas, then I have no idea if you are doing something wrong.

But I expect that the real problem is that you are mistaken to believe that
to_datetime('2015-02-29', coerce=False) raises ValueError.




-- 
Steven



From jon+usenet at unequivocal.co.uk  Wed May 27 21:38:46 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Thu, 28 May 2015 01:38:46 +0000 (UTC)
Subject: Decoding JSON file using python
References: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>
 <mailman.108.1432768474.5151.python-list@python.org>
 <75320b35-72b6-4aae-a47f-630bd7bef812@googlegroups.com>
Message-ID: <slrnmmcsfd.1t4.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-27, Karthik Sharma <karthik.sharma at gmail.com> wrote:
> I tried modifying the program as follows as per your
> suggestion.Doesn't seem to work.

That's because you didn't modify the program as per their suggestion,
you made completely different changes that bore no relation to what
they said.

> import cjson

What do you think 'import json' (or 'import cjson', or whatever) does?
Hint: it doesn't cause Python to henceforth treat all strings as if
they might contain JSON and to interpret them as such just in case.
As Cameron already said, you have to actually *call* the JSON decoder
to convert the data from its JSON string representation into Python
objects.


From steve at pearwood.info  Wed May 27 21:39:47 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 28 May 2015 11:39:47 +1000
Subject: SyntaxError on progress module
References: <csjitvF6cptU1@mid.individual.net>
 <mailman.58.1432655950.5151.python-list@python.org>
 <csla4lFjduaU2@mid.individual.net> <87y4kammxa.fsf@Equus.decebal.nl>
 <mailman.90.1432734279.5151.python-list@python.org>
 <87siaim7p0.fsf@Equus.decebal.nl>
 <mailman.91.1432738329.5151.python-list@python.org>
 <87oal6m4kn.fsf@Equus.decebal.nl>
 <mailman.98.1432757038.5151.python-list@python.org>
Message-ID: <556671e3$0$13003$c3e8da3$5496439d@news.astraweb.com>

On Thu, 28 May 2015 06:03 am, Mark Lawrence wrote:

> You make the statement regarding Python3 "even while it is 10 to 20
> percent slower". Where is your evidence to support this statement?


Its well known that Python 3 is generally slower than Python 2. Cecil's
claim shouldn't be controversial, any more than "Python is generally about
10 to 100 times slower than C" should be controversial.


But for what it's worth:

[steve at ando ~]$ python2.7 -m timeit "i=1; L=sorted([30,20,50,10,40]);
L[i+1]"
1000000 loops, best of 3: 1.54 usec per loop

[steve at ando ~]$ python3.3 -m timeit "i=1; L=sorted([30,20,50,10,40]);
L[i+1]"
1000000 loops, best of 3: 1.62 usec per loop

which is an 8% slowdown.

On the other hand:

[steve at ando ~]$ python2.7 -m timeit "i='1'; L=sorted([30,20,50,10,40]);
L[int(i)+1]"
100000 loops, best of 3: 2.6 usec per loop

[steve at ando ~]$ python3.3 -m timeit "i='1'; L=sorted([30,20,50,10,40]);
L[int(i)+1]"
100000 loops, best of 3: 2.29 usec per loop

which is a 12% speed up. So the lesson is, micro-benchmarks are not a good
guide to whole-application benchmarks.





-- 
Steven



From steve at pearwood.info  Wed May 27 22:16:12 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 28 May 2015 12:16:12 +1000
Subject: Returning a custom file object (Python 3)
Message-ID: <55667a6d$0$13002$c3e8da3$5496439d@news.astraweb.com>

I'd like to return a custom file object, say my own subclass. I can easily
subclass the file object:


from io import TextIOWrapper
class MyFile(TextIOWrapper):
    pass


but how do I tell open() to use MyFile?


Answers for Python 3, thanks.




-- 
Steven



From ben+python at benfinney.id.au  Wed May 27 22:34:49 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 28 May 2015 12:34:49 +1000
Subject: Returning a custom file object (Python 3)
References: <55667a6d$0$13002$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <85iobdwhti.fsf@benfinney.id.au>

Steven D'Aprano <steve at pearwood.info> writes:

> from io import TextIOWrapper
> class MyFile(TextIOWrapper):
>     pass
>
> but how do I tell open() to use MyFile?

I haven't used it, but does the ?opener? parameter do what you want?

    open(file, mode='r', buffering=-1, encoding=None,
             errors=None, newline=None, closefd=True, opener=None) ->
             file object

    [?]

    A custom opener can be used by passing a callable as *opener*. The
    underlying file descriptor for the file object is then obtained by
    calling *opener* with (*file*, *flags*). *opener* must return an
    open file descriptor (passing os.open as *opener* results in
    functionality similar to passing None).

-- 
 \          ?Our products just aren't engineered for security.? ?Brian |
  `\             Valentine, senior vice-president of Microsoft Windows |
_o__)                                                      development |
Ben Finney



From ben+python at benfinney.id.au  Wed May 27 22:40:53 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 28 May 2015 12:40:53 +1000
Subject: Returning a custom file object (Python 3)
References: <55667a6d$0$13002$c3e8da3$5496439d@news.astraweb.com>
 <85iobdwhti.fsf@benfinney.id.au>
Message-ID: <85egm1whje.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> Steven D'Aprano <steve at pearwood.info> writes:
>
> > but how do I tell open() to use MyFile?
>
> I haven't used it, but does the ?opener? parameter do what you want?

No, it doesn't; the ?opener? parameter doesn't have any say in the type
of object returned from ?open?.

It seems the existing ?open? implementation doesn't allow you to
override the type of object returned. You may have to resort to
monkey-patching the ?io? attributes to put your preferred classes there.

-- 
 \       ?Science shows that belief in God is not only obsolete. It is |
  `\                        also incoherent.? ?Victor J. Stenger, 2001 |
_o__)                                                                  |
Ben Finney



From cs at zip.com.au  Wed May 27 23:32:39 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 28 May 2015 13:32:39 +1000
Subject: Decoding JSON file using python
In-Reply-To: <slrnmmcsfd.1t4.jon+usenet@frosty.unequivocal.co.uk>
References: <slrnmmcsfd.1t4.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <20150528033239.GA83090@cskk.homeip.net>

On 28May2015 01:38, Jon Ribbens <jon+usenet at unequivocal.co.uk> wrote:
>On 2015-05-27, Karthik Sharma <karthik.sharma at gmail.com> wrote:
>> I tried modifying the program as follows as per your
>> suggestion.Doesn't seem to work.
>
>That's because you didn't modify the program as per their suggestion,
>you made completely different changes that bore no relation to what
>they said.

Actually, his changes looked good to me. He does print from data first, but 
only for debugging. Then he goes:

  message = json.loads(data)

and tried to access message['Message'].

However I am having trouble reproducing his issue because his quoted code is 
incorrect. I've tried to fix it, as listed below, but I don't know what is 
really meant to be in the 'data" string.

Hint: Karthik, use raw strings (r'foo') for complicated strings - it avoids a 
lot of backslashing etc, and thus avoids errors in backslashing.

Karthik, please correct the code below. Currently I do not get your exception, 
I get an exception from the JSON module parsing the "data" string.

Code below,
Cameron Simpson <cs at zip.com.au>

import json

json_input = { "msgType": "0",
    "tid": "1",
    "data": r'[{"Severity":"warn","Subject":"Reporting ","Message":"tdetails": {"Product":"Gecko","CPUs":8,"Language":"en-GB","isEven t":">}]',
    "Timestamp": "1432703193431",
    "Host": "myserver.com",
    "Agent": "Experience",
    "AppName": "undefined",
    "AppInstance": "my_server",
    "Group": "UndefinedGroup"
}

print('json input original  {} \n\n'.format(json_input))

data = json_input['data']
print('data {} \n\n'.format(data))

message = json.loads(data)
print('message {} \n\n'.format(message['Message']))


From random832 at fastmail.us  Wed May 27 23:48:32 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Wed, 27 May 2015 23:48:32 -0400
Subject: Decoding JSON file using python
In-Reply-To: <75320b35-72b6-4aae-a47f-630bd7bef812@googlegroups.com>
References: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>
 <mailman.108.1432768474.5151.python-list@python.org>
 <75320b35-72b6-4aae-a47f-630bd7bef812@googlegroups.com>
Message-ID: <1432784912.4090367.280075065.228F64D1@webmail.messagingengine.com>



On Wed, May 27, 2015, at 19:51, Karthik Sharma wrote:
> I get the following error.
> 
> Traceback (most recent call last):
>   File "test_json.py", line 23, in <module>
>     print('message {} \n\n'.format(message['Message']))
> TypeError: list indices must be integers, not str
> karthik.sharma at aukksharma2:~$ vim test_json.py 

The string in data appears to be a json _list_ (note the first character
is a square bracket) - try message[0]['Message'].

You should really be able to figure this out yourself.


From python at mrabarnett.plus.com  Wed May 27 23:54:23 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 28 May 2015 04:54:23 +0100
Subject: Decoding JSON file using python
In-Reply-To: <75320b35-72b6-4aae-a47f-630bd7bef812@googlegroups.com>
References: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>
 <mailman.108.1432768474.5151.python-list@python.org>
 <75320b35-72b6-4aae-a47f-630bd7bef812@googlegroups.com>
Message-ID: <5566916F.7010908@mrabarnett.plus.com>

On 2015-05-28 00:51, Karthik Sharma wrote:
> I tried modifying the program as follows as per your suggestion.Doesn't seem to work.
>
> import simplejson as json
> import cjson
>
> json_input = { "msgType": "0",
>      "tid": "1",
>      "data": "[{\"Severity\":\"warn\",\"Subject\":\"Reporting \",\"Message\":\"tdetails:{\\\"Product\\\":\\\"Gecko\\\",\\\"CPUs\\\":8,\\\"Language\\\":\\\"en-GB\\\",\\\"isEvent\\\":\\\">
>      "Timestamp": "1432703193431",
>      "Host": "myserver.com",
>      "Agent": "Experience",
>      "AppName": "undefined",
>      "AppInstance": "my_server",
>      "Group": "UndefinedGroup"
> }
>
> print('json input original  {} \n\n'.format(json_input))
>
> data = json_input['data']
>
> print('data {} \n\n'.format(data))
>
> message = json.loads(data)
>
> print('message {} \n\n'.format(message['Message']))
>
> I get the following error.
>
> Traceback (most recent call last):
>    File "test_json.py", line 23, in <module>
>      print('message {} \n\n'.format(message['Message']))
> TypeError: list indices must be integers, not str
> karthik.sharma at aukksharma2:~$ vim test_json.py
>
You have a different error message now. It says that 'message' is a
list.

Have a look at the JSON data. It represents a list that contains a dict.

>
> On Thursday, 28 May 2015 11:14:44 UTC+12, Cameron Simpson  wrote:
>> On 27May2015 15:23, Karthik Sharma <karthik.sharma at gmail.com> wrote:
>> >I have the JSON structure shown below and the python code shown below to manipulate the JSON structure.
>> >
>> >    import json
>> >
>> >    json_input = {
>> >        "msgType": "0",
>> >        "tid": "1",
>> >        "data": "[{\"Severity\":\"warn\",\"Subject\":\"Reporting \",\"Message\":\"tdetails:{\\\"Product\\\":\\\"Gecko\\\",\\\"CPUs\\\":8,\\\"Language\\\":\\\"en-GB\\\",\\\"isEvent\\\":\\\">
>> >        "Timestamp": "1432703193431",
>> >        "Host": "myserver.com",
>> >        "Agent": "Experience",
>> >        "AppName": "undefined",
>> >        "AppInstance": "my_server",
>> >        "Group": "UndefinedGroup"
>> >    }
>> >
>> >
>> >    data = json_input['data']
>> >    tdetails = data['Message']
>> >    print('json_input {} \n\ndata {} \n\n tdetails {}\n\n'.format(json_input,data,tdetails))
>> >
>> >I am getting the error.
>> >
>> >    Traceback (most recent call last):
>> >      File "test_json.py", line 19, in <module>
>> >        tdetails = data['Message']
>> >    TypeError: string indices must be integers, not str
>>
>> That will be because of this line:
>>
>>   tdetails = data['Message']
>>
>> because "data" is just the string from your dict.
>>
>> >The JSON structure is valid as shown by http://jsonlint.com/
>> >   I want to be able to access the different fields inside `data` such as `severity`, `subject` and also fields inside `tdetails` such as `CPUs` and `Product`. How do I do this?
>>
>> Then you need to decode "data". Example (untested):
>>
>>   data_decoded = json.loads(data)
>>
>> and then access:
>>
>>   data_decoded['Message']
>>



From lac at openend.se  Thu May 28 00:47:46 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 28 May 2015 06:47:46 +0200
Subject: OpenCV with Python (cv or cv2)
In-Reply-To: Message from Markos <markos@c2o.pro.br>
 of "Wed, 27 May 2015 19:56:03 -0300." <55664B83.30809@c2o.pro.br>
References: <5564740E.3090206@c2o.pro.br>
 <201505261421.t4QELBUd015477@fido.openend.se><55664B83.30809@c2o.pro.br>
Message-ID: <201505280447.t4S4lk3J006383@fido.openend.se>

Your cmake output doesn't mention that it tried to build libcv.so
so, I guess the reason you cannot find it is that it never tried
to make it.

You may already have fixed your problem by just installing the relevant
debian package.  If not, it may be that you need to install this one
https://packages.debian.org/squeeze/libcv-dev to get libcv.

You have another problem. Squeeze is _very_ old.
According to http://stackoverflow.com/questions/5212728/libcxcore-so-2-missing-in-opencv
the opencv project renamed a bunch of libraries, so if at all possible you
should upgrade your debian distribution.  You may have all sorts of
mismatches between the source you just built and the libraries you
need -- which you will only find by trial and error.

Laura


From marko at pacujo.net  Thu May 28 01:29:05 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 28 May 2015 08:29:05 +0300
Subject: Returning a custom file object (Python 3)
References: <55667a6d$0$13002$c3e8da3$5496439d@news.astraweb.com>
 <85iobdwhti.fsf@benfinney.id.au>
 <mailman.113.1432780860.5151.python-list@python.org>
Message-ID: <87wpztclsu.fsf@elektro.pacujo.net>

Ben Finney <ben+python at benfinney.id.au>:

> It seems the existing ?open? implementation doesn't allow you to
> override the type of object returned.

The question is, can you assign to the builtin namespace. I'm guessing
you can't.

Within a module, you can simply do:

   open = MyFile

Also, in other modules, you can:

   from myfile import open


Marko


From rosuav at gmail.com  Thu May 28 01:49:19 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 28 May 2015 15:49:19 +1000
Subject: Returning a custom file object (Python 3)
In-Reply-To: <87wpztclsu.fsf@elektro.pacujo.net>
References: <55667a6d$0$13002$c3e8da3$5496439d@news.astraweb.com>
 <85iobdwhti.fsf@benfinney.id.au>
 <mailman.113.1432780860.5151.python-list@python.org>
 <87wpztclsu.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmqMH+mYVKHqEcb9TjFvqfU5L=OW6vymDM36xnzNhUeA3g@mail.gmail.com>

On Thu, May 28, 2015 at 3:29 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Ben Finney <ben+python at benfinney.id.au>:
>
>> It seems the existing ?open? implementation doesn't allow you to
>> override the type of object returned.
>
> The question is, can you assign to the builtin namespace. I'm guessing
> you can't.
>
> Within a module, you can simply do:
>
>    open = MyFile
>
> Also, in other modules, you can:
>
>    from myfile import open

Well, you can...

>>> import builtins
>>> builtins.open
<built-in function open>
>>> builtins.open = "not your grandfather's open() function"
>>> open
"not your grandfather's open() function"

... but I don't think replacing all of open() is what Steven has in
mind; it's a function that does a lot of work, most of which is what's
wanted.

Depending on how brutal you want to be, though, you _could_ hack around it.

>>> from io import TextIOWrapper
>>> class MyFile(TextIOWrapper):
...     def demo(self): print("Hello, world!")
...
>>> f = open("/tmp/dummy", "w", encoding="utf-8")
>>> f
<_io.TextIOWrapper name='/tmp/dummy' mode='w' encoding='utf-8'>
>>> f.__class__
<class '_io.TextIOWrapper'>
>>> f.__class__ = MyFile
>>> f.demo()
Hello, world!

This does appear to work. Whether or not it's a good idea is a
separate question. And of course, you could replace open() with
something like this:

>>> _orig_open = open
>>> def open(*args, **kw):
...   cls = kw.pop("use_class", None)
...   f = _orig_open(*args, **kw)
...   if cls is not None: f.__class__ = cls
...   return f
...
>>> f = open("/tmp/dummy", "w", encoding="utf-8", use_class=MyFile)
>>> f
<_io.TextIOWrapper name='/tmp/dummy' mode='w' encoding='utf-8'>
>>> f.demo()
Hello, world!

But again, I'm really not sure this is a good way to do things.

ChrisA


From gherron at digipen.edu  Thu May 28 01:56:41 2015
From: gherron at digipen.edu (Gary Herron)
Date: Wed, 27 May 2015 22:56:41 -0700
Subject: Returning a custom file object (Python 3)
In-Reply-To: <87wpztclsu.fsf@elektro.pacujo.net>
References: <55667a6d$0$13002$c3e8da3$5496439d@news.astraweb.com>
 <85iobdwhti.fsf@benfinney.id.au>
 <mailman.113.1432780860.5151.python-list@python.org>
 <87wpztclsu.fsf@elektro.pacujo.net>
Message-ID: <5566AE19.2060503@digipen.edu>

On 05/27/2015 10:29 PM, Marko Rauhamaa wrote:
> Ben Finney <ben+python at benfinney.id.au>:
>
>> It seems the existing ?open? implementation doesn't allow you to
>> override the type of object returned.
> The question is, can you assign to the builtin namespace. I'm guessing
> you can't.

Of course you can.

Python2:
 >>> __builtins__.open = "whatever"
 >>>

Python3:
 >>> import builtins
 >>> builtins.open = "whatever"
 >>>

Of course doing so is like shooting yourself in the foot:  Any 
subsequent pain is your own fault and probably well deserved.

Gary Herron



>
> Within a module, you can simply do:
>
>     open = MyFile
>
> Also, in other modules, you can:
>
>     from myfile import open
>
>
> Marko


-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418




From steve+comp.lang.python at pearwood.info  Thu May 28 02:52:25 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 28 May 2015 16:52:25 +1000
Subject: Returning a custom file object (Python 3)
References: <55667a6d$0$13002$c3e8da3$5496439d@news.astraweb.com>
 <85iobdwhti.fsf@benfinney.id.au>
 <mailman.113.1432780860.5151.python-list@python.org>
 <87wpztclsu.fsf@elektro.pacujo.net>
 <mailman.120.1432792612.5151.python-list@python.org>
Message-ID: <5566bb2a$0$11126$c3e8da3@news.astraweb.com>

On Thursday 28 May 2015 15:56, Gary Herron wrote:

> On 05/27/2015 10:29 PM, Marko Rauhamaa wrote:
>> Ben Finney <ben+python at benfinney.id.au>:
>>
>>> It seems the existing ?open? implementation doesn't allow you to
>>> override the type of object returned.
>> The question is, can you assign to the builtin namespace. I'm guessing
>> you can't.
> 
> Of course you can.
> 
> Python2:
>  >>> __builtins__.open = "whatever"


Don't use __builtins__ with an "s". That's an implementation of CPython and 
may not exist in the future, and doesn't exist in other implementations.

Instead, you should import __builtin__ with no "s". Confusing? It certainly 
is, which is why Python 3 renamed __builtin__ to builtins.




-- 
Steve



From steve+comp.lang.python at pearwood.info  Thu May 28 03:04:18 2015
From: steve+comp.lang.python at pearwood.info (Steven D'Aprano)
Date: Thu, 28 May 2015 17:04:18 +1000
Subject: Returning a custom file object (Python 3)
References: <55667a6d$0$13002$c3e8da3$5496439d@news.astraweb.com>
 <85iobdwhti.fsf@benfinney.id.au>
 <mailman.113.1432780860.5151.python-list@python.org>
 <87wpztclsu.fsf@elektro.pacujo.net>
 <mailman.119.1432792558.5151.python-list@python.org>
Message-ID: <5566bdf4$0$13013$c3e8da3$5496439d@news.astraweb.com>

On Thursday 28 May 2015 15:49, Chris Angelico wrote:

> ... but I don't think replacing all of open() is what Steven has in
> mind; it's a function that does a lot of work, most of which is what's
> wanted.

Correct. If I wanted to replace open(), I would have just shadowed it, or 
monkey-patched it, or just used myopen().

I want open() to return my own subclass instead of the standard one. And I 
think you have the solution I was looking for:


> Depending on how brutal you want to be, though, you _could_ hack around
> it.
> 
>>>> from io import TextIOWrapper
>>>> class MyFile(TextIOWrapper):
> ...     def demo(self): print("Hello, world!")
> ...
>>>> f = open("/tmp/dummy", "w", encoding="utf-8")
>>>> f
> <_io.TextIOWrapper name='/tmp/dummy' mode='w' encoding='utf-8'>
>>>> f.__class__
> <class '_io.TextIOWrapper'>
>>>> f.__class__ = MyFile
>>>> f.demo()
> Hello, world!
> 
> This does appear to work. Whether or not it's a good idea is a
> separate question.


And this is EXACTLY the sort of use-case that having __class__ be writable 
is intended to solve. So this is exactly the solution I was after, thank 
you. Er... except for one little problem... in Python 3.3:

py> f = open("/tmp/a", "r")
py> f.__class__ = MyFile
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __class__ assignment: only for heap types


So it doesn't work for me. What version of Python are you using?



-- 
Steve



From lhs3356 at naver.com  Thu May 28 04:03:30 2015
From: lhs3356 at naver.com (=?UTF-8?B?7J207ZiE7IOB?=)
Date: Thu, 28 May 2015 17:03:30 +0900 (KST)
Subject: =?UTF-8?B?SGkuIEkgaGF2ZSBxdWVzdGlvbnMgYWJvdXQgUHl0aG9u?=
Message-ID: <c124ff1d8b657b20837c88c87c6b38@cweb28.nm.nhnsystem.com>

Hi.?Please note that do not speak english well.
Do you know Python 2 vs Python3 MultiProcessing the difference ?Multiprocessing is better performance?
Search results do not make a big difference.
Why it will occur following the performance issues ?http://www.reddit.com/r/Python/comments/37khen/is_that_why_python3_slower_than_python2/

Thank you.
???????? ????
??? ???? ??? ?? ???.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150528/9015c352/attachment.html>

From rosuav at gmail.com  Thu May 28 05:06:09 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 28 May 2015 19:06:09 +1000
Subject: Returning a custom file object (Python 3)
In-Reply-To: <5566bdf4$0$13013$c3e8da3$5496439d@news.astraweb.com>
References: <55667a6d$0$13002$c3e8da3$5496439d@news.astraweb.com>
 <85iobdwhti.fsf@benfinney.id.au>
 <mailman.113.1432780860.5151.python-list@python.org>
 <87wpztclsu.fsf@elektro.pacujo.net>
 <mailman.119.1432792558.5151.python-list@python.org>
 <5566bdf4$0$13013$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAPTjJmpu=xCwYGZEhENga5AZ9K8GQZj+5qMiwpZnV0AxG1x08w@mail.gmail.com>

On Thu, May 28, 2015 at 5:04 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
>> This does appear to work. Whether or not it's a good idea is a
>> separate question.
>
>
> And this is EXACTLY the sort of use-case that having __class__ be writable
> is intended to solve. So this is exactly the solution I was after, thank
> you. Er... except for one little problem... in Python 3.3:
>
> py> f = open("/tmp/a", "r")
> py> f.__class__ = MyFile
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: __class__ assignment: only for heap types
>
>
> So it doesn't work for me. What version of Python are you using?

Huh, didn't even think to check other versions.

$ python3
Python 3.5.0b1+ (default:7255af1a1c50+, May 26 2015, 00:39:06)
[GCC 4.9.2] on linux

It's a build fresh from source control around about this weekend (some
time after feature freeze, obviously, but beyond that all I know is
what you see there). I guess this is pretty new; trying the same thing
on 3.4.2 doesn't work. But hey! 3.5 will be out soon...

ChrisA


From oscar.j.benjamin at gmail.com  Thu May 28 05:29:27 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 28 May 2015 10:29:27 +0100
Subject: Returning a custom file object (Python 3)
In-Reply-To: <55667a6d$0$13002$c3e8da3$5496439d@news.astraweb.com>
References: <55667a6d$0$13002$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAHVvXxTqiUnG7qq5T_nWb-o_FU99LyokA7Y5vxnhthC7ZW9tiA@mail.gmail.com>

On 28 May 2015 at 03:16, Steven D'Aprano <steve at pearwood.info> wrote:
> I'd like to return a custom file object, say my own subclass. I can easily
> subclass the file object:
>
>
> from io import TextIOWrapper
> class MyFile(TextIOWrapper):
>     pass
>
>
> but how do I tell open() to use MyFile?

Does the below do what you want?

#!/usr/bin/env python3

class Wrapper:
    def __init__(self, wrapped):
        self._wrapped = wrapped
    def __getattr__(self, attrname):
        return getattr(self._wrapped, attrname)
    # Special methods are not resolved by __getattr__
    def __iter__(self):
        return self._wrapped.__iter__()
    def __enter__(self):
        return self._wrapped.__enter__()
    def __exit__(self, *args):
        return self._wrapped.__exit__(*args)

class WrapFile(Wrapper):
    def write(self):
        return "Writing..."

f = WrapFile(open('wrap.py'))
print(f.readline())
with f:
    print(len(list(f)))
print(f.write())

>
> Answers for Python 3, thanks.

Tested on 3.2.


--
Oscar


From alex at duboaa.net  Thu May 28 05:50:13 2015
From: alex at duboaa.net (Alexis Dubois)
Date: Thu, 28 May 2015 02:50:13 -0700 (PDT)
Subject: [QT] Scroll multiple lists with one scrollbar
Message-ID: <53f83314-12a6-4368-894f-13e756b5a888@googlegroups.com>

Hello everyone!

My QT GUI contains 8 QlistWidgets filled with a large amount of data.
Each item of a list is related to items of other lists in the same row.
So I want to scroll the 8 lists with only one vertical scrollbar (and vice-versa)

But I don't know how to manage this.
Do you have an idea?

(I'm quite new in python, I'm translating my tcl/tk app to this code)
thanks in advance


From lac at openend.se  Thu May 28 06:11:43 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 28 May 2015 12:11:43 +0200
Subject: [QT] Scroll multiple lists with one scrollbar
In-Reply-To: Message from Alexis Dubois <alex@duboaa.net> of "Thu,
 28 May 2015 02:50:13 -0700."
 <53f83314-12a6-4368-894f-13e756b5a888@googlegroups.com>
References: <53f83314-12a6-4368-894f-13e756b5a888@googlegroups.com>
Message-ID: <201505281011.t4SABhgj014562@fido.openend.se>

You want to define a scrollable area and then add your labels to it.
The code here: http://stackoverflow.com/questions/22255994/pyqt-adding-widgets-to-scrollarea-during-the-runtime

should show you what you need to do, though for you the job is easier if you
do not need to add the labels after the app is running.

Laura


From christian.ibala at gmail.com  Thu May 28 06:34:22 2015
From: christian.ibala at gmail.com (Serge Christian Ibala)
Date: Thu, 28 May 2015 12:34:22 +0200
Subject: Question About Image Processing in Python
Message-ID: <CABsq6izAZ5HNy7gT-WSTAxrUaGNexhbLUhk92Gwtu563ys3Jow@mail.gmail.com>

Hello All,


I want to know which version of Python is compatible (or can be associated
with which version of which "tools or package" for image processing)



I am working under Window and it is so complicated to find out which
version of which tool goes with which other version?



I want to use the following package

?numpy, matplotib, mahotas, ipython   OpenCV and SciPy"



Let say I use





Python 2.7 or Python 2.10 which version of the following tools



?numpy, matplotib, mahotas, ipython, OpenCV and SciPy?



should I use? and where can I find them?


Are there any package available for image processing using Python 3.4.3?



Or what is the recommendation of Python for image processing?



Do you know any good Python book on the subject or tutorial ?




Thank you enormously,


Christian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150528/978c45c5/attachment.html>

From alex at duboaa.net  Thu May 28 06:44:22 2015
From: alex at duboaa.net (Alexis Dubois)
Date: Thu, 28 May 2015 03:44:22 -0700 (PDT)
Subject: [QT] Scroll multiple lists with one scrollbar
In-Reply-To: <mailman.124.1432807913.5151.python-list@python.org>
References: <53f83314-12a6-4368-894f-13e756b5a888@googlegroups.com>
 <mailman.124.1432807913.5151.python-list@python.org>
Message-ID: <6a5519e7-be54-4650-98bc-0446bd2316f0@googlegroups.com>

Le jeudi 28 mai 2015 12:12:42 UTC+2, Laura Creighton a ?crit?:
> You want to define a scrollable area and then add your labels to it.
> The code here: http://stackoverflow.com/questions/22255994/pyqt-adding-widgets-to-scrollarea-during-the-runtime
> 
> should show you what you need to do, though for you the job is easier if you
> do not need to add the labels after the app is running.
> 
> Laura

Thanks Laura.
But as I understand the scrollable area, it's useful to scroll widgets.
But here, in my case, I want to scroll inside my widgets.
My QlistWidgets are fixed in size and data are append inside.
Because I want only one scrollbar for all lists, I set the "setVerticalScrollBarPolicy" lists property to "ScrollBarAlwaysOff" and tried to do what I need with a QScrollBar.


From lac at openend.se  Thu May 28 06:54:14 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 28 May 2015 12:54:14 +0200
Subject: [QT] Scroll multiple lists with one scrollbar
In-Reply-To: Message from Alexis Dubois <alex@duboaa.net> of "Thu,
 28 May 2015 03:44:22 -0700."
 <6a5519e7-be54-4650-98bc-0446bd2316f0@googlegroups.com>
References: <53f83314-12a6-4368-894f-13e756b5a888@googlegroups.com>
 <mailman.124.1432807913.5151.python-list@python.org><6a5519e7-be54-4650-98bc-0446bd2316f0@googlegroups.com>
Message-ID: <201505281054.t4SAsEMW015569@fido.openend.se>

In a message of Thu, 28 May 2015 03:44:22 -0700, Alexis Dubois writes:
>Le jeudi 28 mai 2015 12:12:42 UTC+2, Laura Creighton a ?crit?:
>> You want to define a scrollable area and then add your labels to it.
>> The code here: http://stackoverflow.com/questions/22255994/pyqt-adding-widgets-to-scrollarea-during-the-runtime
>> 
>> should show you what you need to do, though for you the job is easier if you
>> do not need to add the labels after the app is running.
>> 
>> Laura
>
>Thanks Laura.
>But as I understand the scrollable area, it's useful to scroll widgets.
>But here, in my case, I want to scroll inside my widgets.
>My QlistWidgets are fixed in size and data are append inside.
>Because I want only one scrollbar for all lists, I set the "setVerticalScrollBarPolicy" lists property to "ScrollBarAlwaysOff" and tried to do what I need with a QScrollBar.

Ah, sorry, I misunderstood the question.

Hmmm.  I've never tried to do that.  Let me see if I can get something
to work.

Back later,
hack hack hack!
Laura


From tjreedy at udel.edu  Thu May 28 07:05:21 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 28 May 2015 07:05:21 -0400
Subject: Question About Image Processing in Python
In-Reply-To: <CABsq6izAZ5HNy7gT-WSTAxrUaGNexhbLUhk92Gwtu563ys3Jow@mail.gmail.com>
References: <CABsq6izAZ5HNy7gT-WSTAxrUaGNexhbLUhk92Gwtu563ys3Jow@mail.gmail.com>
Message-ID: <mk6sq5$ahe$1@ger.gmane.org>

On 5/28/2015 6:34 AM, Serge Christian Ibala wrote:

> I want to know which version of Python is compatible (or can be
> associated with which version of which "tools or package" for image
> processing)

pillow is a one standard for image processing but I see that mahotas 
does different things. pillow is both 2.x and 3.x.

> I am working under Window and it is so complicated to find out which
> version of which tool goes with which other version?

The magic site for Windows binaries is
http://www.lfd.uci.edu/~gohlke/pythonlibs/
Now pip installable with an extra arg if not automatic.

> I want to use the following package
>
> ?numpy, matplotib, mahotas, ipython   OpenCV and SciPy"

opencv seems to be the only one not available for 3.x.


> Python 2.7 or Python 2.10 which version of the following tools

no 2.10, just 2.7.10, which is what you should install.

> ?numpy, matplotib, mahotas, ipython, OpenCV and SciPy?

just try pip install xyz and let it pick but see above if necessary for 
Windows binaries.

> Are there any package available for image processing using Python 3.4.3?



-- 
Terry Jan Reedy




From oscar.j.benjamin at gmail.com  Thu May 28 07:24:32 2015
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 28 May 2015 12:24:32 +0100
Subject: Question About Image Processing in Python
In-Reply-To: <CABsq6izAZ5HNy7gT-WSTAxrUaGNexhbLUhk92Gwtu563ys3Jow@mail.gmail.com>
References: <CABsq6izAZ5HNy7gT-WSTAxrUaGNexhbLUhk92Gwtu563ys3Jow@mail.gmail.com>
Message-ID: <CAHVvXxStOtO11R+oMhn8NfnMiwMSs0EyofRiwz9P6S6d_00EAQ@mail.gmail.com>

On 28 May 2015 at 11:34, Serge Christian Ibala
<christian.ibala at gmail.com> wrote:
> Hello All,
>
> I want to know which version of Python is compatible (or can be associated
> with which version of which "tools or package" for image processing)
>
> I am working under Window and it is so complicated to find out which version
> of which tool goes with which other version?
>
> I want to use the following package
>
> ?numpy, matplotib, mahotas, ipython   OpenCV and SciPy"

I would just install the most recent released versions unless I saw
some information that suggested a problem with that.

You may want to consider installing something like the Anaconda Python
distribution which includes most of these packages (all except
mahotas) with a single installer:
http://docs.continuum.io/anaconda/install.html

You can see the list of packages included here:
http://docs.continuum.io/anaconda/pkg-docs.html

It will also install the mingw compiler that you could use to install
mahotas using pip/conda.


--
Oscar


From toddrjen at gmail.com  Thu May 28 07:46:28 2015
From: toddrjen at gmail.com (Todd)
Date: Thu, 28 May 2015 13:46:28 +0200
Subject: Question About Image Processing in Python
In-Reply-To: <mk6sq5$ahe$1@ger.gmane.org>
References: <CABsq6izAZ5HNy7gT-WSTAxrUaGNexhbLUhk92Gwtu563ys3Jow@mail.gmail.com>
 <mk6sq5$ahe$1@ger.gmane.org>
Message-ID: <CAFpSVpJS3K2jcKVXt=ZswgiWznUonOC9MRC6dSGx1s3FbRKNdA@mail.gmail.com>

On Thu, May 28, 2015 at 1:05 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 5/28/2015 6:34 AM, Serge Christian Ibala wrote:
>
>
>  I want to use the following package
>>
>> ?numpy, matplotib, mahotas, ipython   OpenCV and SciPy"
>>
>
> opencv seems to be the only one not available for 3.x.
>
>
OpenCV 3 (which is in RC1 now) supports Python 3.x.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150528/edfa1bff/attachment.html>

From lac at openend.se  Thu May 28 08:18:03 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 28 May 2015 14:18:03 +0200
Subject: [QT] Scroll multiple lists with one scrollbar
In-Reply-To: Message from Alexis Dubois <alex@duboaa.net> of "Thu,
 28 May 2015 02:50:13 -0700."
 <53f83314-12a6-4368-894f-13e756b5a888@googlegroups.com>
References: <53f83314-12a6-4368-894f-13e756b5a888@googlegroups.com>
Message-ID: <201505281218.t4SCI3dD017636@fido.openend.se>

Looks like what you need to do is to 
connect verticalScrollBar().valueChanged (in one widget) to
verticalScrollBar().setValue in all of the others.

So is this code on the right track?  It uses lists, not labels, and
doesn't hide the scrollbars, but is this the behaviour you need (
and weren't getting)?  It should work for labels too..


from PyQt4 import QtGui, QtCore

class Window(QtGui.QWidget):
    def __init__(self):
        QtGui.QWidget.__init__(self)
        self.listA = QtGui.QListWidget(self)
        self.listB = QtGui.QListWidget(self)
        self.listC = QtGui.QListWidget(self)

        widgets=[self.listA, self.listB, self.listC]
        layout = QtGui.QHBoxLayout(self)

        for w1 in widgets:
            layout.addWidget(w1)
            for index in range(100):
                w1.addItem('This is line number %d' % (index +1))
            for w2 in widgets:
                if w1 != w2:
                    w1.verticalScrollBar().valueChanged.connect(w2.verticalScrollBar().setValue)
            
if __name__ == '__main__':

    import sys
    app = QtGui.QApplication(sys.argv)
    window = Window()
    window.show()
    sys.exit(app.exec_())


From __peter__ at web.de  Thu May 28 08:35:23 2015
From: __peter__ at web.de (Peter Otten)
Date: Thu, 28 May 2015 14:35:23 +0200
Subject: [QT] Scroll multiple lists with one scrollbar
References: <53f83314-12a6-4368-894f-13e756b5a888@googlegroups.com>
Message-ID: <mk722c$tee$1@ger.gmane.org>

Alexis Dubois wrote:

> My QT GUI contains 8 QlistWidgets filled with a large amount of data.
> Each item of a list is related to items of other lists in the same row.
> So I want to scroll the 8 lists with only one vertical scrollbar (and
> vice-versa)
> 
> But I don't know how to manage this.
> Do you have an idea?
> 
> (I'm quite new in python, I'm translating my tcl/tk app to this code)
> thanks in advance

Are you perhaps looking for a table widget?

https://srinikom.github.io/pyside-docs/PySide/QtGui/QTableWidget.html



From alex at duboaa.net  Thu May 28 08:53:34 2015
From: alex at duboaa.net (Alexis Dubois)
Date: Thu, 28 May 2015 05:53:34 -0700 (PDT)
Subject: [QT] Scroll multiple lists with one scrollbar
In-Reply-To: <mailman.130.1432815493.5151.python-list@python.org>
References: <53f83314-12a6-4368-894f-13e756b5a888@googlegroups.com>
 <mailman.130.1432815493.5151.python-list@python.org>
Message-ID: <b5b2a476-644f-4bae-be5a-d3c2329de0d3@googlegroups.com>

Thank you Laura, it works.
(Even if connecting the scrollbar of one list to other lists is not what I exactly want to do, but no matter, I change the line "w1.verticalScrollBar().valueChanged.connect(w2.verticalScrollBar().setValue)" a little bit to fit my need. )


From alex at duboaa.net  Thu May 28 08:54:55 2015
From: alex at duboaa.net (Alexis Dubois)
Date: Thu, 28 May 2015 05:54:55 -0700 (PDT)
Subject: [QT] Scroll multiple lists with one scrollbar
In-Reply-To: <mailman.131.1432816538.5151.python-list@python.org>
References: <53f83314-12a6-4368-894f-13e756b5a888@googlegroups.com>
 <mailman.131.1432816538.5151.python-list@python.org>
Message-ID: <049fb2f7-ce86-4817-849f-8f934fc69b05@googlegroups.com>

Thanks Peter, but no, even if it could be possible to use a QTableWidget instead of my 8 lists.


From as at sci.fi  Thu May 28 10:07:11 2015
From: as at sci.fi (Anssi Saari)
Date: Thu, 28 May 2015 17:07:11 +0300
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net>
Message-ID: <vg3k2vsvlrk.fsf@coffee.modeemi.fi>

Marko Rauhamaa <marko at pacujo.net> writes:

> Ned Batchelder <ned at nedbatchelder.com>:
>
>> I would find it much clearer to not use a nested class at all, and
>> instead to pass the object into the constructor:
>
> Nested classes are excellent and expressing the state pattern <URL:
> http://en.wikipedia.org/wiki/State_pattern>.

Do you have an example of state pattern using nested classes and python?
With a quick look I didn't happen to find one in any language.


From sturla.molden at gmail.com  Thu May 28 10:33:32 2015
From: sturla.molden at gmail.com (Sturla Molden)
Date: Thu, 28 May 2015 14:33:32 +0000 (UTC)
Subject: Question About Image Processing in Python
References: <CABsq6izAZ5HNy7gT-WSTAxrUaGNexhbLUhk92Gwtu563ys3Jow@mail.gmail.com>
Message-ID: <558491027454515958.449518sturla.molden-gmail.com@news.gmane.org>

Serge Christian Ibala <christian.ibala at gmail.com> wrote:

> Or what is the recommendation of Python for image processing?

Basic setup everyone should have:

Python
NumPy
SciPy (e.g. scipy.ndimage)
Cython
C and C++ compiler
matplotlib
scikit-image
scikit-learn
pillow

Also consider:
mahotas
tifffile (by Christoph Gohlke)
OpenCV
PyOpenGL
VTK
mayavi
pyglet
PyGame
PyQt

Abandonware:
PIL



From sturla.molden at gmail.com  Thu May 28 10:38:56 2015
From: sturla.molden at gmail.com (Sturla Molden)
Date: Thu, 28 May 2015 14:38:56 +0000 (UTC)
Subject: Hi. I have questions about Python
References: <c124ff1d8b657b20837c88c87c6b38@cweb28.nm.nhnsystem.com>
Message-ID: <1898092925454516537.250527sturla.molden-gmail.com@news.gmane.org>

??? <lhs3356 at naver.com> wrote:
> Hi.?Please note that do not speak english well.
> Do you know Python 2 vs Python3 MultiProcessing the difference
> ?Multiprocessing is better performance?

The main difference is that multiprocessing on Python 3.4 (and later) will
allow you to use APIs that are not "forksafe" on Linux or Mac. One example
is Accelerate Framework on Mac OSX (e.g. used by numpy.linalg).



From marko at pacujo.net  Thu May 28 11:01:46 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 28 May 2015 18:01:46 +0300
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net>
 <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
Message-ID: <87siagagpx.fsf@elektro.pacujo.net>

Anssi Saari <as at sci.fi>:

> Do you have an example of state pattern using nested classes and
> python? With a quick look I didn't happen to find one in any language.

Here's an sampling from my mail server:

========================================================================
class SMTPServerConnection(ServerConnection):
    #:     :     :
    #:     :     :

    def __init__(self, server, sock, domain, peer):
        super().__init__(server, sock)
        conn = self
        client_ip = peer[0]

        class STATE:
            def __str__(self):
                return self.__class__.__name__
            def handle_command(self, cmd):
                conn.log("handle_command (unexpected): {}".format(cmd))
                assert False
            def handle_bad_command(self, reason):
                conn.log("handle_bad_command (unexpected): {}".format(reason))
                assert False
            def handle_eof(self):
                conn.log("eof (unexpected)")
                assert False
            def terminate(self):
                assert False
        
        class IDLE(STATE):
            def handle_command(self, cmd):
                conn.log("handle_command: {}".format(cmd))
                verb, args = conn.split_cmd(cmd)
                if verb == b'EHLO':
                    self.process_ehlo_or_helo(
                        args,
                        "PIPELINING",
                        "SIZE {:d}".format(conn.MAX_MSG_SIZE),
                        "VRFY",
                        "EXPN",
                        "8BITMIME")
                elif verb == b'HELO':
                    self.process_ehlo_or_helo(args)
                elif verb in [ b'NOOP', b'RSET' ]:
                    conn.respond(250, "OK")
                elif verb == b'HELP':
                    conn.respond(214, "No help available")
                elif verb in [ b'VRFY', b'EXPN' ]:
                    conn.respond(550, "Access denied to you")
                elif verb == b'QUIT':
                    conn.respond(221, "{} Closing channel".format(
                            server.domain))
                    conn.shut_down()
                    conn.set_state(QUITTING)
                elif verb == b'MAIL':
                    self.handle_mail(args)
                elif verb in [ b'RCPT', b'DATA' ]:
                    conn.respond(503, "Bad sequence of commands")
                else:
                    conn.respond(500, "Command unrecognized")

            def process_ehlo_or_helo(self, args, *capabilities):
                if not args:
                    conn.respond(501, "Missing parameter")
                    return
                try:
                    # may be an IP address
                    conn.helo_domain_name = args[0].decode()
                except UnicodeError:
                    conn.respond(501, "Bad encoding in parameter")
                    return
                if conn.local_client:
                    conn.respond(250, server.domain, *capabilities)
                    conn.set_state(IDLE)
                    return
                ## todo: remove the "suspend" concept from mux
                conn.suspend()
                conn.log("authorize helo {} from {}".format(
                        conn.helo_domain_name, client_ip))
                conn.set_state(SPF_HELO)
                def callback(verdict, reason):
                    conn.state.handle_spf_verdict(
                        verdict, reason, *capabilities)
                conn.spf_query = server.spf_client.authorize_helo(
                    server.host, conn.helo_domain_name, server.family,
                    client_ip, callback, xid = id(conn))

            #:     :     :
            #:     :     :

        class SPF_HELO(STATE):
            def terminate(self):
                conn.resume()
                conn.spf_query.cancel()
                conn.close()
                if conn.timer is not None:
                    conn.timer.cancel()
                    conn.timer = None
                conn.set_state(ZOMBIE)

            def handle_spf_verdict(self, verdict, reason, *capabilities):
                conn.resume()
                conn.log("verdict {} reason {}".format(verdict, reason))
                # RFC 4408 ?2.5.5 calls for leniency for SoftFail
                #if verdict in [ SPFClient.FAIL, SPFClient.SOFT_FAIL ]:
                if verdict in [ SPFClient.FAIL ]:
                    conn.respond(550, "SPF check failed: {}".format(reason))
                    conn.set_state(IDLE)
                    return
                conn.respond(250, server.domain, *capabilities)
                conn.set_state(IDLE)

        class SPF_SENDER(STATE):
            def terminate(self):
                conn.resume()
                conn.spf_query.cancel()
                conn.close()
                if conn.timer is not None:
                    conn.timer.cancel()
                    conn.timer = None
                conn.set_state(ZOMBIE)

            def handle_spf_verdict(self, verdict, reason):
                conn.resume()
                conn.log("verdict {} reason {}".format(verdict, reason))
                # RFC 4408 ?2.5.5 calls for leniency for SoftFail
                #if verdict in [ SPFClient.FAIL, SPFClient.SOFT_FAIL ]:
                if verdict in [ SPFClient.FAIL ]:
                    conn.respond(550, "SPF check failed: {}".format(reason))
                    conn.set_state(IDLE)
                    return
                conn.spf_sender_verdict = verdict
                conn.spf_sender_reason = reason
                conn.respond(250, "OK")
                conn.set_state(RECIPIENTS)

        class RECIPIENTS(STATE):
            def handle_command(self, cmd):
                conn.log("handle_command: {}".format(cmd))
                verb, args = conn.split_cmd(cmd)
                if verb in [ b'EHLO', b'HELO', b'MAIL' ]:
                    conn.respond(503, "Bad sequence of commands")
                elif verb == b'NOOP':
                    conn.respond(250, "OK")
                elif verb == b'HELP':
                    conn.respond(214, "No help available")
                elif verb == b'VRFY' or verb == b'EXPN':
                    conn.respond(550, "Access denied to you")
                elif verb == b'RSET':
                    conn.set_state(IDLE)
                    conn.respond(250, "OK")
                elif verb == b'QUIT':
                    conn.respond(221, "{} Closing channel".format(
                            server.domain))
                    conn.shut_down()
                    conn.set_state(QUITTING)
                elif verb == b'RCPT':
                    self.handle_rcpt(args)
                elif verb == b'DATA':
                    self.handle_data(args)
                else:
                    conn.respond(500, "Command unrecognized")

            #:     :     :
            #:     :     :

        self.send(("220 {} Service ready\r\n".format(domain)).encode())
        self.timer = self.mux.after(self.SERVER_TIMEOUT, self.handle_timeout)
        #:     :     :
        #:     :     :
        self.state = IDLE()

    def set_state(self, state):
        new_state = state()
        self.log("set_state: {} -> {}".format(self.state, new_state))
        self.state = new_state

    def handle_command(self, cmd):
        self.state.handle_command(cmd)

    def handle_timeout(self):
        if self.timer is None:
            return
        self.timer = None
        self.log("timeout")
        self.state.terminate()

    def handle_spf_verdict(self, verdict, reason, *capabilities):
        self.state.handle_spf_verdict(verdict, reason, *capabilities)

    #:     :     :
    #:     :     :
========================================================================


Marko


From lac at openend.se  Thu May 28 11:05:57 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 28 May 2015 17:05:57 +0200
Subject: [QT] Scroll multiple lists with one scrollbar
In-Reply-To: Message from Alexis Dubois <alex@duboaa.net> of "Thu,
 28 May 2015 05:53:34 -0700."
 <b5b2a476-644f-4bae-be5a-d3c2329de0d3@googlegroups.com>
References: <53f83314-12a6-4368-894f-13e756b5a888@googlegroups.com>
 <mailman.130.1432815493.5151.python-list@python.org><b5b2a476-644f-4bae-be5a-d3c2329de0d3@googlegroups.com>
Message-ID: <201505281505.t4SF5vPE021551@fido.openend.se>

In a message of Thu, 28 May 2015 05:53:34 -0700, Alexis Dubois writes:
>Thank you Laura, it works.
>(Even if connecting the scrollbar of one list to other lists is not what I exactly want to do, but no matter, I change the line "w1.verticalScrollBar().valueChanged.connect(w2.verticalScrollBar().setValue)" a little bit to fit my need. )

Great, I had to go to a meeting, so things were not in such a nice
state as I would have liked.  In particular, if some of your lists
are never going to have any scroll bars attatched to them, there is
no need to wire them up to the thing.  But I thought  I had found
the little bit of magic you needed; glad I was correct this time.

Laura


From ian.g.kelly at gmail.com  Thu May 28 11:40:32 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 28 May 2015 09:40:32 -0600
Subject: should "self" be changed?
In-Reply-To: <87siagagpx.fsf@elektro.pacujo.net>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net> <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
 <87siagagpx.fsf@elektro.pacujo.net>
Message-ID: <CALwzidmLBoxDtoW8SjGmJ-_w_bFp8v+2GPYLV+rx1Q3yOwjoZw@mail.gmail.com>

On Thu, May 28, 2015 at 9:01 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Anssi Saari <as at sci.fi>:
>
>> Do you have an example of state pattern using nested classes and
>> python? With a quick look I didn't happen to find one in any language.
>
> Here's an sampling from my mail server:

I think I would be more inclined to use enums. This has the advantages
of not creating a new set of state classes for every connection
instance and that each state is a singleton instance, allowing things
like "if self.state is SMTPConnectionState.IDLE". It could look
something like this:

class SMTPConnectionState(Enum):

    class IDLE:
        @classmethod
        def handle_command(cls, conn, cmd):
            # ...

    class SPF_HELO:
        @classmethod
        def terminate(cls, conn):
            # ...


From davidfstr at gmail.com  Thu May 28 12:34:25 2015
From: davidfstr at gmail.com (davidfstr at gmail.com)
Date: Thu, 28 May 2015 09:34:25 -0700 (PDT)
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
Message-ID: <c9b388e8-dec0-4f2b-baab-537622b80992@googlegroups.com>

Thanks for the responses folks. I will briefly summarize them:

> As you say, it is fundamentally not possible to make this work at 
the Python level.

This is pretty effectively demonstrated by "Tav's admirable but failed attempt to sandbox file IO":
* http://tav.espians.com/a-challenge-to-break-python-security.html

Wow there are some impressive ways to confuse the system. I particularly like overriding str's equality function to defeat mode checking code when opening files.

> When we needed this at edX, we wrote CodeJail (https://github.com/edx/codejail). 
It's a wrapper around AppArmor to provide OS-level protection of code 
execution in subprocesses.  It has Python-specific features, but because it 
is based on AppArmor, can sandbox any process, so long as it's configured 
properly. 

This looks promising. I will take a closer look.

> What about launching the Python process in a Docker container?

This may work in combination with other techniques. Certainly faster than spinning up a new VM or snapshot-restoring a fixed VM on a repeated basis. Would need to see whether CPU, Memory, and Disk usage could be constrained at the level of a container.

- David


On Monday, May 25, 2015 at 7:24:32 PM UTC-7, davi... at gmail.com wrote:
> I am writing a web service that accepts Python programs as input, runs the provided program with some profiling hooks, and returns various information about the program's runtime behavior. To do this in a safe manner, I need to be able to create a sandbox that restricts what the submitted Python program can do on the web server.
> 
> Almost all discussion about Python sandboxes I have seen on the internet involves selectively blacklisting functionality that gives access to system resources, such as trying to hide the "open" builtin to restrict access to file I/O. All such approaches are doomed to fail because you can always find a way around a blacklist.
> 
> For my particular sandbox, I wish to allow *only* the following kinds of actions (in a whitelist):
> * reading from stdin & writing to stdout;
> * reading from files, within a set of whitelisted directories;
> * pure Python computation.
> 
> In particular all other operations available through system calls are banned. This includes, but is not limited to:
> * writing to files;
> * manipulating network sockets;
> * communicating with other processes.
> 
> I believe it is not possible to limit such operations at the Python level. The best you could do is try replacing all the standard library modules, but that is again just a blacklist - it won't prevent a determined attacker from doing things like constructing their own 'code' object and executing it.
> 
> It might be necessary to isolate the Python process at the operating system level.
> * A chroot jail on Linux & OS X can limit access to the filesystem. Again this is just a blacklist.
> * No obvious way to block socket creation. Again this would be just a blacklist.
> * No obvious way to detect unapproved system calls and block them.
> 
> In the limit, I could dynamically spin up a virtual machine and execute the Python program in the machine. However that's extremely expensive in computational time.
> 
> Has anyone on this list attempted to sandbox Python programs in a serious fashion? I'd be interested to hear your approach.
> 
> - David


From marko at pacujo.net  Thu May 28 12:59:23 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Thu, 28 May 2015 19:59:23 +0300
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net>
 <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
 <87siagagpx.fsf@elektro.pacujo.net>
 <mailman.135.1432827676.5151.python-list@python.org>
Message-ID: <87oal4d4es.fsf@elektro.pacujo.net>

Ian Kelly <ian.g.kelly at gmail.com>:

> On Thu, May 28, 2015 at 9:01 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
>> Anssi Saari <as at sci.fi>:
>>
>>> Do you have an example of state pattern using nested classes and
>>> python? With a quick look I didn't happen to find one in any
>>> language.
>>
>> Here's an sampling from my mail server:
>
> I think I would be more inclined to use enums. This has the advantages
> of not creating a new set of state classes for every connection
> instance and that each state is a singleton instance, allowing things
> like "if self.state is SMTPConnectionState.IDLE". It could look
> something like this:
>
> class SMTPConnectionState(Enum):
>
>     class IDLE:
>         @classmethod
>         def handle_command(cls, conn, cmd):
>             # ...
>
>     class SPF_HELO:
>         @classmethod
>         def terminate(cls, conn):
>             # ...

Really, the main expressive choice is whether you use an inner class
(and get the advantages of a closure) or an outer class (and get
potential performance advantages).

When I have coded state machines for C or Java, I have noticed that
nothing beats enums and switch statements performance-wise, and
expressively, they're not that bad, either. Python doesn't have a switch
statement, so the natural thing is to ride on method dispatching
(whether via inner or outer classes). However, I must say the exception
"idiom" someone mentioned on this forum way back has its lure:

    try: raise self.state
    except IDLE:
        #...
    except SPF_HELO:
        #...


Marko


From rosuav at gmail.com  Thu May 28 13:06:01 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 29 May 2015 03:06:01 +1000
Subject: should "self" be changed?
In-Reply-To: <87oal4d4es.fsf@elektro.pacujo.net>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net>
 <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
 <87siagagpx.fsf@elektro.pacujo.net>
 <mailman.135.1432827676.5151.python-list@python.org>
 <87oal4d4es.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmqR=YWvYoGf2X_kgf492=qW4Cckizy4v4rYN8-NtkVMtA@mail.gmail.com>

On Fri, May 29, 2015 at 2:59 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> When I have coded state machines for C or Java, I have noticed that
> nothing beats enums and switch statements performance-wise, and
> expressively, they're not that bad, either. Python doesn't have a switch
> statement, so the natural thing is to ride on method dispatching
> (whether via inner or outer classes). However, I must say the exception
> "idiom" someone mentioned on this forum way back has its lure:
>
>     try: raise self.state
>     except IDLE:
>         #...
>     except SPF_HELO:
>         #...

I take exception to that idiom. :)

ChrisA


From alan.gauld at btinternet.com  Thu May 28 13:16:05 2015
From: alan.gauld at btinternet.com (Alan Gauld)
Date: Thu, 28 May 2015 18:16:05 +0100
Subject: Question About Image Processing in Python
In-Reply-To: <CABsq6izAZ5HNy7gT-WSTAxrUaGNexhbLUhk92Gwtu563ys3Jow@mail.gmail.com>
References: <CABsq6izAZ5HNy7gT-WSTAxrUaGNexhbLUhk92Gwtu563ys3Jow@mail.gmail.com>
Message-ID: <mk7igk$9f9$1@ger.gmane.org>

On 28/05/15 11:34, Serge Christian Ibala wrote:

> I want to know which version of Python is compatible (or can be
> associated with which version of which "tools or package" for image
> processing)

It would help if you told us what kind of image processing.
If you mean programmatic manipulation of images similar
to what would be done using GIMP/Photoshop then the most
common tools are ImageMagick and Pillow. But you mention
neither, so I'm guessing you are looking at doing
something else - maybe trying to analyze content?

Given the list of tools you list I'd suggest fetching
an all-in-one distro like Anaconda or Canopy. That
way somebody else does the dependency dance for you.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos




From stefan_ml at behnel.de  Thu May 28 14:41:05 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 28 May 2015 20:41:05 +0200
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
Message-ID: <mk7ng1$uq3$1@ger.gmane.org>

davidfstr at gmail.com schrieb am 26.05.2015 um 04:24:
> Has anyone on this list attempted to sandbox Python programs in a
> serious fashion? I'd be interested to hear your approach.

Not quite sandboxing Python, but I've seen people use my Lupa [1] library
for this. They're writing all their code in Python, and then let users
embed their own Lua code into it to script their API. The Lua runtime is
apparently quite good at sandboxing, and it's really small, just some 600KB
or so. Lupa then lets you easily control the access to your Python code at
a whitelist level by intercepting all Python attribute lookups.

It doesn't add much to your application to embed Lua (or even LuaJIT) in
Python, and it gives users a nicely object oriented language to call and
orchestrate your Python objects.

Stefan


[1] https://pypi.python.org/pypi/lupa



From denismfmcmahon at gmail.com  Thu May 28 14:48:58 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Thu, 28 May 2015 18:48:58 +0000 (UTC)
Subject: Decoding JSON file using python
References: <11dbf982-efc1-4ef0-b286-e3d08ba5030e@googlegroups.com>
Message-ID: <mk7nuq$qj$2@dont-email.me>

On Wed, 27 May 2015 15:23:31 -0700, Karthik Sharma wrote:

> The JSON structure is valid as shown by http://jsonlint.com/

Not when I paste it in it's not. The "data" attribute is an unterminated 
string and is not followed by a comma.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From denismfmcmahon at gmail.com  Thu May 28 14:49:35 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Thu, 28 May 2015 18:49:35 +0000 (UTC)
Subject: Decoding JSON file using python
References: <slrnmmcsfd.1t4.jon+usenet@frosty.unequivocal.co.uk>
 <mailman.117.1432785446.5151.python-list@python.org>
Message-ID: <mk7nvv$qj$3@dont-email.me>

On Thu, 28 May 2015 13:32:39 +1000, Cameron Simpson wrote:

> On 28May2015 01:38, Jon Ribbens <jon+usenet at unequivocal.co.uk> wrote:
>>On 2015-05-27, Karthik Sharma <karthik.sharma at gmail.com> wrote:
>>> I tried modifying the program as follows as per your
>>> suggestion.Doesn't seem to work.
>>
>>That's because you didn't modify the program as per their suggestion,
>>you made completely different changes that bore no relation to what they
>>said.
> 
> Actually, his changes looked good to me. He does print from data first,
> but only for debugging. Then he goes:
> 
>   message = json.loads(data)
> 
> and tried to access message['Message'].
> 
> However I am having trouble reproducing his issue because his quoted
> code is incorrect. I've tried to fix it, as listed below, but I don't
> know what is really meant to be in the 'data" string.

it looks like data is a broken array of one object, part of which is a 
further quoted json string.

There should be a string value after the isEvent but I have no idea what 
it should be, nor what else should come after.

"message":"tdetails":{att:val pairs}

is also wrong in the first level of inner json.

I think he wants data[0]['message'], but the inner json strings are 
broken too, and should look more like.

data: "[{\"Severity\":\"warn\",\"Subject\":\"Reporting\",\"Message\":
\"tdetails\",\"attr_name\":\"{\\\"Product\\\":\\\"Gecko\\\",\\\"CPUs\\
\":8,\\\"Language\\\":\\\"en-GB\\\",\\\"isEvent\\\":\\\"attr_value\\\"}\"}
]",

In this model, if the data attribute is a json string, then

data maps to a list / array

data[0] maps to an object / dictionary

data[0]["Message"] maps to the string literal "tdetails"

data[0]["attr_name"] maps to a string representation of a json ob with 
another level of escaping.

That string can then be loaded, eg:

attr_name = json.loads(data[0]["attr_name"])

See: http:/www.sined.co.uk/python/nested_json.py.txt


-- 
Denis McMahon, denismfmcmahon at gmail.com


From rosuav at gmail.com  Thu May 28 14:51:07 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 29 May 2015 04:51:07 +1000
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <mk7ng1$uq3$1@ger.gmane.org>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
Message-ID: <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>

On Fri, May 29, 2015 at 4:41 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> davidfstr at gmail.com schrieb am 26.05.2015 um 04:24:
>> Has anyone on this list attempted to sandbox Python programs in a
>> serious fashion? I'd be interested to hear your approach.
>
> Not quite sandboxing Python, but I've seen people use my Lupa [1] library
> for this. They're writing all their code in Python, and then let users
> embed their own Lua code into it to script their API. The Lua runtime is
> apparently quite good at sandboxing, and it's really small, just some 600KB
> or so. Lupa then lets you easily control the access to your Python code at
> a whitelist level by intercepting all Python attribute lookups.
>
> It doesn't add much to your application to embed Lua (or even LuaJIT) in
> Python, and it gives users a nicely object oriented language to call and
> orchestrate your Python objects.

Lua's a much weaker language than Python is, though. Can it handle
arbitrary-precision integers? Unicode? Dare I even ask,
arbitrary-precision rationals (fractions.Fraction)? Security comes at
a price, I guess.

ChrisA


From skybuck2000 at hotmail.com  Thu May 28 17:50:00 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 28 May 2015 23:50:00 +0200
Subject: Logic problem: need better logic for desired thruth table.
Message-ID: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>

Hello,

I was just coding and ran into a little logic problem which is as follows:

There are two booleans/variables which can be either false or true.

The desired thrutle table is:

A = input
B = input
C = output

A B C:
-------
F F T
F T F
T F T
T T T

Surpisingly enough I don't think there is a casual/common operator for this 
thruth table.

AND does not apply.
OR does not apply.
XOR does not apply.

So I would need some combined operators to give the desired result.

I tried logic below... but funny enough it failed, now I feel like a noob 
lol and share this funny little fail logic with you.

Can you improve/fix the logic ?

This is python code, but this^ logic/thruth table problem basically applies 
to any programming language:

# loop has to run if:
# while DesiredResult==True:
# Desired truth table for BotWaitForCooldown and CooldownDetected
# BotWaitForCooldown:  CooldownDetected: Desired Result:
# False       False    True
# False       True     False
# True     False    True
# True     True     True
# desired/suiting logic:
# (BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected))

def TestLogic( BotWaitForCooldown, CooldownDetected ):
return BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected) 
# this logic is flawed, please improve logic.

if TestLogic( False, False ) == True:
print "test 1 ok"
else:
print "test 1 failed"

if TestLogic( False, True ) == False:
print "test 2 ok"
else:
print "test 2 failed"

if TestLogic( True, False ) == True:
print "test 3 ok"
else:
print "test 3 failed"

if TestLogic( True, True ) == True:
print "test 4 ok"
else:
print "test 4 failed"

Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Thu May 28 17:55:09 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Thu, 28 May 2015 23:55:09 +0200
Subject: Logic problem: need better logic for desired thruth table.
 (thruth table to logic/boolean operations generator sought/needed)
In-Reply-To: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
Message-ID: <2ac80$55678eb8$5419aafe$56778@news.ziggo.nl>

I think I have run into this problem before... but solved it with some 
seperate if statements.

However in this case/this time I would like to not solve it with if 
statements, but simply and/or/not/xor, in other words, boolean operators.

So what would help is a "thruth table to logic" convertor/generator ?!

Anybody know one that is suited for boolean logic/software 
programming/programming languages/boolean operations ?

Bye,
  Skybuck.

Original posting:

"Skybuck Flying"  wrote in message 
news:3794b$55678d83$5419aafe$56138 at news.ziggo.nl...

Hello,

I was just coding and ran into a little logic problem which is as follows:

There are two booleans/variables which can be either false or true.

The desired thrutle table is:

A = input
B = input
C = output

A B C:
-------
F F T
F T F
T F T
T T T

Surpisingly enough I don't think there is a casual/common operator for this
thruth table.

AND does not apply.
OR does not apply.
XOR does not apply.

So I would need some combined operators to give the desired result.

I tried logic below... but funny enough it failed, now I feel like a noob
lol and share this funny little fail logic with you.

Can you improve/fix the logic ?

This is python code, but this^ logic/thruth table problem basically applies
to any programming language:

# loop has to run if:
# while DesiredResult==True:
# Desired truth table for BotWaitForCooldown and CooldownDetected
# BotWaitForCooldown:  CooldownDetected: Desired Result:
# False       False    True
# False       True     False
# True     False    True
# True     True     True
# desired/suiting logic:
# (BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected))

def TestLogic( BotWaitForCooldown, CooldownDetected ):
return BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected)
# this logic is flawed, please improve logic.

if TestLogic( False, False ) == True:
print "test 1 ok"
else:
print "test 1 failed"

if TestLogic( False, True ) == False:
print "test 2 ok"
else:
print "test 2 failed"

if TestLogic( True, False ) == True:
print "test 3 ok"
else:
print "test 3 failed"

if TestLogic( True, True ) == True:
print "test 4 ok"
else:
print "test 4 failed"

Bye,
  Skybuck.



From sohcahtoa82 at gmail.com  Thu May 28 17:58:19 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Thu, 28 May 2015 14:58:19 -0700 (PDT)
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
Message-ID: <67de616e-5af7-417d-88a9-db1c665df473@googlegroups.com>

On Thursday, May 28, 2015 at 2:50:18 PM UTC-7, Skybuck Flying wrote:
> Hello,
> 
> I was just coding and ran into a little logic problem which is as follows:
> 
> There are two booleans/variables which can be either false or true.
> 
> The desired thrutle table is:
> 
> A = input
> B = input
> C = output
> 
> A B C:
> -------
> F F T
> F T F
> T F T
> T T T
> 
> Surpisingly enough I don't think there is a casual/common operator for this 
> thruth table.
> 
> AND does not apply.
> OR does not apply.
> XOR does not apply.
> 
> So I would need some combined operators to give the desired result.
> 
> I tried logic below... but funny enough it failed, now I feel like a noob 
> lol and share this funny little fail logic with you.
> 
> Can you improve/fix the logic ?
> 
> This is python code, but this^ logic/thruth table problem basically applies 
> to any programming language:
> 
> # loop has to run if:
> # while DesiredResult==True:
> # Desired truth table for BotWaitForCooldown and CooldownDetected
> # BotWaitForCooldown:  CooldownDetected: Desired Result:
> # False       False    True
> # False       True     False
> # True     False    True
> # True     True     True
> # desired/suiting logic:
> # (BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected))
> 
> def TestLogic( BotWaitForCooldown, CooldownDetected ):
> return BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected) 
> # this logic is flawed, please improve logic.
> 
> if TestLogic( False, False ) == True:
> print "test 1 ok"
> else:
> print "test 1 failed"
> 
> if TestLogic( False, True ) == False:
> print "test 2 ok"
> else:
> print "test 2 failed"
> 
> if TestLogic( True, False ) == True:
> print "test 3 ok"
> else:
> print "test 3 failed"
> 
> if TestLogic( True, True ) == True:
> print "test 4 ok"
> else:
> print "test 4 failed"
> 
> Bye,
>   Skybuck.

I think the logic you're really looking for is:

return BotWaitForCooldown or (not (BotWaitForCooldown or CooldownDetected))


From drsalists at gmail.com  Thu May 28 17:58:26 2015
From: drsalists at gmail.com (Dan Stromberg)
Date: Thu, 28 May 2015 14:58:26 -0700
Subject: Pep 484 Type Hint Checker - Python releases supported?
Message-ID: <CAGGBd_pvXnpj0Hhz83TLEa+5fYriYn2jgMshE3jO8LeWsNmZMw@mail.gmail.com>

I believe I heard that the PEP 484 type checker is to go into CPython 3.5.

Since type annotations have been there since 3.0, is it at all likely
that CPython versions < 3.5 will also be supported by this checker?

Thanks!


From invalid at invalid.invalid  Thu May 28 18:01:10 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 28 May 2015 22:01:10 +0000 (UTC)
Subject: Logic problem: need better logic for desired thruth table.
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
Message-ID: <mk8376$2qq$1@reader1.panix.com>

On 2015-05-28, Skybuck Flying <skybuck2000 at hotmail.com> wrote:

> I tried logic below... but funny enough it failed, now I feel like a
> noob lol and share this funny little fail logic with you.
>
> Can you improve/fix the logic ?

> # while DesiredResult==True:
> # Desired truth table for BotWaitForCooldown and CooldownDetected
> # BotWaitForCooldown:  CooldownDetected: Desired Result:
> # False                False             True
> # False                True              False
> # True                 False             True
> # True                 True              True
> # desired/suiting logic:
> # (BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected))

I don't see why you think that's the desired logic, since it doesn't
match your truth table or your test.

> def TestLogic( BotWaitForCooldown, CooldownDetected ):
> return BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected)
> # this logic is flawed, please improve logic.

def TestLogic( BotWaitForCooldown, CooldownDetected ):
    return not ((not BotWaitForCooldown) and CooldownDetected)

works for me...    

-- 
Grant Edwards               grant.b.edwards        Yow! Alright, you!!
                                  at               Imitate a WOUNDED SEAL
                              gmail.com            pleading for a PARKING
                                                   SPACE!!


From python.list at tim.thechases.com  Thu May 28 18:02:29 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Thu, 28 May 2015 17:02:29 -0500
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
Message-ID: <20150528170229.0b5e0f61@bigbox.christie.dr>

On 2015-05-28 23:50, Skybuck Flying wrote:
> A = input
> B = input
> C = output
> 
> A B C:
> -------
> F F T
> F T F
> T F T
> T T T
> 
> Surpisingly enough I don't think there is a casual/common operator
> for this thruth table.
> 
> AND does not apply.
> OR does not apply.
> XOR does not apply.

Sounds like you want "a or not b"

>>> print("\n".join("A=%s B=%s: %s" % (a, b, a or not b) for a in (True, False) for b in (True, False)))
A=True B=True: True
A=True B=False: True
A=False B=True: False
A=False B=False: True

-tkc




From zachary.ware+pylist at gmail.com  Thu May 28 18:06:59 2015
From: zachary.ware+pylist at gmail.com (Zachary Ware)
Date: Thu, 28 May 2015 17:06:59 -0500
Subject: Pep 484 Type Hint Checker - Python releases supported?
In-Reply-To: <CAGGBd_pvXnpj0Hhz83TLEa+5fYriYn2jgMshE3jO8LeWsNmZMw@mail.gmail.com>
References: <CAGGBd_pvXnpj0Hhz83TLEa+5fYriYn2jgMshE3jO8LeWsNmZMw@mail.gmail.com>
Message-ID: <CAKJDb-NpPScOcnChVCNRKg9969axLe7-7Buy-fc1NWhhc7-4FA@mail.gmail.com>

On Thu, May 28, 2015 at 4:58 PM, Dan Stromberg <drsalists at gmail.com> wrote:
> I believe I heard that the PEP 484 type checker is to go into CPython 3.5.
>
> Since type annotations have been there since 3.0, is it at all likely
> that CPython versions < 3.5 will also be supported by this checker?

All that's going into 3.5 is the typing.py module, which will also be
distributed on PyPI.  The actual type checking is being left to
third-party tools like MyPy (which was the basis for the typing.py
module).

I'm not sure if there is any plan to ever include an actual type
checker with the base Python distribution.

-- 
Zach


From skybuck2000 at hotmail.com  Thu May 28 18:07:36 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Fri, 29 May 2015 00:07:36 +0200
Subject: Logic problem: need better logic for desired thruth table.
 (thruth table to logic/boolean operations generator sought/needed)
In-Reply-To: <2ac80$55678eb8$5419aafe$56778@news.ziggo.nl>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <2ac80$55678eb8$5419aafe$56778@news.ziggo.nl>
Message-ID: <c8624$556791a3$5419aafe$58453@news.ziggo.nl>

This is a start lol:

https://www.youtube.com/watch?v=lKqTSBKmWA4

I wonder if it can be simplied... I'll give it a try.

Basically it comes down to creating a logic expression for each true result 
in the desired output and or-ing with each other.

The variables leading to the true result in the desired output need to be 
kept if true, and negated if false.

So for example:
A B C
F T T

((NOT A) AND (B) AND ETC) OR ETC.

What the video didn't really explain is probably to "and" the variables... 
but it did mention "multiply".

I guess "AND" is the closest thing to a multiply ;)

Makes sense... only AND gives a 1 output if all variables are true, 
otherwise it would zero out...

Bye,
  Skybuck. 



From skybuck2000 at hotmail.com  Thu May 28 18:08:51 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Fri, 29 May 2015 00:08:51 +0200
Subject: Logic problem: need better logic for desired thruth table.
 (thruth table to logic/boolean operations generator sought/needed)
In-Reply-To: <c8624$556791a3$5419aafe$58453@news.ziggo.nl>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <2ac80$55678eb8$5419aafe$56778@news.ziggo.nl>
 <c8624$556791a3$5419aafe$58453@news.ziggo.nl>
Message-ID: <c48b1$556791ee$5419aafe$58588@news.ziggo.nl>

However I can already see I am not happy with this video solution.

I have 3 true outputs, and only 1 false output.

That would require a lot of logic.

I guess I can turn it around and negate the whole thing... and focus on the 
false output.

Bye,
  Skybuck. 



From invalid at invalid.invalid  Thu May 28 18:11:36 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 28 May 2015 22:11:36 +0000 (UTC)
Subject: Logic problem: need better logic for desired thruth table.
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <mk8376$2qq$1@reader1.panix.com>
Message-ID: <mk83qo$2qq$3@reader1.panix.com>

On 2015-05-28, Grant Edwards <invalid at invalid.invalid> wrote:
> On 2015-05-28, Skybuck Flying <skybuck2000 at hotmail.com> wrote:
>
>> I tried logic below... but funny enough it failed, now I feel like a
>> noob lol and share this funny little fail logic with you.
>>
>> Can you improve/fix the logic ?
>
>> # while DesiredResult==True:
>> # Desired truth table for BotWaitForCooldown and CooldownDetected
>> # BotWaitForCooldown:  CooldownDetected: Desired Result:
>> # False                False             True
>> # False                True              False
>> # True                 False             True
>> # True                 True              True
>> # desired/suiting logic:
>> # (BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected))
>
> I don't see why you think that's the desired logic, since it doesn't
> match your truth table or your test.
>
>> def TestLogic( BotWaitForCooldown, CooldownDetected ):
>> return BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected)
>> # this logic is flawed, please improve logic.
>
> def TestLogic( BotWaitForCooldown, CooldownDetected ):
>     return not ((not BotWaitForCooldown) and CooldownDetected)
>
> works for me...    

While I think that's the most "obvious" solution and can be verified
by inspection: there's only out input state that is "false", so write an expression
for that one state and invert it.

However, you can apply De Morgan's law to simplify it:

       not ((not BotWaitForCooldown) and CooldownDetected)
       
is same as

       (BotWaitForCooldown or (not CooldownDetected))

-- 
Grant Edwards               grant.b.edwards        Yow! Do you have exactly
                                  at               what I want in a plaid
                              gmail.com            poindexter bar bat??


From skybuck2000 at hotmail.com  Thu May 28 18:12:00 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Fri, 29 May 2015 00:12:00 +0200
Subject: Logic problem: need better logic for desired thruth table.
 (thruth table to logic/boolean operations generator sought/needed)
In-Reply-To: <c48b1$556791ee$5419aafe$58588@news.ziggo.nl>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <2ac80$55678eb8$5419aafe$56778@news.ziggo.nl>
 <c8624$556791a3$5419aafe$58453@news.ziggo.nl>
 <c48b1$556791ee$5419aafe$58588@news.ziggo.nl>
Message-ID: <7b1ef$556792ab$5419aafe$58931@news.ziggo.nl>

Ok, problem solved for now, it seems:

I used video tutorial method and inverted it for the false case ;)

But anyway... I would not only need a "thruth table to logic/boolean 
operations converter" but also a "boolean operations optimizer" ;)

# loop has to run if:
# while DesiredResult==True:
# Desired truth table for BotWaitForCooldown and CooldownDetected
# BotWaitForCooldown:  CooldownDetected: Desired Result:
# False       False    True
# False       True     False
# True     False    True
# True     True     True
# desired/suiting logic:
# (BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected))



def TestLogic( BotWaitForCooldown, CooldownDetected ):
# return BotWaitForCooldown or ((not BotWaitForCooldown) and 
CooldownDetected) # this logic is flawed, please improve logic.
return (not ((not BotWaitForCooldown) and CooldownDetected))  # fixes it.

if TestLogic( False, False ) == True:
print "test 1 ok"
else:
print "test 1 failed"

if TestLogic( False, True ) == False:
print "test 2 ok"
else:
print "test 2 failed"

if TestLogic( True, False ) == True:
print "test 3 ok"
else:
print "test 3 failed"

if TestLogic( True, True ) == True:
print "test 4 ok"
else:
print "test 4 failed"

Bye,
  Skybuck. 



From jpiitula at ling.helsinki.fi  Thu May 28 18:13:04 2015
From: jpiitula at ling.helsinki.fi (Jussi Piitulainen)
Date: Fri, 29 May 2015 01:13:04 +0300
Subject: Logic problem: need better logic for desired thruth table.
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
Message-ID: <lf5zj4o1hcf.fsf@ling.helsinki.fi>

Skybuck Flying writes:

> There are two booleans/variables which can be either false or true.
>
> The desired thrutle table is:
>
> A = input
> B = input
> C = output
>
> A B C:
> -------
> F F T
> F T F
> T F T
> T T T

That's A >= B, where True >= False:

  >>> BB = False, True
  >>> print(*((A, B, A >= B) for A in BB for B in BB), sep = '\n')
(False, False, True)
(False, True, False)
(True, False, True)
(True, True, True)
  >>> 


From invalid at invalid.invalid  Thu May 28 18:13:08 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 28 May 2015 22:13:08 +0000 (UTC)
Subject: Logic problem: need better logic for desired thruth table.
 (thruth table to logic/boolean operations generator sought/needed)
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <2ac80$55678eb8$5419aafe$56778@news.ziggo.nl>
 <c8624$556791a3$5419aafe$58453@news.ziggo.nl>
 <c48b1$556791ee$5419aafe$58588@news.ziggo.nl>
Message-ID: <mk83tk$2qq$4@reader1.panix.com>

On 2015-05-28, Skybuck Flying <skybuck2000 at hotmail.com> wrote:

> However I can already see I am not happy with this video solution.
>
> I have 3 true outputs, and only 1 false output.
>
> That would require a lot of logic.
>
> I guess I can turn it around and negate the whole thing... and focus on the 
> false output.

Don't they teach Karnaugh mapping any more?

http://en.wikipedia.org/wiki/Karnaugh_map

-- 
Grant Edwards               grant.b.edwards        Yow! I want the presidency
                                  at               so bad I can already taste
                              gmail.com            the hors d'oeuvres.


From invalid at invalid.invalid  Thu May 28 18:15:14 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 28 May 2015 22:15:14 +0000 (UTC)
Subject: Logic problem: need better logic for desired thruth table.
 (thruth table to logic/boolean operations generator sought/needed)
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <2ac80$55678eb8$5419aafe$56778@news.ziggo.nl>
 <c8624$556791a3$5419aafe$58453@news.ziggo.nl>
 <c48b1$556791ee$5419aafe$58588@news.ziggo.nl>
 <mk83tk$2qq$4@reader1.panix.com>
Message-ID: <mk841i$2qq$5@reader1.panix.com>

On 2015-05-28, Grant Edwards <invalid at invalid.invalid> wrote:
> On 2015-05-28, Skybuck Flying <skybuck2000 at hotmail.com> wrote:
>
>> However I can already see I am not happy with this video solution.
>>
>> I have 3 true outputs, and only 1 false output.
>>
>> That would require a lot of logic.
>>
>> I guess I can turn it around and negate the whole thing... and focus on the 
>> false output.
>
> Don't they teach Karnaugh mapping any more?
>
> http://en.wikipedia.org/wiki/Karnaugh_map

BTW, your example truth table is shown in the paragraph '2-variable
map examples'

-- 
Grant Edwards               grant.b.edwards        Yow! Oh my GOD -- the
                                  at               SUN just fell into YANKEE
                              gmail.com            STADIUM!!


From torriem at gmail.com  Thu May 28 18:16:44 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 28 May 2015 16:16:44 -0600
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <67de616e-5af7-417d-88a9-db1c665df473@googlegroups.com>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <67de616e-5af7-417d-88a9-db1c665df473@googlegroups.com>
Message-ID: <556793CC.3020002@gmail.com>

On 05/28/2015 03:58 PM, sohcahtoa82 at gmail.com wrote:
> I think the logic you're really looking for is:
> 
> return BotWaitForCooldown or (not (BotWaitForCooldown or CooldownDetected))

Yes this is the simplest form.  For more complicated truth tables you
can create a K map and then apply a reduction algorithm to it.  This is
commonly done in logic circuits to reduce the number of gates to the
minimum.  While not faster, it can be expressed as:

return BotWaitForCooldown or not BotWaitForCooldown and not
CooldownDectected

Order of operations puts the ands above the ors.




From drsalists at gmail.com  Thu May 28 18:17:49 2015
From: drsalists at gmail.com (Dan Stromberg)
Date: Thu, 28 May 2015 15:17:49 -0700
Subject: Accessing DataSocket Server with Python
In-Reply-To: <CAO2fJ-+UjZNqo2yQRRkuSsEJwFAGOnbB9JPCCWX7zX7jt_5POg@mail.gmail.com>
References: <CAO2fJ-+UjZNqo2yQRRkuSsEJwFAGOnbB9JPCCWX7zX7jt_5POg@mail.gmail.com>
Message-ID: <CAGGBd_pEf0E6skDgzVuWLsPEX73OvpTkQ=CwgjKF6Yz+fC6bsw@mail.gmail.com>

I have no idea about the protocol used by NI DataSockets, but you
might be able to reverse engineer the protocol by using the official
client with a sniffer.

Also, be aware that TCP/IP guarantees that you get the correct data in
the correct order, but it doesn't guarantee anything about the sizes
of the chunks in which that data arrives.  So you could send 100
bytes, 100 bytes, 100 bytes, but on the other end receive 100 bytes,
50 bytes, 75 bytes, 75 bytes.  When you catenate them all together,
it's still the same data though.

HTH.

On Wed, May 27, 2015 at 4:30 AM, Garrone, Corrado
<corrado.garrone at roche.com> wrote:
> Dear Python Team,
>
> currently I am working on a research project for my bachelor degree. A
> LabVIEW application is used for current and power measurements, whereas the
> measured data are sent to DataSocket Server, a technology by National
> Instruments used for data exchange between computers and applications.
> DataSocket is based on TCP/IP and thus requesting data from DataSocket
> should be similar to an internet request.
> I know with the socket library in Python it is possible with to establish
> sockets, send internet requests and communicate between clients and servers.
> Is there a possibility to access NI DataSocket and get measurement data with
> Python on the same computer where Python is installed and the codes are
> executed? Can you maybe send me an example code where such a connection with
> DataSocket is established?
>
> If you got any queries, please do not hesitate to contact me.
>
> Thank you very much for your efforts.
>
> Kind regards,
>
> Corrado Garrone
> DH-Student Fachrichtung Elektrotechnik / Co-op Student B.Eng. Electrical
> Engineering
>
> Roche Diagnostics GmbH
> DFGHMV8Y6164
> Sandhofer Strasse 116
> 68305 Mannheim / Germany
>
> Phone: apprentice
> mailto:corrado.garrone at roche.com
>
> Roche Diagnostics GmbH
> Sandhofer Stra?e 116; D?68305 Mannheim; Telefon +49?621?759?0; Telefax
> +49?621?759?2890
> Sitz der Gesellschaft: Mannheim - Registergericht: AG Mannheim HRB 3962 -
> Gesch?ftsf?hrung: Dr. Ursula Redeker, Sprecherin; Edgar Vieth -
> Aufsichtsratsvorsitzender: Dr. Severin Schwan
>
> Confidentiality Note
> This message is intended only for the use of the named recipient(s) and may
> contain confidential and/or privileged information. If you are not the
> intended recipient, please contact the sender and delete the message. Any
> unauthorized use of the information contained in this message is prohibited.
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>


From rosuav at gmail.com  Thu May 28 18:26:30 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 29 May 2015 08:26:30 +1000
Subject: Pep 484 Type Hint Checker - Python releases supported?
In-Reply-To: <CAKJDb-NpPScOcnChVCNRKg9969axLe7-7Buy-fc1NWhhc7-4FA@mail.gmail.com>
References: <CAGGBd_pvXnpj0Hhz83TLEa+5fYriYn2jgMshE3jO8LeWsNmZMw@mail.gmail.com>
 <CAKJDb-NpPScOcnChVCNRKg9969axLe7-7Buy-fc1NWhhc7-4FA@mail.gmail.com>
Message-ID: <CAPTjJmqFbhCtGwRwVA-OaQu1rWguBDQ8aTx5oqRFz8026fFhnQ@mail.gmail.com>

On Fri, May 29, 2015 at 8:06 AM, Zachary Ware
<zachary.ware+pylist at gmail.com> wrote:
> On Thu, May 28, 2015 at 4:58 PM, Dan Stromberg <drsalists at gmail.com> wrote:
>> I believe I heard that the PEP 484 type checker is to go into CPython 3.5.
>>
>> Since type annotations have been there since 3.0, is it at all likely
>> that CPython versions < 3.5 will also be supported by this checker?
>
> All that's going into 3.5 is the typing.py module, which will also be
> distributed on PyPI.  The actual type checking is being left to
> third-party tools like MyPy (which was the basis for the typing.py
> module).
>
> I'm not sure if there is any plan to ever include an actual type
> checker with the base Python distribution.

And MyPy is quite capable of probing code that belongs on older
interpreters than 3.5 (obviously, since it already exists); it even
has some measure of support for probing Py2 code, although you do need
to have a Python 3.2+ to actually run MyPy on.

ChrisA


From lew.pitcher at digitalfreehold.ca  Thu May 28 18:39:51 2015
From: lew.pitcher at digitalfreehold.ca (Lew Pitcher)
Date: Thu, 28 May 2015 18:39:51 -0400
Subject: Logic problem: need better logic for desired thruth table.
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
Message-ID: <YOM9x.125121$AJ4.46857@fx02.iad>

On Thursday May 28 2015 17:50, in comp.lang.c, "Skybuck Flying"
<skybuck2000 at hotmail.com> wrote:

> Hello,
> 
> I was just coding and ran into a little logic problem which is as follows:
> 
> There are two booleans/variables which can be either false or true.
> 
> The desired thrutle table is:
> 
> A = input
> B = input
> C = output
> 
> A B C:
> -------
> F F T
> F T F
> T F T
> T T T

Seems simple enough: C == A || !B

18:38 $ cat testlogic.c
#include <stdio.h>
#include <stdlib.h>

/*
** A = input
** B = input
** C = output
** 
** A B C:
** -------
** F F T
** F T F
** T F T
** T T T
*/

int testlogic(int a, int b)
{
  return (a || !b);
}

int main(void)
{
                      /* A B C */
  int ttable[4][3] = {  {0,0,1},        /* F F T */
                        {0,1,0},        /* F T F */
                        {1,0,1},        /* T F T */
                        {1,1,1}         /* T T T */
                     };
  int rc = EXIT_SUCCESS;
  int i, max;

  for (i = 0, max = sizeof(ttable) / sizeof(ttable[0]); i < max ; ++i)
    if (testlogic(ttable[i][0],ttable[i][1]) != ttable[i][2])
    {
      printf("testlogic failed on test %d\n",i);
      rc = EXIT_FAILURE;
    }

  if (rc == EXIT_SUCCESS) puts("SUCCESS");

  return rc;
}
18:39 $ cc -o testlogic testlogic.c
18:39 $ ./testlogic
SUCCESS


-- 
Lew Pitcher
"In Skills, We Trust"
PGP public key available upon request



From rosuav at gmail.com  Thu May 28 18:50:10 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 29 May 2015 08:50:10 +1000
Subject: No subject
In-Reply-To: <BLU403-EAS29442C643780B181B828175F1CA0@phx.gbl>
References: <BLU403-EAS29442C643780B181B828175F1CA0@phx.gbl>
Message-ID: <CAPTjJmqS_uTd0aY7JvDe=Cs3f8ikB1zQi5bLTgSOCCeoqnVYig@mail.gmail.com>

2015-05-29 9:44 GMT+10:00 yasna araya <ansay_27 at hotmail.com>:
> hola, escribo para solicitar el uso del programa Python para trabajarlo con
> mi hijo. desde ya muchas gracias
>
> Enviado desde Correo de Windows

Hello!

This is an English-language list, and most of the people here - myself
included - are not fluent in Spanish. But you can download Python from
here:

https://www.python.org/downloads/

If that wasn't what you were looking for, I apologize; possibly
someone else here speaks better Spanish, or perhaps you could rephrase
your question in English.

All the best!

ChrisA


From invalid at invalid.invalid  Thu May 28 18:54:23 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 28 May 2015 22:54:23 +0000 (UTC)
Subject: Logic problem: need better logic for desired thruth table.
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <67de616e-5af7-417d-88a9-db1c665df473@googlegroups.com>
 <mailman.143.1432851416.5151.python-list@python.org>
Message-ID: <mk86av$7bc$1@reader1.panix.com>

On 2015-05-28, Michael Torrie <torriem at gmail.com> wrote:
> On 05/28/2015 03:58 PM, sohcahtoa82 at gmail.com wrote:
>> I think the logic you're really looking for is:
>> 
>> return BotWaitForCooldown or (not (BotWaitForCooldown or CooldownDetected))
>
> Yes this is the simplest form.

Not really.


In old school notation, that's
            ___
        A + A+B      

Apply De Morgan's law to the second term        
            _   _
       A + (A * B)

Invert
       ________
            _ _
       A + (A*B)

Apply Demorgan's law once
            _____
       _    _   _
       A * (A * B)

and again for the second term
       _       
       A * (A + B)

Apply the distributive property:

       _     _
       A*A + A*B

The first term is always false, so we can drop it:
       _
       A*B

Now invert it again to cancel out the one we did back a few steps:

       ___
       _
       A*B

De Morgan one last time:
           _
       A + B
       
-- 
Grant Edwards               grant.b.edwards        Yow! Xerox your lunch
                                  at               and file it under "sex
                              gmail.com            offenders"!


From breamoreboy at yahoo.co.uk  Thu May 28 18:56:11 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 28 May 2015 23:56:11 +0100
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <YOM9x.125121$AJ4.46857@fx02.iad>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <YOM9x.125121$AJ4.46857@fx02.iad>
Message-ID: <mk86ed$j5v$1@ger.gmane.org>

On 28/05/2015 23:39, Lew Pitcher wrote:
> On Thursday May 28 2015 17:50, in comp.lang.c, "Skybuck Flying"
> <skybuck2000 at hotmail.com> wrote:
>
>> Hello,
>>
>> I was just coding and ran into a little logic problem which is as follows:
>>
>> There are two booleans/variables which can be either false or true.
>>
>> The desired thrutle table is:
>>
>> A = input
>> B = input
>> C = output
>>
>> A B C:
>> -------
>> F F T
>> F T F
>> T F T
>> T T T
>
> Seems simple enough: C == A || !B
>
> 18:38 $ cat testlogic.c
> #include <stdio.h>
> #include <stdlib.h>
>
> /*
> ** A = input
> ** B = input
> ** C = output
> **
> ** A B C:
> ** -------
> ** F F T
> ** F T F
> ** T F T
> ** T T T
> */
>
> int testlogic(int a, int b)
> {
>    return (a || !b);
> }
>
> int main(void)
> {
>                        /* A B C */
>    int ttable[4][3] = {  {0,0,1},        /* F F T */
>                          {0,1,0},        /* F T F */
>                          {1,0,1},        /* T F T */
>                          {1,1,1}         /* T T T */
>                       };
>    int rc = EXIT_SUCCESS;
>    int i, max;
>
>    for (i = 0, max = sizeof(ttable) / sizeof(ttable[0]); i < max ; ++i)
>      if (testlogic(ttable[i][0],ttable[i][1]) != ttable[i][2])
>      {
>        printf("testlogic failed on test %d\n",i);
>        rc = EXIT_FAILURE;
>      }
>
>    if (rc == EXIT_SUCCESS) puts("SUCCESS");
>
>    return rc;
> }
> 18:39 $ cc -o testlogic testlogic.c
> 18:39 $ ./testlogic
> SUCCESS
>
>

Strangest looking Python I've ever seen.  Or is it a case of "Get thee 
behind me, Satan" :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From invalid at invalid.invalid  Thu May 28 19:03:53 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Thu, 28 May 2015 23:03:53 +0000 (UTC)
Subject: Logic problem: need better logic for desired thruth table.
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <67de616e-5af7-417d-88a9-db1c665df473@googlegroups.com>
 <mailman.143.1432851416.5151.python-list@python.org>
 <mk86av$7bc$1@reader1.panix.com>
Message-ID: <mk86sp$7bc$2@reader1.panix.com>

On 2015-05-28, Grant Edwards <invalid at invalid.invalid> wrote:
> On 2015-05-28, Michael Torrie <torriem at gmail.com> wrote:
>> On 05/28/2015 03:58 PM, sohcahtoa82 at gmail.com wrote:
>>> I think the logic you're really looking for is:
>>> 
>>> return BotWaitForCooldown or (not (BotWaitForCooldown or CooldownDetected))
>>
>> Yes this is the simplest form.
>
> Not really.
>
>
> In old school notation, that's
>             ___
>         A + A+B      
>
> [...]

That derivation was excessively round-about.  Here is the same thing
in fewer steps:

Invert:
         _______
             ___
         A + A+B      
         
De Morgan:
         _  
         A * (A + B)

Distribute:
         _     _
         A*A + A*B

Drop the null term:
          _
          A*B

Invert (to cancel out the first step)
          ___
          _
          A*B

De Morgan:
            _
          A+B

-- 
Grant Edwards               grant.b.edwards        Yow! I'm reporting for duty
                                  at               as a modern person.  I want
                              gmail.com            to do the Latin Hustle now!


From nestorjb at gmail.com  Thu May 28 19:19:24 2015
From: nestorjb at gmail.com (=?UTF-8?B?TsOpc3RvciBCb3Njw6Fu?=)
Date: Thu, 28 May 2015 18:49:24 -0430
Subject: No subject
In-Reply-To: <CAPTjJmqS_uTd0aY7JvDe=Cs3f8ikB1zQi5bLTgSOCCeoqnVYig@mail.gmail.com>
References: <BLU403-EAS29442C643780B181B828175F1CA0@phx.gbl>
 <CAPTjJmqS_uTd0aY7JvDe=Cs3f8ikB1zQi5bLTgSOCCeoqnVYig@mail.gmail.com>
Message-ID: <CALyLaQ+KRpZ_WpP+csooxNBVkrnb1mLm8AiaFEpPNd3iBgNcBw@mail.gmail.com>

Hola Yasna

Chris te escribe que puedes bajar la soluci?n Python desde este link:

https://www.python.org/downloads/

Tambi?n escribe que el foro es en ingl?s ya que la mayor?a de las personas
que forman parte de la lista hablan ingl?s.

Saludos y ?xitos

Translated:

Hi Yasna

Chris wrote that you can download Python from the following link:

https://www.python.org/downloads/

He also wrote that this forum is in english because the majority of the
people that are part of the list also speak english.

Regards

On Thu, May 28, 2015 at 6:20 PM, Chris Angelico <rosuav at gmail.com> wrote:

> 2015-05-29 9:44 GMT+10:00 yasna araya <ansay_27 at hotmail.com>:
> > hola, escribo para solicitar el uso del programa Python para trabajarlo
> con
> > mi hijo. desde ya muchas gracias
> >
> > Enviado desde Correo de Windows
>
> Hello!
>
> This is an English-language list, and most of the people here - myself
> included - are not fluent in Spanish. But you can download Python from
> here:
>
> https://www.python.org/downloads/
>
> If that wasn't what you were looking for, I apologize; possibly
> someone else here speaks better Spanish, or perhaps you could rephrase
> your question in English.
>
> All the best!
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150528/f1f256e3/attachment.html>

From flebber.crue at gmail.com  Thu May 28 19:20:40 2015
From: flebber.crue at gmail.com (Sayth Renshaw)
Date: Thu, 28 May 2015 16:20:40 -0700 (PDT)
Subject: refer to xpath attributes generally
Message-ID: <c1f6abdd-d280-458b-a8e2-a695bc22d233@googlegroups.com>

Is there a way to specify to all attributes in xpath? Instead of directly Eg //element/@attr

So that you could effectively loop and filter if an element had more than one Attribute?

So items = tree.xpath(@all_attrs)

>From lxml import etree 

Tree = etree.Xpath(//root)

For k, v in items:
    Choose your k's


Thanks for your time in advance
Please forgive some formatting errors phone auto correct 
Sayth


From nospam at needed.com  Thu May 28 19:21:33 2015
From: nospam at needed.com (Paul)
Date: Thu, 28 May 2015 19:21:33 -0400
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
Message-ID: <mk87rk$i6r$1@dont-email.me>

Skybuck Flying wrote:
> Hello,
> 
> I was just coding and ran into a little logic problem which is as follows:
> 
> There are two booleans/variables which can be either false or true.
> 
> The desired thrutle table is:
> 
> A = input
> B = input
> C = output
> 
> A B C:
> -------
> F F T
> F T F
> T F T
> T T T
> 
> Surpisingly enough I don't think there is a casual/common operator for 
> this thruth table.
> 
> AND does not apply.
> OR does not apply.
> XOR does not apply.
> 
> So I would need some combined operators to give the desired result.
> 
> I tried logic below... but funny enough it failed, now I feel like a 
> noob lol and share this funny little fail logic with you.
> 
> Can you improve/fix the logic ?
> 
> This is python code, but this^ logic/thruth table problem basically 
> applies to any programming language:
> 
> # loop has to run if:
> # while DesiredResult==True:
> # Desired truth table for BotWaitForCooldown and CooldownDetected
> # BotWaitForCooldown:  CooldownDetected: Desired Result:
> # False       False    True
> # False       True     False
> # True     False    True
> # True     True     True
> # desired/suiting logic:
> # (BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected))
> 
> def TestLogic( BotWaitForCooldown, CooldownDetected ):
> return BotWaitForCooldown or ((not BotWaitForCooldown) and 
> CooldownDetected) # this logic is flawed, please improve logic.
> 
> if TestLogic( False, False ) == True:
> print "test 1 ok"
> else:
> print "test 1 failed"
> 
> if TestLogic( False, True ) == False:
> print "test 2 ok"
> else:
> print "test 2 failed"
> 
> if TestLogic( True, False ) == True:
> print "test 3 ok"
> else:
> print "test 3 failed"
> 
> if TestLogic( True, True ) == True:
> print "test 4 ok"
> else:
> print "test 4 failed"
> 
> Bye,
>  Skybuck.

If you ever have a really complicated truth table,
you can use Quine McCluskey minimization. At work, I
had no tool for minimizing boolean equations, so I got
some code from another engineer, code that ran on a
mainframe computer. And I converted the code to run
on a personal computer. The computers were so slow
back then, it might take ten to fifteen minutes to
minimize a ten variable truth table. On the mainframe
you could have a 2MB array for storage, whereas on my
personal computer at the time, the memory was segmented
and required some tricks to get enough.

http://en.wikipedia.org/wiki/Quine%E2%80%93McCluskey_algorithm

If it isn't a scam, the source code should be
reasonably short. The program I was using, might have
been on the order of 120 lines of source. This example
is a bit longer, and the couple programs I've been looking
at the source, I don't recognize what they're doing.

http://sourceforge.net/projects/mini-qmc/files/?source=navbar

Your problem isn't large enough to need this sort
of thing, but I thought I'd throw it in as a topic
of general interest. If it's one thing I've learned
over the years, hand-optimization of boolean equations
frequently leads to errors. And it's when you start
engaging your brain, and saying stuff like "I know the
answer", instead of sticking with your boolean algebra,
that errors creep in.

If you need a test case for your QM code, enter an
"XOR tree", as an XOR tree is irreducible and
should cough out the same info, as you entered in
the first place. That was one of my test cases,
when porting the code maybe 30 years ago. (And
no, I didn't keep a copy of the code. We didn't
do stuff like that back then. I didn't even have
a computer at home back then.)

    Paul


From denismfmcmahon at gmail.com  Thu May 28 19:25:02 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Thu, 28 May 2015 23:25:02 +0000 (UTC)
Subject: Logic problem: need better logic for desired thruth table.
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <67de616e-5af7-417d-88a9-db1c665df473@googlegroups.com>
Message-ID: <mk884e$gth$1@dont-email.me>

On Thu, 28 May 2015 14:58:19 -0700, sohcahtoa82 wrote:

> On Thursday, May 28, 2015 at 2:50:18 PM UTC-7, Skybuck Flying wrote:
>> Hello,

>> # Desired truth table for BotWaitForCooldown and CooldownDetected 
>> # BotWaitForCooldown:  CooldownDetected: Desired Result:
>> # False                False             True 
>> # False                True              False 
>> # True                 False             True 
>> # True                 True              True 

> I think the logic you're really looking for is:

> return BotWaitForCooldown or (not (BotWaitForCooldown or
> CooldownDetected))

Nope, it simplifies to:

BotWaitForCooldown or not CooldownDetected

-- 
Denis McMahon, denismfmcmahon at gmail.com


From ansay_27 at hotmail.com  Thu May 28 19:44:26 2015
From: ansay_27 at hotmail.com (yasna araya)
Date: Thu, 28 May 2015 23:44:26 +0000
Subject: No subject
Message-ID: <BLU403-EAS29442C643780B181B828175F1CA0@phx.gbl>

hola, escribo para solicitar el uso del programa Python para trabajarlo con mi hijo. desde ya muchas gracias






Enviado desde Correo de Windows
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150528/17395d66/attachment.html>

From mr.smittye at gmail.com  Thu May 28 20:03:47 2015
From: mr.smittye at gmail.com (IronManMark20)
Date: Thu, 28 May 2015 17:03:47 -0700 (PDT)
Subject: Getting an image from a file on windows
Message-ID: <2dc4ea02-496f-41da-a287-63f178450c0e@googlegroups.com>

Hello,

I have been working on a function that gets a bitmap of the thumbnail for a file. I have had problems getting a large image (256x256) and I was wondering if someone could help me on one object initialization that is driving me nuts.

I have code here: https://gist.github.com/IronManMark20/a83fe7ff7ea9b40c09af

I need to initialize a *void pointer for the third argument for SHGetImageList (ref https://msdn.microsoft.com/en-us/library/bb762185). 

What should the initialization look like (please give me example code, not just "Oh it should be a ctypes.c_void_p").

Thanks!!!


From sohcahtoa82 at gmail.com  Thu May 28 20:15:15 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Thu, 28 May 2015 17:15:15 -0700 (PDT)
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <mailman.143.1432851416.5151.python-list@python.org>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <67de616e-5af7-417d-88a9-db1c665df473@googlegroups.com>
 <mailman.143.1432851416.5151.python-list@python.org>
Message-ID: <aa769234-4987-4593-8359-45ea04c533b4@googlegroups.com>

On Thursday, May 28, 2015 at 3:17:10 PM UTC-7, Michael Torrie wrote:
> On 05/28/2015 03:58 PM, sohcahtoa82 at gmail.com wrote:
> > I think the logic you're really looking for is:
> > 
> > return BotWaitForCooldown or (not (BotWaitForCooldown or CooldownDetected))
> 
> Yes this is the simplest form.  For more complicated truth tables you
> can create a K map and then apply a reduction algorithm to it.  This is
> commonly done in logic circuits to reduce the number of gates to the
> minimum.  While not faster, it can be expressed as:
> 
> return BotWaitForCooldown or not BotWaitForCooldown and not
> CooldownDectected
> 
> Order of operations puts the ands above the ors.

Order of operations might put the ands above the ors, but I still like using parentheses in expressions like that as I think it makes it more clear.


From python at mrabarnett.plus.com  Thu May 28 20:36:46 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 29 May 2015 01:36:46 +0100
Subject: Getting an image from a file on windows
In-Reply-To: <2dc4ea02-496f-41da-a287-63f178450c0e@googlegroups.com>
References: <2dc4ea02-496f-41da-a287-63f178450c0e@googlegroups.com>
Message-ID: <5567B49E.6040302@mrabarnett.plus.com>

On 2015-05-29 01:03, IronManMark20 wrote:
> Hello,
>
> I have been working on a function that gets a bitmap of the thumbnail
> for a file. I have had problems getting a large image (256x256) and I
> was wondering if someone could help me on one object initialization
> that is driving me nuts.
>
> I have code here:
> https://gist.github.com/IronManMark20/a83fe7ff7ea9b40c09af
>
> I need to initialize a *void pointer for the third argument for
> SHGetImageList (ref
> https://msdn.microsoft.com/en-us/library/bb762185).
>
> What should the initialization look like (please give me example
> code, not just "Oh it should be a ctypes.c_void_p").
>
> Thanks!!!
>
Have you tried byref(hico)?


From torriem at gmail.com  Thu May 28 20:52:22 2015
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 28 May 2015 18:52:22 -0600
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <mk86sp$7bc$2@reader1.panix.com>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <67de616e-5af7-417d-88a9-db1c665df473@googlegroups.com>
 <mailman.143.1432851416.5151.python-list@python.org>
 <mk86av$7bc$1@reader1.panix.com> <mk86sp$7bc$2@reader1.panix.com>
Message-ID: <5567B846.8000201@gmail.com>

On 05/28/2015 05:03 PM, Grant Edwards wrote:
> On 2015-05-28, Grant Edwards <invalid at invalid.invalid> wrote:
>> On 2015-05-28, Michael Torrie <torriem at gmail.com> wrote:
>>> On 05/28/2015 03:58 PM, sohcahtoa82 at gmail.com wrote:
>>>> I think the logic you're really looking for is:
>>>>
>>>> return BotWaitForCooldown or (not (BotWaitForCooldown or CooldownDetected))
>>>
>>> Yes this is the simplest form.
>>
>> Not really.

True enough. My skills from EE are pretty rusty.




From mr.smittye at gmail.com  Thu May 28 21:39:41 2015
From: mr.smittye at gmail.com (IronManMark20)
Date: Thu, 28 May 2015 18:39:41 -0700 (PDT)
Subject: Getting an image from a file on windows
In-Reply-To: <mailman.150.1432859817.5151.python-list@python.org>
References: <2dc4ea02-496f-41da-a287-63f178450c0e@googlegroups.com>
 <mailman.150.1432859817.5151.python-list@python.org>
Message-ID: <bed60d72-ccc8-459d-b8d0-df43d1c87d83@googlegroups.com>

On Thursday, May 28, 2015 at 5:37:07 PM UTC-7, MRAB wrote:
> On 2015-05-29 01:03, IronManMark20 wrote:
> > Hello,
> >
> > I have been working on a function that gets a bitmap of the thumbnail
> > for a file. I have had problems getting a large image (256x256) and I
> > was wondering if someone could help me on one object initialization
> > that is driving me nuts.
> >
> > I have code here:
> > https://gist.github.com/IronManMark20/a83fe7ff7ea9b40c09af
> >
> > I need to initialize a *void pointer for the third argument for
> > SHGetImageList (ref
> > https://msdn.microsoft.com/en-us/library/bb762185).
> >
> > What should the initialization look like (please give me example
> > code, not just "Oh it should be a ctypes.c_void_p").
> >
> > Thanks!!!
> >
> Have you tried byref(hico)?

Thanks so much! That seems to have done it. Now I get to figure out how to get the image. 


From steve at pearwood.info  Thu May 28 22:00:32 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 29 May 2015 12:00:32 +1000
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net> <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
 <87siagagpx.fsf@elektro.pacujo.net>
Message-ID: <5567c840$0$12998$c3e8da3$5496439d@news.astraweb.com>

On Fri, 29 May 2015 01:01 am, Marko Rauhamaa wrote:

> Anssi Saari <as at sci.fi>:
> 
>> Do you have an example of state pattern using nested classes and
>> python? With a quick look I didn't happen to find one in any language.
> 
> Here's an sampling from my mail server:


I haven't studied this in close detail, but first impressions is that this
is not well-written Python code. The obvious problems that come to mind:


> class SMTPServerConnection(ServerConnection):
>     #:     :     :
>     #:     :     :
> 
>     def __init__(self, server, sock, domain, peer):
>         super().__init__(server, sock)
>         conn = self
>         client_ip = peer[0]
> 
>         class STATE:
>             def __str__(self):
>                 return self.__class__.__name__

A minor criticism: each time you instantiate a new SMTP server connection,
it creates new STATE, IDLE etc. classes. That's wasteful and inefficient
unless you have a good reason for it, although you may not care if you only
have a single connection at a time.

More serious problem: by nesting the state classes, it is much harder to
test the state classes in isolation from a connection.


>             def handle_command(self, cmd):
>                 conn.log("handle_command (unexpected): {}".format(cmd))
>                 assert False

At first glance, this appears to be an abuse of `assert` and risks breaking
your code when running under -O which turns debugging (and hence asserts)
off. Since the intent is to make these abstract methods which must be
overridden, I'd prefer to define an AbstractMethodError, then:

    raise AbstractMethodError("unexpected %s" % cmd)

then do the logging elsewhere.

This has the advantage(?) of eliminating the need to refer to conn, which
means that the methods are no longer closures and the entire inner class
can be moved out of the SMTPServerConnection body.

[...]
>         class IDLE(STATE):
>             def handle_command(self, cmd):
>                 conn.log("handle_command: {}".format(cmd))

Using a closure for information hiding is a solid functional equivalent to
using private fields in an OOP context. However, "private" should be
considered an anti-testing idiom. I would seriously consider this entire
design to be an anti-pattern, and would prefer to one of two alternate
designs:

(1) Pull the state classes out of the SMTPServerConnection class. On
initialisation, each state takes a conn argument. All references to "conn"
are replaced by "self.conn". That's basically the conventional OOP
approach.

(2) Again, pull the state classes out of the SMTPServerConnection class,
except this time there is no need to instantiate the state classes at all!
Decorate all the methods with @classmethod, and have them take the
connection as an explicit argument. The caller (namely the
SMTPServerConnection  instance) provides itself as argument to the state
methods, like so:

    self.state = IDLE  # No need to instantiate.
    self.state.process_ehlo_or_helo(self, other, args, go, here)

This reduces implicit coupling, makes it explicit that the state methods
depend on the connection, avoids reference loops, and enables easy mocking
for testing.

The only downsides are that people coming from a conventional (e.g. Java)
OOP background will freak at seeing you using a class as a first class
value (pun intended), and it will be ever-so-slightly tiresome to have to
decorate each and every method with classmethod.

(Perhaps a class decorator is the solution to that second issue?)

The "design pattern" community is dominated by Java, where classes are not
values, so this idiom is impossible in Java and unlikely to have a DP name.
But it really should have one -- in a language where classes are themselves
values, there is no reason why a class *must* be instantiated, particularly
if you're only using a single instance of the class. Anyone ever come
across a named design pattern that involves using classes directly without
instantiating them?

I'm basically looking for a less inelegant term for "instanceless class" --
not so much a singleton as a zeroton.



-- 
Steven



From flebber.crue at gmail.com  Thu May 28 22:28:50 2015
From: flebber.crue at gmail.com (Sayth Renshaw)
Date: Thu, 28 May 2015 19:28:50 -0700 (PDT)
Subject: refer to xpath attributes generally
In-Reply-To: <c1f6abdd-d280-458b-a8e2-a695bc22d233@googlegroups.com>
References: <c1f6abdd-d280-458b-a8e2-a695bc22d233@googlegroups.com>
Message-ID: <fd038caf-a0f5-40cd-acb5-299a64ff8461@googlegroups.com>

On Friday, 29 May 2015 09:21:01 UTC+10, Sayth Renshaw  wrote:
> Is there a way to specify to all attributes in xpath? Instead of directly Eg //element/@attr
> 
> So that you could effectively loop and filter if an element had more than one Attribute?
> 
> So items = tree.xpath(@all_attrs)
> 
> From lxml import etree 
> 
> Tree = etree.Xpath(//root)
> 
> For k, v in items:
>     Choose your k's
> 
> 
> Thanks for your time in advance
> Please forgive some formatting errors phone auto correct 
> Sayth

It appears this is the best answer

n [17]: with open('C:/Users/Sayth/Desktop/20150530RHIL0.xml', 'rb') as f:
   ....:     tree = etree.parse(f)
   ....:     for attrResult in tree.xpath('/meeting/race/@*'):
   ....:         print(attrResult.attrname)


From skybuck2000 at hotmail.com  Thu May 28 22:45:01 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Fri, 29 May 2015 04:45:01 +0200
Subject: Logic problem: need better logic for desired thruth table.
 (thruth table to logic/boolean operations generator sought/needed)
In-Reply-To: <7b1ef$556792ab$5419aafe$58931@news.ziggo.nl>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <2ac80$55678eb8$5419aafe$56778@news.ziggo.nl>
 <c8624$556791a3$5419aafe$58453@news.ziggo.nl>
 <c48b1$556791ee$5419aafe$58588@news.ziggo.nl>
 <7b1ef$556792ab$5419aafe$58931@news.ziggo.nl>
Message-ID: <dee1f$5567d2a8$5419aafe$38671@news.ziggo.nl>

Interestingly enough the shortest I have seen so far is ***:

def TestLogic( BotWaitForCooldown, CooldownDetected ):

# return BotWaitForCooldown or ((not BotWaitForCooldown) and 
CooldownDetected) # this logic is flawed, please improve logic.
# return (not ((not BotWaitForCooldown) and CooldownDetected)) # fixes it.
# return  (BotWaitForCooldown or (not CooldownDetected)) # optimization but 
looks weird :)
return (BotWaitForCooldown >= CooldownDetected) # *** even shorter, wow 
cool, thanks to Jussi Piitulainen

Apperently there is a short-coming/flaw in the reasoning about boolean 
logic, if boolean logic is converted to numbers/0/1 and comparisions allowed 
then apperently there are even shorter forms ?!? Am I correct ? Or am I 
missing something ? Perhaps branching don't count ? Hmmmm..... Is this 
branching ? Or something else... hmmm.... I think the comparison result 
could be used without requiring branching... so I think my conclusion might 
be correct.

Bye,
  Skybuck.

"Skybuck Flying"  wrote in message 
news:7b1ef$556792ab$5419aafe$58931 at news.ziggo.nl...

Ok, problem solved for now, it seems:

I used video tutorial method and inverted it for the false case ;)

But anyway... I would not only need a "thruth table to logic/boolean
operations converter" but also a "boolean operations optimizer" ;)

# loop has to run if:
# while DesiredResult==True:
# Desired truth table for BotWaitForCooldown and CooldownDetected
# BotWaitForCooldown:  CooldownDetected: Desired Result:
# False       False    True
# False       True     False
# True     False    True
# True     True     True
# desired/suiting logic:
# (BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected))



def TestLogic( BotWaitForCooldown, CooldownDetected ):
# return BotWaitForCooldown or ((not BotWaitForCooldown) and
CooldownDetected) # this logic is flawed, please improve logic.
return (not ((not BotWaitForCooldown) and CooldownDetected))  # fixes it.

if TestLogic( False, False ) == True:
print "test 1 ok"
else:
print "test 1 failed"

if TestLogic( False, True ) == False:
print "test 2 ok"
else:
print "test 2 failed"

if TestLogic( True, False ) == True:
print "test 3 ok"
else:
print "test 3 failed"

if TestLogic( True, True ) == True:
print "test 4 ok"
else:
print "test 4 failed"

Bye,
  Skybuck.



From skybuck2000 at hotmail.com  Thu May 28 22:46:52 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Fri, 29 May 2015 04:46:52 +0200
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <mk884e$gth$1@dont-email.me>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl><67de616e-5af7-417d-88a9-db1c665df473@googlegroups.com>
 <mk884e$gth$1@dont-email.me>
Message-ID: <9dfb3$5567d317$5419aafe$39452@news.ziggo.nl>



"Denis McMahon"  wrote in message news:mk884e$gth$1 at dont-email.me... 

On Thu, 28 May 2015 14:58:19 -0700, sohcahtoa82 wrote:

> On Thursday, May 28, 2015 at 2:50:18 PM UTC-7, Skybuck Flying wrote:
>> Hello,

>> # Desired truth table for BotWaitForCooldown and CooldownDetected 
>> # BotWaitForCooldown:  CooldownDetected: Desired Result:
>> # False                False             True 
>> # False                True              False 
>> # True                 False             True 
>> # True                 True              True 

> I think the logic you're really looking for is:

> return BotWaitForCooldown or (not (BotWaitForCooldown or
> CooldownDetected))

"
Nope, it simplifies to:

BotWaitForCooldown or not CooldownDetected
"

Apperently it can be simplied even further:

BotWaitForCooldown >= CooldownDetected

Interesting isn't it ? :)

Bye,
  Skybuck.


From skybuck2000 at hotmail.com  Thu May 28 22:56:07 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Fri, 29 May 2015 04:56:07 +0200
Subject: Logic problem: need better logic for desired thruth table.
 (thruth table to logic/boolean operations generator sought/needed)
In-Reply-To: <dee1f$5567d2a8$5419aafe$38671@news.ziggo.nl>
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
 <2ac80$55678eb8$5419aafe$56778@news.ziggo.nl>
 <c8624$556791a3$5419aafe$58453@news.ziggo.nl>
 <c48b1$556791ee$5419aafe$58588@news.ziggo.nl>
 <7b1ef$556792ab$5419aafe$58931@news.ziggo.nl>
 <dee1f$5567d2a8$5419aafe$38671@news.ziggo.nl>
Message-ID: <a7145$5567d542$5419aafe$43256@news.ziggo.nl>

I am not so sure anymore about my conclusion, I will investigate this 
further tomorrow.

It seems safe to conclude that at least the following operators have their 
own thruth tables:

=
<>
>
<
>=
<=

These are the comparision operators.

Assume True is greater than False allows them to be used as well.

The question that remains is:

How many "gates" or "basic" operations or "wires" would they require.

Are some of these perhaps very "efficient" and could thus lead to even 
shorter gate designs ?

However for my purposes, reducing code, the answer is already: YES

For software logic/boolean reduction the answer is already YES, funny and 
interestingly enough ! ;) :)

Bye,
  Skybuck.





From random832 at fastmail.us  Thu May 28 23:20:19 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Thu, 28 May 2015 23:20:19 -0400
Subject: Logic problem: need better logic for desired thruth table.
Message-ID: <1432869619.871453.281061217.70B58077@webmail.messagingengine.com>

On Thu, May 28, 2015, at 17:50, Skybuck Flying wrote:
> Surpisingly enough I don't think there is a casual/common operator for
> this 
> thruth table.
> 
> AND does not apply.
> OR does not apply.
> XOR does not apply.

All sixteen possible logical operators have formal names. This one is
called "B implies A". It can be implemented, as others have mentioned
with "A or not B".

The possibility of spelling these with the comparison operators, as some
have suggested, is a consequence of Python's implementation where True
== 1 and False == 0. In other languages bool may not be relatable (or at
least not orderable), or False may be == -1.


From mr.smittye at gmail.com  Thu May 28 23:41:40 2015
From: mr.smittye at gmail.com (IronManMark20)
Date: Thu, 28 May 2015 20:41:40 -0700 (PDT)
Subject: Getting an image from a file on windows
In-Reply-To: <mailman.150.1432859817.5151.python-list@python.org>
References: <2dc4ea02-496f-41da-a287-63f178450c0e@googlegroups.com>
 <mailman.150.1432859817.5151.python-list@python.org>
Message-ID: <b1b18b70-18ba-4012-8ce9-e6f543277ffc@googlegroups.com>

On Thursday, May 28, 2015 at 5:37:07 PM UTC-7, MRAB wrote:
> On 2015-05-29 01:03, IronManMark20 wrote:
> > Hello,
> >
> > I have been working on a function that gets a bitmap of the thumbnail
> > for a file. I have had problems getting a large image (256x256) and I
> > was wondering if someone could help me on one object initialization
> > that is driving me nuts.
> >
> > I have code here:
> > https://gist.github.com/IronManMark20/a83fe7ff7ea9b40c09af
> >
> > I need to initialize a *void pointer for the third argument for
> > SHGetImageList (ref
> > https://msdn.microsoft.com/en-us/library/bb762185).
> >
> > What should the initialization look like (please give me example
> > code, not just "Oh it should be a ctypes.c_void_p").
> >
> > Thanks!!!
> >
> Have you tried byref(hico)?

Could you suggest how I could get an hIcon from the newly drawn image? I have been trying, but I haven't found anything.


From lac at openend.se  Thu May 28 23:47:34 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 29 May 2015 05:47:34 +0200
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
Message-ID: <201505290347.t4T3lYjC006610@fido.openend.se>

webmaster at python.org just got some mail from some poor embarrased
soul who ran this program and broke their Python install.

They are running Mac OSX 10.7.5

They are getting:

     Utility has encountered a fatal error, and will now terminate.  A
     Python runtime could not be located. You may need to install a
     framework build of Python or edit the PyRuntimeLocations array in this
     applications info.plist file.  Then there are two oblong circles. One
     says Open Console. The other says Terminate.

So https://docs.python.org/2/using/mac.html says:

   The Apple-provided build of Python is installed in
   /System/Library/Frameworks/Python.framework and /usr/bin/python,
   respectively. You should never modify or delete these, as they are
   Apple-controlled and are used by Apple- or third-party software.

So, I assume this poor soul has done precisely that.

What do I tell her to do now?

Laura




From rosuav at gmail.com  Thu May 28 23:48:55 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 29 May 2015 13:48:55 +1000
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <1432869619.871453.281061217.70B58077@webmail.messagingengine.com>
References: <1432869619.871453.281061217.70B58077@webmail.messagingengine.com>
Message-ID: <CAPTjJmpRycvcU_v4W3PQcn3gB2qeReQnTKg3u6beXT-BQrBDwg@mail.gmail.com>

On Fri, May 29, 2015 at 1:20 PM,  <random832 at fastmail.us> wrote:
> The possibility of spelling these with the comparison operators, as some
> have suggested, is a consequence of Python's implementation where True
> == 1 and False == 0. In other languages bool may not be relatable (or at
> least not orderable), or False may be == -1.

True. That said, though, using 0 for False and 1 for True is easily
the most common convention in use today, and the next most likely case
is that comparing booleans would give a simple and immediate error. So
it's most likely to be safe to do. Cross-language compatibility is a
tricky thing anyway; there are all sorts of odd edge cases, even with
otherwise-similar languages (Pike and Python, for instance, have
slightly different handling of slice ranges), so anything that's done
in Python is meant to be interpreted with Python semantics.

ChrisA


From skybuck2000 at hotmail.com  Thu May 28 23:49:15 2015
From: skybuck2000 at hotmail.com (Skybuck Flying)
Date: Fri, 29 May 2015 05:49:15 +0200
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <mailman.152.1432869623.5151.python-list@python.org>
References: <mailman.152.1432869623.5151.python-list@python.org>
Message-ID: <a30da$5567e1b6$5419aafe$6950@news.ziggo.nl>

wrote in message news:mailman.152.1432869623.5151.python-list at python.org...

On Thu, May 28, 2015, at 17:50, Skybuck Flying wrote:
> Surpisingly enough I don't think there is a casual/common operator for
> this
> thruth table.
>
> AND does not apply.
> OR does not apply.
> XOR does not apply.

"
All sixteen possible logical operators have formal names. This one is
called "B implies A". It can be implemented, as others have mentioned
with "A or not B".
"

Ok thanks for this information.

I was just wondering how many thruth table combinations there can be for a 
typical thruth table with 2 inputs and 1 output.

Since there are 2 inputs, this means 4 possible outputs, which means 2 to 
the power of 4 different thruth tables possible, which is indeed 16.

Perhaps you have a link to all possible 16 thruth tables and their formal 
names ? That would help !

Bye,
  Skybuck. 



From lac at openend.se  Fri May 29 00:44:44 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 29 May 2015 06:44:44 +0200
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: Message from "Skybuck Flying" <skybuck2000@hotmail.com> of "Fri,
 29 May 2015 05:49:15 +0200." <a30da$5567e1b6$5419aafe$6950@news.ziggo.nl>
References: <mailman.152.1432869623.5151.python-list@python.org><a30da$5567e1b6$5419aafe$6950@news.ziggo.nl>
Message-ID: <201505290444.t4T4iiFC008571@fido.openend.se>

You may be interested in PyEda
https://pypi.python.org/pypi/pyeda

It is for electronic design automation.  But it will minimise
truth tables for you.

http://pyeda.readthedocs.org/en/latest/search.html?q=truth+tables&check_keywords=yes&area=default

Laura


From nimbiotics at gmail.com  Fri May 29 01:09:49 2015
From: nimbiotics at gmail.com (Mario R. Osorio)
Date: Thu, 28 May 2015 22:09:49 -0700 (PDT)
Subject: No subject
In-Reply-To: <mailman.147.1432853419.5151.python-list@python.org>
References: <BLU403-EAS29442C643780B181B828175F1CA0@phx.gbl>
 <mailman.147.1432853419.5151.python-list@python.org>
Message-ID: <26259e7a-2fff-4da6-aab1-1dd9fbb9f728@googlegroups.com>

Chris. Este grupo es en Ingles. La verdad no se si existen grupos en espa?ol, pero juraria que si.

Entiendo que quieres ense?arle python a tu hijo. Aca te envio algunos recursos. Espero que te sirvan:

https://silvercorp.wordpress.com/2012/05/27/pasos-de-instalacion-de-python-en-windows/

http://www.slideshare.net/CAChemE/instalar-python-27-y-3-en-windows-anaconda

https://devcode.la/tutoriales/instalacion-de-python-27-en-windows-8/

http://codehero.co/python-desde-cero-instalacion-y-configuracion/

http://codehero.co/series/python-desde-cero.html

http://www.ehowenespanol.com/instalar-python-windows-como_44790/

https://www.youtube.com/watch?v=NEo4tDu6tU8

http://es.diveintopython.net/toc.html

http://mundogeek.net/tutorial-python/

http://pyspanishdoc.sourceforge.net/index.html

https://www.youtube.com/watch?v=GbfXj0iQQH0

https://www.youtube.com/watch?v=W7xZvUQhQWg

https://www.youtube.com/watch?v=DVD17s7hAj8


From lac at openend.se  Fri May 29 01:33:45 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 29 May 2015 07:33:45 +0200
Subject: No subject
In-Reply-To: Message from "Mario R. Osorio" <nimbiotics@gmail.com> of "Thu,
 28 May 2015 22:09:49 -0700."
 <26259e7a-2fff-4da6-aab1-1dd9fbb9f728@googlegroups.com>
References: <BLU403-EAS29442C643780B181B828175F1CA0@phx.gbl>
 <mailman.147.1432853419.5151.python-list@python.org><26259e7a-2fff-4da6-aab1-1dd9fbb9f728@googlegroups.com>
Message-ID: <201505290533.t4T5XjvQ009762@fido.openend.se>

Sabe usted acerca de estas p?ginas?
https://mail.python.org/mailman/listinfo/python-es
https://wiki.python.org/moin/SpanishLanguage

Laura




From steve at pearwood.info  Fri May 29 01:46:28 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 29 May 2015 15:46:28 +1000
Subject: should "self" be changed?
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net> <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
 <87siagagpx.fsf@elektro.pacujo.net>
 <5567c840$0$12998$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <5567fd36$0$13006$c3e8da3$5496439d@news.astraweb.com>

On Fri, 29 May 2015 12:00 pm, Steven D'Aprano wrote:

> I haven't studied this in close detail, but first impressions is that this
> is not well-written Python code. The obvious problems that come to mind:

Well, first impressions can be misleading... I wrote the above, thinking
that there were more problems with the code than there actually are.
(Specifically, I didn't notice that the inner classes used closures for
their methods, then forgot to go back and revise that sentence.) So the
code actually was not as bad as I first thought, not withstanding the
issues that I described in my previous post.



-- 
Steven



From steve at pearwood.info  Fri May 29 02:00:00 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 29 May 2015 16:00:00 +1000
Subject: Logic problem: need better logic for desired thruth table.
References: <mailman.152.1432869623.5151.python-list@python.org>
 <a30da$5567e1b6$5419aafe$6950@news.ziggo.nl>
Message-ID: <55680061$0$12995$c3e8da3$5496439d@news.astraweb.com>

On Fri, 29 May 2015 01:49 pm, Skybuck Flying wrote:

> wrote in message
> news:mailman.152.1432869623.5151.python-list at python.org...
> 
> On Thu, May 28, 2015, at 17:50, Skybuck Flying wrote:
>> Surpisingly enough I don't think there is a casual/common operator for
>> this
>> thruth table.
>>
>> AND does not apply.
>> OR does not apply.
>> XOR does not apply.
> 
> "
> All sixteen possible logical operators have formal names. This one is
> called "B implies A". It can be implemented, as others have mentioned
> with "A or not B".
> "
> 
> Ok thanks for this information.
> 
> I was just wondering how many thruth table combinations there can be for a
> typical thruth table with 2 inputs and 1 output.

The word is TRUTH not THRUTH. I don't know if you're a native English
speaker, but "thruth" sounds like the name of a bird (thrush) spoken with a
lisp.

> Since there are 2 inputs, this means 4 possible outputs, which means 2 to
> the power of 4 different thruth tables possible, which is indeed 16.
> 
> Perhaps you have a link to all possible 16 thruth tables and their formal
> names ? That would help !

http://en.wikipedia.org/wiki/Truth_table




-- 
Steven



From nad at acm.org  Fri May 29 02:07:42 2015
From: nad at acm.org (Ned Deily)
Date: Thu, 28 May 2015 23:07:42 -0700
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
References: <201505290347.t4T3lYjC006610@fido.openend.se>
Message-ID: <nad-9B828D.23074228052015@news.gmane.org>

In article <201505290347.t4T3lYjC006610 at fido.openend.se>,
 Laura Creighton <lac at openend.se> wrote:
> webmaster at python.org just got some mail from some poor embarrased
> soul who ran this program and broke their Python install.
> 
> They are running Mac OSX 10.7.5
> 
> They are getting:
> 
>      Utility has encountered a fatal error, and will now terminate.  A
>      Python runtime could not be located. You may need to install a
>      framework build of Python or edit the PyRuntimeLocations array in this
>      applications info.plist file.  Then there are two oblong circles. One
>      says Open Console. The other says Terminate.
> 
> So https://docs.python.org/2/using/mac.html says:
> 
>    The Apple-provided build of Python is installed in
>    /System/Library/Frameworks/Python.framework and /usr/bin/python,
>    respectively. You should never modify or delete these, as they are
>    Apple-controlled and are used by Apple- or third-party software.
> 
> So, I assume this poor soul has done precisely that.
> 
> What do I tell her to do now?

It would be helpful to know what utility it is that is encountering the 
fatal error; that message is not familiar and I suspect it is coming 
from trying to run some third-party application.  It may be that the 
application was depending on a third-party framework installation of 
Python (in /Library/Frameworks rather than the system Python in 
/System/Library/Frameworks), such as provided by the python.org OS X 
installers, but without knowing more information, like what particular 
version of Python is needed, it would only be speculation.  Perhaps the 
best thing to do is to suggest the OP to participate directly here or an 
Apple users' forum like http://apple.stackexchange.com.

-- 
 Ned Deily,
 nad at acm.org



From stefan_ml at behnel.de  Fri May 29 02:18:34 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 29 May 2015 08:18:34 +0200
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
Message-ID: <mk90bq$dn4$1@ger.gmane.org>

Chris Angelico schrieb am 28.05.2015 um 20:51:
> On Fri, May 29, 2015 at 4:41 AM, Stefan Behnel wrote:
>> davidfstr at gmail.com schrieb am 26.05.2015 um 04:24:
>>> Has anyone on this list attempted to sandbox Python programs in a
>>> serious fashion? I'd be interested to hear your approach.
>>
>> Not quite sandboxing Python, but I've seen people use my Lupa [1] library
>> for this. They're writing all their code in Python, and then let users
>> embed their own Lua code into it to script their API. The Lua runtime is
>> apparently quite good at sandboxing, and it's really small, just some 600KB
>> or so. Lupa then lets you easily control the access to your Python code at
>> a whitelist level by intercepting all Python attribute lookups.
>>
>> It doesn't add much to your application to embed Lua (or even LuaJIT) in
>> Python, and it gives users a nicely object oriented language to call and
>> orchestrate your Python objects.
> 
> Lua's a much weaker language than Python is, though. Can it handle
> arbitrary-precision integers? Unicode? Dare I even ask,
> arbitrary-precision rationals (fractions.Fraction)?

All of those and way more, as long as you use it embedded in Python.


> Security comes at a price, I guess.

Sure, but features aren't the price here.

Stefan




From techtonik at gmail.com  Fri May 29 03:14:56 2015
From: techtonik at gmail.com (anatoly techtonik)
Date: Fri, 29 May 2015 10:14:56 +0300
Subject: Subscribe to get an answer vs automatic CC Was: Fwd: Lossless
 bulletproof conversion to unicode (backslashing) (fwd)
Message-ID: <CAPkN8xK_iGT+K5cwjkCBg95oG+-pYZg17xSHxATn7ttk4kToYg@mail.gmail.com>

On Wed, May 27, 2015 at 3:57 PM, Laura Creighton <lac at openend.se> wrote:
> Chris Angelico apparantly has a problem with cc'd people who aren't
> on the list.

I thought that CC in this case works automatically? If that's not
the case, then I'll be annoyed by this too. So, thanks for CCing.
=)

Also, https://mail.python.org/mailman/listinfo/python-list - this
page should probably mention that you need to subscribe to
ask a question (people like me won't read it anyway, but it
may help others).

> python-list is very quiet these days, so if you
> subscribe it won't be drinking from the firehose.

1688 / 29 = 58.206896551724135 msg/day don't look quiet,
and I don't want to be distracted by interesting side threads,
because I will accomplish nothing, get stressed, and things
get worse at the end of the day.

> And you can
> always turn off delivery when you are done. Or you can just
> go read the archives: https://mail.python.org/pipermail/python-list/2015-May/thread.html

I wish I could also commend from the web interface. I've
heard Mailman 3.0 is stable, it will be interesting to see what
is it capable of.


From rosuav at gmail.com  Fri May 29 03:41:10 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 29 May 2015 17:41:10 +1000
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <mk90bq$dn4$1@ger.gmane.org>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
Message-ID: <CAPTjJmrYQtT3TbCHL7ss8nq4JTszuDDc07uPdHOUkiAczVu_+w@mail.gmail.com>

On Fri, May 29, 2015 at 4:18 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> Lua's a much weaker language than Python is, though. Can it handle
>> arbitrary-precision integers? Unicode? Dare I even ask,
>> arbitrary-precision rationals (fractions.Fraction)?
>
> All of those and way more, as long as you use it embedded in Python.

Okay, so how would you go about using Lua-embedded-in-Python to
manipulate Unicode text? I'm not talking about things like the
unicodedata module and the name/codepoint lookups, I'm talking about
the basics of working with international text and the fundamental need
for your native string type to cope with that. Do you have to keep
bouncing back and forth between Python and Lua? And if you do
arithmetic in the single most obvious way, what happens?

http://www.lua.org/pil/2.3.html

Looks to me as if Lua doesn't have integers at all, and so the obvious
form of arithmetic will be equivalent to doing everything in Python
floats. That's not arbitrary precision. There's also a comment that
you can use some other type for numbers, but I suspect that's still
"some other C type", so you still can't do arbitrary precision. (Plus
you get one type for ints and floats, so even if you did select some
magic type that bounces through to a Python int, it'd stop you from
doing any non-integral arithmetic at all.)

http://www.lua.org/pil/2.4.html

Likewise, eight-bit strings, not Unicode. While I could accept some
sort of inter-language thunk for something uncommon, like
fractions.Fraction, forcing people to thunk back and forth for basic
arithmetic and string manipulation is way too much hassle - which
means they won't do it, which means strings will be eight-bit and
numbers will be doubles. And eight-bit strings might be treated as
UTF-8, or might be treated as some arbitrary codepage, or might be
both at once in different contexts, so you can't really depend on them
being anything more than ASCII.

Or can you change all this when you embed Lua?

ChrisA


From techtonik at gmail.com  Fri May 29 04:05:07 2015
From: techtonik at gmail.com (anatoly techtonik)
Date: Fri, 29 May 2015 11:05:07 +0300
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
 (fwd)
In-Reply-To: <201505271257.t4RCv1R2015793@fido.openend.se>
References: <201505271257.t4RCv1R2015793@fido.openend.se>
Message-ID: <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>

On Wed, May 27, 2015 at 3:57 PM, Laura Creighton <lac at openend.se> wrote:
> ------- Forwarded Message
>
> Return-Path: <python-list-bounces+lac=openend.se at python.org>
> Received: from mail.python.org (mail.python.org [82.94.164.166])
>         by theraft.openend.se (8.14.4/8.14.4/Debian-4) with ESMTP id t4RC09ap02From: Chris Angelico <rosuav at gmail.com>
> Cc: "python-list at python.org" <python-list at python.org>
>
>
> On Wed, May 27, 2015 at 9:52 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>> And the short answer is that we need unicode because we are printing this
>> information to the stdout, and stdout is opened in text mode at least on
>> Windows, and without explicit conversion, Python will try to decode stuff
>> as being `ascii` and fail anyway.
>
> So you're working with text.

No. It is unknown.

I am printing Nodes of SCons build graph and I don't know how Nodes are
represented. In my case it appeared that Node contained Russian text, which
led to crash of SCons. It could contain Russian text in cp1251 or in utf-8 or in
KOI-8 and I can't do guessing of all possible encodings there. I just need to
print that tree without crash or information loss.

> That means you HAVE to decode it somehow;
> you fundamentally cannot print bytes to the console. Lossless
> concealment of arbitrary bytes won't help you.

Won't help me with what? I am debugging build scripts to find out the
*structure* of my dependencies and then all of the sudden Python crashes
with UnicodeDecode error leaving me pronouncing bad Russian curses
aloud.

It is not even less forgiving than Java, but is also more treacherous,
because of its run-time nature.

It will surely help to preserve my zen if Python could just flow through
the nodes of this graph. Garbage is okay - I can clean it up or remove if it
stands in the way, just disrupt my flow or say me that now I want to deal
with UnicodeDecode errors. Because I don't.

> If you can't adequately
> decode everything, either backslash-escape the rest, or use a
> replacement character; you can't print out those bytes.

Yes. How to backslash the rest in Python 2? In Python 3 there is
some freaky "surrogateescape" error strategy, but what to do in
Python 2?

Replacement character is not a solution, because it is a data loss,
and if I want to do post processing of graph log, I won't be able to
recover the missing bits.

> And no, I will not cc you. Subscribe to the list if you're going to
> ask a question.

Added Mailman to my suxx tracker:
https://github.com/techtonik/suxx-tracker#mailman

-- 
anatoly t.


From rosuav at gmail.com  Fri May 29 04:19:14 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 29 May 2015 18:19:14 +1000
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
 (fwd)
In-Reply-To: <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
References: <201505271257.t4RCv1R2015793@fido.openend.se>
 <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
Message-ID: <CAPTjJmoTjLmN=WQFjPcyDtNxMEUmQrTOpft4FziJd75HELTXSg@mail.gmail.com>

On Fri, May 29, 2015 at 6:05 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>> On Wed, May 27, 2015 at 9:52 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>>> And the short answer is that we need unicode because we are printing this
>>> information to the stdout, and stdout is opened in text mode at least on
>>> Windows, and without explicit conversion, Python will try to decode stuff
>>> as being `ascii` and fail anyway.
>>
>> So you're working with text.
>
> No. It is unknown.
>
> I am printing Nodes of SCons build graph and I don't know how Nodes are
> represented. In my case it appeared that Node contained Russian text, which
> led to crash of SCons. It could contain Russian text in cp1251 or in utf-8 or in
> KOI-8 and I can't do guessing of all possible encodings there. I just need to
> print that tree without crash or information loss.

You're saying it's text, but you don't know the encoding. You're
trying to display bytes as if they're text, but fundamentally, you're
trying to work with text.

>> That means you HAVE to decode it somehow;
>> you fundamentally cannot print bytes to the console. Lossless
>> concealment of arbitrary bytes won't help you.
>
> Won't help me with what? I am debugging build scripts to find out the
> *structure* of my dependencies and then all of the sudden Python crashes
> with UnicodeDecode error leaving me pronouncing bad Russian curses
> aloud.

Your fundamental problem is not the UnicodeDecodeError, but the
unknown encoding. What you're seeing is that Python refuses to be
sloppy.

>> If you can't adequately
>> decode everything, either backslash-escape the rest, or use a
>> replacement character; you can't print out those bytes.
>
> Yes. How to backslash the rest in Python 2? In Python 3 there is
> some freaky "surrogateescape" error strategy, but what to do in
> Python 2?

Not sure what's so freaky about it. But hey. If Python 2 can't do what
you want, is it so hard to use Python 3? Unicode support really is
better. Alternatively, just do something like this:

b = "some arbitrary byte string that you got from somewhere"
try:
    text = b.decode("utf-8")
except UnicodeDecodeError:
    text = repr(b).decode("ascii")

The repr of a byte string in Py2 should be a safe way to display
arbitrary bytes, without data loss. It will expand the string
significantly (four characters for one \xNN escape, plus adding
backslashes to everything else that needs them), but it does guarantee
safety.

> Replacement character is not a solution, because it is a data loss,
> and if I want to do post processing of graph log, I won't be able to
> recover the missing bits.
>
>> And no, I will not cc you. Subscribe to the list if you're going to
>> ask a question.
>
> Added Mailman to my suxx tracker:
> https://github.com/techtonik/suxx-tracker#mailman

Why? You're trying to fire questions out to a community without being
a part of that community. Why is that the software's problem?

You can either subscribe to the list/ng or follow via some web
interface, but it's unreasonable to ask everyone to cc you. Imagine if
we _did_ all cc you, but we also cc you in on an entire sub-thread
that you're not interested in. Or maybe half of us do and half don't.
What then? You don't get any sort of control over what you get copies
of. Is that really what you want?

ChrisA


From lac at openend.se  Fri May 29 04:41:09 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 29 May 2015 10:41:09 +0200
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
 (fwd)
In-Reply-To: Message from anatoly techtonik <techtonik@gmail.com> of "Fri,
 29 May 2015 11:05:07 +0300."
 <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
References: <201505271257.t4RCv1R2015793@fido.openend.se><CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
Message-ID: <201505290841.t4T8f9Tr014513@fido.openend.se>

In a message of Fri, 29 May 2015 11:05:07 +0300, anatoly techtonik writes:

>Added Mailman to my suxx tracker:
>https://github.com/techtonik/suxx-tracker#mailman

You are damning the wrong piece of software -- this is not a problem
with mailman; mailman doesn't care at all what software you use to
read mail and reply to it with.  The problem is with the various
readers and repliers that people are using.  In particular, people on
the other side of one the usenet -> python-list gateway may not be seeing
this as mail at all, or sending their replies as mail.

But back to your original problem.

I still don't understand why you need to go from some lossless
representation of your filename, back to the original.  You start
with the binary version of the filename  -- a series of bytes which
turns out to be good Cyrillic text, but could be anything.  You store
that as the first so many bytes of your file. If ever you need to have
the original representation of your filename, you already have it,
right there, by reading the first so many bytes of your file.  Why
care about what the user sees as a filename?

Laura




From stefan_ml at behnel.de  Fri May 29 05:23:17 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 29 May 2015 11:23:17 +0200
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <CAPTjJmrYQtT3TbCHL7ss8nq4JTszuDDc07uPdHOUkiAczVu_+w@mail.gmail.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <CAPTjJmrYQtT3TbCHL7ss8nq4JTszuDDc07uPdHOUkiAczVu_+w@mail.gmail.com>
Message-ID: <mk9b65$rp6$1@ger.gmane.org>

Chris Angelico schrieb am 29.05.2015 um 09:41:
> On Fri, May 29, 2015 at 4:18 PM, Stefan Behnel wrote:
>>> Lua's a much weaker language than Python is, though. Can it handle
>>> arbitrary-precision integers? Unicode? Dare I even ask,
>>> arbitrary-precision rationals (fractions.Fraction)?
>>
>> All of those and way more, as long as you use it embedded in Python.
> 
> Okay, so how would you go about using Lua-embedded-in-Python to
> manipulate Unicode text?

Lua only supports byte strings, so Lupa will encode and decode them for
you. If that's not enough, you'll have to work with Python Unicode string
objects through the language interface. (And I just noticed that the
handling can be improved here by overloading Lua operators with Python
operators - not currently implemented.)


> Looks to me as if Lua doesn't have integers at all

The standard number type in Lua is a C double float, i.e. the steady
integer range is somewhere within +/-2^53. That tends to be enough for a
*lot* of use cases. You could change that type in the Lua C code (e.g. to a
64 bit int), but that's usually a bad idea. The same comment as above
applies: if you need Python object features, use Python objects.

Embedding Lua in Python gives you access to all of Python's objects and
ecosystem. It may not always be as cool to use as from Python, but in that
case, why not code it in Python in the first place? You wouldn't use
Lua/Lupa to write whole applications, just the user defined parts of them.
The rest can happily remain in Python. And should, for your own sanity.

Stefan




From rosuav at gmail.com  Fri May 29 05:38:21 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 29 May 2015 19:38:21 +1000
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <mk9b65$rp6$1@ger.gmane.org>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <CAPTjJmrYQtT3TbCHL7ss8nq4JTszuDDc07uPdHOUkiAczVu_+w@mail.gmail.com>
 <mk9b65$rp6$1@ger.gmane.org>
Message-ID: <CAPTjJmqF_wm7mCVg3i4KWCemyMXWLwVEZXor1m+8wrw8PCjmvg@mail.gmail.com>

On Fri, May 29, 2015 at 7:23 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Chris Angelico schrieb am 29.05.2015 um 09:41:
>> On Fri, May 29, 2015 at 4:18 PM, Stefan Behnel wrote:
>>>> Lua's a much weaker language than Python is, though. Can it handle
>>>> arbitrary-precision integers? Unicode? Dare I even ask,
>>>> arbitrary-precision rationals (fractions.Fraction)?
>>>
>>> All of those and way more, as long as you use it embedded in Python.
>>
>> Okay, so how would you go about using Lua-embedded-in-Python to
>> manipulate Unicode text?
>
> Lua only supports byte strings, so Lupa will encode and decode them for
> you. If that's not enough, you'll have to work with Python Unicode string
> objects through the language interface. (And I just noticed that the
> handling can be improved here by overloading Lua operators with Python
> operators - not currently implemented.)
>
>
>> Looks to me as if Lua doesn't have integers at all
>
> The standard number type in Lua is a C double float, i.e. the steady
> integer range is somewhere within +/-2^53. That tends to be enough for a
> *lot* of use cases. You could change that type in the Lua C code (e.g. to a
> 64 bit int), but that's usually a bad idea. The same comment as above
> applies: if you need Python object features, use Python objects.

Unicode strings shouldn't involve the hassle of bouncing through an
interface layer. Nobody will bother, and the result will be code
that's ASCII-only. That happens often enough even in Python 2, where
u"foo" is a Unicode string.

> Embedding Lua in Python gives you access to all of Python's objects and
> ecosystem. It may not always be as cool to use as from Python, but in that
> case, why not code it in Python in the first place? You wouldn't use
> Lua/Lupa to write whole applications, just the user defined parts of them.
> The rest can happily remain in Python. And should, for your own sanity.

The point was to sandbox something inside Python. Otherwise, yes, just
write it in Python. But if you do have to sandbox like this, you lose
language-level Unicode support, language-level arbitrary precision
integers, etcetera, etcetera, etcetera. So I stand by my previous
statement: The price of security is functionality.

ChrisA


From techtonik at gmail.com  Fri May 29 05:46:01 2015
From: techtonik at gmail.com (anatoly techtonik)
Date: Fri, 29 May 2015 12:46:01 +0300
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
 (fwd)
In-Reply-To: <201505290841.t4T8f9Tr014513@fido.openend.se>
References: <201505271257.t4RCv1R2015793@fido.openend.se> <techtonik@gmail.com>
 <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
 <201505290841.t4T8f9Tr014513@fido.openend.se>
Message-ID: <CAPkN8xLz1ykBXSxEp+jrURgEGkubcnq2CCbrxkpmQvUhcPOvOQ@mail.gmail.com>

On Fri, May 29, 2015 at 11:41 AM, Laura Creighton <lac at openend.se> wrote:
> In a message of Fri, 29 May 2015 11:05:07 +0300, anatoly techtonik writes:
>
>>Added Mailman to my suxx tracker:
>>https://github.com/techtonik/suxx-tracker#mailman
>
> You are damning the wrong piece of software -- this is not a problem
> with mailman; mailman doesn't care at all what software you use to
> read mail and reply to it with.  The problem is with the various
> readers and repliers that people are using.  In particular, people on
> the other side of one the usenet -> python-list gateway may not be seeing
> this as mail at all, or sending their replies as mail.

Sounds legit. But middle ux in suxx stands for user experience,
and Mailman still doesn't improve it. If Mailman could subscribe
me automatically to the thread I am starting, that would resolve
all the problems.

> But back to your original problem.
>
> I still don't understand why you need to go from some lossless
> representation of your filename, back to the original.

It is just happened that the only way to get graph out of SCons
is to print its tree representation. That worked fine until we
switched to from StringIO to its io.StringIO unicode equivalent.

Dumping binary stuff in text form is a very common and reliable
way to backup and process data. Starting from SQL dumps to
SVN dumps - all these formats are convenient to store, transmit
and process.

> You start
> with the binary version of the filename  -- a series of bytes which
> turns out to be good Cyrillic text, but could be anything.

Right, good Cyrillic text in utf-8, and Python 2.x uses 'ascii', so if
Python 2.x used 'utf-8' as its default encoding, there won't be an
issue. For now. But I realize that it is not enough, so I want 100%
protection from unwanted crashes and data loss, so I want to
backslash non-utf-8 bytes when converting the data to unicode.

> You store
> that as the first so many bytes of your file. If ever you need to have
> the original representation of your filename, you already have it,
> right there, by reading the first so many bytes of your file.  Why
> care about what the user sees as a filename?

Not sure that I understand. I don't store anything in file. Build graph
is a representation of filesystem structure with entries that may or
may not exist. Node in build graph can also be a string that is never
written to disk. When I dump graph, I have no idea how I will
process it, but when I will need to identify some Node, grep it, find
a reference to it, I want its representation (which may as well serve
as ID) to be preserved to avoid conflicts and wrong interpretation
due to data loss

Hopefully now that my user story is clear, can you tell me how can I
do this bulletproof unicode conversion in Python 2? =)
-- 
anatoly t.


From lac at openend.se  Fri May 29 05:56:17 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 29 May 2015 11:56:17 +0200
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: Message from Chris Angelico <rosuav@gmail.com> of "Fri,
 29 May 2015 19:38:21 +1000."
 <CAPTjJmqF_wm7mCVg3i4KWCemyMXWLwVEZXor1m+8wrw8PCjmvg@mail.gmail.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <CAPTjJmrYQtT3TbCHL7ss8nq4JTszuDDc07uPdHOUkiAczVu_+w@mail.gmail.com>
 <mk9b65$rp6$1@ger.gmane.org><CAPTjJmqF_wm7mCVg3i4KWCemyMXWLwVEZXor1m+8wrw8PCjmvg@mail.gmail.com>
Message-ID: <201505290956.t4T9uHx3016422@fido.openend.se>

In a message of Fri, 29 May 2015 19:38:21 +1000, Chris Angelico writes:
>The point was to sandbox something inside Python. Otherwise, yes, just
>write it in Python. But if you do have to sandbox like this, you lose
>language-level Unicode support, language-level arbitrary precision
>integers, etcetera, etcetera, etcetera. So I stand by my previous
>statement: The price of security is functionality.
>
>ChrisA

You can run a pypy sandbox from inside your CPython app, if that
is what you want to do.  http://pypy.readthedocs.org/en/latest/sandbox.html

Just FYI.

Laura


From lac at openend.se  Fri May 29 05:59:51 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 29 May 2015 11:59:51 +0200
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
In-Reply-To: Message from Ned Deily <nad@acm.org> of "Thu,
 28 May 2015 23:07:42 -0700." <nad-9B828D.23074228052015@news.gmane.org>
References: <201505290347.t4T3lYjC006610@fido.openend.se><nad-9B828D.23074228052015@news.gmane.org>
Message-ID: <201505290959.t4T9xpDK016524@fido.openend.se>

In a message of Thu, 28 May 2015 23:07:42 -0700, Ned Deily writes:
>It would be helpful to know what utility it is that is encountering the 
>fatal error; that message is not familiar and I suspect it is coming 
>from trying to run some third-party application.  It may be that the 
>application was depending on a third-party framework installation of 
>Python (in /Library/Frameworks rather than the system Python in 
>/System/Library/Frameworks), such as provided by the python.org OS X 
>installers, but without knowing more information, like what particular 
>version of Python is needed, it would only be speculation.  Perhaps the 
>best thing to do is to suggest the OP to participate directly here or an 
>Apple users' forum like http://apple.stackexchange.com.
>
>-- 
> Ned Deily,
> nad at acm.org

I asked her to come here, but I fear she is feeling a tad too
embarassed to do that right now.  I don't know how to find out
the name of the Utility -- the Error message really does say
'Utility' -- with no name for it.  Apparantly there is some log
file you can go take a look at to find out, but I don't know
what it is ...

Laura



From ian.g.kelly at gmail.com  Fri May 29 06:02:41 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 29 May 2015 04:02:41 -0600
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
 (fwd)
In-Reply-To: <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
References: <201505271257.t4RCv1R2015793@fido.openend.se>
 <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
Message-ID: <CALwzidkR2Efo-S_pBWa9qH41W+X=wbocnR=f2g7BwiPdy1gCLA@mail.gmail.com>

On Fri, May 29, 2015 at 2:05 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> Added Mailman to my suxx tracker:
> https://github.com/techtonik/suxx-tracker#mailman

What a useless tool. Instead of tiredly complaining that things suck,
why not take some initiative to make them better?

I'm curious about your complaint about virtualenv. How do you envision
that "logging in" to the env would be any different from activating
it?


From qh.resu01 at gmail.com  Fri May 29 06:17:52 2015
From: qh.resu01 at gmail.com (Kevin Peterson)
Date: Fri, 29 May 2015 10:17:52 +0000
Subject: stdout of child process as an input of another thread in Python?
Message-ID: <abcd1234abc123ab12a0000088758000010000001052@gmail.com>

Hi, 

I want to use the stdout of child process as an input of another thread, but some how I am not able to read the stdout.  Using Popen I have created a child process and stdout of it I have redirected to PIPE (because I don't want that to be printed on with my main process).  Now using this statement,"Line=Proc.stdout.readline()" I want to read stdout of child process and I have to do operation on that. but somehow i am not able to read from stdout of child process. 

Following is the code snapshot - 

    Proc = Popen(["python.exe","filename.py"],stdout=PIPE)
    
    while True:
                Line = Proc.stdout.readline()
                print Line

Here,
Filename.py(Child process) is continuously printing "Hello World!!" in place of reading stdout of child process. 



Thanks,
Kevin Peterson




From breamoreboy at yahoo.co.uk  Fri May 29 06:28:58 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 29 May 2015 11:28:58 +0100
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
 (fwd)
In-Reply-To: <CALwzidkR2Efo-S_pBWa9qH41W+X=wbocnR=f2g7BwiPdy1gCLA@mail.gmail.com>
References: <201505271257.t4RCv1R2015793@fido.openend.se>
 <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
 <CALwzidkR2Efo-S_pBWa9qH41W+X=wbocnR=f2g7BwiPdy1gCLA@mail.gmail.com>
Message-ID: <mk9f1c$s8h$1@ger.gmane.org>

On 29/05/2015 11:02, Ian Kelly wrote:
> On Fri, May 29, 2015 at 2:05 AM, anatoly techtonik <techtonik at gmail.com> wrote:
>> Added Mailman to my suxx tracker:
>> https://github.com/techtonik/suxx-tracker#mailman
>
> What a useless tool. Instead of tiredly complaining that things suck,
> why not take some initiative to make them better?
>

The guy who refuses to sign the CLA taking some initiative to make 
things better?  Is that an entire air force of flying pigs I observe 
going past my window, or merely one US atomic powered carrier's worth?

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From jon+usenet at unequivocal.co.uk  Fri May 29 06:44:14 2015
From: jon+usenet at unequivocal.co.uk (Jon Ribbens)
Date: Fri, 29 May 2015 10:44:14 +0000 (UTC)
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
 (fwd)
References: <201505271257.t4RCv1R2015793@fido.openend.se>
 <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
 <mailman.169.1432893804.5151.python-list@python.org>
Message-ID: <slrnmmggq6.1r6.jon+usenet@frosty.unequivocal.co.uk>

On 2015-05-29, Ian Kelly <ian.g.kelly at gmail.com> wrote:
> On Fri, May 29, 2015 at 2:05 AM, anatoly techtonik <techtonik at gmail.com> wrote:
>> Added Mailman to my suxx tracker:
>> https://github.com/techtonik/suxx-tracker#mailman
>
> What a useless tool. Instead of tiredly complaining that things suck,
> why not take some initiative to make them better?
>
> I'm curious about your complaint about virtualenv. How do you envision
> that "logging in" to the env would be any different from activating
> it?

Please Do Not Feed The Troll.


From python.list at tim.thechases.com  Fri May 29 06:53:15 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Fri, 29 May 2015 05:53:15 -0500
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <CAPTjJmpRycvcU_v4W3PQcn3gB2qeReQnTKg3u6beXT-BQrBDwg@mail.gmail.com>
References: <1432869619.871453.281061217.70B58077@webmail.messagingengine.com>
 <CAPTjJmpRycvcU_v4W3PQcn3gB2qeReQnTKg3u6beXT-BQrBDwg@mail.gmail.com>
Message-ID: <20150529055315.5334cfeb@bigbox.christie.dr>

On 2015-05-29 13:48, Chris Angelico wrote:
> That said, though, using 0 for False and 1 for True is easily
> the most common convention in use today, and the next most likely
> case is that comparing booleans would give a simple and immediate
> error. So it's most likely to be safe to do.

There are popular exceptions though. coughshellscriptingcough

-tkc






From lac at openend.se  Fri May 29 07:39:00 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 29 May 2015 13:39:00 +0200
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
 (fwd)
In-Reply-To: Message from anatoly techtonik <techtonik@gmail.com> of "Fri,
 29 May 2015 12:46:01 +0300."
 <CAPkN8xLz1ykBXSxEp+jrURgEGkubcnq2CCbrxkpmQvUhcPOvOQ@mail.gmail.com>
References: <201505271257.t4RCv1R2015793@fido.openend.se> <techtonik@gmail.com>
 <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
 <201505290841.t4T8f9Tr014513@fido.openend.se><CAPkN8xLz1ykBXSxEp+jrURgEGkubcnq2CCbrxkpmQvUhcPOvOQ@mail.gmail.com>
Message-ID: <201505291139.t4TBd0XE018889@fido.openend.se>

Do you know about the codecs module?

reading http://pymotw.com/2/codecs/ may be useful if this is new to you.

Have you read https://www.python.org/dev/peps/pep-0293/ ?

Will backslashreplace do what you want?

Laura



From alister.nospam.ware at ntlworld.com  Fri May 29 07:41:38 2015
From: alister.nospam.ware at ntlworld.com (alister)
Date: Fri, 29 May 2015 11:41:38 +0000 (UTC)
Subject: Logic problem: need better logic for desired thruth table.
References: <1432869619.871453.281061217.70B58077@webmail.messagingengine.com>
 <mailman.154.1432871339.5151.python-list@python.org>
Message-ID: <mk9j9i$ejd$1@speranza.aioe.org>

On Fri, 29 May 2015 13:48:55 +1000, Chris Angelico wrote:

> On Fri, May 29, 2015 at 1:20 PM,  <random832 at fastmail.us> wrote:
>> The possibility of spelling these with the comparison operators, as
>> some have suggested, is a consequence of Python's implementation where
>> True == 1 and False == 0. In other languages bool may not be relatable
>> (or at least not orderable), or False may be == -1.
> 
> True. That said, though, using 0 for False and 1 for True is easily the
> most common convention in use today, and the next most likely case is
> that comparing booleans would give a simple and immediate error. So it's
> most likely to be safe to do. Cross-language compatibility is a tricky
> thing anyway; there are all sorts of odd edge cases, even with
> otherwise-similar languages (Pike and Python, for instance, have
> slightly different handling of slice ranges), so anything that's done in
> Python is meant to be interpreted with Python semantics.
> 
> ChrisA

in the past True has even been -1 with 0 as false
(all bits set in a signed integer)




-- 
I use technology in order to hate it more properly.
		-- Nam June Paik


From random832 at fastmail.us  Fri May 29 08:09:50 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 29 May 2015 08:09:50 -0400
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <CAPTjJmpRycvcU_v4W3PQcn3gB2qeReQnTKg3u6beXT-BQrBDwg@mail.gmail.com>
References: <1432869619.871453.281061217.70B58077@webmail.messagingengine.com>
 <CAPTjJmpRycvcU_v4W3PQcn3gB2qeReQnTKg3u6beXT-BQrBDwg@mail.gmail.com>
Message-ID: <1432901390.2419933.281359833.75A16C8D@webmail.messagingengine.com>

On Thu, May 28, 2015, at 23:48, Chris Angelico wrote:
> On Fri, May 29, 2015 at 1:20 PM,  <random832 at fastmail.us> wrote:
> > The possibility of spelling these with the comparison operators, as some
> > have suggested, is a consequence of Python's implementation where True
> > == 1 and False == 0. In other languages bool may not be relatable (or at
> > least not orderable), or False may be == -1.
> 
> True.

Gah. I said "False may be -1", but what I *meant* was that *True* may be
-1: something that has some merit for bitwise logic (-1 is all-bits-1).
This is, for example, used in pre-.NET incarnations of Visual Basic (and
COM's VARIANT_BOOL type inherits from this). VB, incidentally, has its
own spellings for these operators, with the "Imp" and "Eqv" operators.


From random832 at fastmail.us  Fri May 29 08:12:45 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 29 May 2015 08:12:45 -0400
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <a30da$5567e1b6$5419aafe$6950@news.ziggo.nl>
References: <mailman.152.1432869623.5151.python-list@python.org>
 <a30da$5567e1b6$5419aafe$6950@news.ziggo.nl>
Message-ID: <1432901565.2420794.281362313.06E77498@webmail.messagingengine.com>

On Thu, May 28, 2015, at 23:49, Skybuck Flying wrote:
> Ok thanks for this information.
> 
> I was just wondering how many thruth table combinations there can be for
> a 
> typical thruth table with 2 inputs and 1 output.
> 
> Since there are 2 inputs, this means 4 possible outputs, which means 2 to 
> the power of 4 different thruth tables possible, which is indeed 16.
> 
> Perhaps you have a link to all possible 16 thruth tables and their formal 
> names ? That would help !

http://en.wikipedia.org/wiki/Truth_table#Binary_operations

Scroll down a bit for the formal names.


From flowingtide2 at tiscali.co.uk  Fri May 29 08:42:27 2015
From: flowingtide2 at tiscali.co.uk (David Skardon)
Date: Fri, 29 May 2015 13:42:27 +0100
Subject: python
Message-ID: <414E2DF84E3544769D2597ED4B2002BE@oldsalt>

Absolute rubbish tried all ways to run python keeps saying processor isn't compatible, I have a 6 core 6200 amd processor, so how does one install this program and get its minute brain to operate
regards
Dave


---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150529/10c3dcc1/attachment.html>

From lac at openend.se  Fri May 29 09:11:03 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 29 May 2015 15:11:03 +0200
Subject: python
In-Reply-To: Message from "David Skardon" <flowingtide2@tiscali.co.uk> of "Fri,
 29 May 2015 13:42:27 +0100." <414E2DF84E3544769D2597ED4B2002BE@oldsalt>
References: <414E2DF84E3544769D2597ED4B2002BE@oldsalt>
Message-ID: <201505291311.t4TDB3rM021314@fido.openend.se>

In a message of Fri, 29 May 2015 13:42:27 +0100, "David Skardon" writes:
>Absolute rubbish tried all ways to run python keeps saying processor isn't compatible, I have a 6 core 6200 amd processor, so how does one install this program and get its minute brain to operate
>regards
>Dave

We need more information, including
What operating system? Python 2 or Python 3?
Are you trying to build your own Python from source, or install
a binary?  Or are you embedding?

Laura


From cfkaran2 at gmail.com  Fri May 29 09:12:52 2015
From: cfkaran2 at gmail.com (Cem Karan)
Date: Fri, 29 May 2015 09:12:52 -0400
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
In-Reply-To: <201505290347.t4T3lYjC006610@fido.openend.se>
References: <201505290347.t4T3lYjC006610@fido.openend.se>
Message-ID: <1DD77C2F-FA02-4604-A1AF-7FDE7FFC47E9@gmail.com>


On May 28, 2015, at 11:47 PM, Laura Creighton <lac at openend.se> wrote:

> webmaster at python.org just got some mail from some poor embarrased
> soul who ran this program and broke their Python install.
> 
> They are running Mac OSX 10.7.5
> 
> They are getting:
> 
>     Utility has encountered a fatal error, and will now terminate.  A
>     Python runtime could not be located. You may need to install a
>     framework build of Python or edit the PyRuntimeLocations array in this
>     applications info.plist file.  Then there are two oblong circles. One
>     says Open Console. The other says Terminate.
> 
> So https://docs.python.org/2/using/mac.html says:
> 
>   The Apple-provided build of Python is installed in
>   /System/Library/Frameworks/Python.framework and /usr/bin/python,
>   respectively. You should never modify or delete these, as they are
>   Apple-controlled and are used by Apple- or third-party software.
> 
> So, I assume this poor soul has done precisely that.
> 
> What do I tell her to do now?

Does she have a recent Time Machine backup that she can restore from?  Otherwise the solutions are all fairly painful:

1) Install Python 2.7 from scratch (easy).  Then figure out where to put symlinks that point back to the install (mildly annoying/hard).  Note that Python 3 won't work; none of the built-in scripts expect it.

2) OS X recovery - http://www.macworld.co.uk/how-to/mac/how-reinstall-mac-os-x-using-internet-recovery-3593641/ I've never had to do that, so I have no idea how easy/reliable it is.  I **think** its supposed to save all the data on the drive, but again, I've not done this, so I can't make any guarantees.

3) Wipe it clean and reinstall from scratch.

Honestly, I hope she has a time machine backup.  I've had to do recoveries a couple of times, and it can really save you.

Good luck,
Cem Karan

From wrw at mac.com  Fri May 29 09:37:15 2015
From: wrw at mac.com (William Ray Wing)
Date: Fri, 29 May 2015 09:37:15 -0400
Subject: Accessing DataSocket Server with Python
In-Reply-To: <CAGGBd_pEf0E6skDgzVuWLsPEX73OvpTkQ=CwgjKF6Yz+fC6bsw@mail.gmail.com>
References: <CAO2fJ-+UjZNqo2yQRRkuSsEJwFAGOnbB9JPCCWX7zX7jt_5POg@mail.gmail.com>
 <CAGGBd_pEf0E6skDgzVuWLsPEX73OvpTkQ=CwgjKF6Yz+fC6bsw@mail.gmail.com>
Message-ID: <BA77FC3C-223E-4503-AA51-FED9842A5324@mac.com>


> On May 28, 2015, at 6:17 PM, Dan Stromberg <drsalists at gmail.com> wrote:
> 
> I have no idea about the protocol used by NI DataSockets, but you
> might be able to reverse engineer the protocol by using the official
> client with a sniffer.
> 
> Also, be aware that TCP/IP guarantees that you get the correct data in
> the correct order, but it doesn't guarantee anything about the sizes
> of the chunks in which that data arrives.  So you could send 100
> bytes, 100 bytes, 100 bytes, but on the other end receive 100 bytes,
> 50 bytes, 75 bytes, 75 bytes.  When you catenate them all together,
> it's still the same data though.
> 
> HTH.
> 
> On Wed, May 27, 2015 at 4:30 AM, Garrone, Corrado

While that?s certainly possible in a routed network (and even then can be overridden with the ?do not fragment? bit), it won?t happen in a LAN or self-contained instrument set-up.  These days, even routed networks tend to deliver anything less than a 1500 byte packet as a single entity.  With fiber backbones and high-speed LANs, it is more work for a router to fragment a packet then to simply pass it on.  The days of 480 byte packets pretty much went away with dial-up modems.

Bill



> <corrado.garrone at roche.com> wrote:
>> Dear Python Team,
>> 
>> currently I am working on a research project for my bachelor degree. A
>> LabVIEW application is used for current and power measurements, whereas the
>> measured data are sent to DataSocket Server, a technology by National
>> Instruments used for data exchange between computers and applications.
>> DataSocket is based on TCP/IP and thus requesting data from DataSocket
>> should be similar to an internet request.
>> I know with the socket library in Python it is possible with to establish
>> sockets, send internet requests and communicate between clients and servers.
>> Is there a possibility to access NI DataSocket and get measurement data with
>> Python on the same computer where Python is installed and the codes are
>> executed? Can you maybe send me an example code where such a connection with
>> DataSocket is established?
>> 
>> If you got any queries, please do not hesitate to contact me.
>> 
>> Thank you very much for your efforts.
>> 
>> Kind regards,
>> 
>> Corrado Garrone
>> DH-Student Fachrichtung Elektrotechnik / Co-op Student B.Eng. Electrical
>> Engineering
>> 
>> Roche Diagnostics GmbH
>> DFGHMV8Y6164
>> Sandhofer Strasse 116
>> 68305 Mannheim / Germany
>> 
>> Phone: apprentice
>> mailto:corrado.garrone at roche.com
>> 
>> Roche Diagnostics GmbH
>> Sandhofer Stra?e 116; D?68305 Mannheim; Telefon +49?621?759?0; Telefax
>> +49?621?759?2890
>> Sitz der Gesellschaft: Mannheim - Registergericht: AG Mannheim HRB 3962 -
>> Gesch?ftsf?hrung: Dr. Ursula Redeker, Sprecherin; Edgar Vieth -
>> Aufsichtsratsvorsitzender: Dr. Severin Schwan
>> 
>> Confidentiality Note
>> This message is intended only for the use of the named recipient(s) and may
>> contain confidential and/or privileged information. If you are not the
>> intended recipient, please contact the sender and delete the message. Any
>> unauthorized use of the information contained in this message is prohibited.
>> 
>> 
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list



From steve at pearwood.info  Fri May 29 09:40:46 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 29 May 2015 23:40:46 +1000
Subject: python
References: <mailman.176.1432903425.5151.python-list@python.org>
Message-ID: <55686c5f$0$12977$c3e8da3$5496439d@news.astraweb.com>

On Fri, 29 May 2015 10:42 pm, David Skardon wrote:

> Absolute rubbish tried all ways to run python keeps saying processor isn't
> compatible, I have a 6 core 6200 amd processor, so how does one install
> this program and get its minute brain to operate regards Dave

Clearly a genius like yourself has discovered our plot. Python is a
practical joke, it doesn't actually run for anyone. We all hand around here
pretending to have conversations about an imaginary programming language,
just to play a prank on people like you.

Nah, just kidding.

It sounds like you are trying to run a binary compiled for one CPU on a
different CPU, just like the error message says. I see in my crystal ball
that you downloaded the Mac version of Python and are trying to run it on
your Windows system, or perhaps it is the other way around, my crystal ball
sometimes gets things reversed like that.

Perhaps it would help if you would stop with the stupid insults about minute
brains and absolute rubbish, and instead describe your problem in more
detail. Starting with what platform you are using (e.g. 32 bit Windows XP,
64-bit Linux, something else?), what you did to install Python, where you
got it from, and the *exact* error message.

Then perhaps I can stop relying on my crystal ball and give you some more
useful advice.




-- 
Steven



From subhabrata.banerji at gmail.com  Fri May 29 09:41:49 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Fri, 29 May 2015 06:41:49 -0700 (PDT)
Subject: Inverted Comma Search
Message-ID: <c5d0d595-7699-427f-b680-c67589cec2fe@googlegroups.com>

Dear Group,

I am trying to implement two searches, inverted comma search and search within brackets. I am trying to implement them in Whoosh, but not finding good tutorial or examples. If any one may kindly help me with.

Regards,
Subhabrata Banerjee. 


From wrw at mac.com  Fri May 29 09:55:31 2015
From: wrw at mac.com (William Ray Wing)
Date: Fri, 29 May 2015 09:55:31 -0400
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
In-Reply-To: <1DD77C2F-FA02-4604-A1AF-7FDE7FFC47E9@gmail.com>
References: <201505290347.t4T3lYjC006610@fido.openend.se>
 <1DD77C2F-FA02-4604-A1AF-7FDE7FFC47E9@gmail.com>
Message-ID: <7A936BDF-5D1D-43E7-9ED5-FF50B55A7D4D@mac.com>


> On May 29, 2015, at 9:12 AM, Cem Karan <cfkaran2 at gmail.com> wrote:
> 
> 
> On May 28, 2015, at 11:47 PM, Laura Creighton <lac at openend.se> wrote:
> 
>> webmaster at python.org just got some mail from some poor embarrased
>> soul who ran this program and broke their Python install.
>> 
>> They are running Mac OSX 10.7.5
>> 
>> They are getting:
>> 
>>    Utility has encountered a fatal error, and will now terminate.  A
>>    Python runtime could not be located. You may need to install a
>>    framework build of Python or edit the PyRuntimeLocations array in this
>>    applications info.plist file.  Then there are two oblong circles. One
>>    says Open Console. The other says Terminate.
>> 
>> So https://docs.python.org/2/using/mac.html says:
>> 
>>  The Apple-provided build of Python is installed in
>>  /System/Library/Frameworks/Python.framework and /usr/bin/python,
>>  respectively. You should never modify or delete these, as they are
>>  Apple-controlled and are used by Apple- or third-party software.
>> 
>> So, I assume this poor soul has done precisely that.
>> 
>> What do I tell her to do now?

As a minor addendum - you can point out that ?Clean My Mac? is a well-known piece of crap-ware that is so badly written as to be labeled by most people as malware.  Best possible solution would be a clean boot and restore from a recent clone (either CarbonCopy Cloner or SuperDuper) followed by a restore of intervening files from Time Machine.  Failing that, Cem?s suggestions are good.

-Bill

> 
> Does she have a recent Time Machine backup that she can restore from?  Otherwise the solutions are all fairly painful:
> 
> 1) Install Python 2.7 from scratch (easy).  Then figure out where to put symlinks that point back to the install (mildly annoying/hard).  Note that Python 3 won't work; none of the built-in scripts expect it.
> 
> 2) OS X recovery - http://www.macworld.co.uk/how-to/mac/how-reinstall-mac-os-x-using-internet-recovery-3593641/ I've never had to do that, so I have no idea how easy/reliable it is.  I **think** its supposed to save all the data on the drive, but again, I've not done this, so I can't make any guarantees.
> 
> 3) Wipe it clean and reinstall from scratch.
> 
> Honestly, I hope she has a time machine backup.  I've had to do recoveries a couple of times, and it can really save you.
> 
> Good luck,
> Cem Karan
> -- 
> https://mail.python.org/mailman/listinfo/python-list



From lac at openend.se  Fri May 29 09:57:16 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 29 May 2015 15:57:16 +0200
Subject: Inverted Comma Search
In-Reply-To: Message from subhabrata.banerji@gmail.com of "Fri,
 29 May 2015 06:41:49 -0700."
 <c5d0d595-7699-427f-b680-c67589cec2fe@googlegroups.com>
References: <c5d0d595-7699-427f-b680-c67589cec2fe@googlegroups.com>
Message-ID: <201505291357.t4TDvGWZ022436@fido.openend.se>

In a message of Fri, 29 May 2015 06:41:49 -0700, subhabrata.banerji at gmail.com w
rites:
>Dear Group,
>
>I am trying to implement two searches, inverted comma search and search within brackets. I am trying to implement them in Whoosh, but not finding good tutorial or examples. If any one may kindly help me with.
>
>Regards,
>Subhabrata Banerjee. 
>--

I think you may get better answers if you try the Whoosh Google group.
https://groups.google.com/forum/#!forum/whoosh

But maybe people here know, too.

(alas, I don't).
Laura




From markos at c2o.pro.br  Fri May 29 10:29:03 2015
From: markos at c2o.pro.br (Markos)
Date: Fri, 29 May 2015 11:29:03 -0300
Subject: OpenCV with Python (cv or cv2)
In-Reply-To: <201505280447.t4S4lk3J006383@fido.openend.se>
References: <5564740E.3090206@c2o.pro.br>
 <201505261421.t4QELBUd015477@fido.openend.se><55664B83.30809@c2o.pro.br>
 <201505280447.t4S4lk3J006383@fido.openend.se>
Message-ID: <556877AF.5050202@c2o.pro.br>

Hi Laura,

I will follow your advice and install Debian 8.0. I was postponing this 
upgrade. :^)

I just find that Jessie is the new stable release.

According to the tutorial:

http://milq.github.io/install-opencv-ubuntu-debian/

the simplest way to install opencv in jessie is:

apt-get install python-dev libopencv-opencv

And from what I saw the opencv package available in the repository is 2.4.9

https://packages.debian.org/jessie/python-opencv

And from what I saw on the site:

https://packages.debian.org/jessie/python3
and
https://packages.debian.org/jessie/python

There are packages in the repository for python 3.4.2-2 and 2.7.9-1

My doubt is which version of Python (3.4.2-2 or 2.7.9-1) is compatible 
with the opencv library 2.4.9 available in the repository?

Thanks for your attention,
Markos


On 28-05-2015 01:47, Laura Creighton wrote:
> Your cmake output doesn't mention that it tried to build libcv.so
> so, I guess the reason you cannot find it is that it never tried
> to make it.
>
> You may already have fixed your problem by just installing the relevant
> debian package.  If not, it may be that you need to install this one
> https://packages.debian.org/squeeze/libcv-dev to get libcv.
>
> You have another problem. Squeeze is _very_ old.
> According to http://stackoverflow.com/questions/5212728/libcxcore-so-2-missing-in-opencv
> the opencv project renamed a bunch of libraries, so if at all possible you
> should upgrade your debian distribution.  You may have all sorts of
> mismatches between the source you just built and the libraries you
> need -- which you will only find by trial and error.
>
> Laura
>
>    



From p.f.moore at gmail.com  Fri May 29 10:29:51 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 29 May 2015 07:29:51 -0700 (PDT)
Subject: Is it possible to find python34.lib from within Python?
Message-ID: <68710660-0f24-403a-8c3d-996c06a26a35@googlegroups.com>

I want to set up a script to build some C code. I need to link it with python34.lib, but I'm not sure how to locate that file without hard-coding it.

There doesn't seem to be a sysconfig path I can use - best ways I can think of for finding it in a way that works even if I'm in a virtualenv is:

os.path.join(getattr(sys, 'real_prefix', sys.prefix), 'libs', 'python%s%s.lib' % sys.version_info[:2])

Is that the best way, or is there something better I could use?

In particular, this only works on Windows, which is OK for my needs, but it'd be nice if I could make it cross-platform, just in case.

Paul


From rosuav at gmail.com  Fri May 29 10:38:58 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 30 May 2015 00:38:58 +1000
Subject: Logic problem: need better logic for desired thruth table.
In-Reply-To: <20150529055315.5334cfeb@bigbox.christie.dr>
References: <1432869619.871453.281061217.70B58077@webmail.messagingengine.com>
 <CAPTjJmpRycvcU_v4W3PQcn3gB2qeReQnTKg3u6beXT-BQrBDwg@mail.gmail.com>
 <20150529055315.5334cfeb@bigbox.christie.dr>
Message-ID: <CAPTjJmqTfU8wGk=3ULsm6bXDQT4bt_dCp60K5i3SjiZpqgghPg@mail.gmail.com>

On Fri, May 29, 2015 at 8:53 PM, Tim Chase
<python.list at tim.thechases.com> wrote:
> On 2015-05-29 13:48, Chris Angelico wrote:
>> That said, though, using 0 for False and 1 for True is easily
>> the most common convention in use today, and the next most likely
>> case is that comparing booleans would give a simple and immediate
>> error. So it's most likely to be safe to do.
>
> There are popular exceptions though. coughshellscriptingcough

Yes, so you'd have to be careful if you port your complex script to
bash [1]. I'm also not sure how you'd even implement the comparison.
But shell scripting is still dwarfed by languages in which false is 0
and true is (usually) 1. Using Redmonk rankings:

1 JavaScript
2 Java
3 PHP
4 Python
5 C#
5 C++
5 Ruby
8 CSS
9 C
10 Objective-C
11 Perl
11 Shell

I'm not sure about Ruby, and I don't think CSS has a concept of
boolean arithmetic, but ten languages outrank shell scripting (Perl is
tied) and of them eight (maybe nine) use 0/1. Using langpop.com stats,
shell scripting comse up rather higher, but it's still beaten out -
and by considerable margins - by C, Java, PHP, JavaScript, C++, and
Python, all of which use 0/1. The Tiobe index seems to split up
several shells, which penalizes them severely (Bourne shell and bash
both come up in the 51-100 range), but even if they were merged,
they're unlikely to be able to nudge out the top five of Java, C, C++,
Objective-C, and C#. So while it's clearly false to state that 0/1 is
the _only_ convention in use today, it's fairly justifiably the _most
common_.

And of course, there are plenty of historical systems that were
different, as have been mentioned. But most programmers aren't going
to come across those... I hope!

ChrisA

[1] You'd also have to be insane. A bash script should not, in my
opinion, exceed about a page or two of code. I'm not sure how large a
bash script has to be before it collapses under its own gravity and
forms a black hole out of which no comprehension or sanity can escape,
but I'm taking no chances.


From techtonik at gmail.com  Fri May 29 10:41:30 2015
From: techtonik at gmail.com (anatoly techtonik)
Date: Fri, 29 May 2015 17:41:30 +0300
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
 (fwd)
In-Reply-To: <201505291139.t4TBd0XE018889@fido.openend.se>
References: <201505271257.t4RCv1R2015793@fido.openend.se>
 <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
 <201505290841.t4T8f9Tr014513@fido.openend.se> <techtonik@gmail.com>
 <CAPkN8xLz1ykBXSxEp+jrURgEGkubcnq2CCbrxkpmQvUhcPOvOQ@mail.gmail.com>
 <201505291139.t4TBd0XE018889@fido.openend.se>
Message-ID: <CAPkN8xK9G1ZqR26H2_Z6yB8r2xnUeGNKn3JYnp_z7X1HYxEM8w@mail.gmail.com>

On Fri, May 29, 2015 at 2:39 PM, Laura Creighton <lac at openend.se> wrote:
> Do you know about the codecs module?
>
> reading http://pymotw.com/2/codecs/ may be useful if this is new to you.

Does that work for Python 2 and Python 3?

> Have you read https://www.python.org/dev/peps/pep-0293/ ?

No.

> Will backslashreplace do what you want?

I don't know. I am sorry, but what is there the code that
does this:

  binary -> escaped utf-8 string -> unicode -> binary

I know about coding module, but I am not seeing a solution
to crash-proof output from Python. Is inserting a custom codec
class into every piece of code that I want to debug is the only
solution?
-- 
anatoly t.


From rosuav at gmail.com  Fri May 29 10:46:35 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 30 May 2015 00:46:35 +1000
Subject: Accessing DataSocket Server with Python
In-Reply-To: <BA77FC3C-223E-4503-AA51-FED9842A5324@mac.com>
References: <CAO2fJ-+UjZNqo2yQRRkuSsEJwFAGOnbB9JPCCWX7zX7jt_5POg@mail.gmail.com>
 <CAGGBd_pEf0E6skDgzVuWLsPEX73OvpTkQ=CwgjKF6Yz+fC6bsw@mail.gmail.com>
 <BA77FC3C-223E-4503-AA51-FED9842A5324@mac.com>
Message-ID: <CAPTjJmo1yRoDi6OZz3-VehQOiBncU3_=wctcvKQq-zpb5VG5SA@mail.gmail.com>

On Fri, May 29, 2015 at 11:37 PM, William Ray Wing <wrw at mac.com> wrote:
>> On May 28, 2015, at 6:17 PM, Dan Stromberg <drsalists at gmail.com> wrote:
>>
>> I have no idea about the protocol used by NI DataSockets, but you
>> might be able to reverse engineer the protocol by using the official
>> client with a sniffer.
>>
>> Also, be aware that TCP/IP guarantees that you get the correct data in
>> the correct order, but it doesn't guarantee anything about the sizes
>> of the chunks in which that data arrives.  So you could send 100
>> bytes, 100 bytes, 100 bytes, but on the other end receive 100 bytes,
>> 50 bytes, 75 bytes, 75 bytes.  When you catenate them all together,
>> it's still the same data though.
>>
>> HTH.
>>
>> On Wed, May 27, 2015 at 4:30 AM, Garrone, Corrado
>
> While that?s certainly possible in a routed network (and even then can be overridden with the ?do not fragment? bit), it won?t happen in a LAN or self-contained instrument set-up.  These days, even routed networks tend to deliver anything less than a 1500 byte packet as a single entity.  With fiber backbones and high-speed LANs, it is more work for a router to fragment a packet then to simply pass it on.  The days of 480 byte packets pretty much went away with dial-up modems.
>

It's uncommon to receive 100, 50, 75, 75, to be sure, but it's
certainly possible for multiple writes to be gathered together into
single reads; there are buffers at both ends, and if there's any delay
anywhere, stuff can get combined while it's waiting. (Even stuff that
had previously been sent out as multiple packets, if the data has to
be re-sent, might get grouped.) So you still need to consider your
protocol to be a stream, and delimit messages explicitly rather than
depending on the length of a socket-read. Most of these kinds of
protocols seem to have some kind of message boundary system, though,
so this shouldn't be a problem. (I'm not familiar with DataSocket
itself, but I've used several others that are like that. Even
venerable TELNET effectively does - a "message" is a line of text, so
it's delimited by newline.)

ChrisA


From ian.g.kelly at gmail.com  Fri May 29 10:47:44 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 29 May 2015 08:47:44 -0600
Subject: Fwd: Lossless bulletproof conversion to unicode (backslashing)
 (fwd)
In-Reply-To: <slrnmmggq6.1r6.jon+usenet@frosty.unequivocal.co.uk>
References: <201505271257.t4RCv1R2015793@fido.openend.se>
 <CAPkN8xJxdumt1cNoistwNagUtvze-cFD2y8_3+Z4hyuTfTLmdA@mail.gmail.com>
 <mailman.169.1432893804.5151.python-list@python.org>
 <slrnmmggq6.1r6.jon+usenet@frosty.unequivocal.co.uk>
Message-ID: <CALwzidnTVH=L1wNHRBYHDnkuA=0vD-boteDBmw=Spzr5GZDBcQ@mail.gmail.com>

On Fri, May 29, 2015 at 4:44 AM, Jon Ribbens
<jon+usenet at unequivocal.co.uk> wrote:
> On 2015-05-29, Ian Kelly <ian.g.kelly at gmail.com> wrote:
>> On Fri, May 29, 2015 at 2:05 AM, anatoly techtonik <techtonik at gmail.com> wrote:
>>> Added Mailman to my suxx tracker:
>>> https://github.com/techtonik/suxx-tracker#mailman
>>
>> What a useless tool. Instead of tiredly complaining that things suck,
>> why not take some initiative to make them better?
>>
>> I'm curious about your complaint about virtualenv. How do you envision
>> that "logging in" to the env would be any different from activating
>> it?
>
> Please Do Not Feed The Troll.

It's not a troll if the discussion is potentially useful and not just
disruptive.


From rosuav at gmail.com  Fri May 29 10:51:08 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 30 May 2015 00:51:08 +1000
Subject: OpenCV with Python (cv or cv2)
In-Reply-To: <556877AF.5050202@c2o.pro.br>
References: <5564740E.3090206@c2o.pro.br>
 <201505261421.t4QELBUd015477@fido.openend.se>
 <55664B83.30809@c2o.pro.br>
 <201505280447.t4S4lk3J006383@fido.openend.se>
 <556877AF.5050202@c2o.pro.br>
Message-ID: <CAPTjJmr15LHikTsy-2cLk+o_MpLYCZFUaHXnGefqEV4d1hkGqQ@mail.gmail.com>

On Sat, May 30, 2015 at 12:29 AM, Markos <markos at c2o.pro.br> wrote:
> the simplest way to install opencv in jessie is:
>
> apt-get install python-dev libopencv-opencv
>
> And from what I saw the opencv package available in the repository is 2.4.9
>
> https://packages.debian.org/jessie/python-opencv
>
> And from what I saw on the site:
>
> https://packages.debian.org/jessie/python3
> and
> https://packages.debian.org/jessie/python
>
> There are packages in the repository for python 3.4.2-2 and 2.7.9-1
>
> My doubt is which version of Python (3.4.2-2 or 2.7.9-1) is compatible with
> the opencv library 2.4.9 available in the repository?

Since it's managed by the Debian repo, you can simply look at its
dependencies. In this case, python-opencv is built for Python 2, not
Python 3, so it'll pull in Python 2.7.9-1. Unfortunately there isn't a
corresponding python3-opencv package, so I don't know how hard it
would be to install for Python 3.4.2; but personally, I'd just try
using pip to install it.

$ apt-cache show python-opencv
Version: 2.4.9.1+dfsg-1+b1
Depends: python (<< 2.8), python:any (>= 2.7.5-5~), python (>= 2.7~),
libc6 (>= 2.14), libgcc1 (>= 1:4.1.1), libopencv-calib3d2.4,
libopencv-contrib2.4, libopencv-core2.4, libopencv-features2d2.4,
libopencv-flann2.4, libopencv-highgui2.4, libopencv-imgproc2.4,
libopencv-legacy2.4, libopencv-ml2.4, libopencv-objdetect2.4,
libopencv-photo2.4, libopencv-video2.4, libpython2.7 (>= 2.7),
libstdc++6 (>= 4.1.1), python-numpy-abi9, python-numpy (>= 1:1.6.1)

If you simply "apt-get install python-opencv", it'll make sure you
have the right Python installed.

ChrisA


From nimbiotics at gmail.com  Fri May 29 11:52:12 2015
From: nimbiotics at gmail.com (Mario R. Osorio)
Date: Fri, 29 May 2015 11:52:12 -0400
Subject: No subject
In-Reply-To: <201505290533.t4T5XjvQ009762@fido.openend.se>
References: <BLU403-EAS29442C643780B181B828175F1CA0@phx.gbl>
 <mailman.147.1432853419.5151.python-list@python.org> <nimbiotics@gmail.com>
 <26259e7a-2fff-4da6-aab1-1dd9fbb9f728@googlegroups.com>
 <201505290533.t4T5XjvQ009762@fido.openend.se>
Message-ID: <CAO2PNV4cCvpZXrMF_o0NEUAo9hVENToWuM8+puCuHTSACLOVpw@mail.gmail.com>

?Quien es Usted y por que pregunta?


Dtb/Gby
=======
Mario R. Osorio

?If I had asked people what they wanted, they would have said faster
horses.?
 ? Henry Ford



On Fri, May 29, 2015 at 1:33 AM, Laura Creighton <lac at openend.se> wrote:

> Sabe usted acerca de estas p?ginas?
> https://mail.python.org/mailman/listinfo/python-es
> https://wiki.python.org/moin/SpanishLanguage
>
> Laura
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150529/abc57506/attachment.html>

From kyosohma at gmail.com  Fri May 29 12:01:55 2015
From: kyosohma at gmail.com (Mike Driscoll)
Date: Fri, 29 May 2015 09:01:55 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
Message-ID: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>

Hi,

I've been asked on several occasions to write about intermediate or advanced topics in Python and I was wondering what the community considers to be "intermediate" or "advanced". I realize we're all growing in our abilities with the language, so this is going to be very subjective, but I am still curious what my fellow Python developers think about this topic.

Thanks,
Mike


From joel.goldstick at gmail.com  Fri May 29 12:08:04 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Fri, 29 May 2015 12:08:04 -0400
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
Message-ID: <CAPM-O+zsAb=1AWG3rkSdEwdX_vGqfH8xq2zg3h+vcXHPF2d54g@mail.gmail.com>

Maybe itertools or generators

On Fri, May 29, 2015 at 12:01 PM, Mike Driscoll <kyosohma at gmail.com> wrote:
> Hi,
>
> I've been asked on several occasions to write about intermediate or advanced topics in Python and I was wondering what the community considers to be "intermediate" or "advanced". I realize we're all growing in our abilities with the language, so this is going to be very subjective, but I am still curious what my fellow Python developers think about this topic.
>
> Thanks,
> Mike
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Joel Goldstick
http://joelgoldstick.com


From rosuav at gmail.com  Fri May 29 12:09:53 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 30 May 2015 02:09:53 +1000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
Message-ID: <CAPTjJmrJzs7skxTRcSH0d3UEe=6S_etY+fzEfSwTWDot5oL6=g@mail.gmail.com>

On Sat, May 30, 2015 at 2:01 AM, Mike Driscoll <kyosohma at gmail.com> wrote:
> I've been asked on several occasions to write about intermediate or advanced topics in Python and I was wondering what the community considers to be "intermediate" or "advanced". I realize we're all growing in our abilities with the language, so this is going to be very subjective, but I am still curious what my fellow Python developers think about this topic.
>

Good fun! A few ideas:

How to write decorators, particularly those that take parameters.

The differences between the various number types (int, float, complex,
Fraction, Decimal) and when you'd want each one.

(CPython-specific) The dis.dis() function and what it can tell you
about how Python operates

These are all topics that have come up with my students; they're
advanced enough to be outside the scope of the course itself (the
course _uses_ decorators, but doesn't explain how to actually build
them), but not beyond the grasp of someone who's mastered Python's
fundamentals.

ChrisA


From nad at acm.org  Fri May 29 12:11:17 2015
From: nad at acm.org (Ned Deily)
Date: Fri, 29 May 2015 09:11:17 -0700
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
References: <201505290347.t4T3lYjC006610@fido.openend.se>
 d-9B828D.23074228052015@news.gmmmmmmmmm
 <201505290959.t4T9xpDK016524@fido.openend.se>
Message-ID: <nad-6BFC76.09111729052015@news.gmane.org>

In article <201505290959.t4T9xpDK016524 at fido.openend.se>,
 Laura Creighton <lac at openend.se> wrote:
> I asked her to come here, but I fear she is feeling a tad too
> embarassed to do that right now.  I don't know how to find out
> the name of the Utility -- the Error message really does say
> 'Utility' -- with no name for it.  Apparantly there is some log
> file you can go take a look at to find out, but I don't know
> what it is ...

Well, she could just download a current Python 2.7.x for OS X from 
python.org, install it, and see if that solves the problem.  That would 
be likely the easiest thing to do and is unlikely to make matters worse.  
If that doesn't work, restoring files from a backup would be in order, 
again most likely /Library/Frameworks/Python.framework/*.  As others 
have noted, running third-party apps like CleanMyMac is probably not a 
good idea, but, even if it is as crappy as its sounds, I would think it 
unlikely that it would be fooling with the Apple-supplied system Python 
in /System/Library/Frameworks.  If it did though, then the safest 
approach is to do a full Time Machine or other backup and restore the 
base system from the recovery partition and use the Migration Assistant 
to restore user files and settings from the backup.  But that's a very 
long and somewhat risky process so should only be done as a last resort.

https://support.apple.com/en-us/HT201314

-- 
 Ned Deily,
 nad at acm.org



From nad at acm.org  Fri May 29 12:19:55 2015
From: nad at acm.org (Ned Deily)
Date: Fri, 29 May 2015 09:19:55 -0700
Subject: Is it possible to find python34.lib from within Python?
References: <68710660-0f24-403a-8c3d-996c06a26a35@googlegroups.com>
Message-ID: <nad-445E49.09195529052015@news.gmane.org>

In article <68710660-0f24-403a-8c3d-996c06a26a35 at googlegroups.com>,
 Paul  Moore <p.f.moore at gmail.com> wrote:
> I want to set up a script to build some C code. I need to link it with 
> python34.lib, but I'm not sure how to locate that file without hard-coding 
> it.
[...]

If you are embedding Python, refer to the "Extending and Embedding" 
document in the Python documentation set and the description of 
python-config:

https://docs.python.org/3/extending/embedding.html#compiling-and-linking-
under-unix-like-systems

(Of course, if you are extending Python, Distutils should provide all 
the necessary flags.)

-- 
 Ned Deily,
 nad at acm.org



From invalid at invalid.invalid  Fri May 29 12:29:26 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Fri, 29 May 2015 16:29:26 +0000 (UTC)
Subject: Accessing DataSocket Server with Python
References: <CAO2fJ-+UjZNqo2yQRRkuSsEJwFAGOnbB9JPCCWX7zX7jt_5POg@mail.gmail.com>
 <CAGGBd_pEf0E6skDgzVuWLsPEX73OvpTkQ=CwgjKF6Yz+fC6bsw@mail.gmail.com>
 <mailman.181.1432910260.5151.python-list@python.org>
Message-ID: <mka456$iek$1@reader1.panix.com>

On 2015-05-29, William Ray Wing <wrw at mac.com> wrote:

> While that?s certainly possible in a routed network (and even then
> can be overridden with the ?do not fragment? bit), it won?t happen in
> a LAN or self-contained instrument set-up.

You don't know that.

> These days, even routed networks tend to deliver anything less than a
> 1500 byte packet as a single entity.

Doesn't matter.

It can still happen in the network stack on either end.  If you
quickly call send() which small amounts of data, it's quite possible
that the network stack will combine them into a single packet.  Under
some conditions this is done intentionally to reduce the overall
network overhead.

On the receiving end, if several small packets are received between
calls to recv(), then a single call to recv() will return data from
multiple packets.

> With fiber backbones and high-speed LANs, it is more work for a
> router to fragment a packet then to simply pass it on.  The days of
> 480 byte packets pretty much went away with dial-up modems.

But modern TCP/IP stacks will still combine packets to save on network
overhead.

If you assume TCP read/write operations are atomic and "message"
boundaries are preserved, your code is wrong.  It will eventually
fail.  Period.

-- 
Grant Edwards               grant.b.edwards        Yow! Inside, I'm already
                                  at               SOBBING!
                              gmail.com            


From skip.montanaro at gmail.com  Fri May 29 12:39:50 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Fri, 29 May 2015 11:39:50 -0500
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
Message-ID: <CANc-5UytC3MsJ04ans2ueYyzY7QpXLy9zS-7aeWP15xMJNZ7kg@mail.gmail.com>

On Fri, May 29, 2015 at 11:01 AM, Mike Driscoll <kyosohma at gmail.com> wrote:
> ... I was wondering what the community considers to be "intermediate" or
"advanced".

Just about any topic on which Dave Beazley has given a keynote talk
<https://www.google.com/search?q=Dave+Beazley+keynote&oq=Dave+Beazley+keynote&aqs=chrome..69i57.5895j0j7&sourceid=chrome&es_sm=122&ie=UTF-8#q=Dave+Beazley+keynote&tbm=vid>.
:-)

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150529/77a18bb2/attachment.html>

From rosuav at gmail.com  Fri May 29 12:40:06 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 30 May 2015 02:40:06 +1000
Subject: Accessing DataSocket Server with Python
In-Reply-To: <mka456$iek$1@reader1.panix.com>
References: <CAO2fJ-+UjZNqo2yQRRkuSsEJwFAGOnbB9JPCCWX7zX7jt_5POg@mail.gmail.com>
 <CAGGBd_pEf0E6skDgzVuWLsPEX73OvpTkQ=CwgjKF6Yz+fC6bsw@mail.gmail.com>
 <mailman.181.1432910260.5151.python-list@python.org>
 <mka456$iek$1@reader1.panix.com>
Message-ID: <CAPTjJmpmjdRtx4Emwqb_i3jDPv0_pFh+NZD2A3kqfdOPs5qnUA@mail.gmail.com>

On Sat, May 30, 2015 at 2:29 AM, Grant Edwards <invalid at invalid.invalid> wrote:
> If you assume TCP read/write operations are atomic and "message"
> boundaries are preserved, your code is wrong.  It will eventually
> fail.  Period.

Indeed. That said, though, if your writes are all smaller than one
packet, and you perfectly alternate a write and a read, a write and a
read, at both ends, then you can go a very long way without ever
running into this. In that case, you could have proper parsing and
buffering as a failure case, with the normal case expecting a
termination mark at the end of the socket-read - something like this:

write(query)
data = read(as_much_as_possible)
if data[-1] != "\n":
    # Huh, stuff got split.
    while True:
        more_data = read(as_much_as_possible)
        data += moredata
        if data[-1] == "\n": break
        cope_with_possibility_of(socket_disconnection)
handle_response(data)

Alternating writes and reads ensures that two messages can't be
combined, and if a single message fits inside a packet, it'll usually
be sent that way. But you still can't guarantee that.

ChrisA


From skip.montanaro at gmail.com  Fri May 29 12:44:29 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Fri, 29 May 2015 11:44:29 -0500
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
In-Reply-To: <nad-6BFC76.09111729052015@news.gmane.org>
References: <201505290347.t4T3lYjC006610@fido.openend.se>
 <201505290959.t4T9xpDK016524@fido.openend.se>
 <nad-6BFC76.09111729052015@news.gmane.org>
Message-ID: <CANc-5Uy3Shm870E7G0tM2S64aX8MC_6jqzdZ0T3=KaRNiTZQCw@mail.gmail.com>

On Fri, May 29, 2015 at 11:11 AM, Ned Deily <nad at acm.org> wrote:

> Well, she could just download a current Python 2.7.x for OS X from
> python.org, install it, and see if that solves the problem.  That would
> be likely the easiest thing to do and is unlikely to make matters worse.
>

Might that appear to work at first, but leave some subtle issues which are
not at first apparent? I don't know how intertwined Apple's Python instance
is in the day-to-day operation of the operating system (it certainly seems
to be on some Linux distros), but it's possible that some Apple-specific
package isn't available at part of the stock Python 2.7 distribution.

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150529/dafbf6d4/attachment.html>

From rosuav at gmail.com  Fri May 29 12:50:04 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 30 May 2015 02:50:04 +1000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <CANc-5UytC3MsJ04ans2ueYyzY7QpXLy9zS-7aeWP15xMJNZ7kg@mail.gmail.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <CANc-5UytC3MsJ04ans2ueYyzY7QpXLy9zS-7aeWP15xMJNZ7kg@mail.gmail.com>
Message-ID: <CAPTjJmre7M7Ft8JRmEptK9CEGGhVTM9D9CujjgWukvWuWHKw6Q@mail.gmail.com>

On Sat, May 30, 2015 at 2:39 AM, Skip Montanaro
<skip.montanaro at gmail.com> wrote:
> On Fri, May 29, 2015 at 11:01 AM, Mike Driscoll <kyosohma at gmail.com> wrote:
>> ... I was wondering what the community considers to be "intermediate" or
>> "advanced".
>
> Just about any topic on which Dave Beazley has given a keynote talk. :-)

Hehe. First hit is this one, with its awesome slide...

https://youtu.be/l_HBRhcgeuQ?t=3m34s

I showed that to my parents and siblings, most of whom are not
particularly adept with Python, and they found it highly amusing. As a
participant in python-ideas, I found it hilarious... and absolutely
right.

ChrisA


From kyosohma at gmail.com  Fri May 29 12:55:53 2015
From: kyosohma at gmail.com (Mike Driscoll)
Date: Fri, 29 May 2015 09:55:53 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mailman.191.1432915687.5151.python-list@python.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.191.1432915687.5151.python-list@python.org>
Message-ID: <549f898c-3269-479c-9f96-6d2d97955c2e@googlegroups.com>

On Friday, May 29, 2015 at 11:08:19 AM UTC-5, Joel Goldstick wrote:
> Maybe itertools or generators
> 

Yeah, I was thinking along those lines. I was also thinking about some of the cool stuff in the collections and contextlib modules.

Mike


From kyosohma at gmail.com  Fri May 29 12:57:42 2015
From: kyosohma at gmail.com (Mike Driscoll)
Date: Fri, 29 May 2015 09:57:42 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mailman.192.1432915802.5151.python-list@python.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
Message-ID: <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>


> 
> Good fun! A few ideas:
> 
> How to write decorators, particularly those that take parameters.


Yes, this one always seems to trip people up.


> 
> The differences between the various number types (int, float, complex,
> Fraction, Decimal) and when you'd want each one.

I hadn't considered this one


> 
> (CPython-specific) The dis.dis() function and what it can tell you
> about how Python operates

I remember seeing on PyMOTW - http://pymotw.com/2/dis/  I agree that it's a module most programmers don't know about.


> 
> These are all topics that have come up with my students; they're
> advanced enough to be outside the scope of the course itself (the
> course _uses_ decorators, but doesn't explain how to actually build
> them), but not beyond the grasp of someone who's mastered Python's
> fundamentals.
> 
> ChrisA


From kyosohma at gmail.com  Fri May 29 12:58:46 2015
From: kyosohma at gmail.com (Mike Driscoll)
Date: Fri, 29 May 2015 09:58:46 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mailman.195.1432917598.5151.python-list@python.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.195.1432917598.5151.python-list@python.org>
Message-ID: <ab88b589-6503-4473-990a-520fd74fea84@googlegroups.com>

On Friday, May 29, 2015 at 11:40:11 AM UTC-5, Skip Montanaro wrote:
> On Fri, May 29, 2015 at 11:01 AM, Mike Driscoll <kyos... at gmail.com> wrote:
> > ... I was wondering what the community considers to be "intermediate" or "advanced".
> 
> Just about any topic on which Dave Beazley has given a keynote talk. :-)
> 
> 
> Skip

I've always enjoyed reading Beazley's blog...and his presentations are always neat too. Good idea. 

Thanks,
Mike


From invalid at invalid.invalid  Fri May 29 12:59:47 2015
From: invalid at invalid.invalid (Grant Edwards)
Date: Fri, 29 May 2015 16:59:47 +0000 (UTC)
Subject: Accessing DataSocket Server with Python
References: <CAO2fJ-+UjZNqo2yQRRkuSsEJwFAGOnbB9JPCCWX7zX7jt_5POg@mail.gmail.com>
 <CAGGBd_pEf0E6skDgzVuWLsPEX73OvpTkQ=CwgjKF6Yz+fC6bsw@mail.gmail.com>
 <mailman.181.1432910260.5151.python-list@python.org>
 <mka456$iek$1@reader1.panix.com>
 <mailman.196.1432917609.5151.python-list@python.org>
Message-ID: <mka5u3$6c6$1@reader1.panix.com>

On 2015-05-29, Chris Angelico <rosuav at gmail.com> wrote:
> On Sat, May 30, 2015 at 2:29 AM, Grant Edwards <invalid at invalid.invalid> wrote:
>
>> If you assume TCP read/write operations are atomic and "message"
>> boundaries are preserved, your code is wrong.  It will eventually
>> fail.  Period.
>
> Indeed. That said, though, if your writes are all smaller than one
> packet, and you perfectly alternate a write and a read, a write and a
> read, at both ends, then you can go a very long way without ever
> running into this.

That's true.  You probably won't see a failure until you do something
like run through some sort of WAN connection (satellite and PPP links
are excellent for exposing bad network code), or somebody configures a
switch, router, or IP interface in a goofy (but entire valid) way, or
somebody changes the app such that it writes more than 1500 bytes, or
it violates the strict alternation of reads/writes.

But someday, somehow, (IME) one of those always happens. And whoever
has to fix it will wish bad things upon you and your descendants.

I've lost count of the times I've had to fix somebody else's code that
broke because they assumed TCP was a datagram service rather than a
byte-stream service.

-- 
Grant Edwards               grant.b.edwards        Yow! UH-OH!!  I put on
                                  at               "GREAT HEAD-ON TRAIN
                              gmail.com            COLLISIONS of the 50's"
                                                   by mistake!!!


From toddrjen at gmail.com  Fri May 29 13:02:00 2015
From: toddrjen at gmail.com (Todd)
Date: Fri, 29 May 2015 19:02:00 +0200
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
Message-ID: <CAFpSVpJv9RD4fgE1Gu4902yf23sD8icOhP7gDuFAoVCxEnx_Bg@mail.gmail.com>

On Fri, May 29, 2015 at 6:01 PM, Mike Driscoll <kyosohma at gmail.com> wrote:

> Hi,
>
> I've been asked on several occasions to write about intermediate or
> advanced topics in Python and I was wondering what the community considers
> to be "intermediate" or "advanced". I realize we're all growing in our
> abilities with the language, so this is going to be very subjective, but I
> am still curious what my fellow Python developers think about this topic.
>
> Thanks,
> Mike
> --
> https://mail.python.org/mailman/listinfo/python-list
>

The ast module and AST hacking
The dist module and understanding the interpreter.
Code objects
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150529/291d9111/attachment.html>

From sohcahtoa82 at gmail.com  Fri May 29 13:03:01 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Fri, 29 May 2015 10:03:01 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
Message-ID: <33b7ab62-8cc2-4a27-b229-64c94bce16b6@googlegroups.com>

On Friday, May 29, 2015 at 9:02:06 AM UTC-7, Mike Driscoll wrote:
> Hi,
> 
> I've been asked on several occasions to write about intermediate or advanced topics in Python and I was wondering what the community considers to be "intermediate" or "advanced". I realize we're all growing in our abilities with the language, so this is going to be very subjective, but I am still curious what my fellow Python developers think about this topic.
> 
> Thanks,
> Mike

Metaclasses.

I've read about them.  I still don't understand them, why you would want them, and what you gain from them.


From toddrjen at gmail.com  Fri May 29 13:03:47 2015
From: toddrjen at gmail.com (Todd)
Date: Fri, 29 May 2015 19:03:47 +0200
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <CAFpSVpJv9RD4fgE1Gu4902yf23sD8icOhP7gDuFAoVCxEnx_Bg@mail.gmail.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <CAFpSVpJv9RD4fgE1Gu4902yf23sD8icOhP7gDuFAoVCxEnx_Bg@mail.gmail.com>
Message-ID: <CAFpSVpKRLwSvv4X=ZNPhEhKFqEPWzajw9RX18fH=9YjrghSgUA@mail.gmail.com>

On Fri, May 29, 2015 at 7:02 PM, Todd <toddrjen at gmail.com> wrote:

> On Fri, May 29, 2015 at 6:01 PM, Mike Driscoll <kyosohma at gmail.com> wrote:
>
>> Hi,
>>
>> I've been asked on several occasions to write about intermediate or
>> advanced topics in Python and I was wondering what the community considers
>> to be "intermediate" or "advanced". I realize we're all growing in our
>> abilities with the language, so this is going to be very subjective, but I
>> am still curious what my fellow Python developers think about this topic.
>>
>> Thanks,
>> Mike
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
> The ast module and AST hacking
> The dist module and understanding the interpreter.
> Code objects
>

Sorry, miss-typed. I meant the "dis" module, not the "dist" module.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150529/edcb3458/attachment.html>

From rosuav at gmail.com  Fri May 29 13:08:21 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 30 May 2015 03:08:21 +1000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
Message-ID: <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>

On Sat, May 30, 2015 at 2:57 AM, Mike Driscoll <kyosohma at gmail.com> wrote:
>>
>> Good fun! A few ideas:
>>
>> How to write decorators, particularly those that take parameters.
>
> Yes, this one always seems to trip people up.

It's like a Sherlock Holmes pronouncement. When you see something like
Flask's app.route(), or functools.wraps(), it's pure magic and
completely incomprehensible. But break it down into little pieces (a
function that takes a function and returns a function, then decorator
syntax, then closures and the ability to call the original, and
finally a decorator factory function, which is what a parameterized
decorator is), and it becomes elementary.

>> The differences between the various number types (int, float, complex,
>> Fraction, Decimal) and when you'd want each one.
>
> I hadn't considered this one

It's not often an actual *problem* - I've never seen anyone pick the
wrong data type and mess up their code, not in Python - but it's a
great way to explore some of the differences between real numbers and
what computers work with. Also, I like talking about Fraction and
Decimal for the simple reason that they're unobvious; you can poke
around with Python and discover int and float, and if ever you need
imaginary/complex numbers, you'll quickly come across complex, but you
might use Python for years and not realize that decimal.Decimal even
exists - nor when you'd want it.

ChrisA


From ethan at stoneleaf.us  Fri May 29 13:17:30 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 29 May 2015 10:17:30 -0700
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <33b7ab62-8cc2-4a27-b229-64c94bce16b6@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <33b7ab62-8cc2-4a27-b229-64c94bce16b6@googlegroups.com>
Message-ID: <55689F2A.3070509@stoneleaf.us>

On 05/29/2015 10:03 AM, sohcahtoa82 at gmail.com wrote:
> On Friday, May 29, 2015 at 9:02:06 AM UTC-7, Mike Driscoll wrote:

>> I've been asked on several occasions to write about intermediate or advanced topics
>>  in Python and I was wondering what the community considers to be "intermediate" or
>>  "advanced". I realize we're all growing in our abilities with the language, so this
>>  is going to be very subjective, but I am still curious what my fellow Python
>>  developers think about this topic.
>
> Metaclasses.
>
> I've read about them.  I still don't understand them, why you would want them, and what you gain from them.

Metaclasses change the way a class behaves.

For example, the new (in 3.4) Enum class uses a metaclass.

   class SomeEnum(Enum):
      first = 1
      second = 2
      third = 3

The metaclass changes normal class behavior to:

   - support iterating: list(SomeEnum) --> [SomeEnum.first, SomeEnum.second, SomeEnum.third]
   - support a length:  len(SomeEnum) --> 3
   - not allow new instances to be created:  --> SomeEnum(1) is SomeEnum(1)  # True

--
~Ethan~


From marko at pacujo.net  Fri May 29 13:21:17 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 29 May 2015 20:21:17 +0300
Subject: Accessing DataSocket Server with Python
References: <CAO2fJ-+UjZNqo2yQRRkuSsEJwFAGOnbB9JPCCWX7zX7jt_5POg@mail.gmail.com>
 <CAGGBd_pEf0E6skDgzVuWLsPEX73OvpTkQ=CwgjKF6Yz+fC6bsw@mail.gmail.com>
 <mailman.181.1432910260.5151.python-list@python.org>
 <mka456$iek$1@reader1.panix.com>
 <mailman.196.1432917609.5151.python-list@python.org>
Message-ID: <877frrcnaq.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> Indeed. That said, though, if your writes are all smaller than one
> packet, and you perfectly alternate a write and a read, a write and a
> read, at both ends, then you can go a very long way without ever
> running into this.

Rare errors are worse than consistent errors.

TCP streams are intercepted by address translators, load balancers,
proxies, protocol translators and what not. They will eagerly take
advantage of the fact that TCP is a full-duplex octet stream.

Also, programs could be plugged into other types of octet stream with
other stream chopping characteristics.

Finally, the MTU size is very unpredictable. You have all kinds of
encapsulation (6to4, VPN, VLAN, tunneling etc) that make it difficult to
be sure about what's a "small" write.

What's more, you should be on the lookout for partial writes not block
until they finish. Typically, it is not enough to rely on TCP's flow
control but also impose a separate application-level, end-to-end flow
control to avoid deadlocks on the one hand and buffer overflows on the
other hand.


Marko


From marko at pacujo.net  Fri May 29 13:24:11 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 29 May 2015 20:24:11 +0300
Subject: What is considered an "advanced" topic in Python?
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <33b7ab62-8cc2-4a27-b229-64c94bce16b6@googlegroups.com>
Message-ID: <87zj4nb8lg.fsf@elektro.pacujo.net>

sohcahtoa82 at gmail.com:

> Metaclasses.
>
> I've read about them. I still don't understand them, why you would
> want them, and what you gain from them.

I don't think you would ever want them.


Marko


From nad at acm.org  Fri May 29 13:28:19 2015
From: nad at acm.org (Ned Deily)
Date: Fri, 29 May 2015 10:28:19 -0700
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
References: <201505290347.t4T3lYjC006610@fido.openend.se>
 <201505290959.t4T9xpDK016524@fido.openend.se>
 <nad-6BFC76.09111729052015@news.gmane.org>
 <CANc-5Uy3Shm870E7G0tM2S64aX8MC_6jqzdZ0T3=KaRNiTZQCw@mail.gmail.com>
Message-ID: <nad-EBD58F.10281929052015@news.gmane.org>

In article 
<CANc-5Uy3Shm870E7G0tM2S64aX8MC_6jqzdZ0T3=KaRNiTZQCw at mail.gmail.com>,
 Skip Montanaro <skip.montanaro at gmail.com> wrote:

> On Fri, May 29, 2015 at 11:11 AM, Ned Deily <nad at acm.org> wrote:
> 
> > Well, she could just download a current Python 2.7.x for OS X from
> > python.org, install it, and see if that solves the problem.  That would
> > be likely the easiest thing to do and is unlikely to make matters worse.
> >
> 
> Might that appear to work at first, but leave some subtle issues which are
> not at first apparent? I don't know how intertwined Apple's Python instance
> is in the day-to-day operation of the operating system (it certainly seems
> to be on some Linux distros), but it's possible that some Apple-specific
> package isn't available at part of the stock Python 2.7 distribution.

Installing a python.org Python (/usr/local/bin/python*) is independent 
of and does not interfere with use of the Apple-supplied system Pythons 
(/usr/bin/python*).  Apple does ship various third-party Python packages 
("distributions") with its system Python, like numpy, but they tend to 
be old and outdated versions and there are a few Apple-only packages.  
But, should that issue arise, that can be resolved by choosing the right 
path (/usr/local/bin vs /usr/bin) or removing /usr/local/bin from $PATH.

The thing is the original message in this thread had this:
> They are getting:
>      Utility has encountered a fatal error, and will now terminate.  A
>      Python runtime could not be located. You may need to install a
>      framework build of Python or edit the PyRuntimeLocations array in this
>      applications info.plist file.  Then there are two oblong circles. One
>      says Open Console. The other says Terminate.

But ... I just did what I should have done earlier: googled for that 
message.  And I find that the message is coming from a py2app-built 
application (and it seems I answered the question 3 years ago!):

http://stackoverflow.com/questions/10184974/py2app-is-not-copying-the-pyt
hon-framework-to-the-new-app-while-using-virutalenv

The py2app glue code could be looking for Pythons in various spots 
including the system Python.  So, let's make sure the system Python is 
still working.  On the most up-to-date 10.7 Lion system (10.7.5), typing 
the following two commands should give results the same as those shown 
(->):

/usr/bin/python2.7 -c 'import sys;print(sys.version)'
-> 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) 
-> [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]

/usr/bin/python2.7 -c 'import numpy;print(numpy.__file__)'
-> 
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/pytho
n/numpy/__init__.py

If not, then it really may be necessary to restore system files which, 
as I noted before, is most safely and accurately done by following 
Apple's directions to restore the system from the recovery partition and 
a good backup of user files.

-- 
 Ned Deily,
 nad at acm.org



From MrJean1 at gmail.com  Fri May 29 13:50:37 2015
From: MrJean1 at gmail.com (MrJean1)
Date: Fri, 29 May 2015 10:50:37 -0700 (PDT)
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
In-Reply-To: <mailman.153.1432871262.5151.python-list@python.org>
References: <mailman.153.1432871262.5151.python-list@python.org>
Message-ID: <bffd4736-f9ff-492e-9069-836f6e7867d8@googlegroups.com>

FWIW, I recently upgraded an older MacBook to Mac OS X 10.7.5 and there are 3 different versions of Python in /System/Library/Frameworks/Python.framework, see:

$ ls /System/Library/Frameworks/Python.framework/Versions/
2.5	2.6	2.7	Current


It is unclear whether MacOS X 10.7.5 installed all four of these Python versions.  The older one(s) may be left over from the previous MacOS X release on the machine.  But all are Apple builds of the Python version, for example:

$ python2.7
Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ^D


In addition, there is a link in /usr/bin to the python2.X, pythonw2.X and python2.X-config file for each version, see

$ ls -la /usr/bin/python*
-rwxr-xr-x  2 root  wheel  62752 May 21 19:06 /usr/bin/python
-rwxr-xr-x  6 root  wheel    925 May 21 19:06 /usr/bin/python-config
lrwxr-xr-x  1 root  wheel     75 May 21 19:06 /usr/bin/python2.5 -> ../../System/Library/Frameworks/Python.framework/Versions/2.5/bin/python2.5
lrwxr-xr-x  1 root  wheel     82 May 21 19:06 /usr/bin/python2.5-config -> ../../System/Library/Frameworks/Python.framework/Versions/2.5/bin/python2.5-config
lrwxr-xr-x  1 root  wheel     75 May 21 19:06 /usr/bin/python2.6 -> ../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6
lrwxr-xr-x  1 root  wheel     82 May 21 19:06 /usr/bin/python2.6-config -> ../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6-config
lrwxr-xr-x  1 root  wheel     75 May 21 19:06 /usr/bin/python2.7 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
lrwxr-xr-x  1 root  wheel     82 May 21 19:06 /usr/bin/python2.7-config -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7-config
-rwxr-xr-x  2 root  wheel  62752 May 21 19:06 /usr/bin/pythonw
lrwxr-xr-x  1 root  wheel     76 May 21 19:06 /usr/bin/pythonw2.5 -> ../../System/Library/Frameworks/Python.framework/Versions/2.5/bin/pythonw2.5
lrwxr-xr-x  1 root  wheel     76 May 21 19:06 /usr/bin/pythonw2.6 -> ../../System/Library/Frameworks/Python.framework/Versions/2.6/bin/pythonw2.6
lrwxr-xr-x  1 root  wheel     76 May 21 19:06 /usr/bin/pythonw2.7 -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/pythonw2.7


Perhaps you can reinstall these Pythons from the CD which came with the Mac.  Let me know if you need  more info.

/Jean




On Thursday, May 28, 2015 at 11:47:52 PM UTC-4, Laura Creighton wrote:
> webmaster at python.org just got some mail from some poor embarrased
> soul who ran this program and broke their Python install.
> 
> They are running Mac OSX 10.7.5
> 
> They are getting:
> 
>      Utility has encountered a fatal error, and will now terminate.  A
>      Python runtime could not be located. You may need to install a
>      framework build of Python or edit the PyRuntimeLocations array in this
>      applications info.plist file.  Then there are two oblong circles. One
>      says Open Console. The other says Terminate.
> 
> So https://docs.python.org/2/using/mac.html says:
> 
>    The Apple-provided build of Python is installed in
>    /System/Library/Frameworks/Python.framework and /usr/bin/python,
>    respectively. You should never modify or delete these, as they are
>    Apple-controlled and are used by Apple- or third-party software.
> 
> So, I assume this poor soul has done precisely that.
> 
> What do I tell her to do now?
> 
> Laura


From steve at pearwood.info  Fri May 29 13:55:36 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 30 May 2015 03:55:36 +1000
Subject: What is considered an "advanced" topic in Python?
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
Message-ID: <5568a81a$0$12976$c3e8da3$5496439d@news.astraweb.com>

On Sat, 30 May 2015 02:01 am, Mike Driscoll wrote:

> Hi,
> 
> I've been asked on several occasions to write about intermediate or
> advanced topics in Python and I was wondering what the community considers
> to be "intermediate" or "advanced". I realize we're all growing in our
> abilities with the language, so this is going to be very subjective, but I
> am still curious what my fellow Python developers think about this topic.


I would consider these advanced topics:


Metaclasses.
Descriptors.
Futures.
Asynchronous processing, including multiprocessing and threads.
Writing C or Fortran extensions.
Function/code object internals.
Byte-code hacking.
Manipulating ASTs.
OS-specific features like the os.exec* functions, os.fork, daemons, etc.
Multiple inheritance, mixins, traits, etc.
Coroutines.
Dynamic programming.
Neural nets.
Distributed processing, including remote procedure calls.
Fuzz testing.


I would consider these intermediate topics:


Closures.
Using classes and functions as first-class values.
Second-order functions.
Decorators.
Functional idioms (map, filter, reduce).
Viewing byte-code using the dis module.
Unicode, code pages, codecs and encodings.
Regular expressions.
SQL and dealing with databases.
The complexities of floating point, including Decimal.
Writing your own context managers.
Generators and iterators.
Unit testing, doctests, etc.
Code generation and metaprogramming.
Writing your own classes.
State machines.
The structure of URLs (they're more than just "http://blahblah.com").
Anything to do with HTTP (client or server).
Commandline argument processing (argparse, optparse, etc.)

Some of the intermediate topics start at an intermediate level, but can go
on to include more advanced uses.



-- 
Steven



From nad at acm.org  Fri May 29 13:59:44 2015
From: nad at acm.org (Ned Deily)
Date: Fri, 29 May 2015 10:59:44 -0700
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
References: <mailman.153.1432871262.5151.python-list@python.org>
 <bffd4736-f9ff-492e-9069-836f6e7867d8@googlegroups.com>
Message-ID: <nad-48A364.10594429052015@news.gmane.org>

In article <bffd4736-f9ff-492e-9069-836f6e7867d8 at googlegroups.com>,
 MrJean1 <MrJean1 at gmail.com> wrote:
> FWIW, I recently upgraded an older MacBook to Mac OS X 10.7.5 and there are 3 
> different versions of Python in /System/Library/Frameworks/Python.framework, 
> see:
> 
> $ ls /System/Library/Frameworks/Python.framework/Versions/
> 2.5	2.6	2.7	Current
> > 
> It is unclear whether MacOS X 10.7.5 installed all four of these Python 
> versions.  The older one(s) may be left over from the previous MacOS X 
> release on the machine

For compatibility with applications linked on older OS X releases, Apple 
customarily ships more than one version of Python frameworks with OS X.  
There are actually three versions of the system Python frameworks there: 
you should see that "Current" is actually a symbolic link to "2.7".  All 
of those versions are shipped as part of OS X 10.7.

-- 
 Ned Deily,
 nad at acm.org



From no.email at nospam.invalid  Fri May 29 14:30:34 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Fri, 29 May 2015 11:30:34 -0700
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <mailman.139.1432839075.5151.python-list@python.org>
Message-ID: <87lhg7xmlx.fsf@jester.gateway.sonic.net>

Chris Angelico <rosuav at gmail.com> writes:
>> It doesn't add much to your application to embed Lua
> Lua's a much weaker language than Python is, though.  Can it handle
> arbitrary-precision integers? Unicode? Dare I even ask,
> arbitrary-precision rationals (fractions.Fraction)? Security comes at
> a price, I guess.

The language features are an orthogonal issue to embeddability.  Lua is
easier to embed securely because its embedding interface was designed
for that.  It's also easy to call C functions from Lua, so if you want
arbitrary precision integers or rationals, you can link GMP into your
application use it from your Lua scripts.  

As another example, Javascript is as powerful as Python (though worse in
many ways due to misdesign), but also by now supports reasonably secure
embedding (or else it wouldn't be usable in browsers).

See
http://lucumr.pocoo.org/2014/8/16/the-python-i-would-like-to-see/#the-damn-interpreter
for Armin Ronacher's comments on CPython's not-so-great embedding
interface.  The interface he contrasts it with is essentially how Lua
and Javascript (at least in some implementations) work.  I haven't
looked at MicroPython or PyPy.


From MrJean1 at gmail.com  Fri May 29 14:32:46 2015
From: MrJean1 at gmail.com (MrJean1)
Date: Fri, 29 May 2015 11:32:46 -0700 (PDT)
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
In-Reply-To: <mailman.204.1432922422.5151.python-list@python.org>
References: <mailman.153.1432871262.5151.python-list@python.org>
 <bffd4736-f9ff-492e-9069-836f6e7867d8@googlegroups.com>
 <mailman.204.1432922422.5151.python-list@python.org>
Message-ID: <8314eb53-96c2-41d8-b4ca-bd7c57f96035@googlegroups.com>

Correct, the "Current" version is just a link to "2.7".  Also, the binaries '/usr/bin/python{,w}" seem to be copies of the "/System/Library/.../Versions/2.7/bin/python{,w}" files.

/Jean


On Friday, May 29, 2015 at 2:01:05 PM UTC-4, Ned Deily wrote:
> In article <bffd4736-f9ff-492e-9069-836f6e7867d8 at googlegroups.com>,
>  MrJean1 <MrJean1 at gmail.com> wrote:
> > FWIW, I recently upgraded an older MacBook to Mac OS X 10.7.5 and there are 3 
> > different versions of Python in /System/Library/Frameworks/Python.framework, 
> > see:
> > 
> > $ ls /System/Library/Frameworks/Python.framework/Versions/
> > 2.5	2.6	2.7	Current
> > > 
> > It is unclear whether MacOS X 10.7.5 installed all four of these Python 
> > versions.  The older one(s) may be left over from the previous MacOS X 
> > release on the machine
> 
> For compatibility with applications linked on older OS X releases, Apple 
> customarily ships more than one version of Python frameworks with OS X.  
> There are actually three versions of the system Python frameworks there: 
> you should see that "Current" is actually a symbolic link to "2.7".  All 
> of those versions are shipped as part of OS X 10.7.
> 
> -- 
>  Ned Deily,
>  nad at acm.org



From no.email at nospam.invalid  Fri May 29 14:33:31 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Fri, 29 May 2015 11:33:31 -0700
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
Message-ID: <87h9qvxmh0.fsf@jester.gateway.sonic.net>

Chris Angelico <rosuav at gmail.com> writes:
> Looks to me as if Lua doesn't have integers at all

They fixed that in Lua 5.3:

  http://www.lua.org/manual/5.3/readme.html#changes

> Likewise, eight-bit strings, not Unicode. 

Also fixed in 5.3 (basic utf-8 support added, per above).


From marko at pacujo.net  Fri May 29 15:12:22 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Fri, 29 May 2015 22:12:22 +0300
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <mailman.139.1432839075.5151.python-list@python.org>
 <87lhg7xmlx.fsf@jester.gateway.sonic.net>
Message-ID: <87siafb3l5.fsf@elektro.pacujo.net>

Paul Rubin <no.email at nospam.invalid>:

> The language features are an orthogonal issue to embeddability.

I doubt that. Guile is designed for embedding but it is a full-fledged
Scheme implementation.

> Lua is easier to embed securely because its embedding interface was
> designed for that.

I have very little experience with Lua. What surprises me is that it is
not as elementary as it could be: I don't really see the value of
metatables and weak tables.


Marko


From random832 at fastmail.us  Fri May 29 15:43:18 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Fri, 29 May 2015 15:43:18 -0400
Subject: Fixing Python install on the Mac after running 'CleanMyMac'
In-Reply-To: <nad-6BFC76.09111729052015@news.gmane.org>
References: <201505290347.t4T3lYjC006610@fido.openend.se>
 <201505290959.t4T9xpDK016524@fido.openend.se>
 <nad-6BFC76.09111729052015@news.gmane.org>
Message-ID: <1432928598.2539083.281734817.797613E8@webmail.messagingengine.com>

On Fri, May 29, 2015, at 12:11, Ned Deily wrote:
> As others 
> have noted, running third-party apps like CleanMyMac is probably not a 
> good idea, but, even if it is as crappy as its sounds, I would think it 
> unlikely that it would be fooling with the Apple-supplied system Python 
> in /System/Library/Frameworks. 

It sounds very likely to me. These "cleaner" apps are often designed to
remove "unnecessary" system components where the author doesn't know
what they are and doesn't think anyone needs them because his system
runs fine without it, along with the mistaken belief that these take up
a lot of space or that their existence slows down performance.

Looking at the version history of this page:
https://software.com/mac/utilities/cleanmymac/1.10.0

It looks like python is removed or otherwise messed-with by a module
called "System Junk" (earlier "Development Junk").

Version 1.9.4 _claims_ that it fixed: ""Sytem Junk" module reliability
improved and fixed issues with python applications." - but this is
basically an admission that it was messing with the sort of thing that
could cause these issues in the first place.


From lac at openend.se  Fri May 29 15:56:18 2015
From: lac at openend.se (Laura Creighton)
Date: Fri, 29 May 2015 21:56:18 +0200
Subject: Fixing Python install on the Mac after running 'CleanMyMac' (fwd)
Message-ID: <201505291956.t4TJuIWX031592@fido.openend.se>

Good news, we are getting closer to understanding what to do.
This in from Ned.  I will continue after the message:

------- Forwarded Message

Return-Path: <python-list-bounces+lac=openend.se at python.org>
From: Ned Deily <nad at acm.org>
Subject: Re: Fixing Python install on the Mac after running 'CleanMyMac'
Date: Fri, 29 May 2015 10:28:19 -0700
Lines: 63

> Might that appear to work at first, but leave some subtle issues which are
> not at first apparent? I don't know how intertwined Apple's Python instance
> is in the day-to-day operation of the operating system (it certainly seems
> to be on some Linux distros), but it's possible that some Apple-specific
> package isn't available at part of the stock Python 2.7 distribution.

Installing a python.org Python (/usr/local/bin/python*) is independent 
of and does not interfere with use of the Apple-supplied system Pythons 
(/usr/bin/python*).  Apple does ship various third-party Python packages 
("distributions") with its system Python, like numpy, but they tend to 
be old and outdated versions and there are a few Apple-only packages.  
But, should that issue arise, that can be resolved by choosing the right 
path (/usr/local/bin vs /usr/bin) or removing /usr/local/bin from $PATH.

The thing is the original message in this thread had this:
> They are getting:
>      Utility has encountered a fatal error, and will now terminate.  A
>      Python runtime could not be located. You may need to install a
>      framework build of Python or edit the PyRuntimeLocations array in this
>      applications info.plist file.  Then there are two oblong circles. One
>      says Open Console. The other says Terminate.

But ... I just did what I should have done earlier: googled for that 
message.  And I find that the message is coming from a py2app-built 
application (and it seems I answered the question 3 years ago!):

http://stackoverflow.com/questions/10184974/py2app-is-not-copying-the-pyt
hon-framework-to-the-new-app-while-using-virutalenv

The py2app glue code could be looking for Pythons in various spots 
including the system Python.  So, let's make sure the system Python is 
still working.  On the most up-to-date 10.7 Lion system (10.7.5), typing 
the following two commands should give results the same as those shown 
(->):

/usr/bin/python2.7 -c 'import sys;print(sys.version)'
- -> 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) 
- -> [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]

/usr/bin/python2.7 -c 'import numpy;print(numpy.__file__)'
- -> 
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/pytho
n/numpy/__init__.py

If not, then it really may be necessary to restore system files which, 
as I noted before, is most safely and accurately done by following 
Apple's directions to restore the system from the recovery partition and 
a good backup of user files.

- -- 
 Ned Deily,
 nad at acm.org

- -- 
https://mail.python.org/mailman/listinfo/python-list

------- End of Forwarded Message

So please type

/usr/bin/python2.7 -c 'import sys;print(sys.version)'

and tell me what you get as an answer.

Then type

/usr/bin/python2.7 -c 'import numpy;print(numpy.__file__)'

and again tell me what you get.

But if it isn't as Ned expects, then you may have to go back to
your backups.

Laura


From no.email at nospam.invalid  Fri May 29 16:15:16 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Fri, 29 May 2015 13:15:16 -0700
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <mailman.139.1432839075.5151.python-list@python.org>
 <87lhg7xmlx.fsf@jester.gateway.sonic.net>
 <87siafb3l5.fsf@elektro.pacujo.net>
Message-ID: <87d21jxhrf.fsf@jester.gateway.sonic.net>

Marko Rauhamaa <marko at pacujo.net> writes:
>> The language features are an orthogonal issue to embeddability.
> I doubt that. Guile is designed for embedding but it is a full-fledged
> Scheme implementation.

Orthogonal means independent, not opposing.

> I have very little experience with Lua. What surprises me is that it is
> not as elementary as it could be: I don't really see the value of
> metatables and weak tables.

It lets you implement various flavors of OOP etc. without different
language mechanisms.


From kyosohma at gmail.com  Fri May 29 16:38:07 2015
From: kyosohma at gmail.com (Mike Driscoll)
Date: Fri, 29 May 2015 13:38:07 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <5568a81a$0$12976$c3e8da3$5496439d@news.astraweb.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <5568a81a$0$12976$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <71fe4ca4-1db0-4fdd-9e8c-0210501f41fd@googlegroups.com>

Hi Steven,

On Friday, May 29, 2015 at 12:55:48 PM UTC-5, Steven D'Aprano wrote:
> On Sat, 30 May 2015 02:01 am, Mike Driscoll wrote:
> 
> > Hi,
> > 
> > I've been asked on several occasions to write about intermediate or
> > advanced topics in Python and I was wondering what the community considers
> > to be "intermediate" or "advanced". I realize we're all growing in our
> > abilities with the language, so this is going to be very subjective, but I
> > am still curious what my fellow Python developers think about this topic.
> 
> 
> I would consider these advanced topics:
> 
> 
> Metaclasses.
> Descriptors.
> Futures.
> Asynchronous processing, including multiprocessing and threads.
> Writing C or Fortran extensions.
> Function/code object internals.
> Byte-code hacking.
> Manipulating ASTs.
> OS-specific features like the os.exec* functions, os.fork, daemons, etc.
> Multiple inheritance, mixins, traits, etc.
> Coroutines.
> Dynamic programming.
> Neural nets.
> Distributed processing, including remote procedure calls.
> Fuzz testing.
> 
> 
> I would consider these intermediate topics:
> 
> 
> Closures.
> Using classes and functions as first-class values.
> Second-order functions.
> Decorators.
> Functional idioms (map, filter, reduce).
> Viewing byte-code using the dis module.
> Unicode, code pages, codecs and encodings.
> Regular expressions.
> SQL and dealing with databases.
> The complexities of floating point, including Decimal.
> Writing your own context managers.
> Generators and iterators.
> Unit testing, doctests, etc.
> Code generation and metaprogramming.
> Writing your own classes.
> State machines.
> The structure of URLs (they're more than just "http://blahblah.com").
> Anything to do with HTTP (client or server).
> Commandline argument processing (argparse, optparse, etc.)
> 
> Some of the intermediate topics start at an intermediate level, but can go
> on to include more advanced uses.
> 
> 
> 
> -- 
> Steven

That's a really good list of topics. Thanks so much for sharing your ideas.

Mike


From thebalancepro at gmail.com  Fri May 29 16:39:21 2015
From: thebalancepro at gmail.com (Nick Mellor)
Date: Fri, 29 May 2015 13:39:21 -0700 (PDT)
Subject: Generating list of unique search sub-phrases
Message-ID: <d7b239fc-c814-4935-8172-5c6a6c782d9c@googlegroups.com>

Hi all,

My own solution works but I'm sure it could be simpler or read better. How would you do it?

Say you've got a list of companies:

Aerosonde Ltd
Amcor
ANCA
Austal Ships
Australia Post
Australian Air Express
Australian Defence Industries
Australian Railroad Group
Australian Submarine Corporation

and you need to extract phrases from the company names that uniquely identify that company. The results for the above list of companies should be:

Company: 'Aerosonde Ltd'
 Aliases: Aerosonde,Ltd,Aerosonde Ltd

Company: 'Amcor'
 Aliases: Amcor

Company: 'ANCA'
 Aliases: ANCA

Company: 'Austal Ships'
 Aliases: Austal,Ships,Austal Ships

Company: 'Australia Post'
 Aliases: Post,Australia Post

Company: 'Australian Air Express'
 Aliases: Air,Express,Australian Air,Air Express,Australian Air Express

Company: 'Australian Defence Industries'
 Aliases: Defence,Industries,Australian Defence,Defence Industries,Australian Defence Industries

Company: 'Australian Railroad Group'
 Aliases: Railroad,Group,Australian Railroad,Railroad Group,Australian Railroad Group

Company: 'Australian Submarine Corporation'
 Aliases: Submarine,Corporation,Australian Submarine,Submarine Corporation,Australian Submarine Corporation

Here's my solution:

from itertools import combinations, chain

companies = [
    "Aerosonde Ltd",
    "Amcor",
    "ANCA",
    "Austal Ships",
    "Australia Post",
    "Australian Air Express",
    "Australian Defence Industries",
    "Australian Railroad Group",
    "Australian Submarine Corporation",
]

def flatten(i):
    return list(chain.from_iterable(i))

companies_as_text_stream = ' '.join(companies)
for company in companies:
        word_combinations = [list(combinations(company.split(), r)) for r in range(1, len(company))]
        phrases = [' '.join(phrase) for phrase in flatten(word_combinations)]
        unique_phrases = [phrase for phrase in phrases if companies_as_text_stream.count(phrase) == 1]
        aliases = ','.join(unique_phrases)
        print("Company: '{0}'\n Aliases: {1}\n".format(company, aliases))


From sohcahtoa82 at gmail.com  Fri May 29 17:06:15 2015
From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com)
Date: Fri, 29 May 2015 14:06:15 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mailman.202.1432919880.5151.python-list@python.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <33b7ab62-8cc2-4a27-b229-64c94bce16b6@googlegroups.com>
 <mailman.202.1432919880.5151.python-list@python.org>
Message-ID: <656488c4-e706-4fea-ab45-3d1454e35a7f@googlegroups.com>

On Friday, May 29, 2015 at 10:18:29 AM UTC-7, Ethan Furman wrote:
> On 05/29/2015 10:03 AM, sohcahtoa82 at gmail.com wrote:
> > On Friday, May 29, 2015 at 9:02:06 AM UTC-7, Mike Driscoll wrote:
> 
> >> I've been asked on several occasions to write about intermediate or advanced topics
> >>  in Python and I was wondering what the community considers to be "intermediate" or
> >>  "advanced". I realize we're all growing in our abilities with the language, so this
> >>  is going to be very subjective, but I am still curious what my fellow Python
> >>  developers think about this topic.
> >
> > Metaclasses.
> >
> > I've read about them.  I still don't understand them, why you would want them, and what you gain from them.
> 
> Metaclasses change the way a class behaves.
> 
> For example, the new (in 3.4) Enum class uses a metaclass.
> 
>    class SomeEnum(Enum):
>       first = 1
>       second = 2
>       third = 3
> 
> The metaclass changes normal class behavior to:
> 
>    - support iterating: list(SomeEnum) --> [SomeEnum.first, SomeEnum.second, SomeEnum.third]
>    - support a length:  len(SomeEnum) --> 3
>    - not allow new instances to be created:  --> SomeEnum(1) is SomeEnum(1)  # True
> 
> --
> ~Ethan~

Regarding the first two, you can implement __iter__ and __len__ functions to create that functionality, though those functions would operate on an instance of the class, not the class itself.

As for the third, can't you override the __new__ function to make attempts to create a new instance just return a previously created instance?


From jason.swails at gmail.com  Fri May 29 17:28:43 2015
From: jason.swails at gmail.com (Jason Swails)
Date: Fri, 29 May 2015 17:28:43 -0400
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <656488c4-e706-4fea-ab45-3d1454e35a7f@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <33b7ab62-8cc2-4a27-b229-64c94bce16b6@googlegroups.com>
 <mailman.202.1432919880.5151.python-list@python.org>
 <656488c4-e706-4fea-ab45-3d1454e35a7f@googlegroups.com>
Message-ID: <CAEk9e3ravd6oppL9MLttyYZiHaN9kqaob_wqWgJmHZwPxoJdAg@mail.gmail.com>

On Fri, May 29, 2015 at 5:06 PM, <sohcahtoa82 at gmail.com> wrote:

>
> > For example, the new (in 3.4) Enum class uses a metaclass.
> >
> >    class SomeEnum(Enum):
> >       first = 1
> >       second = 2
> >       third = 3
> >
> > The metaclass changes normal class behavior to:
> >
> >    - support iterating: list(SomeEnum) --> [SomeEnum.first,
> SomeEnum.second, SomeEnum.third]
> >    - support a length:  len(SomeEnum) --> 3
> >    - not allow new instances to be created:  --> SomeEnum(1) is
> SomeEnum(1)  # True
> >
> > --
> > ~Ethan~
>
> Regarding the first two, you can implement __iter__ and __len__ functions
> to create that functionality, though those functions would operate on an
> instance of the class, not the class itself.
>
> As for the third, can't you override the __new__ function to make attempts
> to create a new instance just return a previously created instance?
>

?Of course, but with metaclasses you don't *have* to (in fact, that's the
type of thing that the metaclass could do behind the scenes).

?In this case, any Enum subclass should *always* behave as Ethan described
-- without metaclasses that wouldn't necessarily be true (e.g., if the
person creating an Enum subclass didn't bother to correctly implement
__new__, __iter__, and __len__ for their subclass).

By using metaclasses, you can declare an Enum subclass as simply as Ethan
showed, since the metaclass does all of the dirty work implementing the
desired behavior at the time the class object is constructed (subclasses
inherit their parent class's metaclass).

In this use-case, you can think of it as a kind of "implicit class
factory".  Suppose you have a prescription for how you modify a class
definition (i.e., by implementing certain behavior in __new__ or __init__)
that you wrap up into some function "tweak_my_class".  The metaclass would
allow the class definition to be the equivalent of something like:

class MyClass(SubClass):
    # whatever

MyClass = tweak_my_class(MyClass)

Or as a class decorator

@tweak_my_class
class MyClass(SubClass):
    # whatever

(In fact, the `six` module allows you to implement metaclasses as
decorators to work with both Python 2 and Python 3, but I think metaclasses
are more powerful in Py3 than they are in Py2).?
?
They are cool ideas, and I've used them in my own code, but they do have a
kind of magic-ness to them -- especially in codes that you didn't write but
are working on.  As a result, I've recently started to prefer alternatives,
but in some rare cases (like Enum, for example), they are just the best
solution.

All the best,
Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150529/07ef4598/attachment.html>

From ethan at stoneleaf.us  Fri May 29 17:39:44 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 29 May 2015 14:39:44 -0700
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <656488c4-e706-4fea-ab45-3d1454e35a7f@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <33b7ab62-8cc2-4a27-b229-64c94bce16b6@googlegroups.com>
 <mailman.202.1432919880.5151.python-list@python.org>
 <656488c4-e706-4fea-ab45-3d1454e35a7f@googlegroups.com>
Message-ID: <5568DCA0.4000004@stoneleaf.us>

On 05/29/2015 02:06 PM, sohcahtoa82 at gmail.com wrote:
> On Friday, May 29, 2015 at 10:18:29 AM UTC-7, Ethan Furman wrote:
>>
>> Metaclasses change the way a class behaves.
>>
>> For example, the new (in 3.4) Enum class uses a metaclass.
>>
>>     class SomeEnum(Enum):
>>        first = 1
>>        second = 2
>>        third = 3
>>
>> The metaclass changes normal class behavior to:
>>
>>     - support iterating: list(SomeEnum) --> [SomeEnum.first, SomeEnum.second, SomeEnum.third]
>>     - support a length:  len(SomeEnum) --> 3
>>     - not allow new instances to be created:  --> SomeEnum(1) is SomeEnum(1)  # True
>>
>> --
>> ~Ethan~
>
> Regarding the first two, you can implement __iter__ and __len__ functions to create that functionality, though those functions would operate on an instance of the class, not the class itself.

Hence the need for a metaclass, as the point is to operate on the class, not the instance.

> As for the third, can't you override the __new__ function to make attempts to create a new instance just return a previously created instance?

Yes.  In the case of Enum, however, it takes any user-defined __new__, which is needed for creating the original instances, and replaces it with a __new__ that only returns the already defined ones.

--
~Ethan~


From ethan at stoneleaf.us  Fri May 29 17:44:07 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 29 May 2015 14:44:07 -0700
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <CAEk9e3ravd6oppL9MLttyYZiHaN9kqaob_wqWgJmHZwPxoJdAg@mail.gmail.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <33b7ab62-8cc2-4a27-b229-64c94bce16b6@googlegroups.com>
 <mailman.202.1432919880.5151.python-list@python.org>
 <656488c4-e706-4fea-ab45-3d1454e35a7f@googlegroups.com>
 <CAEk9e3ravd6oppL9MLttyYZiHaN9kqaob_wqWgJmHZwPxoJdAg@mail.gmail.com>
Message-ID: <5568DDA7.7070000@stoneleaf.us>

On 05/29/2015 02:28 PM, Jason Swails wrote:

> ?[...] e.g., if the person creating an Enum subclass didn't bother to correctly
> implement [...] __iter__, and __len__ for their subclass

For __iter__ and __len__ to work on the Enum /class/ it must be defined in the metaclass.

--
~Ethan~


From VladTheCompiler at gmail.com  Fri May 29 18:04:23 2015
From: VladTheCompiler at gmail.com (Vlad)
Date: Fri, 29 May 2015 15:04:23 -0700 (PDT)
Subject: functools.wraps does not play nice with doc tests?
Message-ID: <c441b480-a5e1-45ff-b4d2-e50bb9d5c5f0@googlegroups.com>

Hello,

So, I know this topic comes up a lot, but I haven't been able to find any discussion on this particular twist on the topic. Perhaps someone has some insight.

So, I have a function, which is decorated. In order for the doctest test finder to find the doc tests in the decorated function, my wrapper needs to use the functools.wraps decorator inside it. So far so good. *However*, the issue is that if the decorator and the decorated function are defined in different modules, and the decorator is defined at a greater line number in its module than the decorated function in its module, and the doc test fails, the test report lists the failure as occurring at an "unknown line number". This is a problem if you have additional logic built on top of the reporting that relies on line numbers of test failures (and even more fundamentally, it's a pain to manually find which test actually failed). 

Here is a specific example, this is running Python 2.7.9, in case that matters.

File decorator.py
     1  import functools
     2
     3
     4
     5
     6
     7
     8
     9  # The decorator has to be defined at a line number greater than the
    10  # decorated function in order to trigger the bug.
    11  def decorator(func):
    12      @functools.wraps(func)
    13      def wrapper(*args, **kwargs):
    14          print('In Wrapper!')
    15          return func(*args, **kwargs)
    16      return wrapper


File decorated.py
     1  from . import decorator
     2
     3
     4  @decorator.decorator
     5  def my_func():
     6      '''
     7      >>> my_func()
     8      '''
     9      print('my_func!')
    10
    11
    12  if __name__ == '__main__':
    13      import doctest
    14      doctest.testmod()


Now, let's try to run the tests:
> python -m decorated
**********************************************************************
File ".../decorated.py", line ?, in __main__.my_func
Failed example:
    my_func()
Expected nothing
Got:
    In Wrapper!
    my_func!
**********************************************************************
1 items had failures:
   1 of   1 in __main__.my_func
***Test Failed*** 1 failures.


So, the test ran, and failed as we wanted, but the line number was not detected correctly. Has anyone run into this issue, and/or know a workaround? Or perhaps I am missing something obvious here?


From rosuav at gmail.com  Fri May 29 18:49:00 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 30 May 2015 08:49:00 +1000
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <87h9qvxmh0.fsf@jester.gateway.sonic.net>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
Message-ID: <CAPTjJmqU21gqD1E7tMy56okosy5U4+Tng1bNOvw+J18tonVxFA@mail.gmail.com>

On Sat, May 30, 2015 at 4:33 AM, Paul Rubin <no.email at nospam.invalid> wrote:
> Chris Angelico <rosuav at gmail.com> writes:
>> Looks to me as if Lua doesn't have integers at all
>
> They fixed that in Lua 5.3:
>
>   http://www.lua.org/manual/5.3/readme.html#changes

That's 64-bit integers, not arbitrary-precision, but that's something
at least. You do still need to worry about what happens when your
numbers get too big; in Python, you simply don't. So it's still not
quite there in terms of functionality.

>> Likewise, eight-bit strings, not Unicode.
>
> Also fixed in 5.3 (basic utf-8 support added, per above).

This is definitely NOT what I'm talking about. From what I can see,
5.3 introduces a new library for working with strings of bytes as if
they were UTF-8. That's on par with PHP's Unicode support - basically,
nothing that isn't explicitly coded. It makes Unicode something that
you tack on with resignation, rather than something that's just an
automatic part of text handling. So, a long way from being there in
functionality.

Do you see what I mean about functionality being sacrificed for
security? There is no way that this could be called fully functional
by comparison with Python.

ChrisA


From __peter__ at web.de  Fri May 29 19:07:04 2015
From: __peter__ at web.de (Peter Otten)
Date: Sat, 30 May 2015 01:07:04 +0200
Subject: Generating list of unique search sub-phrases
References: <d7b239fc-c814-4935-8172-5c6a6c782d9c@googlegroups.com>
Message-ID: <mkareo$m37$1@ger.gmane.org>

Nick Mellor wrote:

> Hi all,
> 
> My own solution works but I'm sure it could be simpler or read better. How
> would you do it?
> 
> Say you've got a list of companies:
> 
> Aerosonde Ltd
> Amcor
> ANCA
> Austal Ships
> Australia Post
> Australian Air Express
> Australian Defence Industries
> Australian Railroad Group
> Australian Submarine Corporation
> 
> and you need to extract phrases from the company names that uniquely
> identify that company. The results for the above list of companies should
> be:
> 
> Company: 'Aerosonde Ltd'
>  Aliases: Aerosonde,Ltd,Aerosonde Ltd
> 
> Company: 'Amcor'
>  Aliases: Amcor
> 
> Company: 'ANCA'
>  Aliases: ANCA
> 
> Company: 'Austal Ships'
>  Aliases: Austal,Ships,Austal Ships
> 
> Company: 'Australia Post'
>  Aliases: Post,Australia Post
> 
> Company: 'Australian Air Express'
>  Aliases: Air,Express,Australian Air,Air Express,Australian Air Express
> 
> Company: 'Australian Defence Industries'
>  Aliases: Defence,Industries,Australian Defence,Defence
>  Industries,Australian Defence Industries
> 
> Company: 'Australian Railroad Group'
>  Aliases: Railroad,Group,Australian Railroad,Railroad Group,Australian
>  Railroad Group
> 
> Company: 'Australian Submarine Corporation'
>  Aliases: Submarine,Corporation,Australian Submarine,Submarine
>  Corporation,Australian Submarine Corporation
> 
> Here's my solution:
> 
> from itertools import combinations, chain
> 
> companies = [
>     "Aerosonde Ltd",
>     "Amcor",
>     "ANCA",
>     "Austal Ships",
>     "Australia Post",
>     "Australian Air Express",
>     "Australian Defence Industries",
>     "Australian Railroad Group",
>     "Australian Submarine Corporation",
> ]
> 
> def flatten(i):
>     return list(chain.from_iterable(i))
> 
> companies_as_text_stream = ' '.join(companies)
> for company in companies:
>         word_combinations = [list(combinations(company.split(), r)) for r
>         in range(1, len(company))] phrases = [' '.join(phrase) for phrase
>         in flatten(word_combinations)] unique_phrases = [phrase for phrase
>         in phrases if companies_as_text_stream.count(phrase) == 1] aliases
>         = ','.join(unique_phrases) print("Company: '{0}'\n Aliases:
>         {1}\n".format(company, aliases))

Here's an alternative that operates on the words rather than the text:

import itertools
from collections import defaultdict


def adjacent_combinations(words):
    words = tuple(words)
    for i in range(1, len(words)):
        for k in range(len(words) + 1 - i):
            yield words[k:k+i]
    yield words


def unordered_combinations(words):
    for i in range(1, len(words) + 1):
        yield from map(frozenset, itertools.combinations(words, i))


def show_aliases(companies, combinations):
    combo_to_companies = defaultdict(list)
    for company in companies:
        for combo in combinations(company.split()):
            combo_to_companies[combo].append(company)

    company_to_uniquecombos = defaultdict(list)
    for combo, candidates in combo_to_companies.items():
        if len(candidates) == 1:
            company_to_uniquecombos[candidates[0]].append(combo)

    for company in companies:
        print("Company:", company)
        print(
            "Aliases:",
            ", ".join(map(" ".join, company_to_uniquecombos.get(company, 
()))))
        print()


if __name__ == "__main__":
    companies = [
        "Aerosonde Ltd",
        "Amcor",
        "ANCA",
        "Austal Ships",
        "Australia Post",
        "Australian Air Express",
        "Australian Defence Industries",
        "Australian Railroad Group",
        "Australian Submarine Corporation",
    ]
    show_aliases(companies, adjacent_combinations)
    print("=" * 40)
    show_aliases(companies, unordered_combinations)

Note that the result using adjacent_combinations() is similar but not 
identical to the one you asked for as "Australia" is sufficient to identify 
"Australia Post". 

The result based on unordered_combinations() accepts 
"Australian Corporation" to identify the "Australian Submarine Corporation" 
and would fail to find an alias if the list of companies contained items 
that differ only in the order of words.

So: different result and less concise -- what more can you wish for?

How about some nitpicking?

- you can avoid materialising some of the intermediate lists
- you can avoid cross-company matches (think "Express Australian") with an 
out-of-band character

Which gives:

def all_len_combinations(words):
    return chain.from_iterable(
        combinations(words, r) for r in range(1, len(words) + 1))

companies_as_text_stream = '\0'.join(companies)
for company in companies:
    word_combinations = all_len_combinations(company.split())
    phrases = (' '.join(phrase) for phrase in word_combinations)
    unique_phrases = (phrase for phrase in phrases
                      if companies_as_text_stream.count(phrase) == 1)
    aliases = ','.join(unique_phrases)
    print("Company: '{0}'\n Aliases: {1}\n".format(company, aliases))

You may also want to try adjacent_combinations() from above which omits 
impossible candidates like "Australian Corporation".



From jamie_ka1lpa at charter.net  Fri May 29 19:07:40 2015
From: jamie_ka1lpa at charter.net (M Philbrook)
Date: Fri, 29 May 2015 19:07:40 -0400
Subject: Logic problem: need better logic for desired thruth table.
References: <3794b$55678d83$5419aafe$56138@news.ziggo.nl>
Message-ID: <MPG.2fd2bb3541a5a620989bf2@news.eternal-september.org>

In article <3794b$55678d83$5419aafe$56138 at news.ziggo.nl>, skybuck2000
@hotmail.com says...
> 
> Hello,
> 
> I was just coding and ran into a little logic problem which is as follows:
> 
> There are two booleans/variables which can be either false or true.
> 
> The desired thrutle table is:
> 
> A = input
> B = input
> C = output
> 
> A B C:
> -------
> F F T
> F T F
> T F T
> T T T
> 
> Surpisingly enough I don't think there is a casual/common operator for this 
> thruth table.
> 
> AND does not apply.
> OR does not apply.
> XOR does not apply.
> 
> So I would need some combined operators to give the desired result.
> 
> I tried logic below... but funny enough it failed, now I feel like a noob 
> lol and share this funny little fail logic with you.
> 
> Can you improve/fix the logic ?
> 
> This is python code, but this^ logic/thruth table problem basically applies 
> to any programming language:
> 
> # loop has to run if:
> # while DesiredResult==True:
> # Desired truth table for BotWaitForCooldown and CooldownDetected
> # BotWaitForCooldown:  CooldownDetected: Desired Result:
> # False       False    True
> # False       True     False
> # True     False    True
> # True     True     True
> # desired/suiting logic:
> # (BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected))
> 
> def TestLogic( BotWaitForCooldown, CooldownDetected ):
> return BotWaitForCooldown or ((not BotWaitForCooldown) and CooldownDetected) 

 First maintain a bit table, something that your code can reference as 
Bools via bit operations..
 In your case it's more less simple..
 
 A combine of 3 bits gives you the value of 7 this this can be used as 
the OR operation
  
 AND operation.
   If value = 7 then.....

 OR Operation.

  If Value <> 0 then....

 Xor Operation, 
  If Value in [1,2,4] then....

 If you need to build the "Value" Product from dangling booleans.

 Value := (Value Shl 1) OR Byte(YourBool); 
 
 Since the compiler only uses the first bit for native booleans this 
works out.
 
 Do the above for all three bools to build a final value..

 Or you can simply mask off a bit as your boolean value from a VALUE
location, which makes it faster in the longrun...

Does that do anything for you ?

Jamie

 


From johnhpote at o2.co.uk  Fri May 29 19:18:02 2015
From: johnhpote at o2.co.uk (John Pote)
Date: Sat, 30 May 2015 00:18:02 +0100
Subject: How to do integers to binary lists and back
In-Reply-To: <555E5CD8.4090704@mrabarnett.plus.com>
References: <33677AE8-B2FA-49F9-9304-C8D93784255D@gmail.com>
 <CAPTjJmpEHQV2PpaZvszuTGn0_GNg0sWV6aLenbQsZrSXCy2YKQ@mail.gmail.com>
 <mailman.18957.1424524392.18130.python-list@python.org>
 <mcargl$h3k$1@reader1.panix.com>
 <mailman.19008.1424611304.18130.python-list@python.org>
 <cl2g0vFfiq9U1@mid.individual.net>
 <mailman.19117.1424755795.18130.python-list@python.org>
 <cl486mFu4l9U1@mid.individual.net>
 <mailman.19181.1424868605.18130.python-list@python.org>
 <cl7pmjF7595U1@mid.individual.net>
 <F521BFFE-506F-4B58-9A15-20D91A019BD0@gmail.com>
 <CALwzidm3AngMF9aG-gLpYur6CfaADM0YXGU=p0=dv-qqYXmmFA@mail.gmail.com>
 <8A3659BC-9100-4A3A-9117-47227B3D290B@gmail.com>
 <CALwzidmgj1y==3iv8Zub+aHyFR3P3okeLBRKu+R92914mDXfqQ@mail.gmail.com>
 <5522439B-AEFA-400D-9B6F-00613528381F@gmail.com> <555E5A44.20006@o2.co.uk>
 <555E5CD8.4090704@mrabarnett.plus.com>
Message-ID: <5568F3AA.7060502@o2.co.uk>



On 21/05/2015 23:31, MRAB wrote:
> On 2015-05-21 23:20, John Pote wrote:
>> Hi everyone.
>> I recently had the problem of converting from an integer to its
>> representation as a list of binary bits, each bit being an integer 1 or
>> 0, and vice versa. E.G.
>> 0x53
>> becomes
>> [ 0, 1, 0, 1, 0, 0, 1, 1 ]
>>
>> This I wanted to do for integers of many tens, if not hundreds, of bits.
>> Python very nicely expands integers to any size required, great feature.
>>
>> Just wondered if there was a neat way of doing this without resorting to
>> a bit bashing loop.
>>
>> Looking forward to some interesting answers,
>> John
>>
>>
> I don't know how efficient you want it to be, but:
>
> >>> number = 0x53
> >>> bin(number)
> '0b1010011'
> >>> bin(number)[2 : ]
> '1010011'
> >>> list(map(int, bin(number)[2 : ]))
> [1, 0, 1, 0, 0, 1, 1]
>
Thanks for the replies. Interesting that the offered solutions involve 
converting to a binary text string and then the individual chars back to 
ints. I had thought this would be a route to solve this problem but it 
seemed a bit 'heavy' hence I thought it worthwhile asking the question.

My solution to converting a list of 1s and 0s back to an int is
         listLen = len( binList )
         n = sum( [ binList[i]*( 2**(listLen-1 - i) ) for i in 
range(listLen)] )

In response to Ben Finney's question, I haven't done homework for 40 
years! Genuine problem, I had decided that the clearest way to write the 
algorithm I was working on was to use lists of 1s and 0s rather than 
normal ints.

Thanks for the help,
John


From no.email at nospam.invalid  Fri May 29 21:28:33 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Fri, 29 May 2015 18:28:33 -0700
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
Message-ID: <878uc6yhtq.fsf@jester.gateway.sonic.net>

Chris Angelico <rosuav at gmail.com> writes:
> Do you see what I mean about functionality being sacrificed for
> security? 

No I don't.  Lua has less functionality because it was designed to have
a small embedding footprint.  Python is much bigger because it was
mostly designed to run as a standalone interpreter.  That has nothing to
do with security.  You haven't shown the slightest connection between
Lua's lower functionality and its higher sandbox security, because there
is none.  The lower functionality is because of a totally independent
reason, namely the desire to make the interpreter smaller.


From rosuav at gmail.com  Fri May 29 22:42:12 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 30 May 2015 12:42:12 +1000
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <878uc6yhtq.fsf@jester.gateway.sonic.net>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
Message-ID: <CAPTjJmr88DGBcN25RgDcqhiHQJMJ_1f+257JcrJZXBTT6nsSYw@mail.gmail.com>

On Sat, May 30, 2015 at 11:28 AM, Paul Rubin <no.email at nospam.invalid> wrote:
> Chris Angelico <rosuav at gmail.com> writes:
>> Do you see what I mean about functionality being sacrificed for
>> security?
>
> No I don't.  Lua has less functionality because it was designed to have
> a small embedding footprint.  Python is much bigger because it was
> mostly designed to run as a standalone interpreter.  That has nothing to
> do with security.  You haven't shown the slightest connection between
> Lua's lower functionality and its higher sandbox security, because there
> is none.  The lower functionality is because of a totally independent
> reason, namely the desire to make the interpreter smaller.

This thread started out as "How can I sandbox Python inside Python?".
One of the responses was "You can't, but try sandboxing Lua inside
Python instead". This has the cost that Lua, unlike Python, simply
lacks features. You can *easily* sandbox something that has very
little functionality - all you have to do is provide a minimalist
"language" that permits only a very few actions, and you know it's
safe. But that security comes at a price.

ChrisA


From no.email at nospam.invalid  Sat May 30 00:48:23 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Fri, 29 May 2015 21:48:23 -0700
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
 <mailman.213.1432953735.5151.python-list@python.org>
Message-ID: <874mmuy8ko.fsf@jester.gateway.sonic.net>

Chris Angelico <rosuav at gmail.com> writes:
> You can *easily* sandbox something that has very little functionality
> - all you have to do is provide a minimalist "language" that permits
> only a very few actions, and you know it's safe. But that security
> comes at a price.

This is a non-sequitur.  The reason they didn't put more features into
Lua is that it would have made the memory footprint bigger and they
pitch it as an embeddable extension engine so they want to keep it
small.  Stuff like bignums and unicode in themselves wouldn't have
affected security.  There's no obstacle to implementing Python the way
Lua is implemented, and for all I know, MicroPython does that.  It just
didn't happen to be done that way with CPython because of Python's
expected mode of use historically.  Armin Ronacher's blog entry that I
linked says a little more about this.

Heck, think of Java, which is monstrously more complicated than Python
and supports applet sandboxing, plus it can run Python programs (under
Jython).  Or Javascript, which has similar complexity to Python and runs
sandboxes in millions (billions?) of browsers.


From dieter at handshake.de  Sat May 30 01:43:19 2015
From: dieter at handshake.de (dieter)
Date: Sat, 30 May 2015 07:43:19 +0200
Subject: stdout of child process as an input of another thread in Python?
References: <abcd1234abc123ab12a0000088758000010000001052@gmail.com>
Message-ID: <87pp5id3ig.fsf@handshake.de>

Kevin Peterson <qh.resu01 at gmail.com> writes:

> I want to use the stdout of child process as an input of another thread, but some how I am not able to read the stdout.  Using Popen I have created a child process and stdout of it I have redirected to PIPE (because I don't want that to be printed on with my main process).  Now using this statement,"Line=Proc.stdout.readline()" I want to read stdout of child process and I have to do operation on that. but somehow i am not able to read from stdout of child process. 
>
> Following is the code snapshot - 
>
>     Proc = Popen(["python.exe","filename.py"],stdout=PIPE)
>     
>     while True:
>                 Line = Proc.stdout.readline()
>                 print Line
>
> Here,
> Filename.py(Child process) is continuously printing "Hello World!!" in place of reading stdout of child process. 

Your description is not sufficiently clear:

  If "filename.py" is your child process, why should it read the stdout
  of the child process (i.e. itself).

I suggest, you describe what "filename.py" (your child) does
and what behavior you observe for the parent.



From __peter__ at web.de  Sat May 30 03:56:38 2015
From: __peter__ at web.de (Peter Otten)
Date: Sat, 30 May 2015 09:56:38 +0200
Subject: Minus operator versus unary minus
Message-ID: <mkbqfn$9p0$1@ger.gmane.org>

The following modification of the collections.Counter implementation

https://hg.python.org/cpython/rev/fe4efc0032b5

was just checked in with the line

result[elem] = 0 - count

Does this have an advantage over the obvious

result[elem] = -count

?



From steve at pearwood.info  Sat May 30 04:19:16 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 30 May 2015 18:19:16 +1000
Subject: Minus operator versus unary minus
References: <mailman.215.1432972613.5151.python-list@python.org>
Message-ID: <55697284$0$12992$c3e8da3$5496439d@news.astraweb.com>

On Sat, 30 May 2015 05:56 pm, Peter Otten wrote:

> The following modification of the collections.Counter implementation
> 
> https://hg.python.org/cpython/rev/fe4efc0032b5
> 
> was just checked in with the line
> 
> result[elem] = 0 - count
> 
> Does this have an advantage over the obvious
> 
> result[elem] = -count
> 
> ?


py> class Thingy:
...     def __neg__(self):
...         return [1, 2, 3, 4]
...
py> -Thingy()
[1, 2, 3, 4]
py> 0 - Thingy()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for -: 'int' and 'Thingy'




-- 
Steven



From __peter__ at web.de  Sat May 30 04:36:48 2015
From: __peter__ at web.de (Peter Otten)
Date: Sat, 30 May 2015 10:36:48 +0200
Subject: Minus operator versus unary minus
References: <mailman.215.1432972613.5151.python-list@python.org>
 <55697284$0$12992$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <mkbsr4$dkk$1@ger.gmane.org>

Steven D'Aprano wrote:

> On Sat, 30 May 2015 05:56 pm, Peter Otten wrote:
> 
>> The following modification of the collections.Counter implementation
>> 
>> https://hg.python.org/cpython/rev/fe4efc0032b5
>> 
>> was just checked in with the line
>> 
>> result[elem] = 0 - count
>> 
>> Does this have an advantage over the obvious
>> 
>> result[elem] = -count
>> 
>> ?
> 
> 
> py> class Thingy:
> ...     def __neg__(self):
> ...         return [1, 2, 3, 4]
> ...
> py> -Thingy()
> [1, 2, 3, 4]
> py> 0 - Thingy()
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: unsupported operand type(s) for -: 'int' and 'Thingy'

Hm, do you have an example of a Counter (or subclass) instance where the 
values are not int?



From storchaka at gmail.com  Sat May 30 04:45:32 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sat, 30 May 2015 11:45:32 +0300
Subject: Minus operator versus unary minus
In-Reply-To: <mkbqfn$9p0$1@ger.gmane.org>
References: <mkbqfn$9p0$1@ger.gmane.org>
Message-ID: <mkbtbd$ksm$1@ger.gmane.org>

On 30.05.15 10:56, Peter Otten wrote:
> The following modification of the collections.Counter implementation
>
> https://hg.python.org/cpython/rev/fe4efc0032b5
>
> was just checked in with the line
>
> result[elem] = 0 - count
>
> Does this have an advantage over the obvious
>
> result[elem] = -count
>
> ?

>>> x = 0.0
>>> -x
-0.0 
 

>>> 0 - x
0.0 
 





From steve at pearwood.info  Sat May 30 05:00:14 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 30 May 2015 19:00:14 +1000
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
 <mailman.213.1432953735.5151.python-list@python.org>
 <874mmuy8ko.fsf@jester.gateway.sonic.net>
Message-ID: <55697c1f$0$13013$c3e8da3$5496439d@news.astraweb.com>

On Sat, 30 May 2015 02:48 pm, Paul Rubin wrote:

> Chris Angelico <rosuav at gmail.com> writes:
>> You can *easily* sandbox something that has very little functionality
>> - all you have to do is provide a minimalist "language" that permits
>> only a very few actions, and you know it's safe. But that security
>> comes at a price.
> 
> This is a non-sequitur. The reason they didn't put more features into 
> Lua is that it would have made the memory footprint bigger and they
> pitch it as an embeddable extension engine so they want to keep it
> small.  

I wouldn't have imagined that the claim "it's easier to secure a small
language with a few features than a big language with lots of features"
would have been so controversial. I wonder if this claim will be equally as
controversial?

There is a rough correlation between the number of lines of code in a code
base, and the number of potential security holes that need to be guarded
against.


> Stuff like bignums and unicode in themselves wouldn't have 
> affected security. 

Do you consider a Denial of Service or Memory Exhaustion attack to be a
security issue? If not, try running this in Python:

100**100**100

(Perhaps not a great idea.) How about defeating cryptographic protection
mechanisms?

https://www.auscert.org.au/21885

Or using Unicode to bypass data validation?

https://capec.mitre.org/data/definitions/71.html

Unicode encoding attacks?

https://www.owasp.org/index.php/Unicode_Encoding
https://cwe.mitre.org/data/definitions/176.html

Unicode spoofing? Buffer overflows? UTF-8 exploits? IDNA exploits? Code
point deletion exploits? Malicious rendering?

http://unicode.org/reports/tr36/
http://unicode.org/faq/security.html


[...]
> Heck, think of Java, which is monstrously more complicated than Python
> and supports applet sandboxing, plus it can run Python programs (under
> Jython).  Or Javascript, which has similar complexity to Python and runs
> sandboxes in millions (billions?) of browsers.

Funny you should mention Javascript...

http://securityevaluators.com/knowledge/papers/engineeringheapoverflow.pdf
http://security.stackexchange.com/questions/41966/
https://nakedsecurity.sophos.com/exploring-the-blackhole-exploit-kit-7/
http://resources.infosecinstitute.com/fbi-tor-exploit/
https://www.mozilla.org/en-US/security/advisories/mfsa2013-53/

Yes, I can see why you think Javascript is securely sandboxed...

*wink*



-- 
Steven



From __peter__ at web.de  Sat May 30 05:09:36 2015
From: __peter__ at web.de (Peter Otten)
Date: Sat, 30 May 2015 11:09:36 +0200
Subject: Minus operator versus unary minus
References: <mkbqfn$9p0$1@ger.gmane.org> <mkbtbd$ksm$1@ger.gmane.org>
Message-ID: <mkbuok$90v$1@ger.gmane.org>

Serhiy Storchaka wrote:

> On 30.05.15 10:56, Peter Otten wrote:
>> The following modification of the collections.Counter implementation
>>
>> https://hg.python.org/cpython/rev/fe4efc0032b5
>>
>> was just checked in with the line
>>
>> result[elem] = 0 - count
>>
>> Does this have an advantage over the obvious
>>
>> result[elem] = -count
>>
>> ?
> 
>>>> x = 0.0
>>>> -x
> -0.0
>  
> 
>>>> 0 - x
> 0.0

That's a subtle difference, but does it matter in the context of the patch? 
The complete code is

+        result = Counter()
+        for elem, count in self.items():
+            if count < 0:
+                result[elem] = 0 - count
+        return result

and

>>> -0.0 < 0
False

so both +0.0 and -0.0 would be skipped anyway.



From JustinThyme at nowhere.com  Sat May 30 05:30:04 2015
From: JustinThyme at nowhere.com (Justin Thyme)
Date: Sat, 30 May 2015 10:30:04 +0100
Subject: Python write to spreadsheet?
Message-ID: <mkbvuu$udo$1@news.albasani.net>

Is it possible to write a Python program that will start MS Excel, 
create a spreadsheet and fill cells A1 to A10 (say) with the data in a 
Python array?  The answer is surely yes, but is there an outline of how 
to do it somewhere?
-- 
Shall we only threaten and be angry for an hour?
   When the storm is ended shall we find
How softly but how swiftly they have sidled back to power
   By the favour and contrivance of their kind?

 From /Mesopotamia/ by Rudyard Kipling


From storchaka at gmail.com  Sat May 30 05:32:50 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sat, 30 May 2015 12:32:50 +0300
Subject: Minus operator versus unary minus
In-Reply-To: <mkbuok$90v$1@ger.gmane.org>
References: <mkbqfn$9p0$1@ger.gmane.org> <mkbtbd$ksm$1@ger.gmane.org>
 <mkbuok$90v$1@ger.gmane.org>
Message-ID: <mkc043$t04$1@ger.gmane.org>

On 30.05.15 12:09, Peter Otten wrote:
> Serhiy Storchaka wrote:
>> On 30.05.15 10:56, Peter Otten wrote:
>>> The following modification of the collections.Counter implementation
>>>
>>> https://hg.python.org/cpython/rev/fe4efc0032b5
>>>
>>> was just checked in with the line
>>>
>>> result[elem] = 0 - count
>>>
>>> Does this have an advantage over the obvious
>>>
>>> result[elem] = -count
>>>
>>> ?
>>
>>>>> x = 0.0
>>>>> -x
>> -0.0
>>
>>
>>>>> 0 - x
>> 0.0
>
> That's a subtle difference, but does it matter in the context of the patch?

I don't know. Perhaps there is more meaningful difference for decimals.

This code is not new (see
https://hg.python.org/cpython/rev/e908ebc160f0).



From mail at timgolden.me.uk  Sat May 30 05:40:50 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Sat, 30 May 2015 10:40:50 +0100
Subject: Python write to spreadsheet?
In-Reply-To: <mkbvuu$udo$1@news.albasani.net>
References: <mkbvuu$udo$1@news.albasani.net>
Message-ID: <556985A2.5020809@timgolden.me.uk>

On 30/05/2015 10:30, Justin Thyme wrote:
> Is it possible to write a Python program that will start MS Excel,
> create a spreadsheet and fill cells A1 to A10 (say) with the data in a
> Python array?  The answer is surely yes, but is there an outline of how
> to do it somewhere?


This is still a good place to start:

   http://www.python-excel.org/

but I know I've seen several other packages not mentioned there which 
either call Excel from Python or vice versa. xlwings (not sure?)

TJG


From joel.goldstick at gmail.com  Sat May 30 07:03:00 2015
From: joel.goldstick at gmail.com (Joel Goldstick)
Date: Sat, 30 May 2015 07:03:00 -0400
Subject: Python write to spreadsheet?
In-Reply-To: <556985A2.5020809@timgolden.me.uk>
References: <mkbvuu$udo$1@news.albasani.net> <556985A2.5020809@timgolden.me.uk>
Message-ID: <CAPM-O+wK9BnSrW-BW6hmBjtYDut5U4YAYHFdRCoC1WRGwguzSA@mail.gmail.com>

On Sat, May 30, 2015 at 5:40 AM, Tim Golden <mail at timgolden.me.uk> wrote:
> On 30/05/2015 10:30, Justin Thyme wrote:
>>
>> Is it possible to write a Python program that will start MS Excel,
>> create a spreadsheet and fill cells A1 to A10 (say) with the data in a
>> Python array?  The answer is surely yes, but is there an outline of how
>> to do it somewhere?
>
>
>
> This is still a good place to start:
>
>   http://www.python-excel.org/
>
> but I know I've seen several other packages not mentioned there which either
> call Excel from Python or vice versa. xlwings (not sure?)
>
> TJG
> --
> https://mail.python.org/mailman/listinfo/python-list

This looks promising:
http://www.dev-explorer.com/articles/excel-spreadsheets-and-python


-- 
Joel Goldstick
http://joelgoldstick.com


From lac at openend.se  Sat May 30 07:15:30 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 30 May 2015 13:15:30 +0200
Subject: Python write to spreadsheet?
References: <mkbvuu$udo$1@news.albasani.net>
Message-ID: <201505301115.t4UBFUCN020021@fido.openend.se>

I use this:
https://pypi.python.org/pypi/xlutils

Documentation here:
http://pythonhosted.org/xlutils/

website full of advice here:
http://www.python-excel.org/

Tutorial I took at Europython where I learned a lot of stuff
http://www.simplistix.co.uk/presentations/python-excel.pdf

Laura


From lac at openend.se  Sat May 30 07:24:49 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 30 May 2015 13:24:49 +0200
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: Message from "Steven D'Aprano" <steve@pearwood.info> of "Sat,
 30 May 2015 19:00:14 +1000."
 <55697c1f$0$13013$c3e8da3$5496439d@news.astraweb.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
 <mailman.213.1432953735.5151.python-list@python.org>
 <874mmuy8ko.fsf@jester.gateway.sonic.net><55697c1f$0$13013$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <201505301124.t4UBOnPb020332@fido.openend.se>

In a message of Sat, 30 May 2015 19:00:14 +1000, "Steven D'Aprano" writes:
>I wouldn't have imagined that the claim "it's easier to secure a small
>language with a few features than a big language with lots of features"
>would have been so controversial. I wonder if this claim will be equally as
>controversial?
>
>There is a rough correlation between the number of lines of code in a code
>base, and the number of potential security holes that need to be guarded
>against.

Maybe these aren't controversial if you are doing langauge level
sandboxing, but you don't have to sandbox like that.  Consider, for a
moment, the sandboxing technique used by PyPy
discussed at

http://pypy.readthedocs.org/en/latest/sandbox.html

You think it is way cool, but, alas, you want to sandbox some other
language than Python.

What do you do?  You write an interpreter for this language in
RPython.  Clearly, writing such a thing will be a lot easier for 'the
toy language that does hardly anything I invented this morning' as
opposed to 'javascript that is expected to operate in the real world'
but this has nothing to do with the security aspects of the two
langauges.  You'd have the exact same problem of difficulty even if
you never intend to sandbox the thing at all.   The sandboxing
aspects will happen, automatically, as soon as you have a
written a working interpreter.  The layer that provides the security
doesn't care about your target language, just as long as it is written in
RPython.

As a point of fact, We've _already got_ Topaz, a Ruby interpreter,
Hippy, a PHP interpreter, a Prolog interpreter, a Smalltalk
interpeter, and a javascript interpreter.  Recently we got Pyket a
Racket compiler.  There also exist plenty of experimental languages
written by academic langauge designers, and other crazy people who
like such things.  But don't ask the PyPy project about hard is it to
sandbox one versus the other.  From our point of view, they all cost
the same -- free, as in _already done for you_, same as you get a JIT
for free, and pluggable garbage collectors for free, etc. etc.

If you find this stuff interesting, come check it out.

Laura


From bc at freeuk.com  Sat May 30 08:06:16 2015
From: bc at freeuk.com (BartC)
Date: Sat, 30 May 2015 13:06:16 +0100
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <mailman.210.1432939749.5151.python-list@python.org>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
Message-ID: <2Jhax.531468$oZ4.527214@fx38.am4>

On 29/05/2015 23:49, Chris Angelico wrote:
> On Sat, May 30, 2015 at 4:33 AM, Paul Rubin <no.email at nospam.invalid> wrote:
>> Chris Angelico <rosuav at gmail.com> writes:
>>> Looks to me as if Lua doesn't have integers at all
>>
>> They fixed that in Lua 5.3:
>>
>>    http://www.lua.org/manual/5.3/readme.html#changes
>
> That's 64-bit integers, not arbitrary-precision, but that's something
> at least. You do still need to worry about what happens when your
> numbers get too big; in Python, you simply don't. So it's still not
> quite there in terms of functionality.

But then the vast majority of integer operations won't require arbitrary 
precision. (Or maybe Python programmers routinely use big integers all 
over the place simply because they can.)

>>> Likewise, eight-bit strings, not Unicode.
>>
>> Also fixed in 5.3 (basic utf-8 support added, per above).

> Do you see what I mean about functionality being sacrificed for
> security? There is no way that this could be called fully functional
> by comparison with Python.

Python seems to have sacrificed some performance. When I questioned why 
3.x was slower than 2.x, merging int and long int (as I understood it) 
was one of the reasons put forward.

(Simplicity seems to work for Lua. The entire distribution (for LuaJIT 
2.0), seems to be about 2MB, including C sources, and the JIT 
interpreter is about 220KB. LuaJIT was also one of the fastest dynamic 
languages I've tried. But you're right that the language is rather sparse.)

-- 
Bartc


From subhabrata.banerji at gmail.com  Sat May 30 08:14:19 2015
From: subhabrata.banerji at gmail.com (subhabrata.banerji at gmail.com)
Date: Sat, 30 May 2015 05:14:19 -0700 (PDT)
Subject: Model Dumping Pickle
Message-ID: <1ce961a3-1af9-4fed-a94d-360a29164d79@googlegroups.com>

Dear Group,

If I use pickle to dump my model as follows, 

>>> from gensim.models import Word2Vec
>>> from nltk.corpus import brown, movie_reviews, treebank
>>> b = Word2Vec(brown.sents())
>>> import pickle
>>> f = open('my_classifier5.pickle', 'wb')
>>> pickle.dump(b, f)
>>> f.close()
>>> f1= open('my_classifier5.pickle')
>>> classifier=pickle.load(f1)
>>> classifier.most_similar('money', topn=5)
[('pay', 0.6861867904663086), ('ready', 0.6255729794502258), ('try', 0.6118115186691284), ('go', 0.609034538269043), ('oppose', 0.6002943515777588)]
>>> f1.close()

Am I going fine? If any one may kindly suggest. 

I am using Python 2.7 on MS-Windows 7 Professional. 
Apology for any indentation error.

Regards,
Subhabrata Banerjee. 



From sturla.molden at gmail.com  Sat May 30 08:15:04 2015
From: sturla.molden at gmail.com (Sturla Molden)
Date: Sat, 30 May 2015 12:15:04 +0000 (UTC)
Subject: What is considered an "advanced" topic in Python?
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
Message-ID: <734955378454680698.607749sturla.molden-gmail.com@news.gmane.org>

Mike Driscoll <kyosohma at gmail.com> wrote:
> Hi,
> 
> I've been asked on several occasions to write about intermediate or
> advanced topics in Python and I was wondering what the community
> considers to be "intermediate" or "advanced". I realize we're all growing
> in our abilities with the language, so this is going to be very
> subjective, but I am still curious what my fellow Python developers think about this topic.

Coroutines, metaclasses and decorators is probably high up on the obscurity
list. Writing C extension modules is high up there as well, not to mention
embedding Python and using subinterpreters.


Sturla



From python.list at tim.thechases.com  Sat May 30 08:24:22 2015
From: python.list at tim.thechases.com (Tim Chase)
Date: Sat, 30 May 2015 07:24:22 -0500
Subject: Python write to spreadsheet?
In-Reply-To: <mkbvuu$udo$1@news.albasani.net>
References: <mkbvuu$udo$1@news.albasani.net>
Message-ID: <20150530072422.147c0881@bigbox.christie.dr>

On 2015-05-30 10:30, Justin Thyme wrote:
> Is it possible to write a Python program that will start MS Excel, 
> create a spreadsheet and fill cells A1 to A10 (say) with the data
> in a Python array?  The answer is surely yes, but is there an
> outline of how to do it somewhere?

it depends on how strictly you want to keep your order of operations.

You *can* use Windows API calls to automate Excel, instructing it to
create a new sheet, then populate the various cells with data.
However, this is fragile, Windows-specific, and requires the target
to have Excel on their machine.

If you're willing to forego your specified ordering and create a
file, then open it in Excel, you can use the add-on xlrd/xlwt packages
(the former reads, the latter writes) to write XLS files that you can
then open in Excel.

Finally, if you're going down that route, the *easiest* way is to
simply use Python's csv module from the standard library.

  import csv
  FILENAME = "out.csv"
  with open(FILENAME, 'wb') as f:
    w = csv.writer(f)
    for item in data:
      w.writerow([
        item,
        #, ""  # optionally add a blank column because
               # some parsers like csv.Sniffer will notice
               # the lack of commas and try to find the most
               # common repeated string
        ])

You can then launch that file in Excel, or even better, whatever the
registered program is:

  import os, sys, subprocess
  def open_file(filename):
    if sys.platform == "win32":
      os.startfile(filename)
    else:
      if sys.platform == "darwin":
        program = "open"
      else:
        program = "xdg-open"
      subprocess.call([program, filename])
  open_file(FILENAME)

-tkc





From rosuav at gmail.com  Sat May 30 08:37:05 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 30 May 2015 22:37:05 +1000
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <2Jhax.531468$oZ4.527214@fx38.am4>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <2Jhax.531468$oZ4.527214@fx38.am4>
Message-ID: <CAPTjJmqFFzopOmK0VEKDsvhr0mP4xiCpy0wGUpSFaoAC3u2nnw@mail.gmail.com>

On Sat, May 30, 2015 at 10:06 PM, BartC <bc at freeuk.com> wrote:
> On 29/05/2015 23:49, Chris Angelico wrote:
>> That's 64-bit integers, not arbitrary-precision, but that's something
>> at least. You do still need to worry about what happens when your
>> numbers get too big; in Python, you simply don't. So it's still not
>> quite there in terms of functionality.
>
>
> But then the vast majority of integer operations won't require arbitrary
> precision. (Or maybe Python programmers routinely use big integers all over
> the place simply because they can.)

It's true that it won't often be an issue, but it's a matter of never
needing to worry about it. Have you ever had to tweak an algorithm to
ensure that you never go past some arbitrary boundary? In Python, with
pure integer arithmetic, you can guarantee that mathematical truths
are going to be maintained. There might still be good reason for doing
operations in one order rather than another, but it's to do with
performance, not correctness; the simple and naive algorithm can be
used to verify correctness of any smarter algorithm. That's an
important feature.

> Python seems to have sacrificed some performance. When I questioned why 3.x
> was slower than 2.x, merging int and long int (as I understood it) was one
> of the reasons put forward.

Yes, but that's an API point. A future version of Python could
re-separate them as an optimization, as long as script-level code
can't tell the difference. That's how Pike works, for instance; its
native "int" type handles both machine words and bignums, with no
visible distinction except performance. The common case (small
numbers) is optimized; but short of probing for timings, there's no
way to notice the actual boundary between the two. [1] This kind of
change could be done to a future Python without breaking any existing
code, and without breaking the expectation that integer arithmetic can
go to infinity.

ChrisA

[1] Well, Pike also has a constant Int.NATIVE_MAX, in case a program
is curious. But you get the idea.


From plucena24 at gmail.com  Sat May 30 09:11:26 2015
From: plucena24 at gmail.com (Pablo Lucena)
Date: Sat, 30 May 2015 09:11:26 -0400
Subject: Python write to spreadsheet?
In-Reply-To: <mkbvuu$udo$1@news.albasani.net>
References: <mkbvuu$udo$1@news.albasani.net>
Message-ID: <CAB_tDZy6gU0uw7Yka6-Tjr19n60NN1-v1EeKJyHBWWz5UUgg8Q@mail.gmail.com>

Try openpyxl - I've found this to be a really nice library for interacting
with MS Excel.

On Sat, May 30, 2015 at 5:30 AM, Justin Thyme <JustinThyme at nowhere.com>
wrote:

> Is it possible to write a Python program that will start MS Excel, create
> a spreadsheet and fill cells A1 to A10 (say) with the data in a Python
> array?  The answer is surely yes, but is there an outline of how to do it
> somewhere?
> --
> Shall we only threaten and be angry for an hour?
>   When the storm is ended shall we find
> How softly but how swiftly they have sidled back to power
>   By the favour and contrivance of their kind?
>
> From /Mesopotamia/ by Rudyard Kipling
> --
> https://mail.python.org/mailman/listinfo/python-list
>



-- 
*Pablo Lucena*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150530/504ea67c/attachment.html>

From DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk  Sat May 30 09:18:23 2015
From: DaveFarrance at OMiTTHiSyahooANDTHiS.co.uk (Dave Farrance)
Date: Sat, 30 May 2015 14:18:23 +0100
Subject: Minus operator versus unary minus
References: <mkbqfn$9p0$1@ger.gmane.org> <mkbtbd$ksm$1@ger.gmane.org>
 <mailman.218.1432976989.5151.python-list@python.org>
Message-ID: <ehcjma1ilmh8jmo9f6svfq6al62qsg6pqk@4ax.com>

Peter Otten <__peter__ at web.de> wrote:

>so both +0.0 and -0.0 would be skipped anyway.

Maybe the coder was simply aiming for visibility.  The unary minus can
be hard to spot in some circumstances.  e.g.: I've sneaked a unary minus
into this maths proof, which makes it horrible (although correct):

u and v are vectors (so have an anti-commutative cross-product)
Show that (u - v) x (u + v) = 2u x v

u x (u + v) - v x (u + v)
u x u + u x v - v x u - v x v
u x v - v x u  [ because u x u = 0, v x v = 0]
u x v - (u x -v)
u x v - (-1)(u x v)
u x v + u x v
2(u x v)
2u x v


From garyr at fidalgo.net  Sat May 30 12:22:17 2015
From: garyr at fidalgo.net (garyr)
Date: Sat, 30 May 2015 09:22:17 -0700
Subject: Building an extension module with SWIG
Message-ID: <mkco9p$gf8$1@speranza.aioe.org>

I'm trying to create an extension module using SWIG. I've
succeeded in generating a pyd file but when I import the module I get the
error message: "SystemError: dynamic module not initialized properly." I
added an initfoo() function but that didn't solve the problem. Below are the
various files, a slightly modified version of a SWIG exmaple.
I'm using Python 2.7

What am I missing?

//foo.c:
#include "foo.h"
double Foo;
void initfoo()
{
    Foo = 3.0;
}
int gcd(int x, int y) {
  int g;
  g = y;
  while (x > 0) {
    g = x;
    x = y % x;
    y = g;
  }
  return g;
}

#foo.h:
extern void initfoo();
extern double Foo;
extern int gcd(int x, int y);

#foo.i:
%module example
%inline %{
extern int    gcd(int x, int y);
extern double Foo;
%}

#setup.py
from setuptools import setup, Extension
setup(name='foo',
    version='0.1',
    ext_modules=[Extension('foo', ['foo.c', 'foo.i'],
                    include_dirs=['.'],
                    depends=['foo.h', 'foo.i'],
                    swig_opts=['-modern', '-I../include'],
                    )],
    )





From stefan_ml at behnel.de  Sat May 30 14:42:49 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 30 May 2015 20:42:49 +0200
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <201505301124.t4UBOnPb020332@fido.openend.se>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
 <mailman.213.1432953735.5151.python-list@python.org>
 <874mmuy8ko.fsf@jester.gateway.sonic.net><55697c1f$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <201505301124.t4UBOnPb020332@fido.openend.se>
Message-ID: <mkd0bb$47a$1@ger.gmane.org>

Laura Creighton schrieb am 30.05.2015 um 13:24:
> As a point of fact, We've _already got_ Topaz, a Ruby interpreter,
> Hippy, a PHP interpreter, a Prolog interpreter, a Smalltalk
> interpeter, and a javascript interpreter.  Recently we got Pyket a
> Racket compiler.  There also exist plenty of experimental languages
> written by academic langauge designers, and other crazy people who
> like such things.  But don't ask the PyPy project about hard is it to
> sandbox one versus the other.  From our point of view, they all cost
> the same -- free, as in _already done for you_, same as you get a JIT
> for free, and pluggable garbage collectors for free, etc. etc.

So here the cost of security is actually rewriting the entire language
runtime and potentially also major parts of its ecosystem? Not exactly a
cheap price either.

Stefan




From stefan_ml at behnel.de  Sat May 30 15:05:39 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 30 May 2015 21:05:39 +0200
Subject: Building an extension module with SWIG
In-Reply-To: <mkco9p$gf8$1@speranza.aioe.org>
References: <mkco9p$gf8$1@speranza.aioe.org>
Message-ID: <mkd1m4$og5$1@ger.gmane.org>

garyr schrieb am 30.05.2015 um 18:22:
> I'm trying to create an extension module using SWIG. I've
> succeeded in generating a pyd file but when I import the module I get the
> error message: "SystemError: dynamic module not initialized properly." I
> added an initfoo() function but that didn't solve the problem. Below are the
> various files, a slightly modified version of a SWIG exmaple.
> I'm using Python 2.7
> 
> What am I missing?
> 
> //foo.c:
> #include "foo.h"
> double Foo;
> void initfoo()
> {
>     Foo = 3.0;
> }

This is wrong and you also won't need that.


> int gcd(int x, int y) {
>   int g;
>   g = y;
>   while (x > 0) {
>     g = x;
>     x = y % x;
>     y = g;
>   }
>   return g;
> }
> [...]

Just in case you're not bound to SWIG yet, here's a Cython [1] version of
your code:

    # put this in a file called "foo.pyx"

    def gcd(int x, int y):
        while x > 0:
            y, x = x, y % x
        return y

Compile it ("cythonize -b foo.pyx") and you'll get an extension module that
executes faster than what SWIG would give you and keeps everything in one
file to improve readability.

Stefan


[1] http://cython.org/




From jonathon.blake at gmail.com  Sat May 30 15:32:09 2015
From: jonathon.blake at gmail.com (jonathon)
Date: Sat, 30 May 2015 19:32:09 +0000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
Message-ID: <556A1039.2030201@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 29/05/2015 16:01, Mike Driscoll wrote:

>I was wondering what the community considers to be "intermediate" or "a
dvanced".

A python script that compiles python code.

jonathon
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJVahA4AAoJEE1PKy9+kxplmwsP/R4wkjcgYZwG8wHj6Jp2Do4C
+y3raHDA7PNko9Y/HSsIr3aXyXm9Yy71px3HDYkOXXsD3aO+GXUS14RbQXynK7OX
95LD+S0U9AtJL5Jlywmvm/QJQw6UuGnPfQSs2Y1HiALjlvZYCDMvoycvT4GY4+oQ
6n4tq/3ExdP1inxcu0CwMm7ZL7CCvvy1S7JkVdr+r13BM4TlXCRH9BAvJNtLnbtA
SvP5iOYRkOTvxYOw8veiKXD1clXK2fy3x4SjBtWpK0XZKSu2akWwlD0qvbVkf9qz
/Cu9aP3BbV4VSzLFkOEKU6b4NoZK1Al3s3e4IVvAjPSWIXI5LsKwuVrqvVZ+06Uv
u0ZYyX/4fnpmKaJXvAMyRWULNA+pDaqvjBGgom3qtvT2i8nfYN1KBkMBMrG/ioXt
4BUkAMhOqRHvsPHT+GXrtVaj5wFj1Q9i174CMfINLelbmUlpNhOTLzobm2hnwD/c
hZAVsDUzAxAQjLYNDQV+AtE2Cw4D0YKUJfILE1T82CerSnibjy3Z60RZ703e5oRU
pa75Fm8HpjAmNS4tbvENSz0v0VH6howH1gQdqc2bVzKBGj9ArXiA8yByxRgZdVwI
vXJMzxzpMBL35ScYBhvVPnD0zroa8WBYgrhTWVJFFJpk+eTjUetalTrvkTFYWgQ+
QLsCw9+wUjX8kVJ8Yhx6
=wU1w
-----END PGP SIGNATURE-----


From no.email at nospam.invalid  Sat May 30 16:00:12 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Sat, 30 May 2015 13:00:12 -0700
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
 <mailman.213.1432953735.5151.python-list@python.org>
 <874mmuy8ko.fsf@jester.gateway.sonic.net>
 <55697c1f$0$13013$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <87zj4lx2cz.fsf@jester.gateway.sonic.net>

Steven D'Aprano <steve at pearwood.info> writes:
> I wouldn't have imagined that the claim "it's easier to secure a small
> language with a few features than a big language with lots of features"
> would have been so controversial.

Consider that if the small language is Turing-complete, you can use it
to implement the big language.  If the small language is also secure (in
the sense of not being able to escape a sandbox), the big language
implemented in it can't escape the sandbox either.  Therefore the size
of the language doesn't inherently affect the sandbox security.

Implementing Python in Lua (with LuaJIT) might even have tolerable
performance, possibly beating CPython.

> I wonder if this claim will be equally as controversial?  There is a
> rough correlation between the number of lines of code in a code base,
> and the number of potential security holes that need to be guarded
> against.

Bigger programs are more likely to have bugs, sure, and Lua might have
those already.  But that's not the issue Python faces regarding
sandboxing, where it's insecure by design.

>> Stuff like bignums and unicode in themselves wouldn't have 
>> affected security. 
>
> Do you consider a Denial of Service or Memory Exhaustion attack to be a
> security issue? 

It's less of an issue on the client side were you don't mind too much if
an attacker DOS's his own machine.  Otherwise you have to consider
memory allocation and CPU cycles to be controlled system resources,
which is not rocket science (every operating system does that).  I'm not
sure where Lua sits with regard to this.

> If not, try running this in Python:
> 100**100**100

That's not an issue with bignums in themselves, but rather it's an
artifact of CPython's implementation.  Exponentiation works by repeated
squaring, and each squaring step only doubles the size of its input and
uses predictable cycles, so a sandboxed implementation could get by with
just checking input sizes before every multiplication.

> (Perhaps not a great idea.) How about defeating cryptographic protection
> mechanisms?...
> Or using Unicode to bypass data validation?...
> https://capec.mitre.org/data/definitions/71.html
> Unicode encoding attacks?... ... ...

None of the stuff you listed appear to be issues inherent with
supporting some feature in a language.  They are mostly application and
library bugs.  I got bored enough that I didn't look at all of them, so
maybe I missed something.


From garyr at fidalgo.net  Sat May 30 16:48:34 2015
From: garyr at fidalgo.net (garyr)
Date: Sat, 30 May 2015 13:48:34 -0700
Subject: Building an extension module with SWIG
References: <mkco9p$gf8$1@speranza.aioe.org>
 <mailman.229.1433012748.5151.python-list@python.org>
Message-ID: <mkd7nk$isi$1@speranza.aioe.org>

*snip*

> Compile it ("cythonize -b foo.pyx") and you'll get an extension module
> that
> executes faster than what SWIG would give you and keeps everything in one
> file to improve readability.
>
> Stefan
>
>
> [1] http://cython.org/
>
>
Thanks for your reply. My interest is not in computing the gcd but to learn
how build an extension module. I have some much more complicated C code I
wish to use.





From lac at openend.se  Sat May 30 16:54:48 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 30 May 2015 22:54:48 +0200
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: Message from Stefan Behnel <stefan_ml@behnel.de>
 of "Sat, 30 May 2015 20:42:49 +0200." <mkd0bb$47a$1@ger.gmane.org>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
 <mailman.213.1432953735.5151.python-list@python.org>
 <874mmuy8ko.fsf@jester.gateway.sonic.net><55697c1f$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <201505301124.t4UBOnPb020332@fido.openend.se><mkd0bb$47a$1@ger.gmane.org>
Message-ID: <201505302054.t4UKsmQP000434@fido.openend.se>

In a message of Sat, 30 May 2015 20:42:49 +0200, Stefan Behnel writes:

>So here the cost of security is actually rewriting the entire language
>runtime and potentially also major parts of its ecosystem? Not exactly a
>cheap price either.
>
>Stefan

Well, the runtime is mostly generated, you don't have to write it by
hand.  But, yes, writing an interpreter is work, no question.  I think
that the problem of writing an interpreter is a much smaller proposition
than playing whack-a-mole with language level sandboxing, but depending
on your language, I could be wrong about that.

Laura


From fomcl at yahoo.com  Sat May 30 18:04:51 2015
From: fomcl at yahoo.com (Albert-Jan Roskam)
Date: Sat, 30 May 2015 15:04:51 -0700
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <556A1039.2030201@gmail.com>
Message-ID: <1433023491.37699.YahooMailMobile@web163803.mail.gq1.yahoo.com>

Metaclasses, abc, asyncio, ast, some of the dunder methods, eg __del__, weakref, perhaps gc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150530/2199c36b/attachment.html>

From rosuav at gmail.com  Sat May 30 18:20:45 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 31 May 2015 08:20:45 +1000
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <87zj4lx2cz.fsf@jester.gateway.sonic.net>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
 <mailman.213.1432953735.5151.python-list@python.org>
 <874mmuy8ko.fsf@jester.gateway.sonic.net>
 <55697c1f$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <87zj4lx2cz.fsf@jester.gateway.sonic.net>
Message-ID: <CAPTjJmo7CCe9VVAubM=U3kyyPegDP3cmePvSHtdHT4jJmysLcw@mail.gmail.com>

On Sun, May 31, 2015 at 6:00 AM, Paul Rubin <no.email at nospam.invalid> wrote:
> Steven D'Aprano <steve at pearwood.info> writes:
>> I wouldn't have imagined that the claim "it's easier to secure a small
>> language with a few features than a big language with lots of features"
>> would have been so controversial.
>
> Consider that if the small language is Turing-complete, you can use it
> to implement the big language.  If the small language is also secure (in
> the sense of not being able to escape a sandbox), the big language
> implemented in it can't escape the sandbox either.  Therefore the size
> of the language doesn't inherently affect the sandbox security.

Turing completeness isn't the whole story. How do you go about
sandboxing a Brainf* implementation such that it can be used to
implement Python, but can't be used to read or arbitrary files from
your file system? Will you reimplement the Python standard library in
Brainf*? Will you implement open(), but nerf it? Will you make sure
there's nothing anywhere in the stdlib that can open files? And if you
_don't_ provide a reimplemented standard library, you either need to
provide an import mechanism (so you can make use of the existing
Python code) or declare that the language as a whole is neutered by a
complete lack of all those features that are implemented in Python.

ChrisA


From rosuav at gmail.com  Sat May 30 18:24:35 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 31 May 2015 08:24:35 +1000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <556A1039.2030201@gmail.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <556A1039.2030201@gmail.com>
Message-ID: <CAPTjJmpxwttzY9ecE5PNJsvDmSy+4nAqj0EPtRMzZLUi874CUA@mail.gmail.com>

On Sun, May 31, 2015 at 5:32 AM, jonathon <jonathon.blake at gmail.com> wrote:
> On 29/05/2015 16:01, Mike Driscoll wrote:
>
>>I was wondering what the community considers to be "intermediate" or "a
> dvanced".
>
> A python script that compiles python code.

What do you mean by "compiles"? Something as simple as an import
statement or exec call will compile arbitrary Python code, but if that
were what you meant, you'd have said "the exec() function". (The
implementation of namedtuple uses exec, and I think that would
definitely count as "advanced".) Or do you mean compiling to AST and
exploring the AST? That's a fairly reasonable intermediate topic,
though again, if that's what you meant, I'm sure you would have worded
it differently. The only other form of compilation I can think of is
going to an executable binary (the way a C compiler does), and that's
some serious work. I don't think PyPy is small enough to be called a
"script". :)

ChrisA


From no.email at nospam.invalid  Sat May 30 18:36:56 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Sat, 30 May 2015 15:36:56 -0700
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
 <mailman.213.1432953735.5151.python-list@python.org>
 <874mmuy8ko.fsf@jester.gateway.sonic.net>
 <55697c1f$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <87zj4lx2cz.fsf@jester.gateway.sonic.net>
 <mailman.233.1433024454.5151.python-list@python.org>
Message-ID: <87vbf9wv3r.fsf@jester.gateway.sonic.net>

Chris Angelico <rosuav at gmail.com> writes:
> Turing completeness isn't the whole story. How do you go about
> sandboxing a Brainf* implementation such that it can be used to
> implement Python, but can't be used to read or arbitrary files from
> your file system?

We're talking about sandboxing, so preventing the sandboxed Python
interpreter written in embedded BF from accessing arbitrary files is the
whole point.  If you want to let a sandboxed program access a file, you
create some kind of managed handle outside the interpreter, and pass
that into the interpreter so the interpreted program can make a
constrained set of calls on it.  That's how Java applets work and it's
basically the opposite of Python's "consenting adults" approach which is
to let everything access everything.


From garyr at fidalgo.net  Sat May 30 18:55:07 2015
From: garyr at fidalgo.net (garyr)
Date: Sat, 30 May 2015 15:55:07 -0700
Subject: Building an extension module with SWIG
References: <mkco9p$gf8$1@speranza.aioe.org>
 <mailman.229.1433012748.5151.python-list@python.org>
 <mkd7nk$isi$1@speranza.aioe.org>
Message-ID: <mkdf53$2fh$1@speranza.aioe.org>


"garyr" <garyr at fidalgo.net> wrote in message
news:mkd7nk$isi$1 at speranza.aioe.org...
> *snip*
>
>> Compile it ("cythonize -b foo.pyx") and you'll get an extension module
>> that
>> executes faster than what SWIG would give you and keeps everything in one
>> file to improve readability.
>>
>> Stefan
>>
>>
>> [1] http://cython.org/
>>
>>
> Thanks for your reply. My interest is not in computing the gcd but to
> learn
> how build an extension module. I have some much more complicated C code I
> wish to use.
>
>
>
Thanks for your reply. I'm using the compiler that came with Anaconda Python
2.7. I too used SWIG a bunch of years ago but it has changed a lot since
then; e.g., it is now included in the Python distribution.





From dreamingforward at gmail.com  Sat May 30 19:10:50 2015
From: dreamingforward at gmail.com (zipher)
Date: Sat, 30 May 2015 16:10:50 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <55655a86$0$12908$c3e8da3$5496439d@news.astraweb.com>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.61.1432661300.5151.python-list@python.org>
 <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>
 <241377fe-2cac-46f8-bbe1-698de608cef4@googlegroups.com>
 <mailman.72.1432701603.5151.python-list@python.org>
 <55655a86$0$12908$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <58948067-4783-4dec-b878-b89ec368deb9@googlegroups.com>

On Wednesday, May 27, 2015 at 12:48:02 AM UTC-5, Steven D'Aprano wrote:
> On Wednesday 27 May 2015 14:39, Ben Finney wrote:
> 
> > zipher <dreamingforward at gmail.com> writes:
> > 
> >> Arrgh.  Sorry, that was meant privately...
> > 
> > I'm glad we saw it publicly, so that we get more of an idea how you
> > treat people.
> > 
> > That kind of homophobic slur is inappropriate from anyone in this
> > community. Kindly cut it out altogether.
> 
> I look forward to the day when people would read the earlier insult and be 
> perplexed as to why it is a slur at all. In the same way as "your mother 
> wears army boots" has become a joke slur, not taken seriously.

Yes, and I look forward to the day when you *bend over* freely to take it all in without complaint.  When such offerings are natural and aren't seen as an insult at all.  

You first.

Mark 

> So long as it does nobody any harm, and all participants have consented, 
> what one adult chooses to take into their mouth is nobody else's business 
> and certainly no reason to look down on them.
> 
> 
> 
> -- 
> Steve



From dreamingforward at gmail.com  Sat May 30 19:13:06 2015
From: dreamingforward at gmail.com (zipher)
Date: Sat, 30 May 2015 16:13:06 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <mailman.72.1432701603.5151.python-list@python.org>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.61.1432661300.5151.python-list@python.org>
 <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>
 <241377fe-2cac-46f8-bbe1-698de608cef4@googlegroups.com>
 <mailman.72.1432701603.5151.python-list@python.org>
Message-ID: <d1df1e76-7add-4dee-95d4-75f94e0f8187@googlegroups.com>

On Tuesday, May 26, 2015 at 11:40:20 PM UTC-5, Ben Finney wrote:
> zipher <dreamingforward at gmail.com> writes:
> 
> > Arrgh.  Sorry, that was meant privately...
> 
> I'm glad we saw it publicly, so that we get more of an idea how you
> treat people.

Ben, he asked for it.  Stop assuming.

> That kind of homophobic slur is inappropriate from anyone in this
> community. Kindly cut it out altogether.

Ben that is so presumtuous of you!  He and my acts were completely homo positive.  I assure you.

Perhaps you are hiding something?

Mark
P.S. You can respond privately.


From dreamingforward at gmail.com  Sat May 30 19:15:42 2015
From: dreamingforward at gmail.com (zipher)
Date: Sat, 30 May 2015 16:15:42 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <874mmybhb7.fsf@elektro.pacujo.net>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mk2kte$opk$2@speranza.aioe.org>
 <mailman.64.1432673145.5151.python-list@python.org>
 <556554d9$0$2796$c3e8da3$76491128@news.astraweb.com>
 <mailman.73.1432708376.5151.python-list@python.org>
 <874mmybhb7.fsf@elektro.pacujo.net>
Message-ID: <1f630a51-cfad-4926-a739-7d009050c97d@googlegroups.com>

On Wednesday, May 27, 2015 at 2:39:21 AM UTC-5, Marko Rauhamaa wrote:
> Chris Angelico <rosuav at gmail.com>:
> 
> > Using some other name in place of "self" should definitely remain
> > *possible*, but not commonly done.
> 
> You are effectively making the argument that Python has made a mistake
> by not giving "self" a special, language-level status.

I'm making that claim.  Guido's arguments are based on a simple lexical definition of the language.  If the lexical definition were sophisticated enough to feed into GCC as a front-end, for example, then it wouldn't have to be so abnormal.  It's a short-cut for a less mature language.

Mark


From dreamingforward at gmail.com  Sat May 30 19:18:44 2015
From: dreamingforward at gmail.com (zipher)
Date: Sat, 30 May 2015 16:18:44 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <mailman.88.1432731626.5151.python-list@python.org>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <mailman.61.1432661300.5151.python-list@python.org>
 <3dca4bcd-3bb3-4c12-9f0d-bdd3433ebc09@googlegroups.com>
 <241377fe-2cac-46f8-bbe1-698de608cef4@googlegroups.com>
 <mailman.72.1432701603.5151.python-list@python.org>
 <55655a86$0$12908$c3e8da3$5496439d@news.astraweb.com>
 <mailman.79.1432726201.5151.python-list@python.org>
 <d4c77505-be5f-4434-a4b3-3d3fe083add6@googlegroups.com>
 <mailman.88.1432731626.5151.python-list@python.org>
Message-ID: <c5cbe7dd-f6b9-4bd8-9c05-75fd0fb842bb@googlegroups.com>

On Wednesday, May 27, 2015 at 8:00:49 AM UTC-5, Todd wrote:
> On Wed, May 27, 2015 at 2:40 PM, zipher <dreamin... at gmail.com> wrote:
> On Wednesday, May 27, 2015 at 6:30:16 AM UTC-5, Ben Finney wrote:
> 
> > Steven D'Aprano <steve+comp.... at pearwood.info> writes:
> 
> >
> 
> > > On Wednesday 27 May 2015 14:39, Ben Finney wrote:
> 
> > >
> 
> > > > That kind of homophobic slur is inappropriate from anyone in this
> 
> > > > community. Kindly cut it out altogether.
> 
> > >
> 
> > > I look forward to the day when people would read the earlier insult
> 
> > > and be perplexed as to why it is a slur at all. In the same way as
> 
> > > "your mother wears army boots" has become a joke slur, not taken
> 
> > > seriously.
> 
> >
> 
> > Yes, let's all work toward an end to the use of gender, sexuality,
> 
> > ethnicity, and other inborn traits as the punchline of insults or jokes.
> 
> 
> 
> Oh God, you people are being idiots.? It's poop. And shall we all so look forward to the day, when people who eat poop are also welcome into the circle of humanity?
>
> If your goal is to get people to stop calling you a troll, you are going about it the wrong way.? If it isn't, why are you even here?? Please remember the first rule of holes: if you find yourself in a hole, stop digging.

Dude, I stopped digging into "holes" a long time ago, but then that was the issue wasn't it?  And since my goal isn't that, let's go back to discussing the ins and outs (ahem) of creating a more complete lexical specification of the language, which would provide many other worthy side benefits, besides saving on typing characters.

Mark


From dreamingforward at gmail.com  Sat May 30 19:39:03 2015
From: dreamingforward at gmail.com (zipher)
Date: Sat, 30 May 2015 16:39:03 -0700 (PDT)
Subject: should "self" be changed?
In-Reply-To: <87oal4d4es.fsf@elektro.pacujo.net>
References: <551c8229-f426-45f0-a0ee-fdad1b161f59@googlegroups.com>
 <877frvf7f2.fsf@elektro.pacujo.net>
 <9f9498a5-8291-4347-aef2-ada324bb47a7@googlegroups.com>
 <87r3q3dqju.fsf@elektro.pacujo.net> <vg3k2vsvlrk.fsf@coffee.modeemi.fi>
 <87siagagpx.fsf@elektro.pacujo.net>
 <mailman.135.1432827676.5151.python-list@python.org>
 <87oal4d4es.fsf@elektro.pacujo.net>
Message-ID: <81b3cc36-0f43-4f03-8319-1eb42ffe21d9@googlegroups.com>

On Thursday, May 28, 2015 at 11:59:36 AM UTC-5, Marko Rauhamaa wrote:
> Ian Kelly <ian.g.kelly at gmail.com>:
> > I think I would be more inclined to use enums. This has the advantages
> > of not creating a new set of state classes for every connection
> > instance and that each state is a singleton instance, allowing things
> > like "if self.state is SMTPConnectionState.IDLE". It could look
> > something like this:
> >
> > class SMTPConnectionState(Enum):
> >
> >     class IDLE:
> >         @classmethod
> >         def handle_command(cls, conn, cmd):
> >             # ...
> >
> >     class SPF_HELO:
> >         @classmethod
> >         def terminate(cls, conn):
> >             # ...
> 
> Really, the main expressive choice is whether you use an inner class
> (and get the advantages of a closure) or an outer class (and get
> potential performance advantages).

Can you tell me: What is the advantage of a closure here?

Also:  why wouldn't you simply put your state classes at the outer scope (i.e. module-level) of your module so that they are clear to anyone else who want to use the module?  Presumably these classes are for making your SMTPServerConnection object more useable, not to hide these state-classes from the users.

MarkJ


From steve at pearwood.info  Sat May 30 19:52:29 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 31 May 2015 09:52:29 +1000
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
 <mailman.213.1432953735.5151.python-list@python.org>
 <874mmuy8ko.fsf@jester.gateway.sonic.net><55697c1f$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <mailman.223.1432985122.5151.python-list@python.org>
Message-ID: <556a4d3f$0$12998$c3e8da3$5496439d@news.astraweb.com>

On Sat, 30 May 2015 09:24 pm, Laura Creighton wrote:

> In a message of Sat, 30 May 2015 19:00:14 +1000, "Steven D'Aprano" writes:
>>I wouldn't have imagined that the claim "it's easier to secure a small
>>language with a few features than a big language with lots of features"
>>would have been so controversial. I wonder if this claim will be equally
>>as controversial?
>>
>>There is a rough correlation between the number of lines of code in a code
>>base, and the number of potential security holes that need to be guarded
>>against.
> 
> Maybe these aren't controversial if you are doing langauge level
> sandboxing, but you don't have to sandbox like that.  Consider, for a
> moment, the sandboxing technique used by PyPy
> discussed at
> 
> http://pypy.readthedocs.org/en/latest/sandbox.html
> 
> You think it is way cool, but, alas, you want to sandbox some other
> language than Python.

How many PyPy sandboxes are being used with hostile users motivated to break
out of the sandbox?

"I wrote a sandbox which I can't break out of" is different from "I wrote a
sandbox which nobody can break out of". Javascript is sandboxed, but due to
bugs in implementations, Javascript-based exploits are now heavily used by
malware. There are possibly even more Javascript-based exploits than buffer
overflow based exploits these days, as C programmers get better at using
automated tools that check for buffer overflows.



-- 
Steven



From modulok at gmail.com  Sat May 30 21:08:37 2015
From: modulok at gmail.com (Modulok)
Date: Sat, 30 May 2015 19:08:37 -0600
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: <556a4d3f$0$12998$c3e8da3$5496439d@news.astraweb.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
 <mailman.213.1432953735.5151.python-list@python.org>
 <874mmuy8ko.fsf@jester.gateway.sonic.net>
 <55697c1f$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <mailman.223.1432985122.5151.python-list@python.org>
 <556a4d3f$0$12998$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <CAN2+EpYwwwf_C9uM2OV224GYH=T9YqeZAXE=Tvi8HXuZndpJbg@mail.gmail.com>

While this thread is indeed a theoretical discussion of the interpreter,
for a practical solution where you control the host environment, one might
look into OS level sandboxing like FreeBSD's Jails (not to be confused with
a simple chroot environment) along with various resource limiting
parameters. You can lock down a 'sandboxed' i.e. jailed environment for
arbitrary data and processes, including python, pretty tightly.

-Kurt-

On Sat, May 30, 2015 at 5:52 PM, Steven D'Aprano <steve at pearwood.info>
wrote:

> On Sat, 30 May 2015 09:24 pm, Laura Creighton wrote:
>
> > In a message of Sat, 30 May 2015 19:00:14 +1000, "Steven D'Aprano"
> writes:
> >>I wouldn't have imagined that the claim "it's easier to secure a small
> >>language with a few features than a big language with lots of features"
> >>would have been so controversial. I wonder if this claim will be equally
> >>as controversial?
> >>
> >>There is a rough correlation between the number of lines of code in a
> code
> >>base, and the number of potential security holes that need to be guarded
> >>against.
> >
> > Maybe these aren't controversial if you are doing langauge level
> > sandboxing, but you don't have to sandbox like that.  Consider, for a
> > moment, the sandboxing technique used by PyPy
> > discussed at
> >
> > http://pypy.readthedocs.org/en/latest/sandbox.html
> >
> > You think it is way cool, but, alas, you want to sandbox some other
> > language than Python.
>
> How many PyPy sandboxes are being used with hostile users motivated to
> break
> out of the sandbox?
>
> "I wrote a sandbox which I can't break out of" is different from "I wrote a
> sandbox which nobody can break out of". Javascript is sandboxed, but due to
> bugs in implementations, Javascript-based exploits are now heavily used by
> malware. There are possibly even more Javascript-based exploits than buffer
> overflow based exploits these days, as C programmers get better at using
> automated tools that check for buffer overflows.
>
>
>
> --
> Steven
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150530/3ac4d5e7/attachment.html>

From chris at cdreimer.com  Sat May 30 21:28:17 2015
From: chris at cdreimer.com (C.D. Reimer)
Date: Sat, 30 May 2015 18:28:17 -0700
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
Message-ID: <556A63B1.1010303@cdreimer.com>

On 5/29/2015 9:01 AM, Mike Driscoll wrote:
> I've been asked on several occasions to write about intermediate or advanced topics in Python and I was wondering what the community considers to be "intermediate" or "advanced".

I'm trying my hand at Cython (http://cython.org/). I just know enough of 
the Python and C languages to understand the book, "Cython" by Kurt W. 
Smith, but trying to understand this on a deeper level by re-reading 
each chapter two or three times is enough to give me a headache.  Which 
is fine since I have no use for a programming book that I can breeze 
through. Then again, maybe this book wasn't the best book to learn 
Cython from.

Chris R.


From garyr at fidalgo.net  Sat May 30 21:54:41 2015
From: garyr at fidalgo.net (garyr)
Date: Sat, 30 May 2015 18:54:41 -0700
Subject: Building an extension module with SWIG
References: <mkco9p$gf8$1@speranza.aioe.org>
Message-ID: <mkdpne$k16$1@speranza.aioe.org>


"garyr" <garyr at fidalgo.net> wrote in message 
news:mkco9p$gf8$1 at speranza.aioe.org...
> I'm trying to create an extension module using SWIG. I've
> succeeded in generating a pyd file but when I import the module I get the
> error message: "SystemError: dynamic module not initialized properly." I
> added an initfoo() function but that didn't solve the problem. Below are 
> the
> various files, a slightly modified version of a SWIG exmaple.
> I'm using Python 2.7
>
> What am I missing?
>
> //foo.c:
> #include "foo.h"
> double Foo;
> void initfoo()
> {
>    Foo = 3.0;
> }
> int gcd(int x, int y) {
>  int g;
>  g = y;
>  while (x > 0) {
>    g = x;
>    x = y % x;
>    y = g;
>  }
>  return g;
> }
>
> #foo.h:
> extern void initfoo();
> extern double Foo;
> extern int gcd(int x, int y);
>
> #foo.i:
> %module example
> %inline %{
> extern int    gcd(int x, int y);
> extern double Foo;
> %}
>
> #setup.py
> from setuptools import setup, Extension
> setup(name='foo',
>    version='0.1',
>    ext_modules=[Extension('foo', ['foo.c', 'foo.i'],
>                    include_dirs=['.'],
>                    depends=['foo.h', 'foo.i'],
>                    swig_opts=['-modern', '-I../include'],
>                    )],
>    )
>
It's working! The first character of the name of the extension must be an 
underscore; e.g.,

>    ext_modules=[Extension('_foo', ['foo.c', 'foo.i'],
and the initfoo() function is not needed.

Thanks to all that replied to my post. 




From no.email at nospam.invalid  Sat May 30 23:13:25 2015
From: no.email at nospam.invalid (Paul Rubin)
Date: Sat, 30 May 2015 20:13:25 -0700
Subject: Creating a reliable sandboxed Python environment
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <c9b388e8-dec0-4f2b-baab-537622b80992@googlegroups.com>
Message-ID: <87iob9wiay.fsf@jester.gateway.sonic.net>

davidfstr at gmail.com writes:
> Thanks for the responses folks. I will briefly summarize them:...

I do think you should look at Geordi (the C++ IRC bot) that I linked.
It seems to have changed its implementation to use Docker, but either
way, lots of the the stuff it did was language independent.


From rustompmody at gmail.com  Sat May 30 23:30:32 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sat, 30 May 2015 20:30:32 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
Message-ID: <a4fc4932-136c-4174-aa06-a8565f72348e@googlegroups.com>

On Friday, May 29, 2015 at 9:32:06 PM UTC+5:30, Mike Driscoll wrote:
> Hi,
> 
> I've been asked on several occasions to write about intermediate or advanced topics in Python and I was wondering what the community considers to be "intermediate" or "advanced". I realize we're all growing in our abilities with the language, so this is going to be very subjective, but I am still curious what my fellow Python developers think about this topic.

[I will assume that this question makes sense in the context of introducing
someone (say beginner/learner/noob/etc) to the language]
The zen of python makes a subtle distinction between complex and complicated.
The answers seem to suggest that people are taking the complicated for the complex.

So at the risk of being simplistic (or glib) let me give a definition and a fact:
Def:
Complex is an 'act of God'
Complicated is a slip of man

Fact:
Programming languages (like programs) are made by humans... who err.

So while the err-ors of programmers are only too well-known and spelt 'bugs'
we dont spend enough time on language-bugs.
In a teaching context this only does injustice to the student because the 
learner -- typically a kid -- suffering something or other and wondering
"Am I not fit for this?? Its so hard!" etc may only need a simple:
"Yeah the language designer goofed of on that one"

Trivial example:
C originally had only & | for both bitwise and logical operators.
Later the shortcircuit operators && || were added and kept adjacent in the
precedence table. MISTAKE.
The bitwise operators are conceptually arithmetic not logical, in that
x & MASK == MASK
can make sense only one way.
C makes sense of it the other way.

And so...
The most useful, honest line for a teacher to take is:
C has goofed on that one... So you need to parenthesize that
IOW the beginner is not wrong in writing
x & MASK == MASK
It is C that is wrong in choosing a meaningless parse of that

Now in all fairness python has less gotchas than C but it has its share.
So to my mind Steven's "iterators and generators" as intermediate represents 
no so much an inherent complexity of the subject as an unfortunate complication
from sub-optimal design.

I would put generators as basic; iterators as intermediate.
The generator
def foo():
  yield 1
  yield 2
  yield 3

is 'just' the list [1,2,3] + laziness - memoization and should be closer
to lists in dificulty-level than to iterators.

The reason it seems to be at the level of iterators is related to language
definition, documentation, ontology eg the 'fact' that foo looks like a function
when conceptually/ontologically/implementationally it is not.

This is a language-goofup not student ignorance

One more minor nit with Steven's intermediate-list:
Command line parsing with optparse/argparse.
Well if you use an obese library you get what you ask.
Whats wrong with getopt?
Seems to me simple enough for a beginner to write simple scripts

As I said this is a little nit.
However the pedagogic principle is important.
argparse may be recommended whereas getopt may be deprecated [dunno if it is]
Nevertheless pedagogic expedience is not the same as real world best practice.

<Admission>
In a recent course I taught, I used a[0] and a[1:] to split arrays and write
recursive functions a la Haskell in Python.
Is it efficient? no
Is it idiomatic python? NO!
Is it good to do that? That depends on one's priority.
In mine, learning recursion is more important than learning idiomatic python
</Admission>


From rosuav at gmail.com  Sun May 31 00:25:27 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 31 May 2015 14:25:27 +1000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <a4fc4932-136c-4174-aa06-a8565f72348e@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <a4fc4932-136c-4174-aa06-a8565f72348e@googlegroups.com>
Message-ID: <CAPTjJmrwPmq3J-K+=kM4OkG=9bsYxPRJdi8LtW0tLVvGBQAquw@mail.gmail.com>

On Sun, May 31, 2015 at 1:30 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> <Admission>
> In a recent course I taught, I used a[0] and a[1:] to split arrays and write
> recursive functions a la Haskell in Python.
> Is it efficient? no
> Is it idiomatic python? NO!
> Is it good to do that? That depends on one's priority.
> In mine, learning recursion is more important than learning idiomatic python
> </Admission>

If recursion is more important than idiomatic Python, why are you
using a Python interpreter? Use Python to teach Python, and use
Haskell to teach Haskell!

ChrisA


From rustompmody at gmail.com  Sun May 31 00:46:50 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sat, 30 May 2015 21:46:50 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mailman.237.1433046335.5151.python-list@python.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <a4fc4932-136c-4174-aa06-a8565f72348e@googlegroups.com>
 <mailman.237.1433046335.5151.python-list@python.org>
Message-ID: <7dfc5cf2-8dde-4d56-ab13-67f4286d07c0@googlegroups.com>

On Sunday, May 31, 2015 at 9:55:45 AM UTC+5:30, Chris Angelico wrote:
> On Sun, May 31, 2015 at 1:30 PM, Rustom Mody wrote:
> > <Admission>
> > In a recent course I taught, I used a[0] and a[1:] to split arrays and write
> > recursive functions a la Haskell in Python.
> > Is it efficient? no
> > Is it idiomatic python? NO!
> > Is it good to do that? That depends on one's priority.
> > In mine, learning recursion is more important than learning idiomatic python
> > </Admission>
> 
> If recursion is more important than idiomatic Python, why are you
> using a Python interpreter? Use Python to teach Python, and use
> Haskell to teach Haskell!

Heh! I would have expected less naivete from you Chris!
Firstly idiomatic haskell is as low priority (maybe more so) than idiomatic python

More to the point no language matches perfectly? everything that a learner needs to learn.
Can you write a kernel module in python? (Or Haskell?)
Can you see details of machine state and transitions in python?
Can you client-script a browser in python?
C is the best fit for the first
Assembly for the second
Javascript for the third.

And probably a dozen other languages for a dozen other key concepts.

Finally the idiomatic vs fundamentals is not a clear binary divide.
If I were interviewing someone claiming to be a C programmer who did not know
the difference between postfix and prefix ++, I'd be darned suspicious.
OTOH if someone who had written significant code in C simply refused to write
a '++' saying "Causes just too much trouble" most people would regard it as odd
but not unacceptable.

? or can. Thats G?del's second theorem for you


From rosuav at gmail.com  Sun May 31 00:58:27 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 31 May 2015 14:58:27 +1000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <7dfc5cf2-8dde-4d56-ab13-67f4286d07c0@googlegroups.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <a4fc4932-136c-4174-aa06-a8565f72348e@googlegroups.com>
 <mailman.237.1433046335.5151.python-list@python.org>
 <7dfc5cf2-8dde-4d56-ab13-67f4286d07c0@googlegroups.com>
Message-ID: <CAPTjJmrLW7K0qN96VA2QTOneJ0W_xYodK_4bctOH6aiGi7z8qA@mail.gmail.com>

On Sun, May 31, 2015 at 2:46 PM, Rustom Mody <rustompmody at gmail.com> wrote:
> More to the point no language matches perfectly? everything that a learner needs to learn.
> Can you write a kernel module in python? (Or Haskell?)
> Can you see details of machine state and transitions in python?
> Can you client-script a browser in python?
> C is the best fit for the first
> Assembly for the second
> Javascript for the third.

And if I want to build a web site, I won't write assembly code using
Python idioms. I will use Python. Why fight against a language?

ChrisA


From rustompmody at gmail.com  Sun May 31 01:18:32 2015
From: rustompmody at gmail.com (Rustom Mody)
Date: Sat, 30 May 2015 22:18:32 -0700 (PDT)
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <mailman.238.1433048310.5151.python-list@python.org>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <a4fc4932-136c-4174-aa06-a8565f72348e@googlegroups.com>
 <mailman.237.1433046335.5151.python-list@python.org>
 <7dfc5cf2-8dde-4d56-ab13-67f4286d07c0@googlegroups.com>
 <mailman.238.1433048310.5151.python-list@python.org>
Message-ID: <823d9d5e-c490-408b-8830-7e3da57e3b6f@googlegroups.com>

On Sunday, May 31, 2015 at 10:28:39 AM UTC+5:30, Chris Angelico wrote:
> On Sun, May 31, 2015 at 2:46 PM, Rustom Mody wrote:
> > More to the point no language matches perfectly? everything that a learner needs to learn.
> > Can you write a kernel module in python? (Or Haskell?)
> > Can you see details of machine state and transitions in python?
> > Can you client-script a browser in python?
> > C is the best fit for the first
> > Assembly for the second
> > Javascript for the third.
> 
> And if I want to build a web site, I won't write assembly code using
> Python idioms. I will use Python. Why fight against a language?

I thought I answered. I try again...
Because its the law.
Why does iron rust?
Why do cars/rockets not go faster than light?
Why do we age/shrivel/wither/die?
Because thats the law.
The law here is G?del's second theorem -- no single formalism can encompass all
formalisms.
Any real solution to a significant problem will use some (set of) languages
which will fit good here and not so good there.
Square pegs in round holes are not ideal
Yet every so often there's no choice but to use a wrench to hammer a screw 
through a brick.

The C/assembly/javascript were extreme black-n-white examples.
Most real cases are more fuzzy.
Linus Torvalds thinks C++ (both the language and programmers) consititute
an asinine choice for writing kernel-code.
I am ready to bet that a democratic election on the subject will leave Torvalds
trailing.  ie the world's C++ programmers likely disagree with him.

Who is right?
Ive no idea. Do you?

The main point lost in this discussion is "What is the point?"
If you ask a beginner to write a function to give fibonacci numbers,
are fibonacci numbers the point?

I dont believe that python is 'the point' much more than fibonacci numbers.


From stefan_ml at behnel.de  Sun May 31 01:28:45 2015
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 31 May 2015 07:28:45 +0200
Subject: Building an extension module with SWIG
In-Reply-To: <mkd7nk$isi$1@speranza.aioe.org>
References: <mkco9p$gf8$1@speranza.aioe.org>
 <mailman.229.1433012748.5151.python-list@python.org>
 <mkd7nk$isi$1@speranza.aioe.org>
Message-ID: <mke66d$li2$1@ger.gmane.org>

garyr schrieb am 30.05.2015 um 22:48:
> *snip*
> 
>> Compile it ("cythonize -b foo.pyx") and you'll get an extension module
>> that
>> executes faster than what SWIG would give you and keeps everything in one
>> file to improve readability.
>>
>> [1] http://cython.org/
> 
> Thanks for your reply. My interest is not in computing the gcd but to learn
> how build an extension module. I have some much more complicated C code I
> wish to use.

You can do that with Cython, too.

http://docs.cython.org/src/tutorial/external.html

http://docs.cython.org/src/tutorial/clibraries.html

I might be a bit biased as a core developer, but if the parts of you C
library's API for which you have an immediate use are not so tremendously
huge that it's entirely infeasible for you to write a nicely usable Python
API for them, I'd always recommend using Cython over a wrapper generator
like SWIG. Once you get to the points where it becomes interesting, you'll
always end up having more fun writing a Cython based integration layer than
fighting your up-hill battle against the way the wrapper generator wants
you to design it.

Stefan




From ch.tanveer13 at gmail.com  Sun May 31 01:35:25 2015
From: ch.tanveer13 at gmail.com (ch.tanveer13 at gmail.com)
Date: Sat, 30 May 2015 22:35:25 -0700 (PDT)
Subject: Basic Info
Message-ID: <178f0eac-1760-40b3-9c10-c2d0075883a5@googlegroups.com>

Hi friends,
M Tanveer, and wanna start to learn python language, i've installed python on my Windows (OS) and set path to it, Now please Guide me which editor is best to use and what instructions should be followed . 
Best Regards: Tanveeer Ahmad
Thanks


From lac at openend.se  Sun May 31 02:14:53 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 31 May 2015 08:14:53 +0200
Subject: Creating a reliable sandboxed Python environment
In-Reply-To: Message from "Steven D'Aprano" <steve@pearwood.info> of "Sun,
 31 May 2015 09:52:29 +1000."
 <556a4d3f$0$12998$c3e8da3$5496439d@news.astraweb.com>
References: <60b424a2-2273-42b2-b60c-92656af0afa5@googlegroups.com>
 <mk7ng1$uq3$1@ger.gmane.org>
 <CAPTjJmpC40fLdmGvMqCGtt82O5QWzNGyHnSGgiX_mNrz3O_LdA@mail.gmail.com>
 <mk90bq$dn4$1@ger.gmane.org>
 <mailman.160.1432885273.5151.python-list@python.org>
 <87h9qvxmh0.fsf@jester.gateway.sonic.net>
 <mailman.210.1432939749.5151.python-list@python.org>
 <878uc6yhtq.fsf@jester.gateway.sonic.net>
 <mailman.213.1432953735.5151.python-list@python.org>
 <874mmuy8ko.fsf@jester.gateway.sonic.net><55697c1f$0$13013$c3e8da3$5496439d@news.astraweb.com>
 <mailman.223.1432985122.5151.python-list@python.org><556a4d3f$0$12998$c3e8da3$5496439d@news.astraweb.com>
Message-ID: <201505310614.t4V6ErOf013371@fido.openend.se>

In a message of Sun, 31 May 2015 09:52:29 +1000, "Steven D'Aprano" writes:
>How many PyPy sandboxes are being used with hostile users motivated to break
>out of the sandbox?
>
>"I wrote a sandbox which I can't break out of" is different from "I wrote a
>sandbox which nobody can break out of". Javascript is sandboxed, but due to
>bugs in implementations, Javascript-based exploits are now heavily used by
>malware. There are possibly even more Javascript-based exploits than buffer
>overflow based exploits these days, as C programmers get better at using
>automated tools that check for buffer overflows.

I don't know, as we don't really have a way of tracking who is using
PyPy for anything.  We know we have some.

Laura



From lac at openend.se  Sun May 31 02:27:59 2015
From: lac at openend.se (Laura Creighton)
Date: Sun, 31 May 2015 08:27:59 +0200
Subject: Basic Info
In-Reply-To: Message from ch.tanveer13@gmail.com of "Sat,
 30 May 2015 22:35:25 -0700."
 <178f0eac-1760-40b3-9c10-c2d0075883a5@googlegroups.com>
References: <178f0eac-1760-40b3-9c10-c2d0075883a5@googlegroups.com>
Message-ID: <201505310627.t4V6Rxhl013979@fido.openend.se>

In a message of Sat, 30 May 2015 22:35:25 -0700, ch.tanveer13 at gmail.com writes:
>Hi friends,
>M Tanveer, and wanna start to learn python language, i've installed python on my Windows (OS) and set path to it, Now please Guide me which editor is best to use and what instructions should be followed . 
>Best Regards: Tanveeer Ahmad
>Thanks
>-- 
>https://mail.python.org/mailman/listinfo/python-list

Hello.  And welcome to Python.

Editors are a matter of personal choice.
https://wiki.python.org/moin/PythonEditors
You may also want an IDE
https://wiki.python.org/moin/IntegratedDevelopmentEnvironments

The webmaster account is really only for questions about the python
website.  For questions about learning python these mailing
lists are a better place to ask questions.

https://mail.python.org/mailman/listinfo/tutor
https://mail.python.org/mailman/listinfo/python-list

See you one of those places --

Laura Creighton


From kukki.kanchana at gmail.com  Sun May 31 03:02:35 2015
From: kukki.kanchana at gmail.com (Pythonista)
Date: Sun, 31 May 2015 00:02:35 -0700 (PDT)
Subject: Python Paramiko between Linux and Windows Server -- no output
 obtained from Windows for "dir" or "cd" commands
Message-ID: <b92b07c3-6d2a-4fc9-a780-5803b28dadc4@googlegroups.com>

Hello There:

I am using Python 2.7.6 and Paramiko module (on a Linux server) to connect to a Windows server and send some commands and get output. I have a connect function which takes IP, username and password of the remote Windows server and I get an sshobj when that happens. How do I use it to send remote calls is my question?

If it were a local system, I would just say "os.system" but not sure about the remote calls. Can someone help?

My code looks like below: sys.path.append("/home/me/code")

import libs.ssh_connect as ssh
ssh_obj = ssh.new_conn(IP, username, password)

stdin, stdout, stderr = ssh_obj.exec_command("dir") #since the remote    system I am SSHing into is Windows.

my "new_conn" looks like this:

import paramiko
def new_conn(IP, username, password):
    ssh_obj = paramiko.SSHClient()
    ssh_conn.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    ssh_conn.connect(IP, username, password), timeout=30)
    return ssh_obj

 	
if I replaced "dir" with "ipconfig" it works fine. I wonder how I can make "dir" work - or for that matter.. with "cd /path/to/dir"?


I know for paramiko's ssh connection to work, i need cygwin installed on a Windows. Objective is to run commands remotely from a Linux to a Windows server and then process the output again on Linux server or on Windows itself.

I am confused because, "ipconfig" sent from Linux to Windows using "ssh_obj.exec_command("ipconfig")" works but not "ssh_obj.exec_command("dir")" - tried giving the path like "cd C:\Users\Administrator" for cmd or "cd C:" followed by "cd Users/Administrator" like in Cygwin. Neither of them work.



From __peter__ at web.de  Sun May 31 08:18:30 2015
From: __peter__ at web.de (Peter Otten)
Date: Sun, 31 May 2015 14:18:30 +0200
Subject: Python Paramiko between Linux and Windows Server -- no output
 obtained from Windows for "dir" or "cd" commands
References: <b92b07c3-6d2a-4fc9-a780-5803b28dadc4@googlegroups.com>
Message-ID: <mkeu6o$7gi$1@ger.gmane.org>

Pythonista wrote:

> Hello There:
> 
> I am using Python 2.7.6 and Paramiko module (on a Linux server) to connect
> to a Windows server and send some commands and get output. I have a
> connect function which takes IP, username and password of the remote
> Windows server and I get an sshobj when that happens. How do I use it to
> send remote calls is my question?
> 
> If it were a local system, I would just say "os.system" but not sure about
> the remote calls. Can someone help?
> 
> My code looks like below: sys.path.append("/home/me/code")
> 
> import libs.ssh_connect as ssh
> ssh_obj = ssh.new_conn(IP, username, password)
> 
> stdin, stdout, stderr = ssh_obj.exec_command("dir") #since the remote   
> system I am SSHing into is Windows.
> 
> my "new_conn" looks like this:
> 
> import paramiko
> def new_conn(IP, username, password):
>     ssh_obj = paramiko.SSHClient()
>     ssh_conn.set_missing_host_key_policy(paramiko.AutoAddPolicy())
>     ssh_conn.connect(IP, username, password), timeout=30)
>     return ssh_obj
> 
>  
> if I replaced "dir" with "ipconfig" it works fine. I wonder how I can make
> "dir" work - or for that matter.. with "cd /path/to/dir"?


dir is an internal command of the windows shell. Try

ssh_obj.exec_command("cmd /c dir")

This will execute the dir command and then close the shell.

cd is also internal, but I don't think it makes sense to invoke it and then 
close the shell...


> I know for paramiko's ssh connection to work, i need cygwin installed on a
> Windows. Objective is to run commands remotely from a Linux to a Windows
> server and then process the output again on Linux server or on Windows
> itself.
> 
> I am confused because, "ipconfig" sent from Linux to Windows using
> "ssh_obj.exec_command("ipconfig")" works but not
> "ssh_obj.exec_command("dir")" - tried giving the path like "cd
> C:\Users\Administrator" for cmd or "cd C:" followed by "cd
> Users/Administrator" like in Cygwin. Neither of them work.




From Cecil at decebal.nl  Sun May 31 08:42:02 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 31 May 2015 14:42:02 +0200
Subject: Using Python instead of Bash
Message-ID: <87h9qsnckl.fsf@Equus.decebal.nl>

I help someone that has problems reading. For this I take photo's of
text, use convert from ImageMagick to make a good contrast (original
paper is grey) and use lpr to print it a little bigger.

Normally I would implement this in Bash, but I thought it a good idea
to implement it in Python. This is my first try:
    import glob
    import subprocess

    treshold = 66
    count = 0
    for input in sorted(glob.glob('*.JPG')):
        count += 1
        output = '{0:02d}.png'.format(count)
        print('Going to convert {0} to {1}'.format(input, output))
        p = subprocess.Popen(['convert', '-threshold', '{0}%'.format(treshold), input, output])
        p.wait()
        print('Going to print {0}'.format(output))
        p = subprocess.Popen(['lpr', '-o', 'fit-to-page', '-o', 'media=A4', output])
        p.wait()

There have to be some improvements: display before printing,
possibility to change threshold, ? But is this a good start, or should
I do it differently?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From skov83 at gmail.com  Sun May 31 09:35:05 2015
From: skov83 at gmail.com (Anders Johansen)
Date: Sun, 31 May 2015 06:35:05 -0700 (PDT)
Subject: Hello Group and how to practice?
Message-ID: <fd5b6515-97a0-40bd-bc44-8e07e7b045d1@googlegroups.com>

Hi my name is Anders I am from Denmark, and I am new to programming and python.

Currently, I am doing the codecademy.com python course, but sometime I feel that the course advances to fast and I lack repeating (practicing) some of the concepts, however I don't feel confident enough to start programming on my own.

Do you guys have some advice to how I can practicing programming and get the concept in under the skin?

Thanks! 


From alain at universite-de-strasbourg.fr.invalid  Sun May 31 10:02:59 2015
From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin)
Date: Sun, 31 May 2015 16:02:59 +0200
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
Message-ID: <87382cn8to.fsf@universite-de-strasbourg.fr.invalid>

Cecil Westerhof <Cecil at decebal.nl> writes:

> I help someone that has problems reading. For this I take photo's of
> text, use convert from ImageMagick to make a good contrast (original
> paper is grey) and use lpr to print it a little bigger.

>     import glob
>     import subprocess
>
>     treshold = 66
>     count = 0
>     for input in sorted(glob.glob('*.JPG')):
>         count += 1
>         output = '{0:02d}.png'.format(count)
>         print('Going to convert {0} to {1}'.format(input, output))
>         p = subprocess.Popen(['convert', '-threshold', '{0}%'.format(treshold), input, output])
>         p.wait()
>         print('Going to print {0}'.format(output))
>         p = subprocess.Popen(['lpr', '-o', 'fit-to-page', '-o', 'media=A4', output])
>         p.wait()

Maybe using check_call() would be simpler, but it will, well, check for
the exit code of convert/lpr (which you should do anyway). And I would
call that variable "threshold" instead of "treshold".

(What I don't see is the advantage you find in writing such scripts in
python instead of sh, but I guess you have your own reasons.)

-- Alain.


From kinsz89 at gmail.com  Sun May 31 10:09:44 2015
From: kinsz89 at gmail.com (Kin Zhao)
Date: Sun, 31 May 2015 10:09:44 -0400
Subject: Python Installation
Message-ID: <CAMziuNgWXgn35jYBi5QuNvd_FosNfjcWCbtJi_6zru=b=Qu7Uw@mail.gmail.com>

Hello,

I had a similar problem as Jim Conyngham (DieInSente) as shown in the link
below.

Uninstalling from the uninstall exe doesn't actually, fully uninstall the
program. It just removes it from the menu and the folder in All Program.
You'll have to uninstall with the initial installation exe. That's what I
did and it removed the wrong version for me so I can install the 64 bit
version of Python 3.4 on my computer. Can be a bit annoying, if you have to
figure out which exe you used after trying so many.

The error message comes up, because it's already downloaded. Even though it
says it's overwriting the older one, it doesn't actually overwrite it and
that's where the problem is.

Link: https://bugs.python.org/issue22028


Thanks,

Kin Zhao

C: (646) 204-2708 | E: kinsz89 at gmail.com

LinkedIn: www.linkedin.com/in/kinzhao1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150531/0ae03827/attachment.html>

From cfkaran2 at gmail.com  Sun May 31 10:14:29 2015
From: cfkaran2 at gmail.com (Cem Karan)
Date: Sun, 31 May 2015 10:14:29 -0400
Subject: Using Python instead of Bash
In-Reply-To: <87h9qsnckl.fsf@Equus.decebal.nl>
References: <87h9qsnckl.fsf@Equus.decebal.nl>
Message-ID: <29325179-A62F-40BF-B87C-BBFD372DA7C3@gmail.com>

> I help someone that has problems reading. For this I take photo's of
> text, use convert from ImageMagick to make a good contrast (original
> paper is grey) and use lpr to print it a little bigger.
> 
> Normally I would implement this in Bash, but I thought it a good idea
> to implement it in Python. This is my first try:
>    import glob
>    import subprocess
> 
>    treshold = 66
>    count = 0
>    for input in sorted(glob.glob('*.JPG')):
>        count += 1
>        output = '{0:02d}.png'.format(count)
>        print('Going to convert {0} to {1}'.format(input, output))
>        p = subprocess.Popen(['convert', '-threshold', '{0}%'.format(treshold), input, output])
>        p.wait()
>        print('Going to print {0}'.format(output))
>        p = subprocess.Popen(['lpr', '-o', 'fit-to-page', '-o', 'media=A4', output])
>        p.wait()
> 
> There have to be some improvements: display before printing,
> possibility to change threshold, ? But is this a good start, or should
> I do it differently?


As a first try, I think its pretty good, but to really answer your question, I think we could use a little more information.  

- Are you using python 2, or python 3?  There are slightly easier ways to do this using concurrent.futures objects, but they are only available under python 3. (See https://docs.python.org/3/library/concurrent.futures.html)

- In either case, subprocess.call(), subprocess.check_call(), or subprocess.check_output() may be easier to use.  That said, your code is perfectly fine!  The only real difference is that subprocess.call() will automatically wait for the call to complete, so you don't need to use p.wait() from above.  (See https://docs.python.org/2.7/library/subprocess.html, and https://docs.python.org/3/library/subprocess.html) 



The following codes does the conversion in parallel, and submits the jobs to the printer serially.  That should ensure that the printed output is also in sorted order, but you might want to double check before relying on it too much.  The major problem with it is that you can't display the output before printing; since everything is running in parallel, you'll have race conditions if you try.  **I DID NOT TEST THIS CODE, I JUST TYPED IT OUT IN MY MAIL CLIENT!**  Please test it carefully before relying on it!

"""
import subprocess
import concurrent.futures
import glob
import os.path

_THRESHOLD = 66

def _collect_filenames():
    files = glob.glob('*.JPG')

    # I build a set of the real paths so that if you have 
    # symbolic links that all point to the same file, they
    # they are automatically collapsed to a single file
    real_files = {os.path.realpath(x) for x in files}
    base_files = [os.path.splitext(x)[0] for x in real_files]
    return base_files

def _convert(base_file_name):
    """
    This code is slightly different from your code.  Instead
    of using numbers as names, I use the base name of file and
    append '.png' to it.  You may need to adjust this to ensure
    you don't overwrite anything.
    """
    input = base_file_name + ".JPG"
    output = base_file_name + ".png"
    subprocess.call(['convert', '-threshold', '{0}%'.format(_THRESHOLD), input, output])

def _print_files_in_order(base_files):
    base_files.sort()
    for f in base_files:
        output = f + ".png"
        subprocess.call(['lpr', '-o', 'fit-to-page', '-o', 'media=A4', output])

def driver():
    base_files = _collect_filenames()

    # If you use an executor as a context manager, then the
    # executor will wait until all of the submitted jobs finish
    # before it returns.  The submitted jobs will execute in
    # parallel.
    with concurrent.futures.ProcessPoolExecutor() as executor:
        for f in base_files:
            executor.submit(_convert_and_print, f)

    _print_files_in_order(base_files)
"""

Thanks,
Cem Karan

From cfkaran2 at gmail.com  Sun May 31 10:21:49 2015
From: cfkaran2 at gmail.com (Cem Karan)
Date: Sun, 31 May 2015 10:21:49 -0400
Subject: Hello Group and how to practice?
In-Reply-To: <fd5b6515-97a0-40bd-bc44-8e07e7b045d1@googlegroups.com>
References: <fd5b6515-97a0-40bd-bc44-8e07e7b045d1@googlegroups.com>
Message-ID: <72568971-7487-46C8-BEFC-3BFE25D5169A@gmail.com>


On May 31, 2015, at 9:35 AM, Anders Johansen <skov83 at gmail.com> wrote:

> Hi my name is Anders I am from Denmark, and I am new to programming and python.
> 
> Currently, I am doing the codecademy.com python course, but sometime I feel that the course advances to fast and I lack repeating (practicing) some of the concepts, however I don't feel confident enough to start programming on my own.
> 
> Do you guys have some advice to how I can practicing programming and get the concept in under the skin?

Choose something that you think is small and easy to do, and try to do it.  When you have trouble, read the docs you find online, and ask us questions; the python community is pretty friendly, and if you show us what you've already tried to do, someone is likely to try to help you out.

The main thing is to not get discouraged.  One of the hardest things to do is figuring out what you CAN do with a computer; some things that look like they should be easy, are actually major research questions.  Just keep trying, and it will get easier over time.

Good luck!
Cem Karan

From edd.cowan at gmail.com  Sun May 31 10:34:04 2015
From: edd.cowan at gmail.com (Eddilbert Macharia)
Date: Sun, 31 May 2015 07:34:04 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
Message-ID: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>

Hello All ,

I'm wrecking my head trying to understand. where the class object comes into play . 

Is it only meant to act as base class and does it mean there is an actual class called object in python which all the objects created using the class type inherit ?

i'm assuming the metaclass if simplified would look something like this :

type('dict', (object,),{})

And when we use the class type as a metaclass are we using the instance version of the class type or are we actually using the type class itself ?

Also when we say everything is an object in python, are we referring to the fact that everything is an instance of the class type or does it have to with the object class inherited ? 

As can be attested by using type() function as below :

>>> type(int)
<class 'type'>
>>> type(list)
<class 'type'>
>>> type(dict)
<class 'type'>
>>> type(type)
<class 'type'>
>>> type(object)
<class 'type'>

>From my understanding this means all of this are instances of the class type. which means the class type was used to create this instances.

Now if i look at the __bases__ of all this objects i get :

>>> type.__base__
<class 'object'>
>>> type.__bases__
(<class 'object'>,)
>>> dict.__bases__
(<class 'object'>,)
>>> list.__bases__
(<class 'object'>,)
>>> int.__bases__
(<class 'object'>,)
>>> object.__bases__
()

This tells me that all of this objects inherit from the class object which has nothing to do with them being instances.


From skov83 at gmail.com  Sun May 31 10:51:50 2015
From: skov83 at gmail.com (Anders Johansen)
Date: Sun, 31 May 2015 07:51:50 -0700 (PDT)
Subject: Hello Group and how to practice?
In-Reply-To: <mailman.245.1433082118.5151.python-list@python.org>
References: <fd5b6515-97a0-40bd-bc44-8e07e7b045d1@googlegroups.com>
 <mailman.245.1433082118.5151.python-list@python.org>
Message-ID: <5ba559fe-b633-4f7d-8584-58227ebb98f0@googlegroups.com>

Den s?ndag den 31. maj 2015 kl. 16.22.10 UTC+2 skrev Cem Karan:
> On May 31, 2015, at 9:35 AM, Anders Johansen <skov83 at gmail.com> wrote:
> 
> > Hi my name is Anders I am from Denmark, and I am new to programming and python.
> > 
> > Currently, I am doing the codecademy.com python course, but sometime I feel that the course advances to fast and I lack repeating (practicing) some of the concepts, however I don't feel confident enough to start programming on my own.
> > 
> > Do you guys have some advice to how I can practicing programming and get the concept in under the skin?
> 
> Choose something that you think is small and easy to do, and try to do it.  When you have trouble, read the docs you find online, and ask us questions; the python community is pretty friendly, and if you show us what you've already tried to do, someone is likely to try to help you out.
> 
> The main thing is to not get discouraged.  One of the hardest things to do is figuring out what you CAN do with a computer; some things that look like they should be easy, are actually major research questions.  Just keep trying, and it will get easier over time.
> 
> Good luck!
> Cem Karan

Thank you Cem Karan for your reply. I will try and follow your advice. I am yet to install python on my computer, do you know of any easy to follow instructions on how to do so? Where do I start?


From cfkaran2 at gmail.com  Sun May 31 11:02:34 2015
From: cfkaran2 at gmail.com (Cem Karan)
Date: Sun, 31 May 2015 11:02:34 -0400
Subject: Hello Group and how to practice?
In-Reply-To: <5ba559fe-b633-4f7d-8584-58227ebb98f0@googlegroups.com>
References: <fd5b6515-97a0-40bd-bc44-8e07e7b045d1@googlegroups.com>
 <mailman.245.1433082118.5151.python-list@python.org>
 <5ba559fe-b633-4f7d-8584-58227ebb98f0@googlegroups.com>
Message-ID: <AD126E6F-8A05-40B5-8390-1FC7CFC75CC7@gmail.com>


On May 31, 2015, at 10:51 AM, Anders Johansen <skov83 at gmail.com> wrote:

> Den s?ndag den 31. maj 2015 kl. 16.22.10 UTC+2 skrev Cem Karan:
>> On May 31, 2015, at 9:35 AM, Anders Johansen <skov83 at gmail.com> wrote:
>> 
>>> Hi my name is Anders I am from Denmark, and I am new to programming and python.
>>> 
>>> Currently, I am doing the codecademy.com python course, but sometime I feel that the course advances to fast and I lack repeating (practicing) some of the concepts, however I don't feel confident enough to start programming on my own.
>>> 
>>> Do you guys have some advice to how I can practicing programming and get the concept in under the skin?
>> 
>> Choose something that you think is small and easy to do, and try to do it.  When you have trouble, read the docs you find online, and ask us questions; the python community is pretty friendly, and if you show us what you've already tried to do, someone is likely to try to help you out.
>> 
>> The main thing is to not get discouraged.  One of the hardest things to do is figuring out what you CAN do with a computer; some things that look like they should be easy, are actually major research questions.  Just keep trying, and it will get easier over time.
>> 
>> Good luck!
>> Cem Karan
> 
> Thank you Cem Karan for your reply. I will try and follow your advice. I am yet to install python on my computer, do you know of any easy to follow instructions on how to do so? Where do I start?

Python 3 installers are here: https://www.python.org/downloads/release/python-343/  Choose the one appropriate for your system, and follow the instructions.  If you have trouble, write to the list.

Good luck,
Cem Karan

From breamoreboy at yahoo.co.uk  Sun May 31 11:12:01 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 31 May 2015 16:12:01 +0100
Subject: Basic Info
In-Reply-To: <178f0eac-1760-40b3-9c10-c2d0075883a5@googlegroups.com>
References: <178f0eac-1760-40b3-9c10-c2d0075883a5@googlegroups.com>
Message-ID: <mkf8c3$ul5$1@ger.gmane.org>

On 31/05/2015 06:35, ch.tanveer13 at gmail.com wrote:
> Hi friends,
> M Tanveer, and wanna start to learn python language, i've installed python on my Windows (OS) and set path to it, Now please Guide me which editor is best to use and what instructions should be followed .
> Best Regards: Tanveeer Ahmad
> Thanks
>

If pywin32 doesn't come preinstalled grab it from here 
http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/

There's a perfectly good editor to get you going, which certainly saves 
looking at all the other options :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From marco.buttu at gmail.com  Sun May 31 11:13:20 2015
From: marco.buttu at gmail.com (Marco Buttu)
Date: Sun, 31 May 2015 17:13:20 +0200
Subject: Everything is an object in python - object class and type class
In-Reply-To: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
Message-ID: <556B2510.7020609@oa-cagliari.inaf.it>

On 31/05/2015 16:34, Eddilbert Macharia wrote:

> Also when we say everything is an object in python, are we referring to the fact that everything is an instance of the class type or does it have to with the object class inherited ?

 From Wikipedia: "In the class-based object-oriented programming 
paradigm, `object` refers to a particular instance of a class"

So, because in Python everything is an instance of the class object 
(isinstance(obj, object) is always True), everything is an object.
Only classes are instance of type. That means is worth to differentiate 
between classes and non-classes, but it is a non-sense to differentiate 
between classes and objects. If it could be useful:

http://marco-buttu.github.io/pycon_objmodel/

> As can be attested by using type() function as below :
>
>>>> >>>type(int)
> <class 'type'>
>>>> >>>type(list)
...
> >From my understanding this means all of this are instances of the 
> class type. which means the class type was used to create this instances. 

Yes, that is true, because all these objects are classes

> Now if i look at the __bases__ of all this objects i get :
>>>> >>>type.__base__
> <class 'object'>
>>>> >>>type.__bases__
> (<class 'object'>,)
> ...

> This tells me that all of this objects inherit from the class object which has nothing to do with them being instances.

Take a look at all the steps that anticipate this diagram:

http://marco-buttu.github.io/pycon_objmodel/?full#Diagram

Every class inherits from object, and that implies every instance of a 
class is an instance of object. And classes are instance of a class 
(thier metaclass), so they are instance of objects too

-- 
Marco Buttu

INAF-Osservatorio Astronomico di Cagliari
Via della Scienza n. 5, 09047 Selargius (CA)
Phone: 070 711 80 217
Email: mbuttu at oa-cagliari.inaf.it



From tjreedy at udel.edu  Sun May 31 11:37:08 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 31 May 2015 11:37:08 -0400
Subject: Everything is an object in python - object class and type class
In-Reply-To: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
Message-ID: <mkf9rv$l9d$1@ger.gmane.org>

On 5/31/2015 10:34 AM, Eddilbert Macharia wrote:

> I'm wrecking my head trying to understand. where the class object comes into play .

Leaving custom metaclasses aside.

1. All python data values are instances of some class.

2. Every class is a instance of the metaclass 'type'.

3. Every class is a subclass of the base class 'object'.

Some implications.

4. All objects, including classes, get their basic instance behaviors 
from 'object', except as over-riden by other classes in the inheritance 
chain.  These basic behaviors include default comparisons and 
representations.  See 'dir(object)'.

5. Classes get their special class behaviors from 'type'.

-- 
Terry Jan Reedy



From Cecil at decebal.nl  Sun May 31 12:36:47 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 31 May 2015 18:36:47 +0200
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <87382cn8to.fsf@universite-de-strasbourg.fr.invalid>
Message-ID: <87bnh0n1pc.fsf@Equus.decebal.nl>

Op Sunday 31 May 2015 16:02 CEST schreef Alain Ketterlin:

> Cecil Westerhof <Cecil at decebal.nl> writes:
>
>> I help someone that has problems reading. For this I take photo's
>> of text, use convert from ImageMagick to make a good contrast
>> (original paper is grey) and use lpr to print it a little bigger.
>
>> import glob
>> import subprocess
>>
>> treshold = 66 count = 0 for input in sorted(glob.glob('*.JPG')):
>> count += 1 output = '{0:02d}.png'.format(count) print('Going to
>> convert {0} to {1}'.format(input, output)) p =
>> subprocess.Popen(['convert', '-threshold', '{0}%'.format(treshold),
>> input, output]) p.wait() print('Going to print {0}'.format(output))
>> p = subprocess.Popen(['lpr', '-o', 'fit-to-page', '-o', 'media=A4',
>> output]) p.wait()
>
> Maybe using check_call() would be simpler, but it will, well, check
> for the exit code of convert/lpr (which you should do anyway).

That is a lot better yes. I did not like that I needed two calls.


> And I would call that variable "threshold" instead of "treshold".

Yep, I saw the mistake after posting. :-(


> (What I don't see is the advantage you find in writing such scripts
> in python instead of sh, but I guess you have your own reasons.)

Several.

The first is that I want to get some Python experience.

But later on I want to display the converted photo and if it is not
correct, change the threshold and convert it again with a new
threshold. 66% is good most of the time, but not always. Sometimes it
has to be bigger or smaller.

And after that I want to put a GUI around it.

So, that is why I am using Python.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From mal at europython.eu  Sun May 31 12:52:07 2015
From: mal at europython.eu (M.-A. Lemburg)
Date: Sun, 31 May 2015 18:52:07 +0200
Subject: EuroPython 2015: More than 200 sessions waiting for you
Message-ID: <556B3C37.9020301@europython.eu>

We are pleased to announce the list of accepted sessions for
EuroPython 2015 in Bilbao:

                 *** EuroPython 2015 Session List ***

          https://ep2015.europython.eu/en/events/sessions/

The sessions were selected on the basis of your talk voting and the
work of the EuroPython program work group.

>From the over 340 proposals, 205 sessions were chosen for EuroPython 2015:

 * 169 talks
 * 11 posters
 * 20 trainings
 * 5 help desks

Many thanks to everyone who submitted proposals. EuroPython wouldn?t
be possible without our speakers.

The program work group will now work on the schedule. Given the number
of sessions, this may take a while, but we?ll try to get it done as
quickly as possible.

Enjoy,
--
EuroPython 2015 Team
http://ep2015.europython.eu/
http://www.europython-society.org/


From Cecil at decebal.nl  Sun May 31 12:58:00 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 31 May 2015 18:58:00 +0200
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <mailman.244.1433081674.5151.python-list@python.org>
Message-ID: <877fron0pz.fsf@Equus.decebal.nl>

Op Sunday 31 May 2015 16:14 CEST schreef Cem Karan:

>> I help someone that has problems reading. For this I take photo's
>> of text, use convert from ImageMagick to make a good contrast
>> (original paper is grey) and use lpr to print it a little bigger.
>>
>> Normally I would implement this in Bash, but I thought it a good
>> idea to implement it in Python. This is my first try: import glob
>> import subprocess
>>
>> treshold = 66 count = 0 for input in sorted(glob.glob('*.JPG')):
>> count += 1 output = '{0:02d}.png'.format(count) print('Going to
>> convert {0} to {1}'.format(input, output)) p =
>> subprocess.Popen(['convert', '-threshold', '{0}%'.format(treshold),
>> input, output]) p.wait() print('Going to print {0}'.format(output))
>> p = subprocess.Popen(['lpr', '-o', 'fit-to-page', '-o', 'media=A4',
>> output]) p.wait()
>>
>> There have to be some improvements: display before printing,
>> possibility to change threshold, ? But is this a good start, or
>> should I do it differently?
>
>
> As a first try, I think its pretty good, but to really answer your
> question, I think we could use a little more information.
>
> - Are you using python 2, or python 3? There are slightly easier
>   ways to do this using concurrent.futures objects, but they are
>   only available under python 3. (See
>   https://docs.python.org/3/library/concurrent.futures.html)

In principal I am using Python 3, but I prefer it to work also under
Python 2.

concurrent.futures is definitely something to look at, but not in this
case I think. First of all the processing does not take much time.
Secondly the order of the printing is important. Lastly, it is not
done now, but the conversion should be checked and maybe done over
with another threshold.


> - In either case, subprocess.call(), subprocess.check_call(), or
>   subprocess.check_output() may be easier to use.

I am using subprocess.check_call now. Much better.


> The following codes does the conversion in parallel, and submits the
> jobs to the printer serially.

Not needed now, but I file it for later use. ;-)


> def _convert(base_file_name): """ This code is slightly different
> from your code. Instead of using numbers as names, I use the base
> name of file and append '.png' to it. You may need to adjust this to

The reason I use numbers is that what I am printing is a booklet. The
input is IMG_4769.JPG and then I find 01.png much better, because then
I know which page it is. :-D


Thanks for the quit interesting read.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From Cecil at decebal.nl  Sun May 31 13:06:40 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 31 May 2015 19:06:40 +0200
Subject: Using Python instead of Bash
References: <87h9qsnckl.fsf@Equus.decebal.nl>
 <mailman.247.1433084763.5151.python-list@python.org>
Message-ID: <87382cn0bj.fsf@Equus.decebal.nl>

Op Sunday 31 May 2015 17:05 CEST schreef Dennis Lee Bieber:

> On Sun, 31 May 2015 14:42:02 +0200, Cecil Westerhof
> <Cecil at decebal.nl> declaimed the following:
>
>> I help someone that has problems reading. For this I take photo's
>> of text, use convert from ImageMagick to make a good contrast
>> (original paper is grey) and use lpr to print it a little bigger.
>>
>> Normally I would implement this in Bash, but I thought it a good
>> idea to implement it in Python. This is my first try:
>
> 	<code snipped>
>
> 	My first impression is that all you did was convert BASH line
> for line to Python -- and are still passing all the real work over
> to external programs ("convert" and "lpr"), using subprocess...
> Instead of a BASH script alternating between running command line
> utilities convert and lpr, you have a BASH (or other) shell running
> a Python process and Python alternating between a pair of command
> line utilities with all the overhead of creating and terminating
> processes.

That is true (the conversion). But there are two reasons for this.
- I want to get some experience with Python.
- I want to implement extra functionality, which would be easier with
Python as with Bash.

Also: I am thinking about using ipython3 instead of bash.


> 	I'll concede submitting jobs to a print queue probably does work
> easier this way, but I'd have to ask if the "convert" could be
> performed using actual Python -- the old Python Imaging Library --
> PIL -- for example (superseded, I think, by pillow)?

That is something to look into. My experience with ImageMagick is very
good, but if Python could do an equal good job, then it would be good
to switch. That also removes a dependency.


> 	Given that you do nothing between creating a subprocess and
> waiting for it to exit, subprocess.Popen() could be overkill -- the
> simpler subprocess.call() could be sufficient (heck, the old
> os.system() is sufficient, but preference seems to be to ignore it
> for subprocess.call() instead).

I already use check_call.


By the way: because of the lpr my script only works on Linux (and
probably Unix and Apple). Can I rewrite it in such a way that it also
works with Windows? Someone else has to test it then, because I do not
have a Windows computer.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From steve at pearwood.info  Sun May 31 14:00:49 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 01 Jun 2015 04:00:49 +1000
Subject: Everything is an object in python - object class and type class
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
Message-ID: <556b4c52$0$13013$c3e8da3$5496439d@news.astraweb.com>

On Mon, 1 Jun 2015 12:34 am, Eddilbert Macharia wrote:

> Hello All ,
> 
> I'm wrecking my head trying to understand. where the class object comes
> into play .

This is just the standard object-oriented concept of class and instances.
Have you programmed in any other OO language? 

Lassie is an individual example of a dog. Lassie is an instance, dog is the
class.

42 is an individual example of an int (integer). 42 is an instance, int is
the class (also known as type).

In some languages, like Java, and Javascript, there are two "kinds" of
values: objects, and primitives. Primitives are the sorts of values you use
in low-level languages like C, Fortran or Pascal, or in assembly language:
typically things like numbers, arrays, strings, booleans (True and False).
These are sometimes called "unboxed" values, "boxed" values being another
name for objects.

Python is not like those languages -- all values in Python are objects, in
the sense of object-oriented programming.



> Is it only meant to act as base class and does it mean there is an actual
> class called object in python which all the objects created using the
> class type inherit ?

Yes, there is an actual class called "object" in Python:

py> object
<class 'object'>

In Python 3, all values are objects in BOTH the sense of OO programming, and
the sense that the inherit from the object class.

In Python 2, for historical reasons, *some* classes do not inherit from
object, but form a separate hierarchy called "classic" or "old-style"
classes. It's best to ignore them, if you can, since they add complexity
for little value. I will say no more about these classic classes, although
if you are really curious, ask and someone (possibly me) will answer.


> i'm assuming the metaclass if simplified would look something like this :
> 
> type('dict', (object,),{})
> 
> And when we use the class type as a metaclass are we using the instance
> version of the class type or are we actually using the type class itself ?

I don't understand what you are trying to ask.


Every instance belongs to a class: 42 is an instance of int, "spam and eggs"
is an instance of str. In Python, classes themselves are also values, and
hence are instances of some other class. This class-of-the-class is called
the metaclass.

int is a class, int is an instance of the metaclass called "type";
str is a class, str is an instance of the metaclass called "type";

etc. Normally, all the classes you come across will belong to the same
metaclass, type.

Since type itself is a class, what's its metaclass? (The "metametaclass",
you might say, although we never do :-) Well, type is special. The chain of
instance <- class <- metaclass <- metametaclass <- ... cannot go on
forever, and so the Python interpreter treats type as special, so that the
chain of meta-meta-meta-meta...classes has a beginning:

type -> int -> 42

The class of type is itself!

py> type(type) is type
True

So, when you create your own class:

class Spam(object):
    def method(self):
        return "spam spam spam"


Python treats this as syntactic sugar for a call to type:

Spam = type("Spam", (object,), {'method': function-object})


That's right, type() does two things! With a single argument, it returns the
type (class) of the value passed in:

py> type(42)
<class 'int'>


With three arguments, it creates a new class.

So if you use a metaclass, Python simply replaces the call to type(...) with
a call to the metaclass.

Typically, you won't use metaclasses *at all*, but for those rare times when
you do, the usual pattern is:

- create a subclass of type that does something different;
- specify that as the metaclass.


class Meta(type):
    ...


# Python 3 syntax
class MyClass(object, metaclass=Meta):
    ...

# Python 2 syntax
class MyClass(object):
    __metaclass__ = Meta
    ...


Then, instead of the MyClass class definition being syntactic sugar for:


MyClass = type("MyClass", (object,), {...})  # the default

it becomes:

MyClass = Meta("MyClass", (object,), {...})

So now MyClass is an instance of Meta, which is a subclass of type.


That's (almost!) all there is to it. I've actually simplified a few of the
hairier details and ignored some of the complexity, but as a broad
overview, this is about everything you need to know about metaclasses.



> Also when we say everything is an object in python, are we referring to
> the fact that everything is an instance of the class type or does it have
> to with the object class inherited ?

Yes to both of those!



> As can be attested by using type() function as below :
[...] 
> From my understanding this means all of this are instances of the class
> type. which means the class type was used to create this instances.

Correct.


> Now if i look at the __bases__ of all this objects i get :
> 
>>>> type.__base__
> <class 'object'>
>>>> type.__bases__
> (<class 'object'>,)
>>>> dict.__bases__
> (<class 'object'>,)
>>>> list.__bases__
> (<class 'object'>,)
>>>> int.__bases__
> (<class 'object'>,)
>>>> object.__bases__
> ()
> 
> This tells me that all of this objects inherit from the class object which
> has nothing to do with them being instances.

Also correct.




-- 
Steven



From rxjwg98 at gmail.com  Sun May 31 14:36:35 2015
From: rxjwg98 at gmail.com (fl)
Date: Sun, 31 May 2015 11:36:35 -0700 (PDT)
Subject: Are there any other better ways to access a single bit of string of
 digits?
Message-ID: <68680e6e-b914-4bef-b16b-1fa48b00eaf2@googlegroups.com>

Hi,

I am new to Python. I would manipulate a string of hex numbers. If the first
digit is bigger than 7, the first two digits are required to add 4.

For example, '8022_3345' will be changed to '8422_3345'. The 
underscore between two 4-digit's was generated previously (i.e.
it is already in the .txt file).

I have not tried to read the .txt file to a list yet. I just try the
following:

tmp
['12345678', '23456789', '3456789a', '456789ab']

Each 8-digit hex number is assigned to a variable, such as:

digit8=tmp[0]

 
I can compare digit8[0] with 7, and so on...



The underscore, I think, can be removed by first a string replace.

My question here is:

Do you have better ways than my tmp, digit8 operation?


Thanks,


From mmanns at gmx.net  Sun May 31 15:05:13 2015
From: mmanns at gmx.net (Martin Manns)
Date: Sun, 31 May 2015 21:05:13 +0200
Subject: [ANN] pyspread 0.4.3
Message-ID: <mailman.272.1433143247.5151.python-announce-list@python.org>

==============
pyspread 0.4.3
==============

Pyspread 0.4.3 is released.

Please note that pyspread 0.4.2 is not available on pypi because of a
deployment issue.

Major changes to 0.4.1:

 * Full screen mode added
 * Some changes to improve results on Debian CI tests
 * Updates to first steps document
 * Linux distribution logos removed from first steps document


About pyspread
==============

Pyspread is a non-traditional spreadsheet application that is based on
and written in the programming language Python. 

The goal of pyspread is to be the most pythonic spreadsheet application.

Pyspread is free software. It is released under the GPL v3.

Project website: https://manns.github.io/pyspread/
Download page:   https://pypi.python.org/pypi/pyspread
Source code:     https://github.com/manns/pyspread

Enjoy

Martin


From rxjwg98 at gmail.com  Sun May 31 15:40:19 2015
From: rxjwg98 at gmail.com (fl)
Date: Sun, 31 May 2015 12:40:19 -0700 (PDT)
Subject: What use for reversed()?
Message-ID: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>

Hi,

I have a string b='1234'. I run: br=reversed(b)

I hope that I can print out '4321' by:

for br in b

but it complains:
SyntaxError: invalid syntax


My questions:
1. What use for reversed(). I do not find an example on web.

2. If reversed() is wrong the my purpose, what method can do it? i.e. '4321'
out.


Thanks,


From denismfmcmahon at gmail.com  Sun May 31 15:51:43 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sun, 31 May 2015 19:51:43 +0000 (UTC)
Subject: Are there any other better ways to access a single bit of
 string of digits?
References: <68680e6e-b914-4bef-b16b-1fa48b00eaf2@googlegroups.com>
Message-ID: <mkfoof$lkf$2@dont-email.me>

On Sun, 31 May 2015 11:36:35 -0700, fl wrote:

> I am new to Python. I would manipulate a string of hex numbers. If the
> first digit is bigger than 7, the first two digits are required to add
> 4.

What happens if the first two digits are ff, does it become 103 or 03.

If you have ffff_ffff_ffff

Do you want to create 103ff_103ff_103ff

or

03ff_03ff_03ff

or

0400_0400_03ff

or

10400_0400_03ff ......

> For example, '8022_3345' will be changed to '8422_3345'. The underscore
> between two 4-digit's was generated previously (i.e.
> it is already in the .txt file).
> 
> I have not tried to read the .txt file to a list yet. I just try the
> following:
> 
> tmp ['12345678', '23456789', '3456789a', '456789ab']
> 
> Each 8-digit hex number is assigned to a variable, such as:
> 
> digit8=tmp[0]
> 
> I can compare digit8[0] with 7, and so on...
> 
> The underscore, I think, can be removed by first a string replace.
> 
> My question here is:
> 
> Do you have better ways than my tmp, digit8 operation?

Yes, if these are numbers, manipulate them as numbers, not strings.

def bing(n):
    n = int(n.replace("_", ""), base=16)  # convert to numbers
    if n > 0x7fffffff:                    # if 0x80000000 or more
        n = n + 0x04000000                # add 0x04000000
    return n                              # and return result

newnums = [ bing(x) for x in oldnums ]

It could probably be done as a single list comprehension, but it might 
get a bit messy.

-- 
Denis McMahon, denismfmcmahon at gmail.com


From denismfmcmahon at gmail.com  Sun May 31 15:58:19 2015
From: denismfmcmahon at gmail.com (Denis McMahon)
Date: Sun, 31 May 2015 19:58:19 +0000 (UTC)
Subject: What use for reversed()?
References: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>
Message-ID: <mkfp4q$lkf$3@dont-email.me>

On Sun, 31 May 2015 12:40:19 -0700, fl wrote:

> Hi,
> 
> I have a string b='1234'. I run: br=reversed(b)
> 
> I hope that I can print out '4321' by:
> 
> for br in b
> 
> but it complains:
> SyntaxError: invalid syntax
> 
> 
> My questions:
> 1. What use for reversed(). I do not find an example on web.
> 
> 2. If reversed() is wrong the my purpose, what method can do it? i.e.
> '4321'
> out.

reversed returns an iterator, not a list, so it returns the reversed list 
of elements one at a time. You can use list() or create a list from 
reversed and then join the result:

$ python
Python 2.7.3 (default, Dec 18 2014, 19:10:20) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> "".join(list(reversed("fred")))
'derf'
>>> "".join([x for x in reversed("fred")])
'derf'

So reversed can do it, but needs a little help

-- 
Denis McMahon, denismfmcmahon at gmail.com


From rxjwg98 at gmail.com  Sun May 31 16:08:26 2015
From: rxjwg98 at gmail.com (fl)
Date: Sun, 31 May 2015 13:08:26 -0700 (PDT)
Subject: Are there any other better ways to access a single bit of string
 of digits?
In-Reply-To: <mkfoof$lkf$2@dont-email.me>
References: <68680e6e-b914-4bef-b16b-1fa48b00eaf2@googlegroups.com>
 <mkfoof$lkf$2@dont-email.me>
Message-ID: <656ab926-ad6e-4050-96fd-0054551f2be8@googlegroups.com>

On Sunday, May 31, 2015 at 12:53:19 PM UTC-7, Denis McMahon wrote:
> On Sun, 31 May 2015 11:36:35 -0700, fl wrote:
> > I am new to Python. I would manipulate a string of hex numbers. If the
> > first digit is bigger than 7, the first two digits are required to add
> > 4.
> What happens if the first two digits are ff, does it become 103 or 03.
> If you have ffff_ffff_ffff
> Do you want to create 103ff_103ff_103ff
> or
> 03ff_03ff_03ff
> or
> newnums = [ bing(x) for x in oldnums ]
> 
> It could probably be done as a single list comprehension, but it might 
> get a bit messy.
> 
> -- 
> Denis McMahon, denismfmcmahon at gmail.com

Thanks for your reply. In fact, these data are from a 256 entries table. 
There is no extreme big data, such as Fxxxx. 


From kukki.kanchana at gmail.com  Sun May 31 16:12:24 2015
From: kukki.kanchana at gmail.com (Pythonista)
Date: Sun, 31 May 2015 13:12:24 -0700 (PDT)
Subject: Python Paramiko between Linux and Windows Server -- no output
 obtained from Windows for "dir" or "cd" commands
In-Reply-To: <mailman.242.1433074722.5151.python-list@python.org>
References: <b92b07c3-6d2a-4fc9-a780-5803b28dadc4@googlegroups.com>
 <mailman.242.1433074722.5151.python-list@python.org>
Message-ID: <3a4707d8-46eb-4ba8-a6d9-3689b3c544f4@googlegroups.com>

On Sunday, May 31, 2015 at 5:18:53 AM UTC-7, Peter Otten wrote:
> Pythonista wrote:
> 
> > Hello There:
> > 
> > I am using Python 2.7.6 and Paramiko module (on a Linux server) to connect
> > to a Windows server and send some commands and get output. I have a
> > connect function which takes IP, username and password of the remote
> > Windows server and I get an sshobj when that happens. How do I use it to
> > send remote calls is my question?
> > 
> > If it were a local system, I would just say "os.system" but not sure about
> > the remote calls. Can someone help?
> > 
> > My code looks like below: sys.path.append("/home/me/code")
> > 
> > import libs.ssh_connect as ssh
> > ssh_obj = ssh.new_conn(IP, username, password)
> > 
> > stdin, stdout, stderr = ssh_obj.exec_command("dir") #since the remote   
> > system I am SSHing into is Windows.
> > 
> > my "new_conn" looks like this:
> > 
> > import paramiko
> > def new_conn(IP, username, password):
> >     ssh_obj = paramiko.SSHClient()
> >     ssh_conn.set_missing_host_key_policy(paramiko.AutoAddPolicy())
> >     ssh_conn.connect(IP, username, password), timeout=30)
> >     return ssh_obj
> > 
> >  
> > if I replaced "dir" with "ipconfig" it works fine. I wonder how I can make
> > "dir" work - or for that matter.. with "cd /path/to/dir"?
> 
> 
> dir is an internal command of the windows shell. Try
> 
> ssh_obj.exec_command("cmd /c dir")
> 
> This will execute the dir command and then close the shell.
> 
> cd is also internal, but I don't think it makes sense to invoke it and then 
> close the shell...
> 
> 
> > I know for paramiko's ssh connection to work, i need cygwin installed on a
> > Windows. Objective is to run commands remotely from a Linux to a Windows
> > server and then process the output again on Linux server or on Windows
> > itself.
> > 
> > I am confused because, "ipconfig" sent from Linux to Windows using
> > "ssh_obj.exec_command("ipconfig")" works but not
> > "ssh_obj.exec_command("dir")" - tried giving the path like "cd
> > C:\Users\Administrator" for cmd or "cd C:" followed by "cd
> > Users/Administrator" like in Cygwin. Neither of them work.

Thanks Peter but I got no output from your suggestion either.!


From edd.cowan at gmail.com  Sun May 31 16:33:26 2015
From: edd.cowan at gmail.com (Eddilbert Macharia)
Date: Sun, 31 May 2015 13:33:26 -0700 (PDT)
Subject: Basic Info
In-Reply-To: <178f0eac-1760-40b3-9c10-c2d0075883a5@googlegroups.com>
References: <178f0eac-1760-40b3-9c10-c2d0075883a5@googlegroups.com>
Message-ID: <6a695101-2911-41fd-a2e7-812f5a322f2d@googlegroups.com>

On Sunday, May 31, 2015 at 8:35:42 AM UTC+3, Tanveer Ahmad wrote:
> Hi friends,
> M Tanveer, and wanna start to learn python language, i've installed python on my Windows (OS) and set path to it, Now please Guide me which editor is best to use and what instructions should be followed . 
> Best Regards: Tanveeer Ahmad
> Thanks

Hello i was where you are last year, and as of now by far my favorite editor is jetbrains pycharm https://www.jetbrains.com/pycharm/download/

actual i like there editors because they have a consistent interface from intelij,phpstorm to pycharm. give it a try


From supportNOSPAM at microsoft.com  Sun May 31 17:26:08 2015
From: supportNOSPAM at microsoft.com (cheshire)
Date: Sun, 31 May 2015 23:26:08 +0200
Subject: What use for reversed()?
References: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>
Message-ID: <1e45fkwb94gzo$.1rtdbc942jqny.dlg@40tude.net>

On Sun, 31 May 2015 12:40:19 -0700 (PDT), fl wrote:

> 2. If reversed() is wrong the my purpose, what method can do it? i.e. '4321'
> out.

Try using slices:

>>> b='1234'
>>> b[::-1]
'4321'

https://docs.python.org/2/whatsnew/2.3.html#extended-slices


From Cecil at decebal.nl  Sun May 31 17:33:36 2015
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Sun, 31 May 2015 23:33:36 +0200
Subject: Best way to prevent zombie processes
Message-ID: <87vbf8l9e7.fsf@Equus.decebal.nl>

At the moment I have the following code:
    os.chdir(directory)
    for document in documents:
        subprocess.Popen(['evince', document])

With this I can open several documents at once. But there is no way to
know when those documents are going to be closed. This could/will lead
to zombie processes. (I run it on Linux.) What is the best solution to
circumvent this?

I was thinking about putting all Popen instances in a list. And then
every five minutes walk through the list and check with poll if the
process has terminated. If it has it can be released from the list.
Of-course I need to synchronise those events. Is that a good way to do
it?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof


From marko at pacujo.net  Sun May 31 18:22:32 2015
From: marko at pacujo.net (Marko Rauhamaa)
Date: Mon, 01 Jun 2015 01:22:32 +0300
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
Message-ID: <87vbf88k0n.fsf@elektro.pacujo.net>

Cecil Westerhof <Cecil at decebal.nl>:

> At the moment I have the following code:
>     os.chdir(directory)
>     for document in documents:
>         subprocess.Popen(['evince', document])
>
> With this I can open several documents at once. But there is no way to
> know when those documents are going to be closed. This could/will lead
> to zombie processes. (I run it on Linux.) What is the best solution to
> circumvent this?
>
> I was thinking about putting all Popen instances in a list. And then
> every five minutes walk through the list and check with poll if the
> process has terminated. If it has it can be released from the list.
> Of-course I need to synchronise those events. Is that a good way to do
> it?

If you don't care to know when child processes exit, you can simply
ignore the SIGCHLD signal:

    import signal
    signal.signal(signal.SIGCHLD, signal.SIG_IGN)

That will prevent zombies from appearing.

On the other hand, if you want to know when a process exits, you have
several options:

 * What you propose would work, but is all but elegant. You want to
   react as soon as child processes die.

 * You could actually trap the SIGCHLD signal by setting a signal
   handler. You should not do any actual processing in the signal
   handler itself but rather convert the signal into a file descriptor
   event (with a pipe; Python doesn't seem to support signal file
   descriptors or event file descriptors).

 * Instead of a using a signal handler, you could capture the output of
   the child process. At its simplest, you capture the standard output,
   but you could also open an extra pipe for the purpose. You keep
   reading the standard output and as soon as you read an EOF, you wait
   the process out.


Marko


From cs at zip.com.au  Sun May 31 18:57:13 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 1 Jun 2015 08:57:13 +1000
Subject: Using Python instead of Bash
In-Reply-To: <bn7mmadupkolhvttkok4dbsa71b9j81gmj@4ax.com>
References: <bn7mmadupkolhvttkok4dbsa71b9j81gmj@4ax.com>
Message-ID: <20150531225713.GA73635@cskk.homeip.net>

On 31May2015 11:05, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>	I'll concede submitting jobs to a print queue probably does work easier
>this way, but I'd have to ask if the "convert" could be performed using
>actual Python -- the old Python Imaging Library -- PIL -- for example
>(superseded, I think, by pillow)?

(Yes, PIL is stale, Pillow is live.)

Or, more direct if Pillow doesn't have what he wants, is to use the ImageMagick 
or GraphicsMagick Python libraries. Anything ImageMagick can do via the command 
line they should be able to do directly.

The advantage of using PIL/Pillow or ImageMagick/GraphicsMagick within Python 
is that, memory permitting, one can avoid making (and therefore needing to 
clean up) temporary files.

>	Given that you do nothing between creating a subprocess and waiting for
>it to exit, subprocess.Popen() could be overkill -- the simpler
>subprocess.call() could be sufficient (heck, the old os.system() is
>sufficient, but preference seems to be to ignore it for subprocess.call()
>instead).

os.system passed a string to the shell. That has quoting/security issues.  
(Exactly like shell=True with subprocess.)

With his specific example this isn't a big deal because all the filenames are 
benign etc, but as a general rule using subprocess and passing explicit command 
line argument strings is safer.

All of which you know, I know. But good to get it said in the thread.

Cheers,
Cameron Simpson <cs at zip.com.au>


From timothy.c.delaney at gmail.com  Sun May 31 19:23:02 2015
From: timothy.c.delaney at gmail.com (Tim Delaney)
Date: Mon, 1 Jun 2015 09:23:02 +1000
Subject: What use for reversed()?
In-Reply-To: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>
References: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>
Message-ID: <CAN8CLgnm3ZVzzZHB4306nQdKh3Xwu8fBpL25P1Cy1+0EpvG=6g@mail.gmail.com>

On 1 June 2015 at 05:40, fl <rxjwg98 at gmail.com> wrote:

> Hi,
>
> I have a string b='1234'. I run: br=reversed(b)
>
> I hope that I can print out '4321' by:
>
> for br in b
>
> but it complains:
> SyntaxError: invalid syntax
>

Any time you get a SyntaxError, it means that you have coded something
which does not match the specified syntax of the language version.

Assuming you copied and pasted the above, I can see an error:

    for br in b

The for statement must have a colon at the end of line e.g. a complete for
statement and block is:

for br in b:
    print br

This will output the characters one per line (on Python 3.x), since that is
what the reversed() iterator will return. You will need to do something
else to get it back to a single string.

Have you read through the python tutorials?

https://docs.python.org/3/tutorial/

or for Python 2.x:

https://docs.python.org/2/tutorial/

Tim Delaney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150601/0c75cf84/attachment.html>

From ian.g.kelly at gmail.com  Sun May 31 20:06:13 2015
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sun, 31 May 2015 18:06:13 -0600
Subject: What use for reversed()?
In-Reply-To: <mkfp4q$lkf$3@dont-email.me>
References: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>
 <mkfp4q$lkf$3@dont-email.me>
Message-ID: <CALwzid=d9wJ4AyGPkotFxS1dEkKtV9WxmjV1k1LZCtJV6ZWzzg@mail.gmail.com>

On Sun, May 31, 2015 at 1:58 PM, Denis McMahon <denismfmcmahon at gmail.com> wrote:
> reversed returns an iterator, not a list, so it returns the reversed list
> of elements one at a time. You can use list() or create a list from
> reversed and then join the result:
>
> $ python
> Python 2.7.3 (default, Dec 18 2014, 19:10:20)
> [GCC 4.6.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> "".join(list(reversed("fred")))
> 'derf'
>>>> "".join([x for x in reversed("fred")])
> 'derf'
>
> So reversed can do it, but needs a little help

The str.join method will happily accept an iterator, so the
intermediate list construction in those examples is unnecessary.


From edd.cowan at gmail.com  Sun May 31 20:29:52 2015
From: edd.cowan at gmail.com (Eddilbert Macharia)
Date: Sun, 31 May 2015 17:29:52 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
Message-ID: <9ad6e2be-b3bf-4676-b486-82eab504fe64@googlegroups.com>


So what im getting from above reponse, everything in python is an object because the are instance of the metaclass type and also because they are subclasses of the class object ?


On Sunday, May 31, 2015 at 5:34:20 PM UTC+3, Eddilbert Macharia wrote:
> Hello All ,
> 
> I'm wrecking my head trying to understand. where the class object comes into play . 
> 
> Is it only meant to act as base class and does it mean there is an actual class called object in python which all the objects created using the class type inherit ?
> 
> i'm assuming the metaclass if simplified would look something like this :
> 
> type('dict', (object,),{})
> 
> And when we use the class type as a metaclass are we using the instance version of the class type or are we actually using the type class itself ?
> 
> Also when we say everything is an object in python, are we referring to the fact that everything is an instance of the class type or does it have to with the object class inherited ? 
> 
> As can be attested by using type() function as below :
> 
> >>> type(int)
> <class 'type'>
> >>> type(list)
> <class 'type'>
> >>> type(dict)
> <class 'type'>
> >>> type(type)
> <class 'type'>
> >>> type(object)
> <class 'type'>
> 
> From my understanding this means all of this are instances of the class type. which means the class type was used to create this instances.
> 
> Now if i look at the __bases__ of all this objects i get :
> 
> >>> type.__base__
> <class 'object'>
> >>> type.__bases__
> (<class 'object'>,)
> >>> dict.__bases__
> (<class 'object'>,)
> >>> list.__bases__
> (<class 'object'>,)
> >>> int.__bases__
> (<class 'object'>,)
> >>> object.__bases__
> ()
> 
> This tells me that all of this objects inherit from the class object which has nothing to do with them being instances.


From breamoreboy at yahoo.co.uk  Sun May 31 20:30:40 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 01 Jun 2015 01:30:40 +0100
Subject: What use for reversed()?
In-Reply-To: <CAN8CLgnm3ZVzzZHB4306nQdKh3Xwu8fBpL25P1Cy1+0EpvG=6g@mail.gmail.com>
References: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>
 <CAN8CLgnm3ZVzzZHB4306nQdKh3Xwu8fBpL25P1Cy1+0EpvG=6g@mail.gmail.com>
Message-ID: <mkg93i$1pd$1@ger.gmane.org>

On 01/06/2015 00:23, Tim Delaney wrote:
> On 1 June 2015 at 05:40, fl <rxjwg98 at gmail.com
> <mailto:rxjwg98 at gmail.com>> wrote:
>
>     Hi,
>
>     I have a string b='1234'. I run: br=reversed(b)
>
>     I hope that I can print out '4321' by:
>
>     for br in b
>
>     but it complains:
>     SyntaxError: invalid syntax
>
>
> Any time you get a SyntaxError, it means that you have coded something
> which does not match the specified syntax of the language version.
>
> Assuming you copied and pasted the above, I can see an error:
>
>      for br in b
>
> The for statement must have a colon at the end of line e.g. a complete
> for statement and block is:
>
> for br in b:
>      print br
>
> This will output the characters one per line (on Python 3.x), since that
> is what the reversed() iterator will return. You will need to do
> something else to get it back to a single string.

Will it indeed?  Perhaps fixing the syntax error will get something to 
print :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence



From cs at zip.com.au  Sun May 31 21:03:01 2015
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 1 Jun 2015 11:03:01 +1000
Subject: Best way to prevent zombie processes
In-Reply-To: <87vbf8l9e7.fsf@Equus.decebal.nl>
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
Message-ID: <20150601010301.GA17827@cskk.homeip.net>

On 31May2015 23:33, Cecil Westerhof <Cecil at decebal.nl> wrote:
>At the moment I have the following code:
>    os.chdir(directory)
>    for document in documents:
>        subprocess.Popen(['evince', document])
>
>With this I can open several documents at once. But there is no way to
>know when those documents are going to be closed. This could/will lead
>to zombie processes. (I run it on Linux.) What is the best solution to
>circumvent this?

The standard trick is to make the process a grandchild instead of a child.  
Fork, kick off subprocess, exit (the forked child).

But provided you will collect the children eventually then zombies are only 
untidy, not very resource wasteful. They are essentially just slots in the 
process table left around so that exit status can be collected; the resources 
associated with the full process (memory, open file, etc) have already been 
freed.

>I was thinking about putting all Popen instances in a list. And then
>every five minutes walk through the list and check with poll if the
>process has terminated. If it has it can be released from the list.
>Of-course I need to synchronise those events. Is that a good way to do
>it?

It is reasonable.

Alternatively, and more responsively, you could have a process collection 
Thread. It would loop indefinitely, calling os.wait(). That will block until a 
child process exits (any of them), so it is not a busy loop. When wait() 
returns, look up the pid in your list of dispatched subprocess children and do 
whatever cleanup you intend. (If it is just zombies, the os.wait() cleans that 
part for you.)

One thing you do need to keep in mind with such a task is that it will 
os.wait() _all_ child processes. If there are other children which exit, not 
spawned by your specific calls (earlier children or children made by some 
library function), you must be prepared to receive pids which (a) are not in 
your list of "evince" etc tasks and (b) to be collecting pids which other 
subsystems might have wanted to collect themselves. The latter is uncommon (at 
least, uncommon for such things to occur and you the programmer to be unaware 
of them).

Anyway, that is simple and effective and immediate.

Cheers,
Cameron Simpson <cs at zip.com.au>

If everyone is thinking alike, then someone isn't thinking.     - Patton


From garyr at fidalgo.net  Sun May 31 21:12:48 2015
From: garyr at fidalgo.net (garyr)
Date: Sun, 31 May 2015 18:12:48 -0700
Subject: Basic Info
References: <178f0eac-1760-40b3-9c10-c2d0075883a5@googlegroups.com>
Message-ID: <mkgbjd$v27$1@speranza.aioe.org>


<ch.tanveer13 at gmail.com> wrote in message 
news:178f0eac-1760-40b3-9c10-c2d0075883a5 at googlegroups.com...
> Hi friends,
> M Tanveer, and wanna start to learn python language, i've installed python 
> on my Windows (OS) and set path to it, Now please Guide me which editor is 
> best to use and what instructions should be followed .
> Best Regards: Tanveeer Ahmad
> Thanks

SciTe is an excellent editor for Python C, C++, etc.
http://www.scintilla.org/SciTE.html




From edd.cowan at gmail.com  Sun May 31 23:09:26 2015
From: edd.cowan at gmail.com (Eddilbert Macharia)
Date: Sun, 31 May 2015 20:09:26 -0700 (PDT)
Subject: Everything is an object in python - object class and type class
In-Reply-To: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
References: <14976c1b-a620-426f-b529-41a3c04e9c1a@googlegroups.com>
Message-ID: <c5d52ef0-e5fc-48b6-a30c-846772ff9618@googlegroups.com>

Thank you for you responses guys.

So what I'm getting from above responses, everything in python is an object because the are instance of the metaclass type (from Steven response) and also because they are subclasses of the class object (from Marco and Terry response)? 
I'm having headache trying to understand the cyclic relationship that exit between the metaclass type, the object class, and the class type.

I'm trying to understand how python makes everything an object.is it because everything is an instance of the metaclass type or is it because everything is a subclass of object class.

if its because of being subclass of object class, does that mean if the class object was named class pyobj. would that make everything python pyobj ?

i know objects created by metaclass are types/classes, this types are then used to create other object.

>From this :  
`>>> isinstance(type, object)
True`  
`>>> isinstance(object,type)
True`  
`>>> issubclass(object,type)
False`  
`>>> issubclass(type,object)
True`  

Is it safe to say that python creates the class object first using the type metaclass (I'm simplifying the metaclass for brevity).

`type('object',(),{})`  

which implies class object is a class of class type and it does not inherit any attributes other class.

Then it creates the class type 

`type('type', (object,),{})`  

implying type class is class of class type and it inherits attributes from the object class.

Then creates the other classes by inheriting from the class object

`type('dict', (object,), {})`  
`type('Animal', (object), {})`

which similar to creating an Animal class as :
    
    `class Animal:
         pass`

Does this mean the metaclass used to create the class object is still the one used to create this Animal class or is the metaclass type used by default ? 

Which type is being used, is it the metaclass type or the type class that was created after object was created ? 

Where does the class type created with the base class object come into play ?

I have also tried to understand what really is going on between the object and the class from all he responses above and in this article http://www.cafepy.com/article/python_types_and_objects/python_types_and_objects.html

But i'm still getting confused.What is the relation between this two class in terms of object creation.

will i ever get this or is it a chicken and egg situation ?




On Sunday, May 31, 2015 at 5:34:20 PM UTC+3, Eddilbert Macharia wrote:
> Hello All ,
> 
> I'm wrecking my head trying to understand. where the class object comes into play . 
> 
> Is it only meant to act as base class and does it mean there is an actual class called object in python which all the objects created using the class type inherit ?
> 
> i'm assuming the metaclass if simplified would look something like this :
> 
> type('dict', (object,),{})
> 
> And when we use the class type as a metaclass are we using the instance version of the class type or are we actually using the type class itself ?
> 
> Also when we say everything is an object in python, are we referring to the fact that everything is an instance of the class type or does it have to with the object class inherited ? 
> 
> As can be attested by using type() function as below :
> 
> >>> type(int)
> <class 'type'>
> >>> type(list)
> <class 'type'>
> >>> type(dict)
> <class 'type'>
> >>> type(type)
> <class 'type'>
> >>> type(object)
> <class 'type'>
> 
> From my understanding this means all of this are instances of the class type. which means the class type was used to create this instances.
> 
> Now if i look at the __bases__ of all this objects i get :
> 
> >>> type.__base__
> <class 'object'>
> >>> type.__bases__
> (<class 'object'>,)
> >>> dict.__bases__
> (<class 'object'>,)
> >>> list.__bases__
> (<class 'object'>,)
> >>> int.__bases__
> (<class 'object'>,)
> >>> object.__bases__
> ()
> 
> This tells me that all of this objects inherit from the class object which has nothing to do with them being instances.



From rxjwg98 at gmail.com  Sun May 31 23:11:33 2015
From: rxjwg98 at gmail.com (fl)
Date: Sun, 31 May 2015 20:11:33 -0700 (PDT)
Subject: What use for reversed()?
In-Reply-To: <mailman.252.1433114589.5151.python-list@python.org>
References: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>
 <mailman.252.1433114589.5151.python-list@python.org>
Message-ID: <1e2831cb-9eff-4912-8c89-fc642b47c3ff@googlegroups.com>

On Sunday, May 31, 2015 at 4:23:19 PM UTC-7, Tim Delaney wrote:
> On 1 June 2015 at 05:40, fl <rxj... at gmail.com> wrote:
> Hi,
> 
> The for statement must have a colon at the end of line e.g. a complete for statement and block is:
> 
> for br in b:
> ? ? print br
> 
> This will output the characters one per line (on Python 3.x), since that is what the reversed() iterator will return. You will need to do something else to get it back to a single string.
> 
> 
> Have you read through the python tutorials?
> 
> https://docs.python.org/3/tutorial/
> 
> 
> or for Python 2.x:
> 
> https://docs.python.org/2/tutorial/
> 
> Tim Delaney?

Thank all of you. This is the third time I learn Python. Even though I had
learnt two times, I haven't grasp it. I hope that I can gain a big jump now.
I had read the help tutorial, but forgot it since the long time. But your
reminding does make me remember these stuff.


From random832 at fastmail.us  Sun May 31 23:18:26 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Sun, 31 May 2015 23:18:26 -0400
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
Message-ID: <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>

On Fri, May 29, 2015, at 13:08, Chris Angelico wrote:
> Also, I like talking about Fraction and
> Decimal for the simple reason that they're unobvious; you can poke
> around with Python and discover int and float, and if ever you need
> imaginary/complex numbers, you'll quickly come across complex, but you
> might use Python for years and not realize that decimal.Decimal even
> exists - nor when you'd want it.

Well, isn't that just a byproduct of what problem space you work in? If
someone _does_ know they need a rational or decimal type (e.g. someone
working with money who's clueful enough to know floats won't do),
they'll find these types relatively quickly from a google search -
there's no glut of third-party implementations, and neither is so
obscure nor easily written from scratch that people wouldn't search for
an existing implementation.


From ben+python at benfinney.id.au  Sun May 31 23:37:25 2015
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 01 Jun 2015 13:37:25 +1000
Subject: Best way to prevent zombie processes
References: <87vbf8l9e7.fsf@Equus.decebal.nl>
 <20150601010301.GA17827@cskk.homeip.net>
Message-ID: <856178w13e.fsf@benfinney.id.au>

Cameron Simpson <cs at zip.com.au> writes:

> The standard trick is to make the process a grandchild instead of a
> child.  Fork, kick off subprocess, exit (the forked child).

For Cecil Westerhof's benefit: If you haven't seen it, the
?python-daemon? library is designed to get this, and other fiddly
aspects of daemonising the current program, correct in Python code.

    <URL:https://pypi.python.org/pypi/python-daemon/>

-- 
 \     ?Injustice is relatively easy to bear; what stings is justice.? |
  `\                                                 ?Henry L. Mencken |
_o__)                                                                  |
Ben Finney



From rosuav at gmail.com  Sun May 31 23:43:31 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 1 Jun 2015 13:43:31 +1000
Subject: What is considered an "advanced" topic in Python?
In-Reply-To: <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
References: <c03de3a7-3429-44a3-8241-5a7d81b9aefc@googlegroups.com>
 <mailman.192.1432915802.5151.python-list@python.org>
 <28cec7df-0f51-4098-b027-72e913a9b656@googlegroups.com>
 <CAPTjJmqZrkSs16SR_ADKZ_cz5tqYBxrVC93sDO3TQ8T+oDEoDA@mail.gmail.com>
 <1433128706.32838.283108057.2C4C8613@webmail.messagingengine.com>
Message-ID: <CAPTjJmpmM-jMszyMPSxch5CT4L43cMsXQbkyGGxvFXVpY2CdkA@mail.gmail.com>

On Mon, Jun 1, 2015 at 1:18 PM,  <random832 at fastmail.us> wrote:
> On Fri, May 29, 2015, at 13:08, Chris Angelico wrote:
>> Also, I like talking about Fraction and
>> Decimal for the simple reason that they're unobvious; you can poke
>> around with Python and discover int and float, and if ever you need
>> imaginary/complex numbers, you'll quickly come across complex, but you
>> might use Python for years and not realize that decimal.Decimal even
>> exists - nor when you'd want it.
>
> Well, isn't that just a byproduct of what problem space you work in? If
> someone _does_ know they need a rational or decimal type (e.g. someone
> working with money who's clueful enough to know floats won't do),
> they'll find these types relatively quickly from a google search -
> there's no glut of third-party implementations, and neither is so
> obscure nor easily written from scratch that people wouldn't search for
> an existing implementation.

Yes, but how many people actually know they need a rational type? Just
now there's a thread on python-ideas that was based around the
expectation that a float could do that, which it can't; the OP just
naturally assumed that float was the data type he should be using.

ChrisA


From random832 at fastmail.us  Sun May 31 23:57:30 2015
From: random832 at fastmail.us (random832 at fastmail.us)
Date: Sun, 31 May 2015 23:57:30 -0400
Subject: Everything is an object in python - object class and type class
Message-ID: <1433131050.44416.283127729.2545CCB0@webmail.messagingengine.com>

On Sun, May 31, 2015, at 10:34, Eddilbert Macharia wrote:
> Hello All ,
> 
> I'm wrecking my head trying to understand. where the class object comes
> into play . 
> 
> Is it only meant to act as base class and does it mean there is an actual
> class called object in python which all the objects created using the
> class type inherit ?

Well, you _can_ create a proper instance if you want.

>>> x = object()
>>> x # incidentally, the repr method is called to produce this output
<object object at 0x1003b8090>
>>> id(x)
4298866832
>>> hash(x)
268679177

The other thing that object does is provide the default implementations
of these methods, so when you inherit from it you don't have to
implement them yourself (equality semantics based on object identity,
str based on repr, repr as shown above)


From timothy.c.delaney at gmail.com  Sun May 31 23:59:01 2015
From: timothy.c.delaney at gmail.com (Tim Delaney)
Date: Mon, 1 Jun 2015 13:59:01 +1000
Subject: What use for reversed()?
In-Reply-To: <mkg93i$1pd$1@ger.gmane.org>
References: <514c05fc-dded-4278-ac86-3e8e3cd0e851@googlegroups.com>
 <CAN8CLgnm3ZVzzZHB4306nQdKh3Xwu8fBpL25P1Cy1+0EpvG=6g@mail.gmail.com>
 <mkg93i$1pd$1@ger.gmane.org>
Message-ID: <CAN8CLgnPepVQ4nVETN7aNjHE8e4nyBVaHVi+r2f_DoQdHhi2dQ@mail.gmail.com>

On 1 June 2015 at 10:30, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:

> On 01/06/2015 00:23, Tim Delaney wrote:
>
>> The for statement must have a colon at the end of line e.g. a complete
>> for statement and block is:
>>
>> for br in b:
>>      print br
>>
>> This will output the characters one per line (on Python 3.x), since that
>> is what the reversed() iterator will return. You will need to do
>> something else to get it back to a single string.
>>
>
> Will it indeed?  Perhaps fixing the syntax error will get something to
> print :)


Indeed - as Mark is so gently alluding to, I've done the reverse of what I
said - given Python 2.x syntax instead of Python 3.x.

That should have been:

for br in b:
     print(br)

Tim Delaney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150601/d5b4df53/attachment.html>