From tjreedy at udel.edu  Thu Nov  1 02:20:01 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 31 Oct 2012 21:20:01 -0400
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <20121031202807.13E8B25016F@webabinitio.net>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
	<20121031202807.13E8B25016F@webabinitio.net>
Message-ID: <k6sioi$930$1@ger.gmane.org>

On 10/31/2012 4:28 PM, R. David Murray wrote:
>"local variable referenced before assignment" *is*
> a pointer to the concept of when global variables become local...perhaps
> there is a better wording, do you have a suggestion?

The current wording is an exact, concise, description of the problem. 
Rather than tinkering with the wording, I think a more general solution 
might be a new HOWTO: Understanding exception messages. It could have 
alphabetically sorted entries for exceptions and messages that people 
find problematic.

UnboundLocalError
   local variable referenced before assignment

Problem: You used a local name 'x' in an expression before you assigned 
it a value. So the interpreter could not compute the value of the 
expression. Remember that assignment makes a name local unless it is 
declared global or nonlocal.

Remedy: If you intend 'x' to refer to a glocal or nonlocal name, add the 
necessary global or nonlocal declaration. If you intend



-- 
Terry Jan Reedy


From rosuav at gmail.com  Thu Nov  1 02:45:44 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 1 Nov 2012 12:45:44 +1100
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <k6sioi$930$1@ger.gmane.org>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
	<20121031202807.13E8B25016F@webabinitio.net>
	<k6sioi$930$1@ger.gmane.org>
Message-ID: <CAPTjJmoGujs43QKU3MdxcpYqNRQLxnbK6gPcqruvZF2qd=NLag@mail.gmail.com>

On Thu, Nov 1, 2012 at 12:20 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> The current wording is an exact, concise, description of the problem. Rather
> than tinkering with the wording, I think a more general solution might be a
> new HOWTO: Understanding exception messages. It could have alphabetically
> sorted entries for exceptions and messages that people find problematic.
>
> UnboundLocalError
>
>   local variable referenced before assignment
> ...
> Remedy: If you intend 'x' to refer to a glocal or nonlocal name, add the
> necessary global or nonlocal declaration. If you intend

+1. Can this be tied in with help(UnboundLocalError) perhaps? At the
moment, that produces a huge screed of details that aren't
particularly helpful to a novice (all its methods etc), and only a
one-line explanation "Local name referenced but not bound to a
value.". If that could be shortened and expanded on, it'd be a logical
place to point people. "You got an error you don't understand? Go to
the interactive interpreter and type help(NameOfError) - that should
tell you what it is."

ChrisA

From tjreedy at udel.edu  Thu Nov  1 06:49:51 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 01 Nov 2012 01:49:51 -0400
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <k6sioi$930$1@ger.gmane.org>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
	<20121031202807.13E8B25016F@webabinitio.net>
	<k6sioi$930$1@ger.gmane.org>
Message-ID: <k6t2ih$f1g$1@ger.gmane.org>

On 10/31/2012 9:20 PM, Terry Reedy wrote:
> On 10/31/2012 4:28 PM, R. David Murray wrote:
>> "local variable referenced before assignment" *is*
>> a pointer to the concept of when global variables become local...perhaps
>> there is a better wording, do you have a suggestion?
>
> The current wording is an exact, concise, description of the problem.
> Rather than tinkering with the wording, I think a more general solution
> might be a new HOWTO: Understanding exception messages. It could have
> alphabetically sorted entries for exceptions and messages that people
> find problematic.
>
> UnboundLocalError
>    local variable referenced before assignment
>
> Problem: You used a local name 'x' in an expression before you assigned
> it a value. So the interpreter could not compute the value of the
> expression. Remember that assignment makes a name local unless it is
> declared global or nonlocal.
>
> Remedy: If you intend 'x' to refer to a glocal or nonlocal name, add the
> necessary global or nonlocal declaration. If you intend
<left out> 'x' to be local, rearrange your code to give it a value 
before you use it.


-- 
Terry Jan Reedy


From rob.cliffe at btinternet.com  Thu Nov  1 09:36:51 2012
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 01 Nov 2012 08:36:51 +0000
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <CAPTjJmoGujs43QKU3MdxcpYqNRQLxnbK6gPcqruvZF2qd=NLag@mail.gmail.com>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
	<20121031202807.13E8B25016F@webabinitio.net>
	<k6sioi$930$1@ger.gmane.org>
	<CAPTjJmoGujs43QKU3MdxcpYqNRQLxnbK6gPcqruvZF2qd=NLag@mail.gmail.com>
Message-ID: <509234A3.8030507@btinternet.com>


On 01/11/2012 01:45, Chris Angelico wrote:
> On Thu, Nov 1, 2012 at 12:20 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> The current wording is an exact, concise, description of the problem. Rather
>> than tinkering with the wording, I think a more general solution might be a
>> new HOWTO: Understanding exception messages. It could have alphabetically
>> sorted entries for exceptions and messages that people find problematic.
>>
>> UnboundLocalError
>>
>>    local variable referenced before assignment
>> ...
>> Remedy: If you intend 'x' to refer to a glocal or nonlocal name, add the
>> necessary global or nonlocal declaration. If you intend
> +1. Can this be tied in with help(UnboundLocalError) perhaps? At the
> moment, that produces a huge screed of details that aren't
> particularly helpful to a novice (all its methods etc), and only a
> one-line explanation "Local name referenced but not bound to a
> value.". If that could be shortened and expanded on, it'd be a logical
> place to point people. "You got an error you don't understand? Go to
> the interactive interpreter and type help(NameOfError) - that should
> tell you what it is."
>
> ChrisA
An excellent idea IMO.  +1
Rob Cliffe
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/rob.cliffe%40btinternet.com
>
>


From andrew.svetlov at gmail.com  Fri Nov  2 15:46:58 2012
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Fri, 2 Nov 2012 16:46:58 +0200
Subject: [Python-Dev] Cut/Copy/Paste items in IDLE right click context menu
Message-ID: <CAL3CFcX6d1OJeit9iBBzGTTvo0-gvOWKB=wLTDmPNFZ4FOjqkQ@mail.gmail.com>

Hi. There are issue for subject: http://bugs.python.org/issue1207589
It has patches, implemented well and committed.
I've pushed it to versions 2.7, 3.2, 3.3 and 3.4.

My thoughts are: the issue is not brand new subject but improvement for
current IDLE functionality.
Added code cannot break any existing program/library I hope, it's related
to humans who use IDLE only.
It is desirable feature and probably IDLE users will be ok with new items
in context menu even they are still 2.7 users.

There are another opinion: it is new feature, not a bug, and the patch
should be applied to 3.4 only.

If you look on discussion for issue (http://bugs.python.org/issue1207589)
you can see we cannot make decision, votes are split.

I want to raise the question on this mailing list (as Terry Reedy
suggested) to ask for community opinion.

Thanks, Andrew Svetlov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121102/9027db22/attachment.html>

From ncoghlan at gmail.com  Fri Nov  2 16:16:12 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 3 Nov 2012 01:16:12 +1000
Subject: [Python-Dev] Cut/Copy/Paste items in IDLE right click context
	menu
In-Reply-To: <CAL3CFcX6d1OJeit9iBBzGTTvo0-gvOWKB=wLTDmPNFZ4FOjqkQ@mail.gmail.com>
References: <CAL3CFcX6d1OJeit9iBBzGTTvo0-gvOWKB=wLTDmPNFZ4FOjqkQ@mail.gmail.com>
Message-ID: <CADiSq7dX3NbOvMuMnYFrMMwOPRn+Sf_rmsUO0L9LnovhTQkyyw@mail.gmail.com>

On Sat, Nov 3, 2012 at 12:46 AM, Andrew Svetlov
<andrew.svetlov at gmail.com> wrote:
> Hi. There are issue for subject: http://bugs.python.org/issue1207589
> It has patches, implemented well and committed.
> I've pushed it to versions 2.7, 3.2, 3.3 and 3.4.
>
> My thoughts are: the issue is not brand new subject but improvement for
> current IDLE functionality.
> Added code cannot break any existing program/library I hope, it's related to
> humans who use IDLE only.
> It is desirable feature and probably IDLE users will be ok with new items in
> context menu even they are still 2.7 users.
>
> There are another opinion: it is new feature, not a bug, and the patch
> should be applied to 3.4 only.
>
> If you look on discussion for issue (http://bugs.python.org/issue1207589)
> you can see we cannot make decision, votes are split.
>
> I want to raise the question on this mailing list (as Terry Reedy suggested)
> to ask for community opinion.

The status quo is that IDLE is covered by the "no new features in
maintenance releases" rule along with the rest of the standard
library. Now, it may be *unreasonable* that this is so, and changing
it would help improve IDLE as a tool. The way to resolve a proposal
like that is to put it forward as a PEP, and explain the rationale for
treating IDLE differently. A PEP also makes it possible to state
exactly which modules are being proposed for exemption from the
no-new-features rule.

The fact that many (most?) distro packaging teams split idle out into
a separate package from their base python installation may be a point
in such a proposal's favour (e.g. in Fedora, there's a separate
"python-tools" package, so the main python package doesn't need to
depend on tkinter).

Until such a PEP has been submitted and approved, any feature
additions on maintenance branches should be reverted.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From rdmurray at bitdance.com  Fri Nov  2 16:34:58 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 02 Nov 2012 11:34:58 -0400
Subject: [Python-Dev] Cut/Copy/Paste items in IDLE right click context
	menu
In-Reply-To: <CADiSq7dX3NbOvMuMnYFrMMwOPRn+Sf_rmsUO0L9LnovhTQkyyw@mail.gmail.com>
References: <CAL3CFcX6d1OJeit9iBBzGTTvo0-gvOWKB=wLTDmPNFZ4FOjqkQ@mail.gmail.com>
	<CADiSq7dX3NbOvMuMnYFrMMwOPRn+Sf_rmsUO0L9LnovhTQkyyw@mail.gmail.com>
Message-ID: <20121102153459.0107B250174@webabinitio.net>

On Sat, 03 Nov 2012 01:16:12 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sat, Nov 3, 2012 at 12:46 AM, Andrew Svetlov
> <andrew.svetlov at gmail.com> wrote:
> > Hi. There are issue for subject: http://bugs.python.org/issue1207589
[...]
> The status quo is that IDLE is covered by the "no new features in
> maintenance releases" rule along with the rest of the standard
> library. Now, it may be *unreasonable* that this is so, and changing
> it would help improve IDLE as a tool. The way to resolve a proposal
> like that is to put it forward as a PEP, and explain the rationale for
> treating IDLE differently. A PEP also makes it possible to state
> exactly which modules are being proposed for exemption from the
> no-new-features rule.

In this particular instance we are not looking to exempt the entire
module, just this changeset (because it does not change callable code).

Exempting IDLE in general is an interesting idea, but is not the immediate
question.

--David

From status at bugs.python.org  Fri Nov  2 18:07:26 2012
From: status at bugs.python.org (Python tracker)
Date: Fri,  2 Nov 2012 18:07:26 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20121102170726.DC5181CA74@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-10-26 - 2012-11-02)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3804 (-19)
  closed 24361 (+77)
  total  28165 (+58)

Open issues with patches: 1714 


Issues opened (29)
==================

#9742: Python 2.7: math module fails to build on Solaris 9
http://bugs.python.org/issue9742  reopened by mark.dickinson

#15478: UnicodeDecodeError on OSError on Windows with undecodable (byt
http://bugs.python.org/issue15478  reopened by skrah

#16218: Python launcher does not support non ascii characters
http://bugs.python.org/issue16218  reopened by jcea

#16333: Trailing whitespace in json dump when using indent
http://bugs.python.org/issue16333  opened by zach.mathew

#16334: Faster unicode-escape and raw-unicode-escape codecs
http://bugs.python.org/issue16334  opened by serhiy.storchaka

#16335: Integer overflow in unicode-escape decoder
http://bugs.python.org/issue16335  opened by serhiy.storchaka

#16336: Check input in surrogatepass error handler
http://bugs.python.org/issue16336  opened by serhiy.storchaka

#16338: pysnmp/asyncore - timeout ineffective?
http://bugs.python.org/issue16338  opened by Trenton.Craig

#16339: Document "exec(stmt, global_dict, local_dict)" form in Python 
http://bugs.python.org/issue16339  opened by mark.dickinson

#16346: readline problem
http://bugs.python.org/issue16346  opened by mathieu37

#16347: configure.ac patch
http://bugs.python.org/issue16347  opened by cavallo71

#16349: Document whether it's safe to use bytes for struct format stri
http://bugs.python.org/issue16349  opened by takluyver

#16350: zlib.Decompress.decompress() after EOF discards existing value
http://bugs.python.org/issue16350  opened by nadeem.vawda

#16353: add function to os module for getting path to default shell
http://bugs.python.org/issue16353  opened by chris.jerdonek

#16354: Remember python version choice on docs.python.org
http://bugs.python.org/issue16354  opened by wichert

#16355: inspect.getcomments() does not work in the interactive shell
http://bugs.python.org/issue16355  opened by marco.buttu

#16357: SSLSocket created from SSLContext.wrap_socket doesn't include 
http://bugs.python.org/issue16357  opened by mcjeff

#16360: argparse: comma in metavar causes assertion failure when forma
http://bugs.python.org/issue16360  opened by bgamari

#16361: HTTPS/TLS Problem in Python 3.3
http://bugs.python.org/issue16361  opened by pventura

#16367: io.FileIO.readall() is not 64-bit safe on Windows
http://bugs.python.org/issue16367  opened by haypo

#16376: wrong type for wintypes.BYTE
http://bugs.python.org/issue16376  opened by techtonik

#16378: venv.EnvBuilder docstring inconsistencies
http://bugs.python.org/issue16378  opened by bfroehle

#16379: SQLite error code not exposed to python
http://bugs.python.org/issue16379  opened by torsten

#16381: Introduce option to force the interpreter to exit upon MemoryE
http://bugs.python.org/issue16381  opened by ctheune

#16382: Better warnings exception for bad category
http://bugs.python.org/issue16382  opened by pelson

#16383: Python 3.3 Permission Error with User Library on Windows
http://bugs.python.org/issue16383  opened by jimp02

#16384: import.c doesn't handle EOFError from PyMarshal_Read*
http://bugs.python.org/issue16384  opened by syeberman

#16385: evaluating literal dict with repeated keys gives no warnings/e
http://bugs.python.org/issue16385  opened by Albert.Ferras

#16386: imp.find_module does not specify registry key it searches on w
http://bugs.python.org/issue16386  opened by dhgmgn



Most recent 15 issues with no replies (15)
==========================================

#16386: imp.find_module does not specify registry key it searches on w
http://bugs.python.org/issue16386

#16384: import.c doesn't handle EOFError from PyMarshal_Read*
http://bugs.python.org/issue16384

#16382: Better warnings exception for bad category
http://bugs.python.org/issue16382

#16379: SQLite error code not exposed to python
http://bugs.python.org/issue16379

#16376: wrong type for wintypes.BYTE
http://bugs.python.org/issue16376

#16360: argparse: comma in metavar causes assertion failure when forma
http://bugs.python.org/issue16360

#16353: add function to os module for getting path to default shell
http://bugs.python.org/issue16353

#16347: configure.ac patch
http://bugs.python.org/issue16347

#16346: readline problem
http://bugs.python.org/issue16346

#16339: Document "exec(stmt, global_dict, local_dict)" form in Python 
http://bugs.python.org/issue16339

#16334: Faster unicode-escape and raw-unicode-escape codecs
http://bugs.python.org/issue16334

#16321: Move eq.h out of stringlib
http://bugs.python.org/issue16321

#16320: Establish order in bytes/string dependencies
http://bugs.python.org/issue16320

#16287: Sporadic test_utime() failures on Solaris
http://bugs.python.org/issue16287

#16283: ctypes.util.find_library does not find all DLLs anymore
http://bugs.python.org/issue16283



Most recent 15 issues waiting for review (15)
=============================================

#16386: imp.find_module does not specify registry key it searches on w
http://bugs.python.org/issue16386

#16382: Better warnings exception for bad category
http://bugs.python.org/issue16382

#16381: Introduce option to force the interpreter to exit upon MemoryE
http://bugs.python.org/issue16381

#16367: io.FileIO.readall() is not 64-bit safe on Windows
http://bugs.python.org/issue16367

#16357: SSLSocket created from SSLContext.wrap_socket doesn't include 
http://bugs.python.org/issue16357

#16350: zlib.Decompress.decompress() after EOF discards existing value
http://bugs.python.org/issue16350

#16336: Check input in surrogatepass error handler
http://bugs.python.org/issue16336

#16335: Integer overflow in unicode-escape decoder
http://bugs.python.org/issue16335

#16334: Faster unicode-escape and raw-unicode-escape codecs
http://bugs.python.org/issue16334

#16333: Trailing whitespace in json dump when using indent
http://bugs.python.org/issue16333

#16329: mimetypes does not support webm type
http://bugs.python.org/issue16329

#16327: subprocess.Popen leaks file descriptors on os.fork() failure
http://bugs.python.org/issue16327

#16326: distutils build_ext fails to set library_dirs in 2.7.2 on Linu
http://bugs.python.org/issue16326

#16324: MIMEText __init__ does not support Charset instance
http://bugs.python.org/issue16324

#16323: Wrong C API documentation for locale encoding
http://bugs.python.org/issue16323



Top 10 most discussed issues (10)
=================================

#16248: Security bug in tkinter allows for untrusted, arbitrary code e
http://bugs.python.org/issue16248  19 msgs

#16381: Introduce option to force the interpreter to exit upon MemoryE
http://bugs.python.org/issue16381  14 msgs

#15478: UnicodeDecodeError on OSError on Windows with undecodable (byt
http://bugs.python.org/issue15478  12 msgs

#16312: New command line option supported by unittest.main() for runni
http://bugs.python.org/issue16312  10 msgs

#16355: inspect.getcomments() does not work in the interactive shell
http://bugs.python.org/issue16355   9 msgs

#16385: evaluating literal dict with repeated keys gives no warnings/e
http://bugs.python.org/issue16385   9 msgs

#9742: Python 2.7: math module fails to build on Solaris 9
http://bugs.python.org/issue9742   8 msgs

#16218: Python launcher does not support non ascii characters
http://bugs.python.org/issue16218   8 msgs

#16350: zlib.Decompress.decompress() after EOF discards existing value
http://bugs.python.org/issue16350   6 msgs

#2005: posixmodule expects sizeof(pid_t/gid_t/uid_t) <= sizeof(long)
http://bugs.python.org/issue2005   5 msgs



Issues closed (71)
==================

#3636: Managing dual 2.x and 3.0 installations on Windows
http://bugs.python.org/issue3636  closed by eric.araujo

#5210: zlib does not indicate end of compressed stream properly
http://bugs.python.org/issue5210  closed by nadeem.vawda

#7098: g formatting for decimal types should always strip trailing ze
http://bugs.python.org/issue7098  closed by mark.dickinson

#8996: Add a default role to allow writing bare `len` instead of :fun
http://bugs.python.org/issue8996  closed by eric.araujo

#9017: doctest option flag to enable/disable some chunk of doctests?
http://bugs.python.org/issue9017  closed by ezio.melotti

#9530: integer undefined behaviors
http://bugs.python.org/issue9530  closed by mark.dickinson

#10189: SyntaxError: no binding for nonlocal doesn't contain a useful 
http://bugs.python.org/issue10189  closed by python-dev

#10836: urllib.request.urlretrieve calls URLError with 3 args
http://bugs.python.org/issue10836  closed by orsenthil

#12890: cgitb displays <p> tags when executed in text mode
http://bugs.python.org/issue12890  closed by r.david.murray

#13701: Remove Decimal Python 2.3 Compatibility
http://bugs.python.org/issue13701  closed by mark.dickinson

#14570: Document json "sort_keys" parameter properly
http://bugs.python.org/issue14570  closed by asvetlov

#14616: subprocess docs should mention pipes.quote/shlex.quote
http://bugs.python.org/issue14616  closed by asvetlov

#14625: Faster utf-32 decoder
http://bugs.python.org/issue14625  closed by python-dev

#14675: make distutils.ccompiler.CCompiler an abstract class
http://bugs.python.org/issue14675  closed by eric.araujo

#14893: Tutorial: Add function annotation example to function tutorial
http://bugs.python.org/issue14893  closed by asvetlov

#14897: struct.pack raises unexpected error message
http://bugs.python.org/issue14897  closed by petri.lehtinen

#14900: cProfile does not take its result headers as sort arguments
http://bugs.python.org/issue14900  closed by asvetlov

#15148: shutil.which() docstring could be clearer
http://bugs.python.org/issue15148  closed by r.david.murray

#15746: test_winsound bombing out on 2003 buildslave
http://bugs.python.org/issue15746  closed by trent

#15889: regrtest --start option raises AttributeError in many scenario
http://bugs.python.org/issue15889  closed by r.david.murray

#15957: README.txt points to broken "contributing" url in python wiki
http://bugs.python.org/issue15957  closed by eric.araujo

#15961: Missing return value in ``system_message``
http://bugs.python.org/issue15961  closed by eric.araujo

#16086: tp_flags: Undefined behaviour with 32 bits long
http://bugs.python.org/issue16086  closed by haypo

#16107: distutils2.version doesn't str() "1.0.post1" correctly
http://bugs.python.org/issue16107  closed by eric.araujo

#16197: Several small errors in winreg documentation
http://bugs.python.org/issue16197  closed by brian.curtin

#16228: JSON crashes during encoding resized lists
http://bugs.python.org/issue16228  closed by pitrou

#16230: select.select crashes on resized lists
http://bugs.python.org/issue16230  closed by pitrou

#16239: PEP8 arithmetic operator examples
http://bugs.python.org/issue16239  closed by gvanrossum

#16243: Add example for inspect module doc
http://bugs.python.org/issue16243  closed by asvetlov

#16250: URLError invoked with reason as filename
http://bugs.python.org/issue16250  closed by orsenthil

#16268: dir(closure) does not find __dir__
http://bugs.python.org/issue16268  closed by benjamin.peterson

#16271: weird dual behavior with changing __qualname__; different valu
http://bugs.python.org/issue16271  closed by python-dev

#16300: Windows raises ValueError for file:///file/not/exists instead 
http://bugs.python.org/issue16300  closed by orsenthil

#16301: localhost() and thishost() in urllib/request.py
http://bugs.python.org/issue16301  closed by orsenthil

#16307: multiprocess.pool.map_async callables not working
http://bugs.python.org/issue16307  closed by hynek

#16316: Support xz compression in mimetypes module
http://bugs.python.org/issue16316  closed by nadeem.vawda

#16325: PEP??8 refers to reference to PEP 8
http://bugs.python.org/issue16325  closed by ezio.melotti

#16330: Use surrogate-related macros
http://bugs.python.org/issue16330  closed by serhiy.storchaka

#16331: Add a version switcher to python docs site
http://bugs.python.org/issue16331  closed by loewis

#16332: Use new exception handling syntax in Subprocess documentation
http://bugs.python.org/issue16332  closed by ezio.melotti

#16337: typo in documentation regarding "Bytearray Objects", chapter 4
http://bugs.python.org/issue16337  closed by ezio.melotti

#16340: Decoding error due to byte-compiling venv scripts at install t
http://bugs.python.org/issue16340  closed by python-dev

#16341: In examples, "except:" should use new syntax
http://bugs.python.org/issue16341  closed by asvetlov

#16342: setup.py not compiling with NDEBUG in non-debug builds
http://bugs.python.org/issue16342  closed by brett.cannon

#16343: PyUnicode_FromFormatV() doesn't support utf-8 text
http://bugs.python.org/issue16343  closed by chris.jerdonek

#16344: Traceback Internationalization Proposal
http://bugs.python.org/issue16344  closed by rhettinger

#16345: dict.fromkeys() assumes 'self' is empty
http://bugs.python.org/issue16345  closed by python-dev

#16348: Decimal.remainder_near documentation incorrect.
http://bugs.python.org/issue16348  closed by mark.dickinson

#16351: Add a function to get GC statistics
http://bugs.python.org/issue16351  closed by pitrou

#16352: Error call
http://bugs.python.org/issue16352  closed by r.david.murray

#16356: cjson dose not decode \/ properly
http://bugs.python.org/issue16356  closed by christian.heimes

#16358: Enhancement for mmap_read: Consistency with standard file read
http://bugs.python.org/issue16358  closed by r.david.murray

#16359: can't print figures 08 and 09
http://bugs.python.org/issue16359  closed by ezio.melotti

#16362: _LegalCharsPatt in cookies.py includes illegal characters
http://bugs.python.org/issue16362  closed by r.david.murray

#16363: super cannot invoke descriptors
http://bugs.python.org/issue16363  closed by r.david.murray

#16364: datetime.strftime and locale.getdefaultlocale conflict on Wind
http://bugs.python.org/issue16364  closed by jwfang

#16365: IDLE for Windows 8
http://bugs.python.org/issue16365  closed by loewis

#16366: _handleError not very informative
http://bugs.python.org/issue16366  closed by python-dev

#16368: error when logging message
http://bugs.python.org/issue16368  closed by r.david.murray

#16369: Global PyTypeObjects not initialized with PyType_Ready(...)
http://bugs.python.org/issue16369  closed by python-dev

#16370: Mention Py_SetProgramName in example for very high level embed
http://bugs.python.org/issue16370  closed by asvetlov

#16371: typo in ctypes
http://bugs.python.org/issue16371  closed by asvetlov

#16372: Initialization strange behavior
http://bugs.python.org/issue16372  closed by mark.dickinson

#16373: Recursion error comparing set() and collections.Set instances
http://bugs.python.org/issue16373  closed by asvetlov

#16374: ConfigParser: Passing a semicolon as a value
http://bugs.python.org/issue16374  closed by lukasz.langa

#16375: Warning in Parser/grammar1.c
http://bugs.python.org/issue16375  closed by benjamin.peterson

#16377: Fix bisect unittest
http://bugs.python.org/issue16377  closed by asvetlov

#16380: true/proper subset
http://bugs.python.org/issue16380  closed by asvetlov

#16387: test_cmd_line_script failures
http://bugs.python.org/issue16387  closed by skrah

#16388: Urllib screws up capitalization in "User-Agent" HTTP Header
http://bugs.python.org/issue16388  closed by r.david.murray

#1207589: IDLE: Right Click Context Menu
http://bugs.python.org/issue1207589  closed by asvetlov

From tjreedy at udel.edu  Fri Nov  2 18:29:47 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 02 Nov 2012 13:29:47 -0400
Subject: [Python-Dev] Cut/Copy/Paste items in IDLE right click context
	menu
In-Reply-To: <CADiSq7dX3NbOvMuMnYFrMMwOPRn+Sf_rmsUO0L9LnovhTQkyyw@mail.gmail.com>
References: <CAL3CFcX6d1OJeit9iBBzGTTvo0-gvOWKB=wLTDmPNFZ4FOjqkQ@mail.gmail.com>
	<CADiSq7dX3NbOvMuMnYFrMMwOPRn+Sf_rmsUO0L9LnovhTQkyyw@mail.gmail.com>
Message-ID: <k70vuu$qi7$1@ger.gmane.org>

On 11/2/2012 11:16 AM, Nick Coghlan wrote:
> On Sat, Nov 3, 2012 at 12:46 AM, Andrew Svetlov
> <andrew.svetlov at gmail.com> wrote:
>> Hi. There are issue for subject: http://bugs.python.org/issue1207589
>> It has patches, implemented well and committed.
>> I've pushed it to versions 2.7, 3.2, 3.3 and 3.4.
>>
>> My thoughts are: the issue is not brand new subject but improvement for
>> current IDLE functionality.
>> Added code cannot break any existing program/library I hope, it's related to
>> humans who use IDLE only.
>> It is desirable feature and probably IDLE users will be ok with new items in
>> context menu even they are still 2.7 users.
>>
>> There are another opinion: it is new feature, not a bug, and the patch
>> should be applied to 3.4 only.
>>
>> If you look on discussion for issue (http://bugs.python.org/issue1207589)
>> you can see we cannot make decision, votes are split.
>>
>> I want to raise the question on this mailing list (as Terry Reedy suggested)
>> to ask for community opinion.
>
> The status quo is that IDLE is covered by the "no new features in
> maintenance releases" rule along with the rest of the standard
> library.

That may be your opinion, but I disagree that the situation is so clear. 
In what PEP (or other document) is the above stated.

IDLE has previously been treated here as an exception to the normal 
rules. Two years ago, we debated *dropping* IDLE from the distribution 
-- beginning with the next release. We would not have had such a 
discussion for any normal library module as simply removing a module 
before Py 4 is against policy.

 > Now, it may be *unreasonable* that this is so, and changing
> it would help improve IDLE as a tool.

If it is 'unreasonable', then perhaps it is not so.

 > The way to resolve a proposal
> like that is to put it forward as a PEP, and explain the rationale for
> treating IDLE differently.

A PEP seems like overkill to me. The matter is a rule clarification, not 
an enhancement proposal. The rationale is simple.

1. The reason for the no-new-features rule does not apply to user 
interface features, certainly not to the right-click context menu.

2. Users will prefer consistency, especially in something like 
right-click behavior (or search/replace boxes, etc).

3. It is often unclear whether a particular change is a fix or an 
enhancement. I would suggest that a) in many cases neither word really 
applies and b) in such cases, given 1) and 2) above, it is not worth the 
effort to force fit a change into either category.

For instance, the existence of a right-click context menu is not 
mentioned in the sketchy Library manual chapter for IDLE. So there can 
neither be consistency nor inconsistency between current behavior and 
the non-existent doc entry. Hence, there is no objective standard for 
classifying a change, and hence there is disagreement. Since
http://bugs.python.org/issue1207589
brings IDLE in line with external standards, I consider it a bug fix. 
Actually, I consider it *both* a bug fix *and* and enhancement, but a 
bug fix for the purpose of deciding where to apply it (given that 
someone, Andrew, was willing to go though the effort of applying it 
everywhere).

Even features that are documented as to existence are not specified. The 
following is typical.
   "Find...
   Open a search dialog box with many options"
There have been or still are proposed changes to Find or Replace that 
could be classified either way, depending on whether, in the absence of 
any specification, one is inclined to make 'bug fix' or 'enhancement' 
the default choice.

> A PEP also makes it possible to state
> exactly which modules are being proposed for exemption from the
> no-new-features rule.

Since the proposal two years ago was to delete the entire idlelib/* 
package, I would start with the entire package. If and when possibly 
generally useful modules (perhaps idlelib/rpc.py -- remote procedure 
calls) are documented for general use, they should be versioned. But 
then they should perhaps be moved elsewhere.

> The fact that many (most?) distro packaging teams split idle out into
> a separate package from their base python installation may be a point
> in such a proposal's favour (e.g. in Fedora, there's a separate
> "python-tools" package, so the main python package doesn't need to
> depend on tkinter).
>
> Until such a PEP has been submitted and approved, any feature
> additions on maintenance branches should be reverted.

And who shall decide what change is what?

-- 
Terry Jan Reedy


From nad at acm.org  Fri Nov  2 18:31:47 2012
From: nad at acm.org (Ned Deily)
Date: Fri, 02 Nov 2012 10:31:47 -0700
Subject: [Python-Dev] Cut/Copy/Paste items in IDLE right click context
	menu
References: <CAL3CFcX6d1OJeit9iBBzGTTvo0-gvOWKB=wLTDmPNFZ4FOjqkQ@mail.gmail.com>
	<CADiSq7dX3NbOvMuMnYFrMMwOPRn+Sf_rmsUO0L9LnovhTQkyyw@mail.gmail.com>
	<20121102153459.0107B250174@webabinitio.net>
Message-ID: <nad-E3721F.10314602112012@news.gmane.org>

In article <20121102153459.0107B250174 at webabinitio.net>,
 "R. David Murray" <rdmurray at bitdance.com> wrote:

> On Sat, 03 Nov 2012 01:16:12 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > On Sat, Nov 3, 2012 at 12:46 AM, Andrew Svetlov
> > <andrew.svetlov at gmail.com> wrote:
> > > Hi. There are issue for subject: http://bugs.python.org/issue1207589
> [...]
> > The status quo is that IDLE is covered by the "no new features in
> > maintenance releases" rule along with the rest of the standard
> > library. Now, it may be *unreasonable* that this is so, and changing
> > it would help improve IDLE as a tool. The way to resolve a proposal
> > like that is to put it forward as a PEP, and explain the rationale for
> > treating IDLE differently. A PEP also makes it possible to state
> > exactly which modules are being proposed for exemption from the
> > no-new-features rule.
> In this particular instance we are not looking to exempt the entire
> module, just this changeset (because it does not change callable code).
> 
> Exempting IDLE in general is an interesting idea, but is not the immediate
> question.

Also, as Roger Serwy has pointed out in the issue, the change also can 
affect third-party IDLE extensions but he thinks the backport is still 
worthwhile.  Since the discussion has progressed primarily on the issue 
tracker and the python-dev interest is probably limited, I would suggest 
keeping the discussion over there rather than both here and there.

-- 
 Ned Deily,
 nad at acm.org


From nad at acm.org  Fri Nov  2 18:47:11 2012
From: nad at acm.org (Ned Deily)
Date: Fri, 02 Nov 2012 10:47:11 -0700
Subject: [Python-Dev] Cut/Copy/Paste items in IDLE right click context
	menu
References: <CAL3CFcX6d1OJeit9iBBzGTTvo0-gvOWKB=wLTDmPNFZ4FOjqkQ@mail.gmail.com>
	<CADiSq7dX3NbOvMuMnYFrMMwOPRn+Sf_rmsUO0L9LnovhTQkyyw@mail.gmail.com>
	<k70vuu$qi7$1@ger.gmane.org>
Message-ID: <nad-FF4D5B.10471102112012@news.gmane.org>

In article <k70vuu$qi7$1 at ger.gmane.org>, Terry Reedy <tjreedy at udel.edu> 
wrote:
> For instance, the existence of a right-click context menu is not 
> mentioned in the sketchy Library manual chapter for IDLE.

Actually, it is documented as of a few weeks ago (see the changes for 
Issue10405).  And the issue under discussion here updated both the 
manual and the IDLE help file.

http://docs.python.org/2/library/idle.html#edit-context-menu

-- 
 Ned Deily,
 nad at acm.org


From pjenvey at underboss.org  Fri Nov  2 19:16:25 2012
From: pjenvey at underboss.org (Philip Jenvey)
Date: Fri, 2 Nov 2012 11:16:25 -0700
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
	but this time more solid numbers)
In-Reply-To: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
Message-ID: <3E792B3D-BEA9-4E9F-9102-7F18FD3771E4@underboss.org>


On Oct 26, 2012, at 12:14 PM, Brett Cannon wrote:

> 
> Worst benchmark is nosite_startup, best is telco. The benchmarks people might want to analyze (i.e. more than 20% slower in Python 3.3) are mako_v2, threaded_count, normal_startup, iterative_count, pathlib, formatted_logging, and simple_logging.

> 
> ### mako_v2 ###
> Min: 0.083660 -> 0.243323: 2.91x slower
> Avg: 0.084634 -> 0.247875: 2.93x slower
> Significant (t=-821.55)
> Stddev: 0.00193 -> 0.00400: 2.0737x larger
> Timeline: b'http://tinyurl.com/98n9fab'

So Mike Bayer and I narrowed down mako_v2's slowness to use of an inline re

This:

http://www.makotemplates.org/trac/changeset/c1468b12f115ac9e469150ce24ea042aeae5e270

brings it down to around:

### mako_v2 ###
Min: 0.087608 -> 0.066748: 1.31x faster
Avg: 0.091348 -> 0.071224: 1.28x faster
Significant (t=26.10)
Stddev: 0.00312 -> 0.00447: 1.4340x larger
Timeline: http://tinyurl.com/as2zedo

The culprit is the lru_cache on re._compile_typed. Notice functools' numbers from the profiler:

http://paste.ofcode.org/yZRKnJfTsHesFR8hMWfc7f

Mike also noticed that the mako fix above does nothing to 2.7's numbers.

--
Philip Jenvey

From brett at python.org  Fri Nov  2 19:42:55 2012
From: brett at python.org (Brett Cannon)
Date: Fri, 2 Nov 2012 14:42:55 -0400
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <3E792B3D-BEA9-4E9F-9102-7F18FD3771E4@underboss.org>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<3E792B3D-BEA9-4E9F-9102-7F18FD3771E4@underboss.org>
Message-ID: <CAP1=2W4VMMDCmQk-7VKEyK3zFCq9R8bsK3pv6hV69sCLA0MQsA@mail.gmail.com>

Issue filed for the performance issue: http://bugs.python.org/issue16390

With that change and running on tip of Mako on my laptop now reports 1.25x
slower which is *much* better than it was. This performance issue might
also explain why all of the regex compilation benchmarks are worse under
Python 3.3 by a decent margin.

On Fri, Nov 2, 2012 at 2:16 PM, Philip Jenvey <pjenvey at underboss.org> wrote:

> lru_cache on re._compile_typed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121102/40045126/attachment.html>

From ericsnowcurrently at gmail.com  Fri Nov  2 21:05:51 2012
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Fri, 2 Nov 2012 14:05:51 -0600
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <20121102170726.DC5181CA74@psf.upfronthosting.co.za>
References: <20121102170726.DC5181CA74@psf.upfronthosting.co.za>
Message-ID: <CALFfu7A+dx5kVxmW-c-5d-egQeqad_ihUE+-JgaEpBLz=hm3tQ@mail.gmail.com>

On Fri, Nov 2, 2012 at 11:07 AM, Python tracker <status at bugs.python.org> wrote:
>
> ACTIVITY SUMMARY (2012-10-26 - 2012-11-02)
> Python tracker at http://bugs.python.org/
>
> To view or respond to any of the issues listed below, click on the issue.
> Do NOT respond to this message.
>
> Issues counts and deltas:
>   open    3804 (-19)

wow!

>   closed 24361 (+77)
>   total  28165 (+58)
>
> Open issues with patches: 1714

From andrew.svetlov at gmail.com  Fri Nov  2 21:24:38 2012
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Fri, 2 Nov 2012 22:24:38 +0200
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <CALFfu7A+dx5kVxmW-c-5d-egQeqad_ihUE+-JgaEpBLz=hm3tQ@mail.gmail.com>
References: <20121102170726.DC5181CA74@psf.upfronthosting.co.za>
	<CALFfu7A+dx5kVxmW-c-5d-egQeqad_ihUE+-JgaEpBLz=hm3tQ@mail.gmail.com>
Message-ID: <CAL3CFcVcfLqo2H=mGSoCVJ2pmuNdzKDt5Dj7zz9UM-yD1gN63A@mail.gmail.com>

On Fri, Nov 2, 2012 at 10:05 PM, Eric Snow <ericsnowcurrently at gmail.com>
wrote:
>
> On Fri, Nov 2, 2012 at 11:07 AM, Python tracker <status at bugs.python.org>
wrote:
> >
> > ACTIVITY SUMMARY (2012-10-26 - 2012-11-02)
> > Python tracker at http://bugs.python.org/
> >
> > To view or respond to any of the issues listed below, click on the
issue.
> > Do NOT respond to this message.
> >
> > Issues counts and deltas:
> >   open    3804 (-19)
>
> wow!
>
Aha!
>
> >   closed 24361 (+77)
> >   total  28165 (+58)
> >
> > Open issues with patches: 1714
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com




--
Thanks,
Andrew Svetlov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121102/07b3a71d/attachment.html>

From ncoghlan at gmail.com  Sat Nov  3 03:04:57 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 3 Nov 2012 12:04:57 +1000
Subject: [Python-Dev] Cut/Copy/Paste items in IDLE right click context
	menu
In-Reply-To: <k70vuu$qi7$1@ger.gmane.org>
References: <CAL3CFcX6d1OJeit9iBBzGTTvo0-gvOWKB=wLTDmPNFZ4FOjqkQ@mail.gmail.com>
	<CADiSq7dX3NbOvMuMnYFrMMwOPRn+Sf_rmsUO0L9LnovhTQkyyw@mail.gmail.com>
	<k70vuu$qi7$1@ger.gmane.org>
Message-ID: <CADiSq7eEN3nSxbrouZ5VgyQC7cKwgTjCM78jQz26NxKjU7ipoA@mail.gmail.com>

On Sat, Nov 3, 2012 at 3:29 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> The way to resolve a proposal
>>
>> like that is to put it forward as a PEP, and explain the rationale for
>> treating IDLE differently.
>
>
> A PEP seems like overkill to me. The matter is a rule clarification, not an
> enhancement proposal. The rationale is simple.

If you don't want to run the risk of needing to rehash this discussion
every time an IDLE feature addition is made in maintenance branches,
write the rules down in a PEP.

I don't actually care all that much about IDLE (except in an abstract
"I know other people care about it" sense), but I *do* care about
developers being able to understand the rules about whether or not a
feature addition is permissible in a maintenance branch. That kind of
thing *cannot* be left to "oh, this change is OK, this change isn't,
but you needed to be around for this discussion that happened 5 years
ago in order to understand why", because it *wastes everybody's time*
explaining the unwritten rules to new developers. If there is a part
of the code base where new features are permitted in maintenance
branches, write them down, get agreement on them, and if anyone
complains "but that's a new feature on a maintenance branch", point
them to the PEP that says its OK.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From tjreedy at udel.edu  Sat Nov  3 04:10:23 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 02 Nov 2012 23:10:23 -0400
Subject: [Python-Dev] Cut/Copy/Paste items in IDLE right click context
	menu
In-Reply-To: <CADiSq7eEN3nSxbrouZ5VgyQC7cKwgTjCM78jQz26NxKjU7ipoA@mail.gmail.com>
References: <CAL3CFcX6d1OJeit9iBBzGTTvo0-gvOWKB=wLTDmPNFZ4FOjqkQ@mail.gmail.com>
	<CADiSq7dX3NbOvMuMnYFrMMwOPRn+Sf_rmsUO0L9LnovhTQkyyw@mail.gmail.com>
	<k70vuu$qi7$1@ger.gmane.org>
	<CADiSq7eEN3nSxbrouZ5VgyQC7cKwgTjCM78jQz26NxKjU7ipoA@mail.gmail.com>
Message-ID: <50948B1F.1030000@udel.edu>

On 11/2/2012 10:04 PM, Nick Coghlan wrote:
> On Sat, Nov 3, 2012 at 3:29 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>>> The way to resolve a proposal
>>>
>>> like that is to put it forward as a PEP, and explain the rationale for
>>> treating IDLE differently.
>>
>>
>> A PEP seems like overkill to me. The matter is a rule clarification, not an
>> enhancement proposal. The rationale is simple.
>
> If you don't want to run the risk of needing to rehash this discussion
> every time an IDLE feature addition is made in maintenance branches,
> write the rules down in a PEP.
[snip reasons]
OK, I am convinced an info PEP would be a good idea.

--
Terry


From ncoghlan at gmail.com  Sat Nov  3 10:28:02 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 3 Nov 2012 19:28:02 +1000
Subject: [Python-Dev] Cut/Copy/Paste items in IDLE right click context
	menu
In-Reply-To: <50948B1F.1030000@udel.edu>
References: <CAL3CFcX6d1OJeit9iBBzGTTvo0-gvOWKB=wLTDmPNFZ4FOjqkQ@mail.gmail.com>
	<CADiSq7dX3NbOvMuMnYFrMMwOPRn+Sf_rmsUO0L9LnovhTQkyyw@mail.gmail.com>
	<k70vuu$qi7$1@ger.gmane.org>
	<CADiSq7eEN3nSxbrouZ5VgyQC7cKwgTjCM78jQz26NxKjU7ipoA@mail.gmail.com>
	<50948B1F.1030000@udel.edu>
Message-ID: <CADiSq7fYXqXO+QWG6eJdxkiCtJLQDdWVJk6e11pv2PssyNBkUA@mail.gmail.com>

On Sat, Nov 3, 2012 at 1:10 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> [snip reasons]
> OK, I am convinced an info PEP would be a good idea.

Unless anyone objects, I'm happy to be BDFL-delegate for such a PEP. I
mainly want to ensure we clearly fence off "IDLE-the-application",
with (in effect) a 6 month release cycle synchronised across versions,
from the rest of the standard library.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Nov  3 10:38:55 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 3 Nov 2012 19:38:55 +1000
Subject: [Python-Dev] [Python-checkins] cpython (3.3): Issue #15814: Use
 hash function that is compatible with the equality
In-Reply-To: <3XtV7q58c6zRMy@mail.python.org>
References: <3XtV7q58c6zRMy@mail.python.org>
Message-ID: <CADiSq7fSaz6uxQeoQeyi0aukH3kLifUAmSpd1ZRJe2KedXD5DQ@mail.gmail.com>

On Sat, Nov 3, 2012 at 3:07 AM, stefan.krah <python-checkins at python.org> wrote:
> +        # equality-hash invariant
> +        x = ndarray(list(range(12)), shape=[12], format='B')
> +        a = memoryview(nd)
> +
> +        y = ndarray(list(range(12)), shape=[12], format='b')
> +        b = memoryview(nd)
> +
> +        z = ndarray(list(bytes(chr(x), 'latin-1') for x in range(12)),
> +                    shape=[12], format='c')
> +        c = memoryview(nd)
> +
> +        if (a == b):
> +            self.assertEqual(hash(a), hash(b))
> +
> +        if (a == c):
> +            self.assertEqual(hash(a), hash(c))
> +
> +        if (b == c):
> +            self.assertEqual(hash(b), hash(c))

These checks could do with a comment explaining why the if statements
are needed (I'm assuming something to do with memory order).

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From stefan at bytereef.org  Sat Nov  3 10:59:23 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Sat, 3 Nov 2012 10:59:23 +0100
Subject: [Python-Dev] [Python-checkins] cpython (3.3): Issue #15814: Use
 hash function that is compatible with the equality
In-Reply-To: <CADiSq7fSaz6uxQeoQeyi0aukH3kLifUAmSpd1ZRJe2KedXD5DQ@mail.gmail.com>
References: <3XtV7q58c6zRMy@mail.python.org>
	<CADiSq7fSaz6uxQeoQeyi0aukH3kLifUAmSpd1ZRJe2KedXD5DQ@mail.gmail.com>
Message-ID: <20121103095923.GA30827@sleipnir.bytereef.org>

Nick Coghlan <ncoghlan at gmail.com> wrote:
> > +        if (b == c):
> > +            self.assertEqual(hash(b), hash(c))
> 
> These checks could do with a comment explaining why the if statements
> are needed (I'm assuming something to do with memory order).

The checks aren't needed; they were supposed to spell out the equality-hash
relationship. But that isn't obvious, so I'll add a comment or remove them.


Stefan Krah



From solipsis at pitrou.net  Sat Nov  3 11:17:48 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 3 Nov 2012 11:17:48 +0100
Subject: [Python-Dev] [Python-checkins] cpython (3.3): Issue #15814: Use
 hash function that is compatible with the equality
References: <3XtV7q58c6zRMy@mail.python.org>
	<CADiSq7fSaz6uxQeoQeyi0aukH3kLifUAmSpd1ZRJe2KedXD5DQ@mail.gmail.com>
	<20121103095923.GA30827@sleipnir.bytereef.org>
Message-ID: <20121103111748.1100c73e@pitrou.net>

On Sat, 3 Nov 2012 10:59:23 +0100
Stefan Krah <stefan at bytereef.org> wrote:
> Nick Coghlan <ncoghlan at gmail.com> wrote:
> > > +        if (b == c):
> > > +            self.assertEqual(hash(b), hash(c))
> > 
> > These checks could do with a comment explaining why the if statements
> > are needed (I'm assuming something to do with memory order).
> 
> The checks aren't needed; they were supposed to spell out the equality-hash
> relationship.

Better use assertEqual(), then.

Regards

Antoine.



From techtonik at gmail.com  Sat Nov  3 12:29:57 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Sat, 3 Nov 2012 14:29:57 +0300
Subject: [Python-Dev] Using Wiki for Python development coordination
Message-ID: <CAPkN8xLkxbV8083NTr=+b_fXpxwJYuiNwihmW43NNZgd2Sm99g@mail.gmail.com>

I don't have access to modify the front page.
http://wiki.python.org/moin/FrontPage

To me it lacks one more section concerning help with core development
infrastructure.

Python Core Development

Python Website <http://wiki.python.org/moin/PythonWebsite>


The entrypoint for everything concerning the language


Bug Tracker <http://wiki.python.org/moin/TrackerDevelopment>


Roundup and code review services we use


Package Index <http://wiki.python.org/moin/CheeseShopDev>


Some building blocks for your own projects (including frameworks for
database, GUI, Web programming)


-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121103/e85fb688/attachment.html>

From fijall at gmail.com  Sat Nov  3 15:48:25 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 3 Nov 2012 16:48:25 +0200
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAP1=2W4VMMDCmQk-7VKEyK3zFCq9R8bsK3pv6hV69sCLA0MQsA@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<3E792B3D-BEA9-4E9F-9102-7F18FD3771E4@underboss.org>
	<CAP1=2W4VMMDCmQk-7VKEyK3zFCq9R8bsK3pv6hV69sCLA0MQsA@mail.gmail.com>
Message-ID: <CAK5idxSzr19zuWEd34EY0YscMBWnG-BU+W3vFd9MSb3ORPiNVg@mail.gmail.com>

On Fri, Nov 2, 2012 at 8:42 PM, Brett Cannon <brett at python.org> wrote:
> Issue filed for the performance issue: http://bugs.python.org/issue16390
>
> With that change and running on tip of Mako on my laptop now reports 1.25x
> slower which is much better than it was. This performance issue might also
> explain why all of the regex compilation benchmarks are worse under Python
> 3.3 by a decent margin.
>
> On Fri, Nov 2, 2012 at 2:16 PM, Philip Jenvey <pjenvey at underboss.org> wrote:
>>
>> lru_cache on re._compile_typed

I would like to warn you about modifying benchmarks like this (or
frameworks). Why is it relevant anyway?

From paul at boddie.org.uk  Sat Nov  3 15:54:45 2012
From: paul at boddie.org.uk (Paul Boddie)
Date: Sat, 3 Nov 2012 15:54:45 +0100
Subject: [Python-Dev] Using Wiki for Python development coordination
In-Reply-To: <CAPkN8xLkxbV8083NTr=+b_fXpxwJYuiNwihmW43NNZgd2Sm99g@mail.gmail.com>
References: <CAPkN8xLkxbV8083NTr=+b_fXpxwJYuiNwihmW43NNZgd2Sm99g@mail.gmail.com>
Message-ID: <201211031554.46007.paul@boddie.org.uk>

On Saturday 03 November 2012 12:29:57 anatoly techtonik wrote:
> I don't have access to modify the front page.
> http://wiki.python.org/moin/FrontPage

Yes, access was restricted a while ago because of spamming.

> To me it lacks one more section concerning help with core development
> infrastructure.
>
> Python Core Development
>
> Python Website <http://wiki.python.org/moin/PythonWebsite>
>
>
> The entrypoint for everything concerning the language
>
>
> Bug Tracker <http://wiki.python.org/moin/TrackerDevelopment>
>
>
> Roundup and code review services we use
>
>
> Package Index <http://wiki.python.org/moin/CheeseShopDev>
>
>
> Some building blocks for your own projects (including frameworks for
> database, GUI, Web programming)

I'll admit that the current content is just a reformatted version of what was 
there before, but tidied up and making better use of vertical space, and it 
could be improved. Certainly, there isn't a core development section, so 
perhaps I'll just add what you suggest. I know how you dislike the Moin 
markup, Anatoly. ;-)

Paul

From techtonik at gmail.com  Sat Nov  3 16:53:57 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Sat, 3 Nov 2012 18:53:57 +0300
Subject: [Python-Dev] Using Wiki for Python development coordination
In-Reply-To: <201211031554.46007.paul@boddie.org.uk>
References: <CAPkN8xLkxbV8083NTr=+b_fXpxwJYuiNwihmW43NNZgd2Sm99g@mail.gmail.com>
	<201211031554.46007.paul@boddie.org.uk>
Message-ID: <CAPkN8xJZVGW0qU+3x8wtUAct8uXG552ukXdi=fvtt57joosQXg@mail.gmail.com>

On Sat, Nov 3, 2012 at 5:54 PM, Paul Boddie <paul at boddie.org.uk> wrote:
>
>
> I'll admit that the current content is just a reformatted version of what was
> there before, but tidied up and making better use of vertical space, and it
> could be improved. Certainly, there isn't a core development section, so
> perhaps I'll just add what you suggest.

Thanks! That brings visibility of that rather critical section up to the highest
point possible. =)

> I know how you dislike the Moin
> markup, Anatoly. ;-)

Yes, I even think about further enhancing "Using this Wiki" part to
erase references to Moin help pages. They are a little bit misleading
to me and if somebody needs help on Moin - there is a side menu. I'd
like to change it from:

----
Feel free to add more useful stuff (see HelpContents and HelpOnEditing
to learn how), but do us a favour and do tests in the WikiSandBox if
you're not accustomed to Wiki technologies. If you're new to Wikis,
please read WikiWikiWeb.

 See WikiGuidelines for details of the policies and rules governing this Wiki.

 See SiteImprovements for a discussion of improvements to this Wiki
and other related sites.

 See RecentChanges for a history, available in RSS format. To see
pages which need writing, take a look at DesiredPages.

To report problems or to help out, please contact the python.org maintainers.
----

to

----
This Wiki is community place to gather and organize all things about
Python. Feel free to exercise your editorial skills and expertise to
improve it into a useful knowledge base and up-to-date reference on
different matters.

There are some WikiGuidelines if you feel lost. SiteImprovements
contains TODO list with various tasks you can help with. Check
RecentChanges frequently to read the news (also available in RSS).

In case of emergency please contact the python.org maintainers, or pop
up in pydotorg-www to say "help".
----

From brett at python.org  Sat Nov  3 17:29:18 2012
From: brett at python.org (Brett Cannon)
Date: Sat, 3 Nov 2012 12:29:18 -0400
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAK5idxSzr19zuWEd34EY0YscMBWnG-BU+W3vFd9MSb3ORPiNVg@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<3E792B3D-BEA9-4E9F-9102-7F18FD3771E4@underboss.org>
	<CAP1=2W4VMMDCmQk-7VKEyK3zFCq9R8bsK3pv6hV69sCLA0MQsA@mail.gmail.com>
	<CAK5idxSzr19zuWEd34EY0YscMBWnG-BU+W3vFd9MSb3ORPiNVg@mail.gmail.com>
Message-ID: <CAP1=2W4MY31QTEPRzkEcYwW7uhsxgcytv9eSy2LhoegFQYio0Q@mail.gmail.com>

I'm not modifying any benchmark or framework. At best I will replace Mako
0.7.2 with Mako 0.7.3 in the benchmark suite since no one is historically
recording the mako_v2 benchmark yet and it should be running with the
newest version until we set it in stone.


On Sat, Nov 3, 2012 at 10:48 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> On Fri, Nov 2, 2012 at 8:42 PM, Brett Cannon <brett at python.org> wrote:
> > Issue filed for the performance issue: http://bugs.python.org/issue16390
> >
> > With that change and running on tip of Mako on my laptop now reports
> 1.25x
> > slower which is much better than it was. This performance issue might
> also
> > explain why all of the regex compilation benchmarks are worse under
> Python
> > 3.3 by a decent margin.
> >
> > On Fri, Nov 2, 2012 at 2:16 PM, Philip Jenvey <pjenvey at underboss.org>
> wrote:
> >>
> >> lru_cache on re._compile_typed
>
> I would like to warn you about modifying benchmarks like this (or
> frameworks). Why is it relevant anyway?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121103/fd85810c/attachment.html>

From solipsis at pitrou.net  Sat Nov  3 18:13:43 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 3 Nov 2012 18:13:43 +0100
Subject: [Python-Dev] cpython: Issue #16218: skip test if filesystem
 doesn't support required encoding
References: <3Xv06h2mWtzMCY@mail.python.org>
Message-ID: <20121103181343.27e690d7@pitrou.net>

On Sat,  3 Nov 2012 13:37:48 +0100 (CET)
andrew.svetlov <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/95d1adf144ee
> changeset:   80187:95d1adf144ee
> user:        Andrew Svetlov <andrew.svetlov at gmail.com>
> date:        Sat Nov 03 14:37:37 2012 +0200
> summary:
>   Issue #16218: skip test if filesystem doesn't support required encoding
> 
> files:
>   Lib/test/test_cmd_line_script.py |  7 ++++++-
>   1 files changed, 6 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py
> --- a/Lib/test/test_cmd_line_script.py
> +++ b/Lib/test/test_cmd_line_script.py
> @@ -366,7 +366,12 @@
>      def test_non_utf8(self):
>          # Issue #16218
>          with temp_dir() as script_dir:
> -            script_basename = '\udcf1\udcea\udcf0\udce8\udcef\udcf2'
> +            script_basename = '\u0441\u043a\u0440\u0438\u043f\u0442'

Why exactly did you change the tested name here?

Regards

Antoine.



From chris.jerdonek at gmail.com  Sat Nov  3 18:36:28 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sat, 3 Nov 2012 10:36:28 -0700
Subject: [Python-Dev] [Python-checkins] peps: Revert PEP 430 to Final.
In-Reply-To: <3XqHjw6HZ6zR4V@mail.python.org>
References: <3XqHjw6HZ6zR4V@mail.python.org>
Message-ID: <CAOTb1wf6U_bEBK5oJ2hdBD4+KPd+02t+MZzGs75WNM23ie0hRw@mail.gmail.com>

On Sun, Oct 28, 2012 at 5:07 AM, nick.coghlan
<python-checkins at python.org> wrote:
>
> http://hg.python.org/peps/rev/1ccf682bdfc9
> changeset:   4575:1ccf682bdfc9
> user:        Nick Coghlan <ncoghlan at gmail.com>
> date:        Sun Oct 28 22:06:46 2012 +1000
> summary:
>   Revert PEP 430 to Final.
> ...
> -To help ensure future stability even of links to the in-development version,
> -the ``/dev/`` subpath will be redirected to the appropriate version specific
> -subpath (currently ``/3.4/``).
> ...
>  * ``http://docs.python.org/x/*``
>  * ``http://docs.python.org/x.y/*``
> +* ``http://docs.python.org/dev/*``
>  * ``http://docs.python.org/release/x.y.z/*``
>  * ``http://docs.python.org/devguide``
> ...
> +The ``/dev/`` URL means the documentation for the default branch in source
> +control.

I noticed on an older issue in the tracker that the following deep,
in-development link is broken:

http://docs.python.org/dev/2.6/library/multiprocessing.html

(in comment http://bugs.python.org/issue4711#msg78151 )

Can something be done to preserve those deep links, or is it not worth
worrying about?  I'm not sure how prevalent such links are or when
they were valid (and when they stopped working, assuming they once
did).

--Chris

From chris.jerdonek at gmail.com  Sat Nov  3 20:36:36 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sat, 3 Nov 2012 12:36:36 -0700
Subject: [Python-Dev] [Python-checkins] peps: Revert PEP 430 to Final.
In-Reply-To: <50955FE9.3000400@udel.edu>
References: <3XqHjw6HZ6zR4V@mail.python.org>
	<CAOTb1wf6U_bEBK5oJ2hdBD4+KPd+02t+MZzGs75WNM23ie0hRw@mail.gmail.com>
	<50955FE9.3000400@udel.edu>
Message-ID: <CAOTb1wdTD6nY5y8CP8EmGPbeimxpPqeFFpbhqOj7uVAEp6iZxg@mail.gmail.com>

On Sat, Nov 3, 2012 at 11:18 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>
> On 11/3/2012 1:36 PM, Chris Jerdonek wrote:
>>
>> I noticed on an older issue in the tracker that the following deep,
>> in-development link is broken:
>>
>> http://docs.python.org/dev/2.6/library/multiprocessing.html
>>
>> (in comment http://bugs.python.org/issue4711#msg78151 )
>>
>> Can something be done to preserve those deep links, or is it not worth
>> worrying about?  I'm not sure how prevalent such links are or when
>> they were valid (and when they stopped working, assuming they once
>> did).
>
>
> I'd say not bother. The message was posted 2008-12-11 and the link was
> already obsolete when Ezio answered 2009-07-01, when he gave a permanent
> one. Fortunately, the OP gave a .png screenshot showing the problem.
>
> Any doc link in the tracker becomes obsolete in a sense as soon as the
> problem noted is fixed. So tracker messages should state the problem
> independently of the link. (After the docs are rebuilt tonight,
> http://docs.python.org/library/multiprocessing.html
> will no longer exhibit the problem either ;-).

Though this link will still exhibit it (which may be what it would redirect to):

http://docs.python.org/2.6/library/multiprocessing.html

So a doc link in the tracker won't become obsolete if we wait long
enough to fix the issue. :)

--Chris

From techtonik at gmail.com  Mon Nov  5 08:31:26 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 5 Nov 2012 10:31:26 +0300
Subject: [Python-Dev] ctypes is not an acceptable implementation strategy
 for modules in the standard library?
Message-ID: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>

>From http://bugs.python.org/issue16410
Subj?

Aren't there any modules in stdlib that access system API through ctypes?

My arguments for ctypes:
1. doesn't require compilation
2. easier to maintain (no C/toolchain knowledge/ownership needed)
3. pure Python is impossible to exploit (unlike pure C)
4. eating your own dogfood helps to make modules complete and notice
such silly/critical/timewasting/drivesmemad errors as
http://bugs.python.org/issue16376 a few years earlier

Maybe it could even help to make ctypes faster (through some caching mechanizm).
--
anatoly t.

From ronaldoussoren at mac.com  Mon Nov  5 10:32:18 2012
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Mon, 05 Nov 2012 10:32:18 +0100
Subject: [Python-Dev] ctypes is not an acceptable implementation
 strategy for modules in the standard library?
In-Reply-To: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>
References: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>
Message-ID: <81A6AADC-39DE-4B46-B6F2-D545ED3C4C72@mac.com>


On 5 Nov, 2012, at 8:31, anatoly techtonik <techtonik at gmail.com> wrote:

> From http://bugs.python.org/issue16410
> Subj?
> 
> Aren't there any modules in stdlib that access system API through ctypes?

uuid uses ctypes to access platform APIs.
> 
> My arguments for ctypes:
> 1. doesn't require compilation
> 2. easier to maintain (no C/toolchain knowledge/ownership needed)
> 3. pure Python is impossible to exploit (unlike pure C)

That's not not quite true, python code that uses ctypes can still cause buffer overflows and the like when you aren't careful, and adds new failure modes (such as incorrect specification of a C function prototype in the Python code that calls that C function)

Ronald

From victor.stinner at gmail.com  Mon Nov  5 12:36:00 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Mon, 5 Nov 2012 12:36:00 +0100
Subject: [Python-Dev] ctypes is not an acceptable implementation
 strategy for modules in the standard library?
In-Reply-To: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>
References: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>
Message-ID: <CAMpsgwZqW1SzJWhFCTj6W0K+=eN2uVQ4dzGc2m-seUa9hLPPWg@mail.gmail.com>

I'm not sure that ctypes is always available (available on all platforms).

Victor

From rdmurray at bitdance.com  Mon Nov  5 12:49:15 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 05 Nov 2012 06:49:15 -0500
Subject: [Python-Dev] ctypes is not an acceptable implementation
	strategy for modules in the standard library?
In-Reply-To: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>
References: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>
Message-ID: <20121105114915.5818225006A@webabinitio.net>

On Mon, 05 Nov 2012 10:31:26 +0300, anatoly techtonik <techtonik at gmail.com> wrote:
> Aren't there any modules in stdlib that access system API through ctypes?

No.  This is a policy.  Changing that policy would require a PEP (*after*
a discussion on python-ideas...which has probably already happened in
the past, I would guess).

--David

From dickinsm at gmail.com  Mon Nov  5 13:08:47 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 5 Nov 2012 12:08:47 +0000
Subject: [Python-Dev] Okay to document the "exec tuple" form of the exec
 statement in Python 2.7?
Message-ID: <CAAu3qLXJNOgLSnHxcTdFPYrfc2eCkRvKsKotw7LH=nzU2Mixpw@mail.gmail.com>

In Python 2, the 'exec' statement supports 'exec'-ing a (statement,
globals, locals) tuple:

>>> exec("print 2", {}, {})
2

This isn't currently documented at:
http://docs.python.org/reference/simple_stmts.html#the-exec-statement.
 It's easy to fix the docs, but in doing so we'd effectively be
blessing this form of exec as an official part of the language.  Do
people think it's acceptable to add this to the docs, or are there
good reasons for the 'exec tuple' form of the exec statement to remain
an undocumented feature?

See also http://bugs.python.org/issue16339.

Mark

From ncoghlan at gmail.com  Mon Nov  5 13:22:26 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 5 Nov 2012 22:22:26 +1000
Subject: [Python-Dev] Okay to document the "exec tuple" form of the exec
 statement in Python 2.7?
In-Reply-To: <CAAu3qLXJNOgLSnHxcTdFPYrfc2eCkRvKsKotw7LH=nzU2Mixpw@mail.gmail.com>
References: <CAAu3qLXJNOgLSnHxcTdFPYrfc2eCkRvKsKotw7LH=nzU2Mixpw@mail.gmail.com>
Message-ID: <CADiSq7fuYDPJawjKejJn6rn6aYpydZGfYyJv5SxoOUVom=i4hQ@mail.gmail.com>

On Mon, Nov 5, 2012 at 10:08 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> In Python 2, the 'exec' statement supports 'exec'-ing a (statement,
> globals, locals) tuple:
>
>>>> exec("print 2", {}, {})
> 2
>
> This isn't currently documented at:
> http://docs.python.org/reference/simple_stmts.html#the-exec-statement.
>  It's easy to fix the docs, but in doing so we'd effectively be
> blessing this form of exec as an official part of the language.  Do
> people think it's acceptable to add this to the docs, or are there
> good reasons for the 'exec tuple' form of the exec statement to remain
> an undocumented feature?
>
> See also http://bugs.python.org/issue16339.

If you can find an existing test for it, then definitely (although the
fact it didn't previously work on Jython suggests there may not be
one).

If there's no test, then I'd still be in favour of making it official
both by testing *and* documenting it, for the forward compatibility
benefits you mention on the tracker.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From dickinsm at gmail.com  Mon Nov  5 14:09:55 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 5 Nov 2012 13:09:55 +0000
Subject: [Python-Dev] Okay to document the "exec tuple" form of the exec
 statement in Python 2.7?
In-Reply-To: <CADiSq7fuYDPJawjKejJn6rn6aYpydZGfYyJv5SxoOUVom=i4hQ@mail.gmail.com>
References: <CAAu3qLXJNOgLSnHxcTdFPYrfc2eCkRvKsKotw7LH=nzU2Mixpw@mail.gmail.com>
	<CADiSq7fuYDPJawjKejJn6rn6aYpydZGfYyJv5SxoOUVom=i4hQ@mail.gmail.com>
Message-ID: <CAAu3qLX_qyiCAtnOzrqceuHj_CmxdM4m=73cwbM8P6tfvqrU+A@mail.gmail.com>

On Mon, Nov 5, 2012 at 12:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> If you can find an existing test for it, then definitely (although the
> fact it didn't previously work on Jython suggests there may not be
> one).

I don't see any *direct* tests for this feature, though there are a
couple of tests that just happen to use that form of 'exec'
(test_pkg.py, test_module.py, test_pep263,py).

> If there's no test, then I'd still be in favour of making it official
> both by testing *and* documenting it, for the forward compatibility
> benefits you mention on the tracker.

Sounds good to me. I'll add a note to the issue about tests.

Mark

From fijall at gmail.com  Mon Nov  5 14:38:23 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 5 Nov 2012 15:38:23 +0200
Subject: [Python-Dev] ctypes is not an acceptable implementation
 strategy for modules in the standard library?
In-Reply-To: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>
References: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>
Message-ID: <CAK5idxQeTySE6w5KMYu_9y2_b_94PLurtuNzGQ0xSTCNrLOOkQ@mail.gmail.com>

On Mon, Nov 5, 2012 at 9:31 AM, anatoly techtonik <techtonik at gmail.com> wrote:
> From http://bugs.python.org/issue16410
> Subj?
>
> Aren't there any modules in stdlib that access system API through ctypes?
>
> My arguments for ctypes:
> 1. doesn't require compilation
> 2. easier to maintain (no C/toolchain knowledge/ownership needed)
> 3. pure Python is impossible to exploit (unlike pure C)
> 4. eating your own dogfood helps to make modules complete and notice
> such silly/critical/timewasting/drivesmemad errors as
> http://bugs.python.org/issue16376 a few years earlier
>
> Maybe it could even help to make ctypes faster (through some caching mechanizm).
> --
> anatoly t.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com

Hi anatoly.

ctypes comes with it's own set of problems that manifest themselves
more or less depending what sort of libary have you tried to wrap.
Have you ever tried to use it seriously? The list of my personal
issues is available here:
http://morepypy.blogspot.com/2012/06/release-01-of-cffi.html

The main problem is API vs ABI and the robustness of checks.

I would not recommend using ctypes for any of the sdtlib (we actually
tried in pypy, it turned out a bit awful).

Cheers,
fijal

From catch-all at masklinn.net  Mon Nov  5 15:14:17 2012
From: catch-all at masklinn.net (Xavier Morel)
Date: Mon, 5 Nov 2012 15:14:17 +0100
Subject: [Python-Dev] ctypes is not an acceptable implementation
	strategy for modules in the standard library?
In-Reply-To: <81A6AADC-39DE-4B46-B6F2-D545ED3C4C72@mac.com>
References: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>
	<81A6AADC-39DE-4B46-B6F2-D545ED3C4C72@mac.com>
Message-ID: <B3F74D2B-F87B-41D1-9478-AA022A969DE5@masklinn.net>

On 2012-11-05, at 10:32 , Ronald Oussoren wrote:
>> My arguments for ctypes:
>> 1. doesn't require compilation
>> 2. easier to maintain (no C/toolchain knowledge/ownership needed)
>> 3. pure Python is impossible to exploit (unlike pure C)
> 
> That's not not quite true, python code that uses ctypes can still cause buffer overflows and the like

Such as segfaulting the interpreter. I seem to reliably segfault
everything every time I try to use ctypes.

From sturla at molden.no  Mon Nov  5 15:47:05 2012
From: sturla at molden.no (Sturla Molden)
Date: Mon, 05 Nov 2012 15:47:05 +0100
Subject: [Python-Dev] ctypes is not an acceptable implementation
 strategy for modules in the standard library?
In-Reply-To: <B3F74D2B-F87B-41D1-9478-AA022A969DE5@masklinn.net>
References: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>
	<81A6AADC-39DE-4B46-B6F2-D545ED3C4C72@mac.com>
	<B3F74D2B-F87B-41D1-9478-AA022A969DE5@masklinn.net>
Message-ID: <5097D169.7060304@molden.no>


On 05.11.2012 15:14, Xavier Morel wrote:

 > Such as segfaulting the interpreter. I seem to reliably segfault
 > everything every time I try to use ctypes.

You can do that with C extensions too, by the way. Apart from that, 
dependency on ABI is more annoying to maintain across platforms than 
dependency on API. Function calls with ctypes are also very slow. For C 
extensions in the stdlib, Cython might be a better choice then ctypes.

ctypes might be a good choice if you are to use a DLL on your own 
computer. Because then you only have one ABI to worry about. Not so for 
Python's standard library.


Sturla

From guido at python.org  Mon Nov  5 15:57:30 2012
From: guido at python.org (Guido van Rossum)
Date: Mon, 5 Nov 2012 06:57:30 -0800
Subject: [Python-Dev] Okay to document the "exec tuple" form of the exec
 statement in Python 2.7?
In-Reply-To: <CADiSq7fuYDPJawjKejJn6rn6aYpydZGfYyJv5SxoOUVom=i4hQ@mail.gmail.com>
References: <CAAu3qLXJNOgLSnHxcTdFPYrfc2eCkRvKsKotw7LH=nzU2Mixpw@mail.gmail.com>
	<CADiSq7fuYDPJawjKejJn6rn6aYpydZGfYyJv5SxoOUVom=i4hQ@mail.gmail.com>
Message-ID: <CAP7+vJKJOH-58tXxLZ9J4jULcqFVMD1xwq=8RudxZqSQEGQsmw@mail.gmail.com>

+1. This probably fell through the cracks because I changed my mind on
this. The function form was old, and I wanted to root it out in favor fo
the statement. But then I changed my mind for py3k. Good idea to document
and add tests in 2.x.

--Guido

On Monday, November 5, 2012, Nick Coghlan wrote:

> On Mon, Nov 5, 2012 at 10:08 PM, Mark Dickinson <dickinsm at gmail.com<javascript:;>>
> wrote:
> > In Python 2, the 'exec' statement supports 'exec'-ing a (statement,
> > globals, locals) tuple:
> >
> >>>> exec("print 2", {}, {})
> > 2
> >
> > This isn't currently documented at:
> > http://docs.python.org/reference/simple_stmts.html#the-exec-statement.
> >  It's easy to fix the docs, but in doing so we'd effectively be
> > blessing this form of exec as an official part of the language.  Do
> > people think it's acceptable to add this to the docs, or are there
> > good reasons for the 'exec tuple' form of the exec statement to remain
> > an undocumented feature?
> >
> > See also http://bugs.python.org/issue16339.
>
> If you can find an existing test for it, then definitely (although the
> fact it didn't previously work on Jython suggests there may not be
> one).
>
> If there's no test, then I'd still be in favour of making it official
> both by testing *and* documenting it, for the forward compatibility
> benefits you mention on the tracker.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com <javascript:;>   |   Brisbane,
> Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <javascript:;>
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121105/901ba3dd/attachment.html>

From g.brandl at gmx.net  Mon Nov  5 19:04:34 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 05 Nov 2012 19:04:34 +0100
Subject: [Python-Dev] cpython (3.3): Add examples for opener argument of
	open (#13424).
In-Reply-To: <3XvCTv2Q38zPjt@mail.python.org>
References: <3XvCTv2Q38zPjt@mail.python.org>
Message-ID: <k78v3i$n6$1@ger.gmane.org>

Am 03.11.2012 22:10, schrieb eric.araujo:
> http://hg.python.org/cpython/rev/17b094c08600
> changeset:   80219:17b094c08600
> branch:      3.3
> parent:      80214:e6d0951f412a
> user:        ?ric Araujo <merwok at netwok.org>
> date:        Sat Nov 03 17:06:52 2012 -0400
> summary:
>   Add examples for opener argument of open (#13424).
> 
> Patch by Guillaume Pratte.
> 
> files:
>   Doc/library/functions.rst |  30 +++++++++++++++++++++++++++
>   Doc/library/io.rst        |   3 ++
>   Misc/ACKS                 |   1 +
>   3 files changed, 34 insertions(+), 0 deletions(-)
> 
> 
> diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
> --- a/Doc/library/functions.rst
> +++ b/Doc/library/functions.rst
> @@ -935,6 +935,36 @@
>     :mod:`os.open` as *opener* results in functionality similar to passing
>     ``None``).
>  
> +   The following example is an alternative implementation for opening files
> +   for exclusive writing.  If we did not have support for the ``'x'`` mode,
> +   we could implement it with this opener::
> +
> +      >>> import os
> +      >>> def open_exclusive(path, mode):
> +      ...     return os.open(path, mode | os.O_CREAT | os.O_EXCL)
> +      ...
> +      >>> filename = 'spam.txt'
> +      >>> fp = open(filename, 'w', opener=open_exclusive)
> +      >>> fp2 = open(filename, 'w', opener=open_exclusive)
> +      Traceback (most recent call last):
> +        ...
> +      FileExistsError: [Errno 17] File exists: 'spam.txt'
> +
> +   This other example uses the :ref:`dir_fd` parameter of the
> +   :func:`os.open` function to open a file relative to a given directory::

Please heed your Sphinx warnings: the :ref:`dir_fd` needs a link caption, since
it can't autogenerate one (the dir_fd anchor does not point to a heading).

Georg


From greg.ewing at canterbury.ac.nz  Mon Nov  5 22:51:01 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 06 Nov 2012 10:51:01 +1300
Subject: [Python-Dev] Okay to document the "exec tuple" form of the exec
 statement in Python 2.7?
In-Reply-To: <CADiSq7fuYDPJawjKejJn6rn6aYpydZGfYyJv5SxoOUVom=i4hQ@mail.gmail.com>
References: <CAAu3qLXJNOgLSnHxcTdFPYrfc2eCkRvKsKotw7LH=nzU2Mixpw@mail.gmail.com>
	<CADiSq7fuYDPJawjKejJn6rn6aYpydZGfYyJv5SxoOUVom=i4hQ@mail.gmail.com>
Message-ID: <509834C5.8060302@canterbury.ac.nz>

Nick Coghlan wrote:
> On Mon, Nov 5, 2012 at 10:08 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> 
>>In Python 2, the 'exec' statement supports 'exec'-ing a (statement,
>>globals, locals) tuple:

If this is a deliberate feature, I'd guess it's because exec
is a statement rather than a function in Python 2, so you
can't use * to pass a tuple of arguments to it.

-- 
Greg



From chris at simplistix.co.uk  Tue Nov  6 07:18:07 2012
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 06 Nov 2012 06:18:07 +0000
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
Message-ID: <5098AB9F.2010806@simplistix.co.uk>

Hi All,

I bumped into this using Michael Foord's Mock library.
It feels like a bug to me, but thought I'd ask here before logging one 
in the tracker in case people know that we won't be able to fix it:

On 05/11/2012 13:43, Michael Foord wrote:
>
>>>>>> class Foo(object):
>>> ...  def __setattr__(s, k, v):
>>> ...   print k
>>> ...
>>>>>> f = Foo()
>>>>>>
>>>>>> f.g = f.x = 3
>>> g
>>> x
>>
>> Hmm, that's definitely counter-intuitive. Worth raising on python-dev?
>
> Well, I guess "it's always been that way", so changing it would be backwards incompatible. Feel free to raise it though. :-)

Here's the actual problem I had:

>>>>>>> mock = Mock()
>>>>>>> inst = mock.Popen.return_value = mock.Popen_instance = Mock(spec=Popen)
>>>
>>>
>>> Here the *return_value* is set first. So when mock.Popen_instance is assigned it already has a parent! (Setting a mock as the return value of another mock also establishes the child / parent relationship.)
>>>
>>> This is why calls to the instance are then not recorded in "method_calls".

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk

From jeanpierreda at gmail.com  Tue Nov  6 09:54:08 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 6 Nov 2012 03:54:08 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <5098AB9F.2010806@simplistix.co.uk>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk>
Message-ID: <CABicbJKY0-P90rgdXfb3xi5sgs9ysH2iYg6HqfXpbuV9jNxCcQ@mail.gmail.com>

On Tue, Nov 6, 2012 at 1:18 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> Hi All,
>
> I bumped into this using Michael Foord's Mock library.
> It feels like a bug to me, but thought I'd ask here before logging one in
> the tracker in case people know that we won't be able to fix it:
>
> On 05/11/2012 13:43, Michael Foord wrote:
>>
>>
>>>>>>> class Foo(object):
>>>>
>>>> ...  def __setattr__(s, k, v):
>>>> ...   print k
>>>> ...
>>>>>>>
>>>>>>> f = Foo()
>>>>>>>
>>>>>>> f.g = f.x = 3
>>>>
>>>> g
>>>> x
>>>

It's terrible and counter-intuitive, but it is documented. See:
http://docs.python.org/2/reference/simple_stmts.html#assignment-statements

Notice that it proclaims that target lists are assigned to from left
to right. That's the behavior you've noticed.

On the other hand, one might easily misread this piece of
documentation, which implies right to left assignment:
http://docs.python.org/2/reference/expressions.html#evaluation-order

But the last note doesn't look normative to me.

(I'm not a dev, just trying to be helpful.)

-- Devin

From tjreedy at udel.edu  Tue Nov  6 10:24:37 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 06 Nov 2012 04:24:37 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <5098AB9F.2010806@simplistix.co.uk>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk>
Message-ID: <k7al0r$iom$1@ger.gmane.org>

On 11/6/2012 1:18 AM, Chris Withers wrote:
> Hi All,
>
> I bumped into this using Michael Foord's Mock library.
> It feels like a bug to me, but thought I'd ask here before logging one
> in the tracker in case people know that we won't be able to fix it:
>
> On 05/11/2012 13:43, Michael Foord wrote:
>>
>>>>>>> class Foo(object):
>>>> ...  def __setattr__(s, k, v):
>>>> ...   print k
>>>> ...
>>>>>>> f = Foo()
>>>>>>>
>>>>>>> f.g = f.x = 3
>>>> g
>>>> x
>>>
>>> Hmm, that's definitely counter-intuitive. Worth raising on python-dev?
>>
>> Well, I guess "it's always been that way", so changing it would be
>> backwards incompatible. Feel free to raise it though. :-)

You are expecting a chained assignment a = b = c to be parsed as
a = (b = c), as in C and as it should be if assignment is an expression. 
But in Python it is not. The right association would be meaningless 
since (b = c) has no value to be assigned to a.

Python actually parses the chained assignment as (a =) (b =) c.
The relevant grammar production is "assignment_stmt ::=
(target_list "=")+ (expression_list | yield_expression)".
The explanation after the grammar begins with this sentence.

"An assignment statement evaluates the expression list (remember that 
this can be a single expression or a comma-separated list, the latter 
yielding a tuple) and assigns the single resulting object to each of the 
target lists, from left to right."

In other words, in my example, c is assigned to a then b. Please do not 
report documented behavior on the tracker as a bug (and don't encourage 
people to). If you think the above is not clear enough, you *can* 
suggest improvement. Perhaps add an example and explanation such as

a = b, (c,d), *e, f = 1, (2, 3), 4, 5, 6

"The tuple on the right is first assigned to a and then unpacked to b, 
c, d, e, and f, giving them the values 1, 2, 3, [4, 5], and 6."

-- 
Terry Jan Reedy


From ncoghlan at gmail.com  Tue Nov  6 13:01:14 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 6 Nov 2012 22:01:14 +1000
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <k7al0r$iom$1@ger.gmane.org>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
Message-ID: <CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>

As noted, it's really only counterintuitive if your intuition is primed to
expect C style right to left chained assignments.

Python, on the other hand, is able to preserve primarily left to right
evaluation in this case with only the far right hand expression needing to
be evaluated out of order.

One example that can really make the intended behaviour clear:

*a = *b = iter(range(3))

a ends up as (0,1,2), b ends up as (), because the first assignment
consumes the entire iterable.

My actual advice, though? If the order of assignment really matters, use
multiple assignment statements rather than relying on readers knowing the
assignment order.

Cheers,
Nick.

--
Sent from my phone, thus the relative brevity :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121106/e9260b3d/attachment.html>

From trent at snakebite.org  Tue Nov  6 15:35:00 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 6 Nov 2012 09:35:00 -0500
Subject: [Python-Dev] ctypes is not an acceptable implementation
 strategy for modules in the standard library?
In-Reply-To: <CAMpsgwZqW1SzJWhFCTj6W0K+=eN2uVQ4dzGc2m-seUa9hLPPWg@mail.gmail.com>
References: <CAPkN8xJCTs=9meZSLEkP=wJ05vNUyx+TnDGQ3z30t0uCfSDmFQ@mail.gmail.com>
	<CAMpsgwZqW1SzJWhFCTj6W0K+=eN2uVQ4dzGc2m-seUa9hLPPWg@mail.gmail.com>
Message-ID: <20121106143458.GA86328@snakebite.org>

On Mon, Nov 05, 2012 at 03:36:00AM -0800, Victor Stinner wrote:
> I'm not sure that ctypes is always available (available on all platforms).

    Indeed.  Every non-x86 Snakebite platform has pretty serious issues
    with ctypes.  I spent a morning looking into one platform, Solaris
    10 on SPARC, and, after building the latest libffi from scratch, it
    failed more tests than that box chews amps.

        Trent.

From rob.cliffe at btinternet.com  Tue Nov  6 16:02:31 2012
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Tue, 06 Nov 2012 15:02:31 +0000
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
Message-ID: <50992687.2000505@btinternet.com>


On 06/11/2012 12:01, Nick Coghlan wrote:
>
> As noted, it's really only counterintuitive if your intuition is 
> primed to expect C style right to left chained assignments.
>
> Python, on the other hand, is able to preserve primarily left to right 
> evaluation in this case with only the far right hand expression 
> needing to be evaluated out of order.
>
>
>
It strikes me that a really intuitive language (at least for Westerners 
who read left-to-right) would write assignments as
     expression --> target
and then the order of assignment in
     expression -> target1 -> target2
could be the natural left-to-right one.
[Sorry, this is more appropriate to Python-ideas, but I couldn't resist 
adding my 2c.]
Rob Cliffe
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/rob.cliffe%40btinternet.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121106/2792fc1f/attachment.html>

From merwok at netwok.org  Tue Nov  6 16:12:54 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 06 Nov 2012 10:12:54 -0500
Subject: [Python-Dev] cpython (3.3): Add examples for opener argument of
 open (#13424).
In-Reply-To: <k78v3i$n6$1@ger.gmane.org>
References: <3XvCTv2Q38zPjt@mail.python.org> <k78v3i$n6$1@ger.gmane.org>
Message-ID: <509928F6.7080906@netwok.org>

Hi,

Le 05/11/2012 13:04, Georg Brandl a ?crit :
> Please heed your Sphinx warnings: the :ref:`dir_fd` needs a link caption, since
> it can't autogenerate one (the dir_fd anchor does not point to a heading).

Okay.  I hadn?t noticed it because I was using my system sphinx-build
instead of a local one and there were many warnings related (I think) to
pyspecific extensions not found.  Will fix (and also simplify the
examples per the reviews I got).

Cheers

From python at mrabarnett.plus.com  Tue Nov  6 16:32:23 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 06 Nov 2012 15:32:23 +0000
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <50992687.2000505@btinternet.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<50992687.2000505@btinternet.com>
Message-ID: <50992D87.20007@mrabarnett.plus.com>

On 2012-11-06 15:02, Rob Cliffe wrote:
>
> On 06/11/2012 12:01, Nick Coghlan wrote:
>>
>> As noted, it's really only counterintuitive if your intuition is
>> primed to expect C style right to left chained assignments.
>>
>> Python, on the other hand, is able to preserve primarily left to right
>> evaluation in this case with only the far right hand expression
>> needing to be evaluated out of order.
>>
> It strikes me that a really intuitive language (at least for Westerners
> who read left-to-right) would write assignments as
>      expression --> target
> and then the order of assignment in
>      expression -> target1 -> target2
> could be the natural left-to-right one.

That would make augmented assignment more difficult. For example, how
would you write the equivalent of "x -= y"?

> [Sorry, this is more appropriate to Python-ideas, but I couldn't resist
> adding my 2c.]
> Rob Cliffe



From guido at python.org  Tue Nov  6 16:37:45 2012
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Nov 2012 07:37:45 -0800
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
Message-ID: <CAP7+vJLv7Q_nCCxas3t94uj2xm+2WY6Fo2gm7Xa0iPOHvE-wpQ@mail.gmail.com>

+1 to what Nick said. And I thought about this carefully when
designing the language. It's not a bug. The note about assignment RHS
being evaluated before LHS is normative -- you just have to interpret
RHS as "after the *last* '=' symbol". Assignment itself is *not* an
expression.

On Tue, Nov 6, 2012 at 4:01 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> As noted, it's really only counterintuitive if your intuition is primed to
> expect C style right to left chained assignments.
>
> Python, on the other hand, is able to preserve primarily left to right
> evaluation in this case with only the far right hand expression needing to
> be evaluated out of order.
>
> One example that can really make the intended behaviour clear:
>
> *a = *b = iter(range(3))
>
> a ends up as (0,1,2), b ends up as (), because the first assignment consumes
> the entire iterable.
>
> My actual advice, though? If the order of assignment really matters, use
> multiple assignment statements rather than relying on readers knowing the
> assignment order.
>
> Cheers,
> Nick.
>
> --
> Sent from my phone, thus the relative brevity :)
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)

From storchaka at gmail.com  Tue Nov  6 17:14:38 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 06 Nov 2012 18:14:38 +0200
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
Message-ID: <k7bd1m$i5b$1@ger.gmane.org>

On 06.11.12 14:01, Nick Coghlan wrote:
> Python, on the other hand, is able to preserve primarily left to right
> evaluation in this case with only the far right hand expression needing
> to be evaluated out of order.

I'm surprised, but it is really so.

   >>> {}[print('foo')] = print('bar')
   bar
   foo

I was expecting "foo" before "bar".

Another counterintuitive (and possible wrong) example:

   >>> {print('foo'): print('bar')}
   bar
   foo
   {None: None}



From rdmurray at bitdance.com  Tue Nov  6 17:26:55 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 06 Nov 2012 11:26:55 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <k7bd1m$i5b$1@ger.gmane.org>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
Message-ID: <20121106162655.A858625006B@webabinitio.net>


On Tue, 06 Nov 2012 18:14:38 +0200, Serhiy Storchaka <storchaka at gmail.com> wrote:
> Another counterintuitive (and possible wrong) example:
> 
>    >>> {print('foo'): print('bar')}
>    bar
>    foo
>    {None: None}

http://bugs.python.org/issue11205

--David

From ned at nedbatchelder.com  Tue Nov  6 18:58:53 2012
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 06 Nov 2012 12:58:53 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <20121106162655.A858625006B@webabinitio.net>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
Message-ID: <50994FDD.8060906@nedbatchelder.com>

On 11/6/2012 11:26 AM, R. David Murray wrote:
> On Tue, 06 Nov 2012 18:14:38 +0200, Serhiy Storchaka <storchaka at gmail.com> wrote:
>> Another counterintuitive (and possible wrong) example:
>>
>>     >>> {print('foo'): print('bar')}
>>     bar
>>     foo
>>     {None: None}
> http://bugs.python.org/issue11205

This seems to me better left undefined, since there's hardly ever a need 
to know the precise evaluation sequence between keys and values, and 
retaining some amount of "unspecified" to allow for implementation 
flexibility is a good thing.

--Ned.

>
> --David
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com
>


From jeanpierreda at gmail.com  Tue Nov  6 19:19:56 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Tue, 6 Nov 2012 13:19:56 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <50994FDD.8060906@nedbatchelder.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
Message-ID: <CABicbJK+_3aFmcfegELAt=Kgd1n89jzW13ukHL3gn=woiAqUUQ@mail.gmail.com>

On Nov 6, 2012 1:05 PM, "Ned Batchelder" <ned at nedbatchelder.com> wrote:
>
> On 11/6/2012 11:26 AM, R. David Murray wrote:
>>
>> On Tue, 06 Nov 2012 18:14:38 +0200, Serhiy Storchaka <storchaka at gmail.com>
wrote:
>>>
>>> Another counterintuitive (and possible wrong) example:
>>>
>>>     >>> {print('foo'): print('bar')}
>>>     bar
>>>     foo
>>>     {None: None}
>>
>> http://bugs.python.org/issue11205
>
>
> This seems to me better left undefined, since there's hardly ever a need
to know the precise evaluation sequence between keys and values, and
retaining some amount of "unspecified" to allow for implementation
flexibility is a good thing.

"Left undefined"? The behavior was defined, but CPython didn't follow the
defined behaviour.

--Devin (phone)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121106/3f9e0091/attachment.html>

From ned at nedbatchelder.com  Tue Nov  6 20:00:17 2012
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Tue, 06 Nov 2012 14:00:17 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CABicbJK+_3aFmcfegELAt=Kgd1n89jzW13ukHL3gn=woiAqUUQ@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CABicbJK+_3aFmcfegELAt=Kgd1n89jzW13ukHL3gn=woiAqUUQ@mail.gmail.com>
Message-ID: <50995E41.5080907@nedbatchelder.com>


On 11/6/2012 1:19 PM, Devin Jeanpierre wrote:
>
>
> On Nov 6, 2012 1:05 PM, "Ned Batchelder" <ned at nedbatchelder.com 
> <mailto:ned at nedbatchelder.com>> wrote:
> >
> > On 11/6/2012 11:26 AM, R. David Murray wrote:
> >>
> >> On Tue, 06 Nov 2012 18:14:38 +0200, Serhiy Storchaka 
> <storchaka at gmail.com <mailto:storchaka at gmail.com>> wrote:
> >>>
> >>> Another counterintuitive (and possible wrong) example:
> >>>
> >>>     >>> {print('foo'): print('bar')}
> >>>     bar
> >>>     foo
> >>>     {None: None}
> >>
> >> http://bugs.python.org/issue11205
> >
> >
> > This seems to me better left undefined, since there's hardly ever a 
> need to know the precise evaluation sequence between keys and values, 
> and retaining some amount of "unspecified" to allow for implementation 
> flexibility is a good thing.
>
> "Left undefined"? The behavior was defined, but CPython didn't follow 
> the defined behaviour.
>

I would change the reference manual to leave it undefined.  Clearly not 
many people have been bothered by the fact that CPython implemented it 
"wrong".  If someone really needs to control whether the keys or values 
are evaluated first, they shouldn't use a dict literal.

--Ned.

> --Devin (phone)
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121106/0a5b1d93/attachment.html>

From greg.ewing at canterbury.ac.nz  Tue Nov  6 22:30:58 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 07 Nov 2012 10:30:58 +1300
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <50992D87.20007@mrabarnett.plus.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<50992687.2000505@btinternet.com> <50992D87.20007@mrabarnett.plus.com>
Message-ID: <50998192.9080907@canterbury.ac.nz>

MRAB wrote:
> That would make augmented assignment more difficult. For example, how
> would you write the equivalent of "x -= y"?

SUBTRACT x FROM y.

CLOSE POST WITH SMILEY.

-- 
Greg

From guido at python.org  Tue Nov  6 23:12:40 2012
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Nov 2012 14:12:40 -0800
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <50994FDD.8060906@nedbatchelder.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
Message-ID: <CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>

On Tue, Nov 6, 2012 at 9:58 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> On 11/6/2012 11:26 AM, R. David Murray wrote:
>>
>> On Tue, 06 Nov 2012 18:14:38 +0200, Serhiy Storchaka <storchaka at gmail.com>
>> wrote:
>>>
>>> Another counterintuitive (and possible wrong) example:
>>>
>>>     >>> {print('foo'): print('bar')}
>>>     bar
>>>     foo
>>>     {None: None}
>>
>> http://bugs.python.org/issue11205
>
>
> This seems to me better left undefined, since there's hardly ever a need to
> know the precise evaluation sequence between keys and values, and retaining
> some amount of "unspecified" to allow for implementation flexibility is a
> good thing.

Maybe. Do note that Python tries to be *different* than your average
C++ standard and actually prescribes evaluation orders in most cases.
The idea being that the kind of optimizations that C++ compilers get
to do by moving evaluation order around aren't worth it in Python
anyway, and it's better for the user if there are no arbitrary
differences in this area between Python implementations. Note that I
didn't say "no surprises" -- the post that started this thread shows
that surprises are still possible.

-- 
--Guido van Rossum (python.org/~guido)

From jiuc at yahoo.cn  Wed Nov  7 06:24:11 2012
From: jiuc at yahoo.cn (=?utf-8?B?6YKT5Lmd56Wl?=)
Date: Wed, 7 Nov 2012 13:24:11 +0800 (CST)
Subject: [Python-Dev] =?utf-8?b?5Zue5aSN77yaIFB5dGhvbi1EZXYgRGlnZXN0LCBW?=
	=?utf-8?q?ol_112=2C_Issue_8?=
In-Reply-To: <mailman.13042.1352237459.27097.python-dev@python.org>
References: <mailman.13042.1352237459.27097.python-dev@python.org>
Message-ID: <1352265851.72557.YahooMailNeo@web92103.mail.cnh.yahoo.com>

I hava some question about Object which "inital" need ,but return "object" in mongo shell. So?
?BSONElement initial = p["initial"];
????????????if ( initial.type() != Object ) {
????????????????errmsg = "initial has to be an object";
????????????????return false;
????????????}
?"initial.type() != Object " will be found ,so "group" command will not run forever. What can'i do
Mybe this is my fase, please give me some advise.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121107/2dbd62f2/attachment.html>

From storchaka at gmail.com  Wed Nov  7 08:49:43 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 07 Nov 2012 09:49:43 +0200
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <50995E41.5080907@nedbatchelder.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CABicbJK+_3aFmcfegELAt=Kgd1n89jzW13ukHL3gn=woiAqUUQ@mail.gmail.com>
	<50995E41.5080907@nedbatchelder.com>
Message-ID: <k7d3qm$sqi$1@ger.gmane.org>

On 06.11.12 21:00, Ned Batchelder wrote:
> If someone really needs to control whether the keys or values
> are evaluated first, they shouldn't use a dict literal.

Not only a dict literal.

 >>> {print('foo'): print('bar') for x in [1]}
bar
foo
{None: None}



From amauryfa at gmail.com  Wed Nov  7 09:48:22 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 7 Nov 2012 09:48:22 +0100
Subject: [Python-Dev]
	=?gb2312?b?u9i4tKO6IFB5dGhvbi1EZXYgRGlnZXN0LCBWb2wg?=
	=?gb2312?b?MTEyLCBJc3N1ZSA4?=
In-Reply-To: <1352265851.72557.YahooMailNeo@web92103.mail.cnh.yahoo.com>
References: <mailman.13042.1352237459.27097.python-dev@python.org>
	<1352265851.72557.YahooMailNeo@web92103.mail.cnh.yahoo.com>
Message-ID: <CAGmFidZSqfvLXA8wL=v6_and0P=adLX0-VqRuhftper=27vnYQ@mail.gmail.com>

Hi,

2012/11/7 ??? <jiuc at yahoo.cn>:
> I hava some question about Object which "inital" need ,but return "object"
> in mongo shell. So

This is the python-dev mailing list, which deals with development of
the Python language.
You should ask your question on some MongoDB forum.

>
>  BSONElement initial = p["initial"];
>             if ( initial.type()
>  != Object ) {
>                 errmsg = "initial has to be an object";
>                 return false;
>             }
>  "initial.type() != Object " will be found ,so "group" command will not run
> forever. What can'i do
> Mybe this is my fase, please give me some advise.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/amauryfa%40gmail.com
>



-- 
Amaury Forgeot d'Arc

From techtonik at gmail.com  Wed Nov  7 11:57:05 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 7 Nov 2012 13:57:05 +0300
Subject: [Python-Dev] urlretrieve regression no.2, now in Python 3.3
Message-ID: <CAPkN8xJtZ71gB3LdgvmQAy0cxKnwsCc7yMhpKj7c4Mb4T8jgmQ@mail.gmail.com>

urlretrieve has a callback parameter, which takes function with the
following prototype:

  def callback(block_number, block_size, total_size):
    pass

Where block_size was constant and block_size*block_number gave an exact
number of transferred bytes.

Recent change in Python 3.3 changed the semantic of block_size and broke my
`wget` package. http://bugs.python.org/issue16409


Can anybody tell me if I can wait for the fix in 3.3.1 or should implement
a long-term workaround myself?
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121107/99b06a29/attachment.html>

From techtonik at gmail.com  Wed Nov  7 12:02:08 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 7 Nov 2012 14:02:08 +0300
Subject: [Python-Dev] Fwd: [issue16424] regression:
	os.path.split('//hostname/foo/bar.txt')
In-Reply-To: <1352285169.2.0.24678892452.issue16424@psf.upfronthosting.co.za>
References: <1352268933.45.0.882120414377.issue16424@psf.upfronthosting.co.za>
	<1352285169.2.0.24678892452.issue16424@psf.upfronthosting.co.za>
Message-ID: <CAPkN8xJCm2FNJYVSrcvFP+zWZ=YOf1kpMnOHrO1dLptLidYB-A@mail.gmail.com>

Forwarding to python-dev.

Does everybody agree that the following behavior is not a regression, but a
feature of os.path.split()?

Python 3:
>>> import os.path as osp
>>> osp.split('//hostname/foo/')
('//hostname/foo/', '')

Python 2:
>>> osp.split('//hostname/foo/')
('//hostname/foo', '')


But Python 3 again:
>>> osp.split('//hostname/foo/bar/')
('//hostname/foo/bar', '')


http://bugs.python.org/issue16424
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121107/d9794b0b/attachment.html>

From ned at nedbatchelder.com  Wed Nov  7 13:13:00 2012
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 07 Nov 2012 07:13:00 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
Message-ID: <509A504C.10807@nedbatchelder.com>


On 11/6/2012 5:12 PM, Guido van Rossum wrote:
> On Tue, Nov 6, 2012 at 9:58 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
>> On 11/6/2012 11:26 AM, R. David Murray wrote:
>>> On Tue, 06 Nov 2012 18:14:38 +0200, Serhiy Storchaka <storchaka at gmail.com>
>>> wrote:
>>>> Another counterintuitive (and possible wrong) example:
>>>>
>>>>      >>> {print('foo'): print('bar')}
>>>>      bar
>>>>      foo
>>>>      {None: None}
>>> http://bugs.python.org/issue11205
>>
>> This seems to me better left undefined, since there's hardly ever a need to
>> know the precise evaluation sequence between keys and values, and retaining
>> some amount of "unspecified" to allow for implementation flexibility is a
>> good thing.
> Maybe. Do note that Python tries to be *different* than your average
> C++ standard and actually prescribes evaluation orders in most cases.
> The idea being that the kind of optimizations that C++ compilers get
> to do by moving evaluation order around aren't worth it in Python
> anyway, and it's better for the user if there are no arbitrary
> differences in this area between Python implementations. Note that I
> didn't say "no surprises" -- the post that started this thread shows
> that surprises are still possible.
>

I think it's unfortunate that the current patch in the referenced bug ( 
http://bugs.python.org/issue11205 ) fixes the "problem" by adding one 
more bytecode to the compiled Python.  The other alternative seems to be 
changing the meaning of two opcodes. Neither of these are great 
alternatives.  I know we don't promise to maintain backward 
compatibility in the meanings of opcodes, but I'll bet actual code will 
be broken by changing the meaning of STORE_MAP and MAP_ADD.  Slowing 
down dict displays (just slightly) to get this right seems unfortunate also.

If the bug report is accurate, CPython and the reference manual have 
disagreed since Python 2.5, and many of us are now surprised to hear it, 
which means there can't have been much broken code.

I understand the point about C compilers having more opportunities to 
take advantage of "undefined" in the spec, but Python implementations 
are getting cleverer about how to implement Python semantics as well, 
perhaps we should have some faith in their future cleverness and give 
them even a little bit of leeway.  I don't imagine this little bit will 
actually be useful to them, but making this undefined will already help 
CPython by avoiding either an extra bytecode or a change in the opcodes.

There are plenty of places where different Python implementations 
differ, and even careful observers had different ideas about how keys 
and values were ordered in dict displays ("I thought it was like a 
function call,"  vs, "I thought it was like an assignment"). We've gone 
out of our way to maintain backward compatibility with the implemented 
behavior before (ordering of dict keys, for example).  The simplest 
change to make here is to update the reference and keep the implementation.

--Ned.

From ulrich.eckhardt at dominolaser.com  Wed Nov  7 14:57:57 2012
From: ulrich.eckhardt at dominolaser.com (Ulrich Eckhardt)
Date: Wed, 07 Nov 2012 14:57:57 +0100
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <5091A31D.4000904@pearwood.info>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
	<5091A31D.4000904@pearwood.info>
Message-ID: <509A68E5.6020403@dominolaser.com>

Am 31.10.2012 23:15, schrieb Steven D'Aprano:
> On 01/11/12 06:57, anatoly techtonik wrote:
> [...]
>> UnboundLocalError: local variable 'FONT_NAMES' referenced before
>> assignment
[...]
>> I wonder if this message can be improved with a
>> pointer to the concept on when global variables become local?
>
> If you have a suggestion for an improved message, please tell us.

I'll take a shot, since I was also bitten by this when trying to learn 
Python. The important point is that some code earlier or later in that 
function does an assignment, so this location should be referenced in 
the error.

How about:

   "UnboundLocalError: Local variable 'FONT_NAMES' (created on
    line 11) referenced before assignment."

What I don't really like is the term "created". Maybe "implicitly 
created on line 11"? Or "implied by line 11"? Or how about "Local 
variable FONT_NAMES (implied by line 11) doesn't refer to an object", to 
avoid the multiple interpretations of the term "assignment"?

=just my 2cc=

Uli

**************************************************************************************
Domino Laser GmbH, Fangdieckstra???e 75a, 22547 Hamburg, Deutschland
Gesch???ftsf???hrer: Hans Robert Dapprich, Amtsgericht Hamburg HR B62 932
**************************************************************************************
Visit our website at http://www.dominolaser.com
**************************************************************************************
Diese E-Mail einschlie???lich s???mtlicher Anh???nge ist nur f???r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf???nger sein sollten. Die E-Mail ist in diesem Fall zu l???schen und darf weder gelesen, weitergeleitet, ver???ffentlicht oder anderweitig benutzt werden.
E-Mails k???nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ???nderungen enthalten. Domino Laser GmbH ist f???r diese Folgen nicht verantwortlich.
**************************************************************************************


From ncoghlan at gmail.com  Wed Nov  7 15:11:41 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 8 Nov 2012 00:11:41 +1000
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <509A504C.10807@nedbatchelder.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
Message-ID: <CADiSq7cB3w+h=_-CMT_9YN65urQ+VXRxf7J3m=3KKvUPsFE3qg@mail.gmail.com>

On Wed, Nov 7, 2012 at 10:13 PM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> There are plenty of places where different Python implementations differ,
> and even careful observers had different ideas about how keys and values
> were ordered in dict displays ("I thought it was like a function call,"  vs,
> "I thought it was like an assignment"). We've gone out of our way to
> maintain backward compatibility with the implemented behavior before
> (ordering of dict keys, for example).  The simplest change to make here is
> to update the reference and keep the implementation.

"The implementation is right, the docs are wrong" sounds good to me,
as it's easy to justify the out of order evaluation in terms of the
equivalent item assignment statements:

    x = {a:b, c:d}

vs

    x = {}
    x[a] = b
    x[c] = d

That relationship is quite logical given that (ignoring namespace
details) dict construction from a display [1] pretty much does the
equivalent of:

    result = {}
    for key_expr, val_expr in display_entries:
        result[eval(key_expr)] = eval(val_expr)

This comment [2] from the dict comprehension implementation makes it
explicit that the behaviour of the equivalent Python item assignment
code was taken to be normative.

[1] http://hg.python.org/cpython/file/default/Python/compile.c#l3319
[2] http://hg.python.org/cpython/file/default/Python/compile.c#l3020

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From sam.partington at gmail.com  Wed Nov  7 15:15:34 2012
From: sam.partington at gmail.com (Sam Partington)
Date: Wed, 7 Nov 2012 14:15:34 +0000
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <509A68E5.6020403@dominolaser.com>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
	<5091A31D.4000904@pearwood.info> <509A68E5.6020403@dominolaser.com>
Message-ID: <CABuPkmThPu929Gek0g85UOb+AoabFTv-7TGcMrrLpAnaeZq+XQ@mail.gmail.com>

On 7 November 2012 13:57, Ulrich Eckhardt
<ulrich.eckhardt at dominolaser.com> wrote:
> Am 31.10.2012 23:15, schrieb Steven D'Aprano:
> I'll take a shot, since I was also bitten by this when trying to learn
> Python. The important point is that some code earlier or later in that
> function does an assignment, so this location should be referenced in the
> error.

+1

> How about:
>
>   "UnboundLocalError: Local variable 'FONT_NAMES' (created on
>    line 11) referenced before assignment."
>
> What I don't really like is the term "created". Maybe "implicitly created on
> line 11"? Or "implied by line 11"? Or how about "Local variable FONT_NAMES
> (implied by line 11) doesn't refer to an object", to avoid the multiple
> interpretations of the term "assignment"?

Why not make use of the well defined words already there in the error
message, but at the line number

   "UnboundLocalError: Local variable 'FONT_NAMES' referenced before
it has been assigned to on line 11."

Sam

From rosuav at gmail.com  Wed Nov  7 15:16:01 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 8 Nov 2012 01:16:01 +1100
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CADiSq7cB3w+h=_-CMT_9YN65urQ+VXRxf7J3m=3KKvUPsFE3qg@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CADiSq7cB3w+h=_-CMT_9YN65urQ+VXRxf7J3m=3KKvUPsFE3qg@mail.gmail.com>
Message-ID: <CAPTjJmqJXATbW8mfC9VO5fCdeFbH8E-30y1POcDv0ohVBGTOwQ@mail.gmail.com>

On Thu, Nov 8, 2012 at 1:11 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> "The implementation is right, the docs are wrong" sounds good to me,
> as it's easy to justify the out of order evaluation in terms of the
> equivalent item assignment statements:

What do other Pythons than CPython do currently? Or is it "The
reference implementation is right, the docs are wrong"?

ChrisA

From skip at pobox.com  Wed Nov  7 15:16:57 2012
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 7 Nov 2012 08:16:57 -0600
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <509A68E5.6020403@dominolaser.com>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
	<5091A31D.4000904@pearwood.info> <509A68E5.6020403@dominolaser.com>
Message-ID: <CANc-5Uyhs0w0VQ-hBAC2nyE0gsbCek=4vR3=H6KdzmKw15wFzA@mail.gmail.com>

> How about:
>
>   "UnboundLocalError: Local variable 'FONT_NAMES' (created on
>    line 11) referenced before assignment."
>
> What I don't really like is the term "created". Maybe "implicitly created
> on line 11"? Or "implied by line 11"? Or how about "Local variable
> FONT_NAMES (implied by line 11) doesn't refer to an object", to avoid the
> multiple interpretations of the term "assignment"?

It's been a long (long) while since I looked at the virtual machine
implementation, but my guess is that at the point where the error is
detected the system won't know what line number corresponds to the
assignment.  There might also be multiple assignments.  How would you
know which one to pick?

As for a better word than "created", I would use "assigned."

Skip

From lrekucki at gmail.com  Wed Nov  7 15:19:55 2012
From: lrekucki at gmail.com (=?UTF-8?Q?=C5=81ukasz_Rekucki?=)
Date: Wed, 7 Nov 2012 15:19:55 +0100
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CAPTjJmqJXATbW8mfC9VO5fCdeFbH8E-30y1POcDv0ohVBGTOwQ@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CADiSq7cB3w+h=_-CMT_9YN65urQ+VXRxf7J3m=3KKvUPsFE3qg@mail.gmail.com>
	<CAPTjJmqJXATbW8mfC9VO5fCdeFbH8E-30y1POcDv0ohVBGTOwQ@mail.gmail.com>
Message-ID: <CAEZs-ELt5-LZ0kBobQF7jaopZw+yuADp3AVCxySxhZZOne9juQ@mail.gmail.com>

On 7 November 2012 15:16, Chris Angelico <rosuav at gmail.com> wrote:
> On Thu, Nov 8, 2012 at 1:11 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> "The implementation is right, the docs are wrong" sounds good to me,
>> as it's easy to justify the out of order evaluation in terms of the
>> equivalent item assignment statements:
>
> What do other Pythons than CPython do currently? Or is it "The
> reference implementation is right, the docs are wrong"?

PyPy and IronPython are the same as CPython. Only Jython (both 2.5 and
2.7a) follows the docs.

Regards,
?ukasz Rekucki

From rdmurray at bitdance.com  Wed Nov  7 15:20:19 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 07 Nov 2012 09:20:19 -0500
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
	variable referenced before assignment"
In-Reply-To: <509A68E5.6020403@dominolaser.com>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
	<5091A31D.4000904@pearwood.info> <509A68E5.6020403@dominolaser.com>
Message-ID: <20121107142019.BD46025006F@webabinitio.net>

On Wed, 07 Nov 2012 14:57:57 +0100, Ulrich Eckhardt <ulrich.eckhardt at dominolaser.com> wrote:
> Am 31.10.2012 23:15, schrieb Steven D'Aprano:
> > On 01/11/12 06:57, anatoly techtonik wrote:
> > [...]
> >> UnboundLocalError: local variable 'FONT_NAMES' referenced before
> >> assignment
> [...]
> >> I wonder if this message can be improved with a
> >> pointer to the concept on when global variables become local?
> >
> > If you have a suggestion for an improved message, please tell us.
> 
> I'll take a shot, since I was also bitten by this when trying to learn 
> Python. The important point is that some code earlier or later in that 
> function does an assignment, so this location should be referenced in 
> the error.
> 
> How about:
> 
>    "UnboundLocalError: Local variable 'FONT_NAMES' (created on
>     line 11) referenced before assignment."
> 
> What I don't really like is the term "created". Maybe "implicitly 
> created on line 11"? Or "implied by line 11"? Or how about "Local 
> variable FONT_NAMES (implied by line 11) doesn't refer to an object", to 
> avoid the multiple interpretations of the term "assignment"?

The problem is that when Python executes the statement that raises the
error it has no idea where the assignment was done.  All it knows is that
the variable is local.  Keeping track of the location of every assignment
that makes a variable local and writing it in to the .pyc file is a very
non-trivial change to how the Python bytecode compiler works, I think, and
probably not worth the overhead in order to improve this error message.

(And note that raising an error at compile time would change Python's
semantics.)

--David

From guido at python.org  Wed Nov  7 15:54:32 2012
From: guido at python.org (Guido van Rossum)
Date: Wed, 7 Nov 2012 06:54:32 -0800
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <509A504C.10807@nedbatchelder.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
Message-ID: <CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>

On Wed, Nov 7, 2012 at 4:13 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
> I think it's unfortunate that the current patch in the referenced bug (
> http://bugs.python.org/issue11205 ) fixes the "problem" by adding one more
> bytecode to the compiled Python.  The other alternative seems to be changing
> the meaning of two opcodes. Neither of these are great alternatives.  I know
> we don't promise to maintain backward compatibility in the meanings of
> opcodes, but I'll bet actual code will be broken by changing the meaning of
> STORE_MAP and MAP_ADD.  Slowing down dict displays (just slightly) to get
> this right seems unfortunate also.

I agree this would be unfortunate and I recommend that you add this to
the bug. I agree that we should be *very* conservative in changing the
meaning of existing opcodes (adding new one is a different story).

But it was fixed before (http://bugs.python.org/issue448679 has a
simple fix that doesn't change opcode meanings and was applied in
2001) -- what happened?

> If the bug report is accurate, CPython and the reference manual have
> disagreed since Python 2.5, and many of us are now surprised to hear it,
> which means there can't have been much broken code.

Give that it was discussed before and fixed before, I think the intent
is clear: we should fix the code, not the docs.

> I understand the point about C compilers having more opportunities to take
> advantage of "undefined" in the spec, but Python implementations are getting
> cleverer about how to implement Python semantics as well, perhaps we should
> have some faith in their future cleverness and give them even a little bit
> of leeway.

Hm. I really don't think that is a good development for Python to
compromise in the area of expression evaluation order where side
effects are involved. A good compiler should be able to detect the
absence of potential side effects. E.g. it might reorder when only
constants and simple variable references are present, or (in the case
of a JIT, which has run-time type information) when it knows enough
about the types of the operands involved to determine that operations
like getattr or getitem are guaranteed side-effect-free.

> I don't imagine this little bit will actually be useful to them,
> but making this undefined will already help CPython by avoiding either an
> extra bytecode or a change in the opcodes.

I haven't looked at the proposed fixes, but I think correctness is
more important than saving an extra bytecode (OTOH keeping the set of
opcodes the same trumps both). I can't imagine that this extra opcode
will be significant in many cases.

> There are plenty of places where different Python implementations differ,
> and even careful observers had different ideas about how keys and values
> were ordered in dict displays ("I thought it was like a function call,"  vs,
> "I thought it was like an assignment"). We've gone out of our way to
> maintain backward compatibility with the implemented behavior before
> (ordering of dict keys, for example).

Not sure what you're referencing here -- didn't we just start
randomizing hashing?

> The simplest change to make here is
> to update the reference and keep the implementation.

In this particular case I disagree.

-- 
--Guido van Rossum (python.org/~guido)

From ncoghlan at gmail.com  Wed Nov  7 16:01:59 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 8 Nov 2012 01:01:59 +1000
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <509A68E5.6020403@dominolaser.com>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
	<5091A31D.4000904@pearwood.info> <509A68E5.6020403@dominolaser.com>
Message-ID: <CADiSq7e6FWbccEB-NOHx-6ZUmG+AVnJVpnpYfkTu6Q1SFaNCAA@mail.gmail.com>

On Wed, Nov 7, 2012 at 11:57 PM, Ulrich Eckhardt
<ulrich.eckhardt at dominolaser.com> wrote:
> How about:
>
>   "UnboundLocalError: Local variable 'FONT_NAMES' (created on
>    line 11) referenced before assignment."
>
> What I don't really like is the term "created". Maybe "implicitly created on
> line 11"? Or "implied by line 11"? Or how about "Local variable FONT_NAMES
> (implied by line 11) doesn't refer to an object", to avoid the multiple
> interpretations of the term "assignment"?

Unfortunately, we don't track the information we would need in order
to emit that kind of error message. However, you did give me an idea:
I believe the compiler is actually in a position to emit SyntaxWarning
for functions that have a high chance of triggering UnboundLocalError
when called. With output pointing to *both* problematic lines,
beginners should stand a better chance of figuring out what is going
on. (http://bugs.python.org/issue16429)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From rosuav at gmail.com  Wed Nov  7 16:06:15 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 8 Nov 2012 02:06:15 +1100
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
Message-ID: <CAPTjJmr6bjdXVDS5D=eZbgsa3cbE3ZOxot1poz78bYOoUddKWQ@mail.gmail.com>

On Thu, Nov 8, 2012 at 1:54 AM, Guido van Rossum <guido at python.org> wrote:
> On Wed, Nov 7, 2012 at 4:13 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
>> We've gone out of our way to
>> maintain backward compatibility with the implemented behavior before
>> (ordering of dict keys, for example).
>
> Not sure what you're referencing here -- didn't we just start
> randomizing hashing?

Hash randomization could have been quietly introduced as a security
fix and backported to old versions. But since applications might have
depended on dict iteration order, it wasn't - old versions of Python
won't randomize hashes unless you set an environment variable. That's
taking care of old code, even when that old code is depending on
non-spec behaviour.

ChrisA

From ncoghlan at gmail.com  Wed Nov  7 16:12:37 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 8 Nov 2012 01:12:37 +1000
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
Message-ID: <CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>

On Thu, Nov 8, 2012 at 12:54 AM, Guido van Rossum <guido at python.org> wrote:
> On Wed, Nov 7, 2012 at 4:13 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
>> If the bug report is accurate, CPython and the reference manual have
>> disagreed since Python 2.5, and many of us are now surprised to hear it,
>> which means there can't have been much broken code.
>
> Give that it was discussed before and fixed before, I think the intent
> is clear: we should fix the code, not the docs.

Almost certainly, it was broken in the migration to the AST compiler
and there was no regression test to pick up the change.

> I haven't looked at the proposed fixes, but I think correctness is
> more important than saving an extra bytecode (OTOH keeping the set of
> opcodes the same trumps both). I can't imagine that this extra opcode
> will be significant in many cases.

Since you've indicated the implementation is in the wrong here and you
also want to preserve opcode semantics, I think Skip's patch is
correct, but also needs to be applied to dict comprehensions (now we
have them). The extra bytecode is only ROT_TWO, which is one of the
cheapest we have kicking around :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From guido at python.org  Wed Nov  7 16:17:41 2012
From: guido at python.org (Guido van Rossum)
Date: Wed, 7 Nov 2012 07:17:41 -0800
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
	<CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>
Message-ID: <CAP7+vJKnN9DDH=vTGs0kQSBwiFA72F8gu6aiOXX-+P4ikEF-jQ@mail.gmail.com>

Ok, somebody go for it! (Also please refer to my pronouncement in the bug
-- I've gotta run.)


On Wed, Nov 7, 2012 at 7:12 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Thu, Nov 8, 2012 at 12:54 AM, Guido van Rossum <guido at python.org>
> wrote:
> > On Wed, Nov 7, 2012 at 4:13 AM, Ned Batchelder <ned at nedbatchelder.com>
> wrote:
> >> If the bug report is accurate, CPython and the reference manual have
> >> disagreed since Python 2.5, and many of us are now surprised to hear it,
> >> which means there can't have been much broken code.
> >
> > Give that it was discussed before and fixed before, I think the intent
> > is clear: we should fix the code, not the docs.
>
> Almost certainly, it was broken in the migration to the AST compiler
> and there was no regression test to pick up the change.
>
> > I haven't looked at the proposed fixes, but I think correctness is
> > more important than saving an extra bytecode (OTOH keeping the set of
> > opcodes the same trumps both). I can't imagine that this extra opcode
> > will be significant in many cases.
>
> Since you've indicated the implementation is in the wrong here and you
> also want to preserve opcode semantics, I think Skip's patch is
> correct, but also needs to be applied to dict comprehensions (now we
> have them). The extra bytecode is only ROT_TWO, which is one of the
> cheapest we have kicking around :)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121107/f71543c4/attachment.html>

From storchaka at gmail.com  Wed Nov  7 18:08:25 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 07 Nov 2012 19:08:25 +0200
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
	<CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>
Message-ID: <k7e4ih$idp$1@ger.gmane.org>

On 07.11.12 17:12, Nick Coghlan wrote:
> Since you've indicated the implementation is in the wrong here and you
> also want to preserve opcode semantics, I think Skip's patch is
> correct, but also needs to be applied to dict comprehensions (now we
> have them). The extra bytecode is only ROT_TWO, which is one of the
> cheapest we have kicking around :)

Not only to dict comprehensions, but also to item assignments.  It will 
be weird if a dict comprehension and a plain loop will be inconsistent.



From tjreedy at udel.edu  Wed Nov  7 20:08:33 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 07 Nov 2012 14:08:33 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
Message-ID: <k7ebjk$lrl$1@ger.gmane.org>

On 11/7/2012 9:54 AM, Guido van Rossum wrote:

> Hm. I really don't think that is a good development for Python to
> compromise in the area of expression evaluation order where side
> effects are involved.

I agreee. I think Python's simple left to right evaluation order is one 
of its virtues.

> A good compiler should be able to detect the
> absence of potential side effects. E.g. it might reorder when only
> constants and simple variable references are present, or (in the case
> of a JIT, which has run-time type information) when it knows enough
> about the types of the operands involved to determine that operations
> like getattr or getitem are guaranteed side-effect-free.

I call this the 'as-if' rule: the compiler can take shortcuts if the 
result is 'as-if' no shortcut.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Wed Nov  7 20:19:06 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 07 Nov 2012 14:19:06 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <CAP7+vJKnN9DDH=vTGs0kQSBwiFA72F8gu6aiOXX-+P4ikEF-jQ@mail.gmail.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
	<CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>
	<CAP7+vJKnN9DDH=vTGs0kQSBwiFA72F8gu6aiOXX-+P4ikEF-jQ@mail.gmail.com>
Message-ID: <k7ec7e$roa$1@ger.gmane.org>

On 11/7/2012 10:17 AM, Guido van Rossum wrote:
> Ok, somebody go for it! (Also please refer to my pronouncement in the
> bug -- I've gotta run.)

Done. http://bugs.python.org/issue11205?@ok_message=msg 175120

-- 
Terry Jan Reedy


From ned at nedbatchelder.com  Wed Nov  7 22:39:28 2012
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 07 Nov 2012 16:39:28 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <k7e4ih$idp$1@ger.gmane.org>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
	<CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>
	<k7e4ih$idp$1@ger.gmane.org>
Message-ID: <509AD510.5060200@nedbatchelder.com>


On 11/7/2012 12:08 PM, Serhiy Storchaka wrote:
> On 07.11.12 17:12, Nick Coghlan wrote:
>> Since you've indicated the implementation is in the wrong here and you
>> also want to preserve opcode semantics, I think Skip's patch is
>> correct, but also needs to be applied to dict comprehensions (now we
>> have them). The extra bytecode is only ROT_TWO, which is one of the
>> cheapest we have kicking around :)
>
> Not only to dict comprehensions, but also to item assignments.  It 
> will be weird if a dict comprehension and a plain loop will be 
> inconsistent.
>
>
Just to be clear: the reference guide says that the behavior *SHOULD BE* 
(but is not yet) this:

    Python 3.3.0
     >>> {print("a"):print("b")}
    a
    b
    {None: None}
     >>> d = {}
     >>> d[print("a")] = print("b")
    b
    a
     >>>

Is this or is this not "weird" to you?

--Ned.

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121107/2474e3be/attachment.html>

From guido at python.org  Wed Nov  7 23:03:07 2012
From: guido at python.org (Guido van Rossum)
Date: Wed, 7 Nov 2012 14:03:07 -0800
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <509AD510.5060200@nedbatchelder.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
	<CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>
	<k7e4ih$idp$1@ger.gmane.org> <509AD510.5060200@nedbatchelder.com>
Message-ID: <CAP7+vJL6t6dKk+zOw9jMaH8AQ0JsK4GF2iR6Q4=uMmeUdSjBJQ@mail.gmail.com>

The dict display is considered an *expression* and thus must follow
the L2R rule. The assignment is explicitly covered by the R2L rule for
assignments (only). Weird or not, those are the rules, and I don't
want to change them.

On Wed, Nov 7, 2012 at 1:39 PM, Ned Batchelder <ned at nedbatchelder.com> wrote:
>
> On 11/7/2012 12:08 PM, Serhiy Storchaka wrote:
>
> On 07.11.12 17:12, Nick Coghlan wrote:
>
> Since you've indicated the implementation is in the wrong here and you
> also want to preserve opcode semantics, I think Skip's patch is
> correct, but also needs to be applied to dict comprehensions (now we
> have them). The extra bytecode is only ROT_TWO, which is one of the
> cheapest we have kicking around :)
>
>
> Not only to dict comprehensions, but also to item assignments.  It will be
> weird if a dict comprehension and a plain loop will be inconsistent.
>
>
> Just to be clear: the reference guide says that the behavior *SHOULD BE*
> (but is not yet) this:
>
> Python 3.3.0
>>>> {print("a"):print("b")}
> a
> b
> {None: None}
>>>> d = {}
>>>> d[print("a")] = print("b")
> b
> a
>>>>
>
> Is this or is this not "weird" to you?
>
> --Ned.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)

From tjreedy at udel.edu  Wed Nov  7 23:11:40 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 07 Nov 2012 17:11:40 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <509AD510.5060200@nedbatchelder.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
	<CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>
	<k7e4ih$idp$1@ger.gmane.org> <509AD510.5060200@nedbatchelder.com>
Message-ID: <k7emb0$okb$1@ger.gmane.org>

On 11/7/2012 4:39 PM, Ned Batchelder wrote:

> Just to be clear: the reference guide says that the behavior *SHOULD BE*
> (but is not yet) this:
>
>     Python 3.3.0
>      >>> {print("a"):print("b")}
>     a
>     b
>     {None: None}
>      >>> d = {}
>      >>> d[print("a")] = print("b")
>     b
>     a
>      >>>
>
> Is this or is this not "weird" to you?

Not weird. Expressions and assignment targets are each consistently 
evaluated left to right (except as *necessarily* alter by precedence), 
with expressions evaluated before targets.

What is weird -- to me ;-) -- is using side-effects in either example above.

-- 
Terry Jan Reedy


From alexandre at peadrop.com  Wed Nov  7 23:21:28 2012
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Wed, 7 Nov 2012 14:21:28 -0800
Subject: [Python-Dev] cpython: Issue #16218: skip test if filesystem
 doesn't support required encoding
In-Reply-To: <20121103181343.27e690d7@pitrou.net>
References: <3Xv06h2mWtzMCY@mail.python.org>
	<20121103181343.27e690d7@pitrou.net>
Message-ID: <CANcUUedKG+P4c+=SxxCG7_GVRJkNmSnnYXtuB=DYeQ2m+P5hLg@mail.gmail.com>

The Unicode code points in the
U+DC00-DFFF<http://www.unicode.org/charts/PDF/UDC00.pdf>
 range (low surrogate area) can't be encoded in UTF-8. Quoting from
RFC 3629<http://tools.ietf.org/html/rfc3629>
:

*The definition of UTF-8 prohibits encoding character numbers between
U+D800 and U+DFFF, which are reserved for use with the UTF-16 encoding form
(as surrogate pairs) and do not directly represent characters.*


It looks like this test was doing something specific with regards to this.
So, I am curious as well about this change.



On Sat, Nov 3, 2012 at 10:13 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sat,  3 Nov 2012 13:37:48 +0100 (CET)
> andrew.svetlov <python-checkins at python.org> wrote:
> > http://hg.python.org/cpython/rev/95d1adf144ee
> > changeset:   80187:95d1adf144ee
> > user:        Andrew Svetlov <andrew.svetlov at gmail.com>
> > date:        Sat Nov 03 14:37:37 2012 +0200
> > summary:
> >   Issue #16218: skip test if filesystem doesn't support required encoding
> >
> > files:
> >   Lib/test/test_cmd_line_script.py |  7 ++++++-
> >   1 files changed, 6 insertions(+), 1 deletions(-)
> >
> >
> > diff --git a/Lib/test/test_cmd_line_script.py
> b/Lib/test/test_cmd_line_script.py
> > --- a/Lib/test/test_cmd_line_script.py
> > +++ b/Lib/test/test_cmd_line_script.py
> > @@ -366,7 +366,12 @@
> >      def test_non_utf8(self):
> >          # Issue #16218
> >          with temp_dir() as script_dir:
> > -            script_basename = '\udcf1\udcea\udcf0\udce8\udcef\udcf2'
> > +            script_basename = '\u0441\u043a\u0440\u0438\u043f\u0442'
>
> Why exactly did you change the tested name here?
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/alexandre%40peadrop.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121107/0c9cbc34/attachment-0001.html>

From steve at pearwood.info  Wed Nov  7 23:29:27 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 08 Nov 2012 09:29:27 +1100
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <509AD510.5060200@nedbatchelder.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
	<CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>
	<k7e4ih$idp$1@ger.gmane.org> <509AD510.5060200@nedbatchelder.com>
Message-ID: <509AE0C7.5080304@pearwood.info>

On 08/11/12 08:39, Ned Batchelder wrote:

> Just to be clear: the reference guide says that the behavior *SHOULD BE* (but is not yet) this:
>
> Python 3.3.0
>> >> {print("a"):print("b")}
> a
> b
> {None: None}


That was the behaviour of Python 2.4:

py> def pr(x):
...     print x
...
py> {pr(1): pr(2), pr(3): pr(4)}
1
2
3
4
{None: None}


2.5 changed to the behaviour seen now, that is, it prints 2 1 4 3
in that order.


>> >> d = {}
>> >> d[print("a")] = print("b")
> b
> a
>
> Is this or is this not "weird" to you?

Not weird to me. The first case has no assignment, so it operates
left to right without exception. The second case has an assignment,
so it operates left to right with a single exception, the right
hand side of the assignment is evaluated before the left hand side(s).

This gives a single, intuitive[1] order of evaluation (left to right),
with the fewest number of exceptions necessary[2]. Using Python 2.4
again:


py> d = {}
py> d[pr(1)] = d[pr(2)] = d[pr(3)] = pr(4) is pr(5)
4
5
1
2
3





[1] Well, intuitive to those whose native language reads left to right.

[2] I assume it is necessary.



-- 
Steven

From tjreedy at udel.edu  Wed Nov  7 23:42:58 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 07 Nov 2012 17:42:58 -0500
Subject: [Python-Dev] urlretrieve regression no.2, now in Python 3.3
In-Reply-To: <CAPkN8xJtZ71gB3LdgvmQAy0cxKnwsCc7yMhpKj7c4Mb4T8jgmQ@mail.gmail.com>
References: <CAPkN8xJtZ71gB3LdgvmQAy0cxKnwsCc7yMhpKj7c4Mb4T8jgmQ@mail.gmail.com>
Message-ID: <k7eo5m$6pc$1@ger.gmane.org>

On 11/7/2012 5:57 AM, anatoly techtonik wrote:
> urlretrieve has a callback parameter, which takes function with the
> following prototype:
>
>    def callback(block_number, block_size, total_size):
>      pass
>
> Where block_size was constant and block_size*block_number gave an exact
> number of transferred bytes.

The 3.2 and 3.3 docs both say "The third argument, if present, is a hook 
function that will be called once on establishment of the network 
connection and once after each block read thereafter. The hook will be 
passed three arguments; a count of blocks transferred so far, a block 
size in bytes, and the total size of the file. The third argument may be 
-1 on older FTP servers which do not return a file size in response to a 
retrieval request."

The word 'constant' does not appear. The product is still the same.

> Recent change in Python 3.3 changed the semantic of block_size and broke
> my `wget` package. http://bugs.python.org/issue16409

The only change is that blocksize is now reported as 0 before any blocks 
have been transmitted. It is a side-effect of commits for
http://bugs.python.org/issue10050.

-- 
Terry Jan Reedy


From victor.stinner at gmail.com  Wed Nov  7 23:47:13 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 7 Nov 2012 23:47:13 +0100
Subject: [Python-Dev] cpython: Issue #16218: skip test if filesystem
 doesn't support required encoding
In-Reply-To: <CANcUUedKG+P4c+=SxxCG7_GVRJkNmSnnYXtuB=DYeQ2m+P5hLg@mail.gmail.com>
References: <3Xv06h2mWtzMCY@mail.python.org>
	<20121103181343.27e690d7@pitrou.net>
	<CANcUUedKG+P4c+=SxxCG7_GVRJkNmSnnYXtuB=DYeQ2m+P5hLg@mail.gmail.com>
Message-ID: <CAMpsgwYys2dWU0S+32TM1Kyq=u14ne50hZRg5myvezR=_5sRXQ@mail.gmail.com>

2012/11/7 Alexandre Vassalotti <alexandre at peadrop.com>:
> The Unicode code points in the U+DC00-DFFF range (low surrogate area) can't
> be encoded in UTF-8. Quoting from RFC 3629:
>
> The definition of UTF-8 prohibits encoding character numbers between U+D800
> and U+DFFF, which are reserved for use with the UTF-16 encoding form (as
> surrogate pairs) and do not directly represent characters.
>
>
> It looks like this test was doing something specific with regards to this.
> So, I am curious as well about this change.

os.fsencode() uses the surrogateescape error handler (PEP 393) on UNIX.

>>> os.fsencode('\udcf1\udcea\udcf0\udce8\udcef\udcf2')
b'\xf1\xea\xf0\xe8\xef\xf2'

I replaced this arbitrary string (and other similar constant strings)
with support.FS_NONASCII which is more portable (should be available
on all locale encodings... except ASCII) and documented.

I rewrote test_cmd_line_script.test_non_ascii() (and other tests) in
Python 3.4 to use support.FS_NONASCII.

This change should improve code coverage on heterogeneous environments.

Victor

From rdmurray at bitdance.com  Thu Nov  8 00:08:45 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 07 Nov 2012 18:08:45 -0500
Subject: [Python-Dev] cpython: Issue #16218: skip test if filesystem
	doesn't support required encoding
In-Reply-To: <CAMpsgwYys2dWU0S+32TM1Kyq=u14ne50hZRg5myvezR=_5sRXQ@mail.gmail.com>
References: <3Xv06h2mWtzMCY@mail.python.org>
	<20121103181343.27e690d7@pitrou.net>
	<CANcUUedKG+P4c+=SxxCG7_GVRJkNmSnnYXtuB=DYeQ2m+P5hLg@mail.gmail.com>
	<CAMpsgwYys2dWU0S+32TM1Kyq=u14ne50hZRg5myvezR=_5sRXQ@mail.gmail.com>
Message-ID: <20121107230845.7A7D225006F@webabinitio.net>

On Wed, 07 Nov 2012 23:47:13 +0100, Victor Stinner <victor.stinner at gmail.com> wrote:
> 2012/11/7 Alexandre Vassalotti <alexandre at peadrop.com>:
> > The Unicode code points in the U+DC00-DFFF range (low surrogate area) can't
> > be encoded in UTF-8. Quoting from RFC 3629:
> >
> > The definition of UTF-8 prohibits encoding character numbers between U+D800
> > and U+DFFF, which are reserved for use with the UTF-16 encoding form (as
> > surrogate pairs) and do not directly represent characters.
> >
> >
> > It looks like this test was doing something specific with regards to this.
> > So, I am curious as well about this change.
> 
> os.fsencode() uses the surrogateescape error handler (PEP 393) on UNIX.
> 
> >>> os.fsencode('\udcf1\udcea\udcf0\udce8\udcef\udcf2')
> b'\xf1\xea\xf0\xe8\xef\xf2'
> 
> I replaced this arbitrary string (and other similar constant strings)
> with support.FS_NONASCII which is more portable (should be available
> on all locale encodings... except ASCII) and documented.
> 
> I rewrote test_cmd_line_script.test_non_ascii() (and other tests) in
> Python 3.4 to use support.FS_NONASCII.
> 
> This change should improve code coverage on heterogeneous environments.

Alexandre's point was that the string did not appear to be arbitrary,
but rather appeared to specifically be a string containing surrogates.
Is this not the case?

--David

From greg at krypto.org  Thu Nov  8 00:48:06 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Wed, 7 Nov 2012 15:48:06 -0800
Subject: [Python-Dev] urlretrieve regression no.2, now in Python 3.3
In-Reply-To: <k7eo5m$6pc$1@ger.gmane.org>
References: <CAPkN8xJtZ71gB3LdgvmQAy0cxKnwsCc7yMhpKj7c4Mb4T8jgmQ@mail.gmail.com>
	<k7eo5m$6pc$1@ger.gmane.org>
Message-ID: <CAGE7PN+3Sr4pgw_TD5JeLSqRhxb=PXBfFU-7ZPLbRtj4t5EcKA@mail.gmail.com>

On Wed, Nov 7, 2012 at 2:42 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 11/7/2012 5:57 AM, anatoly techtonik wrote:
>>
>> urlretrieve has a callback parameter, which takes function with the
>> following prototype:
>>
>>    def callback(block_number, block_size, total_size):
>>      pass
>>
>> Where block_size was constant and block_size*block_number gave an exact
>> number of transferred bytes.
>
>
> The 3.2 and 3.3 docs both say "The third argument, if present, is a hook
> function that will be called once on establishment of the network connection
> and once after each block read thereafter. The hook will be passed three
> arguments; a count of blocks transferred so far, a block size in bytes, and
> the total size of the file. The third argument may be -1 on older FTP
> servers which do not return a file size in response to a retrieval request."
>
> The word 'constant' does not appear. The product is still the same.
>
>
>> Recent change in Python 3.3 changed the semantic of block_size and broke
>> my `wget` package. http://bugs.python.org/issue16409
>
>
> The only change is that blocksize is now reported as 0 before any blocks
> have been transmitted. It is a side-effect of commits for
> http://bugs.python.org/issue10050.

two changes.  the first block size is 0.  the last block size is the
number of bytes read.

It is a trivial two line change to undo this and I think it would be
wise.  It is difficult to even find documentation on what the
arguments to that callback mean.  Look up the urlretrieve() function
docs in 2.7 and 3.3 and it doesn't tell you there or refer you to
anywhere else to find out.

I'm in favor of restoring and documenting the old behavior for this
legacy urlretrieve API to make existing code being ported from 2.7 not
break.

-gps

>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org

From ned at nedbatchelder.com  Thu Nov  8 02:11:21 2012
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Wed, 07 Nov 2012 20:11:21 -0500
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <k7emb0$okb$1@ger.gmane.org>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
	<CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>
	<k7e4ih$idp$1@ger.gmane.org> <509AD510.5060200@nedbatchelder.com>
	<k7emb0$okb$1@ger.gmane.org>
Message-ID: <509B06B9.6000406@nedbatchelder.com>

On 11/7/2012 5:11 PM, Terry Reedy wrote:
> On 11/7/2012 4:39 PM, Ned Batchelder wrote:
>
>> Just to be clear: the reference guide says that the behavior *SHOULD BE*
>> (but is not yet) this:
>>
>>     Python 3.3.0
>>      >>> {print("a"):print("b")}
>>     a
>>     b
>>     {None: None}
>>      >>> d = {}
>>      >>> d[print("a")] = print("b")
>>     b
>>     a
>>      >>>
>>
>> Is this or is this not "weird" to you?
>
> Not weird. Expressions and assignment targets are each consistently 
> evaluated left to right (except as *necessarily* alter by precedence), 
> with expressions evaluated before targets.
>
Sorry, I should have been clearer: I was asking about weird not to say, 
"This is weird and should be changed!", but to get clarification from 
Serhiy about his statement, " It will be weird if a dict comprehension 
and a plain loop will be inconsistent."  I honestly didn't know which 
behavior he considered inconsistent and therefore weird.

--Ned.


From stefan_ml at behnel.de  Thu Nov  8 13:47:06 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 08 Nov 2012 13:47:06 +0100
Subject: [Python-Dev] Make extension module initialisation more like Python
	module initialisation
Message-ID: <k7g9k9$8bl$1@ger.gmane.org>

Hi,

I suspect that this will be put into a proper PEP at some point, but I'd
like to bring this up for discussion first. This came out of issues 13429
and 16392.

http://bugs.python.org/issue13429

http://bugs.python.org/issue16392

Stefan


The problem
===========

Python modules and extension modules are not being set up in the same way.
For Python modules, the module is created and set up first, then the module
code is being executed. For extensions, i.e. shared libraries, the module
init function is executed straight away and does both the creation and
initialisation. This means that it knows neither the __file__ it is being
loaded from nor its package (i.e. its FQMN). This hinders relative imports
and resource loading. In Py3, it's also not being added to sys.modules,
which means that a (potentially transitive) re-import of the module will
really try to reimport it and thus run into an infinite loop when it
executes the module init function again. And without the FQMN, it's not
trivial to correctly add the module to sys.modules either.

We specifically run into this for Cython generated modules, for which it's
not uncommon that the module init code has the same level of complexity as
that of any 'regular' Python module. Also, the lack of a FQMN and correct
file path hinders the compilation of __init__.py modules, i.e. packages,
especially when relative imports are being used at module init time.

The proposal
============

I propose to split the extension module initialisation into two steps in
Python 3.4, in a backwards compatible way.

Step 1: The current module init function can be reduced to just creating
the module instance and returning it (and potentially doing some simple C
level setup). Optionally, after creating the module (and this is the new
part), the module init code can register a C callback function that will be
called after setting up the module.

Step 2: The shared library importer receives the module instance from the
module init function, adds __file__, __path__, __package__ and friends to
the module dict, and then checks for the callback. If non-NULL, it calls it
to continue the module initialisation by user code.

The callback
============

The callback is defined as follows::

    int (*PyModule_init_callback)(PyObject* the_module,
                                  PyModuleInitContext* context)

"PyModuleInitContext" is a struct that is meant mostly for making the
callback more future proof by allowing additional parameters to be passed
in. For now, I can see a use case for the following fields::

    struct PyModuleInitContext {
        char* module_name;
        char* qualified_module_name;
    }

Both names are encoded in UTF-8. As for the file path, I consider it best
to retrieve it from the module's __file__ attribute as a Python string
object to reduce filename encoding problems.

Note that this struct argument is not strictly required, but given that
this proposal would have been much simpler if the module init function had
accepted such an argument in the first place, I consider it a good idea not
to let this chance pass by again.

The registration of the callback uses a new C-API function:

    int PyModule_SetInitFunction(PyObject* module,
                                 PyModule_init_callback callback)

The function name uses "Set" instead of "Register" to make it clear that
there is only one such function per module.

An alternative would be a new module creation function "PyModule_Create3()"
that takes the callback as third argument, in addition to what
"PyModule_Create2()" accepts. This would require users to explicitly pass
in the (second) version argument, which might be considered only a minor issue.

Implementation
==============

The implementation requires local changes to the extension module importer
and a new C-API function. In order to store the callback, it should use a
new field in the module object struct.

Open questions
==============

It is not clear how extensions should be handled that register more than
one module in their module init function, e.g. compiled packages. One
possibility would be to leave the setup to the user, who would have to know
all FQMNs anyway in this case, although not the import file path.
Alternatively, the import machinery could use a stack to remember for which
modules a callback was registered during the last init function call, set
up all of them and then call their callbacks. It's not clear if this meets
the intention of the user.

Alternatives
============

1) It would be possible to make extension modules optionally export another
symbol, e.g. "PyInit2_modulename", that the shared library loader would
call in addition to the required function "PyInit_modulename". This would
remove the need for a new API that registers the above callback. The
drawback is that it also makes it easier to write broken code because a
Python version or implementation that does not support this second symbol
would simply not call it, without error. The new C-API function would let
the build fail instead if it is not supported.

2) The callback could be made available as a Python function in the module
dict, thus also removing the need for an explicit registration API.
However, this approach would add overhead to both sides, the importer code
and the user provided module init code, as it would require additional
dictionary handling and the implementation of a one-time Python function in
user code. It would also suffer from the problem that missing support in
the runtime would pass silently.

3) The callback could be registered statically in the PyModuleDef struct by
adding a new field. This is not trivial to do in a backwards compatible way
because the struct would grow longer without explicit initialisation by
existing user code. Extending PyModuleDef_HEAD_INIT might be possible but
would still break at least binary compatibility.

4) Pass a new context argument into the module init function that contains
all information necessary to properly and completely set up the module at
creation time. This would provide a much simpler and cleaner solution than
the proposed solution. However, it will not be possible before Python 4 as
it breaks backwards compatibility with all existing extension modules at
both the source and binary level.


From mal at egenix.com  Thu Nov  8 14:01:27 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 08 Nov 2012 14:01:27 +0100
Subject: [Python-Dev] Make extension module initialisation more like
 Python module initialisation
In-Reply-To: <k7g9k9$8bl$1@ger.gmane.org>
References: <k7g9k9$8bl$1@ger.gmane.org>
Message-ID: <509BAD27.2030906@egenix.com>

On 08.11.2012 13:47, Stefan Behnel wrote:
> Hi,
> 
> I suspect that this will be put into a proper PEP at some point, but I'd
> like to bring this up for discussion first. This came out of issues 13429
> and 16392.
> 
> http://bugs.python.org/issue13429
> 
> http://bugs.python.org/issue16392
> 
> Stefan
> 
> 
> The problem
> ===========
> 
> Python modules and extension modules are not being set up in the same way.
> For Python modules, the module is created and set up first, then the module
> code is being executed. For extensions, i.e. shared libraries, the module
> init function is executed straight away and does both the creation and
> initialisation. This means that it knows neither the __file__ it is being
> loaded from nor its package (i.e. its FQMN). This hinders relative imports
> and resource loading. In Py3, it's also not being added to sys.modules,
> which means that a (potentially transitive) re-import of the module will
> really try to reimport it and thus run into an infinite loop when it
> executes the module init function again. And without the FQMN, it's not
> trivial to correctly add the module to sys.modules either.
> 
> We specifically run into this for Cython generated modules, for which it's
> not uncommon that the module init code has the same level of complexity as
> that of any 'regular' Python module. Also, the lack of a FQMN and correct
> file path hinders the compilation of __init__.py modules, i.e. packages,
> especially when relative imports are being used at module init time.
> 
> The proposal
> ============
> 
> ... [callbacks] ...
>
> Alternatives
> ============
> ...
> 3) The callback could be registered statically in the PyModuleDef struct by
> adding a new field. This is not trivial to do in a backwards compatible way
> because the struct would grow longer without explicit initialisation by
> existing user code. Extending PyModuleDef_HEAD_INIT might be possible but
> would still break at least binary compatibility.

I think the above is the cleaner approach than the callback mechanism.
There's no problem in adding new slots to the end of the PyModuleDef struct
- we've been doing that for years in many other structs :-)

All you have to do is bump the Python API version number.

(Martin's PEP http://www.python.org/dev/peps/pep-3121/ has the details)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 08 2012)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From stefan_ml at behnel.de  Thu Nov  8 14:20:27 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 08 Nov 2012 14:20:27 +0100
Subject: [Python-Dev] Make extension module initialisation more like
 Python module initialisation
In-Reply-To: <509BAD27.2030906@egenix.com>
References: <k7g9k9$8bl$1@ger.gmane.org> <509BAD27.2030906@egenix.com>
Message-ID: <k7gbir$psq$1@ger.gmane.org>

M.-A. Lemburg, 08.11.2012 14:01:
> On 08.11.2012 13:47, Stefan Behnel wrote:
>> I suspect that this will be put into a proper PEP at some point, but I'd
>> like to bring this up for discussion first. This came out of issues 13429
>> and 16392.
>>
>> http://bugs.python.org/issue13429
>>
>> http://bugs.python.org/issue16392
>>
>> Stefan
>>
>>
>> The problem
>> ===========
>>
>> Python modules and extension modules are not being set up in the same way.
>> For Python modules, the module is created and set up first, then the module
>> code is being executed. For extensions, i.e. shared libraries, the module
>> init function is executed straight away and does both the creation and
>> initialisation. This means that it knows neither the __file__ it is being
>> loaded from nor its package (i.e. its FQMN). This hinders relative imports
>> and resource loading. In Py3, it's also not being added to sys.modules,
>> which means that a (potentially transitive) re-import of the module will
>> really try to reimport it and thus run into an infinite loop when it
>> executes the module init function again. And without the FQMN, it's not
>> trivial to correctly add the module to sys.modules either.
>>
>> We specifically run into this for Cython generated modules, for which it's
>> not uncommon that the module init code has the same level of complexity as
>> that of any 'regular' Python module. Also, the lack of a FQMN and correct
>> file path hinders the compilation of __init__.py modules, i.e. packages,
>> especially when relative imports are being used at module init time.
>>
>> The proposal
>> ============
>>
>> ... [callbacks] ...
>>
>> Alternatives
>> ============
>> ...
>> 3) The callback could be registered statically in the PyModuleDef struct by
>> adding a new field. This is not trivial to do in a backwards compatible way
>> because the struct would grow longer without explicit initialisation by
>> existing user code. Extending PyModuleDef_HEAD_INIT might be possible but
>> would still break at least binary compatibility.
> 
> I think the above is the cleaner approach than the callback mechanism.

Oh, definitely.


> There's no problem in adding new slots to the end of the PyModuleDef struct
> - we've been doing that for years in many other structs :-)
> 
> All you have to do is bump the Python API version number.
> 
> (Martin's PEP http://www.python.org/dev/peps/pep-3121/ has the details)

The difference is that this specific struct is provided by user code and
(typically) initialised statically. There is no guarantee that user code
that does not expect the additional field will initialise it to 0. Failing
that, I don't see how we could trust its value in any way.

Stefan



From stefan_ml at behnel.de  Thu Nov  8 14:51:54 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 08 Nov 2012 14:51:54 +0100
Subject: [Python-Dev] Make extension module initialisation more like
 Python module initialisation
In-Reply-To: <k7gbir$psq$1@ger.gmane.org>
References: <k7g9k9$8bl$1@ger.gmane.org> <509BAD27.2030906@egenix.com>
	<k7gbir$psq$1@ger.gmane.org>
Message-ID: <k7gddo$b2v$1@ger.gmane.org>

Stefan Behnel, 08.11.2012 14:20:
> M.-A. Lemburg, 08.11.2012 14:01:
>> On 08.11.2012 13:47, Stefan Behnel wrote:
>>> I suspect that this will be put into a proper PEP at some point, but I'd
>>> like to bring this up for discussion first. This came out of issues 13429
>>> and 16392.
>>>
>>> http://bugs.python.org/issue13429
>>>
>>> http://bugs.python.org/issue16392
>>>
>>> Stefan
>>>
>>>
>>> The problem
>>> ===========
>>>
>>> Python modules and extension modules are not being set up in the same way.
>>> For Python modules, the module is created and set up first, then the module
>>> code is being executed. For extensions, i.e. shared libraries, the module
>>> init function is executed straight away and does both the creation and
>>> initialisation. This means that it knows neither the __file__ it is being
>>> loaded from nor its package (i.e. its FQMN). This hinders relative imports
>>> and resource loading. In Py3, it's also not being added to sys.modules,
>>> which means that a (potentially transitive) re-import of the module will
>>> really try to reimport it and thus run into an infinite loop when it
>>> executes the module init function again. And without the FQMN, it's not
>>> trivial to correctly add the module to sys.modules either.
>>>
>>> We specifically run into this for Cython generated modules, for which it's
>>> not uncommon that the module init code has the same level of complexity as
>>> that of any 'regular' Python module. Also, the lack of a FQMN and correct
>>> file path hinders the compilation of __init__.py modules, i.e. packages,
>>> especially when relative imports are being used at module init time.
>>>
>>> The proposal
>>> ============
>>>
>>> ... [callbacks] ...
>>>
>>> Alternatives
>>> ============
>>> ...
>>> 3) The callback could be registered statically in the PyModuleDef struct by
>>> adding a new field. This is not trivial to do in a backwards compatible way
>>> because the struct would grow longer without explicit initialisation by
>>> existing user code. Extending PyModuleDef_HEAD_INIT might be possible but
>>> would still break at least binary compatibility.
>>
>> I think the above is the cleaner approach than the callback mechanism.
> 
> Oh, definitely.
> 
> 
>> There's no problem in adding new slots to the end of the PyModuleDef struct
>> - we've been doing that for years in many other structs :-)
>>
>> All you have to do is bump the Python API version number.
>>
>> (Martin's PEP http://www.python.org/dev/peps/pep-3121/ has the details)
> 
> The difference is that this specific struct is provided by user code and
> (typically) initialised statically. There is no guarantee that user code
> that does not expect the additional field will initialise it to 0. Failing
> that, I don't see how we could trust its value in any way.

Hmm - you're actually right. In C, uninitialised fields in a static struct
are set to 0 automatically. Same case as the type structs. That makes your
objection perfectly valid. I'll rewrite and shorten the proposal.

Thanks!

Stefan



From stefan_ml at behnel.de  Thu Nov  8 15:32:34 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 08 Nov 2012 15:32:34 +0100
Subject: [Python-Dev] Update - Re: Make extension module initialisation more
 like Python module initialisation
In-Reply-To: <k7g9k9$8bl$1@ger.gmane.org>
References: <k7g9k9$8bl$1@ger.gmane.org>
Message-ID: <k7gfq0$177$1@ger.gmane.org>

Hi,

here's an updated proposal, adopting Marc-Andre's improvement that uses a
new field in the PyModuleDef struct to register the callback. Note that
this change no longer keeps up binary compatibility, which may or may not
be acceptable for Python 3.4.

Stefan


The problem
===========

Python modules and extension modules are not being set up in the same way.
For Python modules, the module is created and set up first, then the module
code is being executed. For extensions, i.e. shared libraries, the module
init function is executed straight away and does both the creation and
initialisation. This means that it knows neither the __file__ it is being
loaded from nor its package (i.e. its FQMN). This hinders relative imports
and resource loading. In Py3, it's also not being added to sys.modules,
which means that a (potentially transitive) re-import of the module will
really try to reimport it and thus run into an infinite loop when it
executes the module init function again. And without the FQMN, it's not
trivial to correctly add the module to sys.modules either.

We specifically run into this for Cython generated modules, for which it's
not uncommon that the module init code has the same level of complexity as
that of any 'regular' Python module. Also, the lack of a FQMN and correct
file path hinders the compilation of __init__.py modules, i.e. packages,
especially when relative imports are being used at module init time.

The proposal
============

I propose to split the extension module initialisation into two steps in
Python 3.4, in a backwards compatible way.

Step 1: The current module init function can be reduced to just creating
the module instance and returning it (and potentially doing some simple C
level setup). Additionally, and this is the new part, the module init code
can register a C callback function in its PyModuleDef struct that will be
called after setting up the module.

Step 2: The shared library importer receives the module instance from the
module init function, adds __file__, __path__, __package__ and friends to
the module dict, and then checks for the callback. If non-NULL, it calls it
to continue the module initialisation by user code.

The callback
============

The callback is defined as follows::

    int (*PyModule_init_callback)(PyObject* the_module,
                                  PyModuleInitContext* context)

"PyModuleInitContext" is a struct that is meant mostly for making the
callback more future proof by allowing additional parameters to be passed
in. For now, I can see a use case for the following fields::

    struct PyModuleInitContext {
        char* module_name;
        char* qualified_module_name;
    }

Both names are encoded in UTF-8. As for the file path, I consider it best
to retrieve it from the module's __file__ attribute as a Python string
object to reduce filename encoding problems.

Note that this struct argument is not strictly required (it could be a
simple "inquiry" function), but given that this proposal would have been
much simpler if the module init function had accepted such an argument in
the first place, I consider it a good idea not to let this chance pass by
again. The counter arguments would be "keep it simple" and "we already pass
in the whole module (and its dict) anyway". Up for debate!

The registration of the callback uses a new field "m_init" in the
PyModuleDef struct::

    typedef struct PyModuleDef{
      PyModuleDef_Base m_base;
      const char* m_name;
      const char* m_doc;
      Py_ssize_t m_size;
      PyMethodDef *m_methods;
      inquiry m_reload;
      traverseproc m_traverse;
      inquiry m_clear;
      freefunc m_free;          /* --- original fields up to here */
      PyModule_init_callback m_init;   /* post-setup init callback */
    } PyModuleDef;

Implementation
==============

The implementation requires local changes to the extension module importer
and a new field in the PyModuleDef struct.

Open questions
==============

It is not clear how extensions should be handled that register more than
one module in their module init function, e.g. compiled packages. One
possibility would be to leave the setup to the user, who would have to know
all FQMNs anyway in this case, although not the import file path.
Alternatively, the import machinery could use a stack to remember for which
modules a callback was registered during the last init function call, set
up all of them and then call their callbacks. It's not clear if this meets
the intention of the user. It's not guaranteed that all of these modules
will be related to the module that registered them, in the sense that they
should receive the same setup. The best way to fix this correctly might be
to make users pass the setup explicitly into the module creation functions
in Python 4 (see alternatives below), so that the setup and sys.modules
registration can happen directly at this point.

Alternatives
============

1) It would be possible to make extension modules optionally export another
symbol, e.g. "PyInit2_modulename", that the shared library loader would
call in addition to the required function "PyInit_modulename". This would
keep up binary compatibility. The drawback is that it also makes it easier
to write broken code because a Python version or implementation that does
not support this second symbol would simply not call it, without error. The
new struct field would let the build fail instead if it is not supported.

2) The callback could be made available as a Python function in the module
dict, thus also removing the need for an explicit registration API.
However, this approach would add overhead to both sides, the importer code
and the user provided module init code, as it would require additional
dictionary handling and the implementation of a one-time Python function in
user code. It would also suffer from the problem that missing support in
the runtime would pass silently.

3) The original proposal used a new C-API function to register the callback
explicitly, as opposed to extending the PyModuleDef struct. This has the
advantage of keeping up binary compatibility with existing Py3.3
extensions. It has the disadvantage of adding another indirection to the
setup procedure where a static function pointer would suffice.

4) Pass a new context argument into the module init function that contains
all information necessary to properly and completely set up the module at
creation time. This would provide a much simpler and cleaner solution than
the proposed solution. However, it will not be possible before Python 4 as
it breaks backwards compatibility with all existing extension modules at
both the source and binary level.


From brett at python.org  Thu Nov  8 15:41:30 2012
From: brett at python.org (Brett Cannon)
Date: Thu, 8 Nov 2012 09:41:30 -0500
Subject: [Python-Dev] Make extension module initialisation more like
 Python module initialisation
In-Reply-To: <k7g9k9$8bl$1@ger.gmane.org>
References: <k7g9k9$8bl$1@ger.gmane.org>
Message-ID: <CAP1=2W6W1LwY00B+ZJMFTrBcH2_iQ=MGghgxn=nNMc7h0c4sww@mail.gmail.com>

On Thu, Nov 8, 2012 at 7:47 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Hi,
>
> I suspect that this will be put into a proper PEP at some point, but I'd
> like to bring this up for discussion first. This came out of issues 13429
> and 16392.
>
> http://bugs.python.org/issue13429
>
> http://bugs.python.org/issue16392
>
> Stefan
>
>
> The problem
> ===========
>
> Python modules and extension modules are not being set up in the same way.
> For Python modules, the module is created and set up first, then the module
> code is being executed. For extensions, i.e. shared libraries, the module
> init function is executed straight away and does both the creation and
> initialisation. This means that it knows neither the __file__ it is being
> loaded from nor its package (i.e. its FQMN). This hinders relative imports
> and resource loading. In Py3, it's also not being added to sys.modules,
> which means that a (potentially transitive) re-import of the module will
> really try to reimport it and thus run into an infinite loop when it
> executes the module init function again. And without the FQMN, it's not
> trivial to correctly add the module to sys.modules either.
>
> We specifically run into this for Cython generated modules, for which it's
> not uncommon that the module init code has the same level of complexity as
> that of any 'regular' Python module. Also, the lack of a FQMN and correct
> file path hinders the compilation of __init__.py modules, i.e. packages,
> especially when relative imports are being used at module init time.
>

Or to put it another way, importlib doesn't give you a nice class to
inherit from which will handle all of the little details of creating a
blank module (or fetching from sys.modules if you are reloading), setting
__file__, __cached__, __package__, __name__, __loader__, and (optionally)
__path__ for you, and then cleaning up if something goes wrong. It's a pain
to do all of this yourself and to get all the details right (i.e. there's a
reason that @importlib.util.module_for_loader exists).


>
> The proposal
> ============
>
> I propose to split the extension module initialisation into two steps in
> Python 3.4, in a backwards compatible way.
>
> Step 1: The current module init function can be reduced to just creating
> the module instance and returning it (and potentially doing some simple C
> level setup). Optionally, after creating the module (and this is the new
> part), the module init code can register a C callback function that will be
> called after setting up the module.
>

Why even bother with the module creation? Why can't Python do that as well
and then call the callback?


>
> Step 2: The shared library importer receives the module instance from the
> module init function, adds __file__, __path__, __package__ and friends to
> the module dict, and then checks for the callback. If non-NULL, it calls it
> to continue the module initialisation by user code.


> The callback
> ============
>
> The callback is defined as follows::
>
>     int (*PyModule_init_callback)(PyObject* the_module,
>                                   PyModuleInitContext* context)
>
> "PyModuleInitContext" is a struct that is meant mostly for making the
> callback more future proof by allowing additional parameters to be passed
> in. For now, I can see a use case for the following fields::
>
>     struct PyModuleInitContext {
>         char* module_name;
>         char* qualified_module_name;
>     }
>
> Both names are encoded in UTF-8. As for the file path, I consider it best
> to retrieve it from the module's __file__ attribute as a Python string
> object to reduce filename encoding problems.
>
> Note that this struct argument is not strictly required, but given that
> this proposal would have been much simpler if the module init function had
> accepted such an argument in the first place, I consider it a good idea not
> to let this chance pass by again.
>
> The registration of the callback uses a new C-API function:
>
>     int PyModule_SetInitFunction(PyObject* module,
>                                  PyModule_init_callback callback)
>
> The function name uses "Set" instead of "Register" to make it clear that
> there is only one such function per module.
>
> An alternative would be a new module creation function "PyModule_Create3()"
> that takes the callback as third argument, in addition to what
> "PyModule_Create2()" accepts. This would require users to explicitly pass
> in the (second) version argument, which might be considered only a minor
> issue.
>
> Implementation
> ==============
>
> The implementation requires local changes to the extension module importer
> and a new C-API function. In order to store the callback, it should use a
> new field in the module object struct.
>
> Open questions
> ==============
>
> It is not clear how extensions should be handled that register more than
> one module in their module init function, e.g. compiled packages. One
> possibility would be to leave the setup to the user, who would have to know
> all FQMNs anyway in this case, although not the import file path.
> Alternatively, the import machinery could use a stack to remember for which
> modules a callback was registered during the last init function call, set
> up all of them and then call their callbacks. It's not clear if this meets
> the intention of the user.
>
> Alternatives
> ============
>
> 1) It would be possible to make extension modules optionally export another
> symbol, e.g. "PyInit2_modulename", that the shared library loader would
> call in addition to the required function "PyInit_modulename". This would
> remove the need for a new API that registers the above callback. The
> drawback is that it also makes it easier to write broken code because a
> Python version or implementation that does not support this second symbol
> would simply not call it, without error. The new C-API function would let
> the build fail instead if it is not supported.
>

An alternative to the alternative is that if the PyInit2 function exists
it's called instead of the the PyInit function, and then the PyInit
function is nothing more than a single line function call (or whatever the
absolute bare minimum is) into some helper that calls the PyInit2 call
properly for backwards ABI compatibility (i.e. passes in whatever details
are lost by the indirection in function call). That provides an eventual
upgrade path of dropping PyInit and moving over to PyInit2.

-Brett


>
> 2) The callback could be made available as a Python function in the module
> dict, thus also removing the need for an explicit registration API.
> However, this approach would add overhead to both sides, the importer code
> and the user provided module init code, as it would require additional
> dictionary handling and the implementation of a one-time Python function in
> user code. It would also suffer from the problem that missing support in
> the runtime would pass silently.
>
> 3) The callback could be registered statically in the PyModuleDef struct by
> adding a new field. This is not trivial to do in a backwards compatible way
> because the struct would grow longer without explicit initialisation by
> existing user code. Extending PyModuleDef_HEAD_INIT might be possible but
> would still break at least binary compatibility.
>
> 4) Pass a new context argument into the module init function that contains
> all information necessary to properly and completely set up the module at
> creation time. This would provide a much simpler and cleaner solution than
> the proposed solution. However, it will not be possible before Python 4 as
> it breaks backwards compatibility with all existing extension modules at
> both the source and binary level.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121108/dccfb542/attachment-0001.html>

From stefan_ml at behnel.de  Thu Nov  8 16:00:20 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 08 Nov 2012 16:00:20 +0100
Subject: [Python-Dev] Make extension module initialisation more like
 Python module initialisation
In-Reply-To: <CAP1=2W6W1LwY00B+ZJMFTrBcH2_iQ=MGghgxn=nNMc7h0c4sww@mail.gmail.com>
References: <k7g9k9$8bl$1@ger.gmane.org>
	<CAP1=2W6W1LwY00B+ZJMFTrBcH2_iQ=MGghgxn=nNMc7h0c4sww@mail.gmail.com>
Message-ID: <k7ghe3$h0g$1@ger.gmane.org>

Hi Brett,

thanks for the feedback.

Brett Cannon, 08.11.2012 15:41:
> On Thu, Nov 8, 2012 at 7:47 AM, Stefan Behnel wrote:
>> I propose to split the extension module initialisation into two steps in
>> Python 3.4, in a backwards compatible way.
>>
>> Step 1: The current module init function can be reduced to just creating
>> the module instance and returning it (and potentially doing some simple C
>> level setup). Optionally, after creating the module (and this is the new
>> part), the module init code can register a C callback function that will be
>> called after setting up the module.
> 
> Why even bother with the module creation? Why can't Python do that as well
> and then call the callback?
> 
> 
>> Step 2: The shared library importer receives the module instance from the
>> module init function, adds __file__, __path__, __package__ and friends to
>> the module dict, and then checks for the callback. If non-NULL, it calls it
>> to continue the module initialisation by user code.
> [...]
> An alternative to the alternative is that if the PyInit2 function exists
> it's called instead of the the PyInit function, and then the PyInit
> function is nothing more than a single line function call (or whatever the
> absolute bare minimum is) into some helper that calls the PyInit2 call
> properly for backwards ABI compatibility (i.e. passes in whatever details
> are lost by the indirection in function call). That provides an eventual
> upgrade path of dropping PyInit and moving over to PyInit2.

In that case, you'd have to export the PyModuleDef descriptor as well,
because that's what tells CPython how the module behaves and what to do
with it to set it up properly (e.g. allocate module state space on the heap).

In fact, if the module init function became a field in the descriptor, it
would be enough (taking backwards compatibility aside) if *only* the
descriptor was exported and used by the module loader.

With the caveat that this might kill some less common but not necessarily
illegitimate use cases that do more than just creating and initialising a
single module...

Stefan



From brett at python.org  Thu Nov  8 16:06:50 2012
From: brett at python.org (Brett Cannon)
Date: Thu, 8 Nov 2012 10:06:50 -0500
Subject: [Python-Dev] Make extension module initialisation more like
 Python module initialisation
In-Reply-To: <k7ghe3$h0g$1@ger.gmane.org>
References: <k7g9k9$8bl$1@ger.gmane.org>
	<CAP1=2W6W1LwY00B+ZJMFTrBcH2_iQ=MGghgxn=nNMc7h0c4sww@mail.gmail.com>
	<k7ghe3$h0g$1@ger.gmane.org>
Message-ID: <CAP1=2W5qGqn_ig94LUwcK6u6XYR4RpBMpCuyVx4LH_zCd4=QPQ@mail.gmail.com>

On Thu, Nov 8, 2012 at 10:00 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Hi Brett,
>
> thanks for the feedback.
>
> Brett Cannon, 08.11.2012 15:41:
> > On Thu, Nov 8, 2012 at 7:47 AM, Stefan Behnel wrote:
> >> I propose to split the extension module initialisation into two steps in
> >> Python 3.4, in a backwards compatible way.
> >>
> >> Step 1: The current module init function can be reduced to just creating
> >> the module instance and returning it (and potentially doing some simple
> C
> >> level setup). Optionally, after creating the module (and this is the new
> >> part), the module init code can register a C callback function that
> will be
> >> called after setting up the module.
> >
> > Why even bother with the module creation? Why can't Python do that as
> well
> > and then call the callback?
> >
> >
> >> Step 2: The shared library importer receives the module instance from
> the
> >> module init function, adds __file__, __path__, __package__ and friends
> to
> >> the module dict, and then checks for the callback. If non-NULL, it
> calls it
> >> to continue the module initialisation by user code.
> > [...]
> > An alternative to the alternative is that if the PyInit2 function exists
> > it's called instead of the the PyInit function, and then the PyInit
> > function is nothing more than a single line function call (or whatever
> the
> > absolute bare minimum is) into some helper that calls the PyInit2 call
> > properly for backwards ABI compatibility (i.e. passes in whatever details
> > are lost by the indirection in function call). That provides an eventual
> > upgrade path of dropping PyInit and moving over to PyInit2.
>
> In that case, you'd have to export the PyModuleDef descriptor as well,
> because that's what tells CPython how the module behaves and what to do
> with it to set it up properly (e.g. allocate module state space on the
> heap).
>

True.


>
> In fact, if the module init function became a field in the descriptor, it
> would be enough (taking backwards compatibility aside) if *only* the
> descriptor was exported and used by the module loader.
>
>
Also true.


> With the caveat that this might kill some less common but not necessarily
> illegitimate use cases that do more than just creating and initialising a
> single module...
>

You mean creating another module in the init function? That's fine, but
that should be a call to __import__ anyway and that should handle things
properly. Else you are circumventing the import system and you can do
everything from scratch. I don't see why this would stop you from doing
anything you want, it just simplifies the common case.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121108/3f6c9db5/attachment.html>

From stefan_ml at behnel.de  Thu Nov  8 16:30:05 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 08 Nov 2012 16:30:05 +0100
Subject: [Python-Dev] Make extension module initialisation more like
 Python module initialisation
In-Reply-To: <CAP1=2W5qGqn_ig94LUwcK6u6XYR4RpBMpCuyVx4LH_zCd4=QPQ@mail.gmail.com>
References: <k7g9k9$8bl$1@ger.gmane.org>
	<CAP1=2W6W1LwY00B+ZJMFTrBcH2_iQ=MGghgxn=nNMc7h0c4sww@mail.gmail.com>
	<k7ghe3$h0g$1@ger.gmane.org>
	<CAP1=2W5qGqn_ig94LUwcK6u6XYR4RpBMpCuyVx4LH_zCd4=QPQ@mail.gmail.com>
Message-ID: <k7gj5s$2n1$1@ger.gmane.org>

Brett Cannon, 08.11.2012 16:06:
> On Thu, Nov 8, 2012 at 10:00 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> 
>> Hi Brett,
>>
>> thanks for the feedback.
>>
>> Brett Cannon, 08.11.2012 15:41:
>>> On Thu, Nov 8, 2012 at 7:47 AM, Stefan Behnel wrote:
>>>> I propose to split the extension module initialisation into two steps in
>>>> Python 3.4, in a backwards compatible way.
>>>>
>>>> Step 1: The current module init function can be reduced to just creating
>>>> the module instance and returning it (and potentially doing some simple
>> C
>>>> level setup). Optionally, after creating the module (and this is the new
>>>> part), the module init code can register a C callback function that
>> will be
>>>> called after setting up the module.
>>>
>>> Why even bother with the module creation? Why can't Python do that as
>> well
>>> and then call the callback?
>>>
>>>
>>>> Step 2: The shared library importer receives the module instance from
>> the
>>>> module init function, adds __file__, __path__, __package__ and friends
>> to
>>>> the module dict, and then checks for the callback. If non-NULL, it
>> calls it
>>>> to continue the module initialisation by user code.
>>> [...]
>>> An alternative to the alternative is that if the PyInit2 function exists
>>> it's called instead of the the PyInit function, and then the PyInit
>>> function is nothing more than a single line function call (or whatever
>> the
>>> absolute bare minimum is) into some helper that calls the PyInit2 call
>>> properly for backwards ABI compatibility (i.e. passes in whatever details
>>> are lost by the indirection in function call). That provides an eventual
>>> upgrade path of dropping PyInit and moving over to PyInit2.
>>
>> In that case, you'd have to export the PyModuleDef descriptor as well,
>> because that's what tells CPython how the module behaves and what to do
>> with it to set it up properly (e.g. allocate module state space on the
>> heap).
> 
> True.
> 
>> In fact, if the module init function became a field in the descriptor, it
>> would be enough (taking backwards compatibility aside) if *only* the
>> descriptor was exported and used by the module loader.
>
> Also true.
> 
>> With the caveat that this might kill some less common but not necessarily
>> illegitimate use cases that do more than just creating and initialising a
>> single module...
> 
> You mean creating another module in the init function? That's fine, but
> that should be a call to __import__ anyway and that should handle things
> properly.

Ok.


> Else you are circumventing the import system and you can do
> everything from scratch.

I guess I'd be ok with putting that burden on users in this case.


> I don't see why this would stop you from doing
> anything you want, it just simplifies the common case.

The only problematic case I see here would be a module that calculates the
size of its state space at init time, e.g. based on some platform specifics
or environment parameters, anything from the platform specific size of some
data type to the runtime configured number of OpenMP threads.

That would make the PyModuleDef a compile time static thing - not sure if
that's currently required.

Stefan



From ncoghlan at gmail.com  Thu Nov  8 16:55:15 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 9 Nov 2012 01:55:15 +1000
Subject: [Python-Dev] Update - Re: Make extension module initialisation
 more like Python module initialisation
In-Reply-To: <k7gfq0$177$1@ger.gmane.org>
References: <k7g9k9$8bl$1@ger.gmane.org>
	<k7gfq0$177$1@ger.gmane.org>
Message-ID: <CADiSq7cKArRVpmV4Ah_31d-xXeQfY_Yny-64FLdLe2TjqTndxQ@mail.gmail.com>

On Fri, Nov 9, 2012 at 12:32 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> here's an updated proposal, adopting Marc-Andre's improvement that uses a
> new field in the PyModuleDef struct to register the callback. Note that
> this change no longer keeps up binary compatibility, which may or may not
> be acceptable for Python 3.4.
>

It's not acceptable, as PyModuleDef is part of PEP 384's stable ABI. All
such public structures are locked at their original size.

3) The original proposal used a new C-API function to register the callback
> explicitly, as opposed to extending the PyModuleDef struct. This has the
> advantage of keeping up binary compatibility with existing Py3.3
> extensions. It has the disadvantage of adding another indirection to the
> setup procedure where a static function pointer would suffice.
>

Module initialisation is (and must be) part of the stable ABI. Indirection
(especially through Python) is a *good* thing, as, ideally, any new
interfaces should be defined in a way that doesn't increase the maintenance
burden for the stable ABI.

I don't agree that the use of a new init API can fail silently, so long as
it completely *replaces* the old API, rather than being an addition. That
way, since you won't be defining the *old* init function at all, old
versions will correctly refuse to load your module.

So I propose that we simply *fix* extension module loading to work the same
way as everything else: the loader creates the module object, and passes it
in to a new init function to be fully populated. __file__ and __name__
would be passed in as preinitialised module attributes. The existing
PyModule_Create functions would be complemented by a PyModule_SetDef
function which allowed a PyModuleDef to be configured on a pre-existing
module.

Extension modules that wanted to create multiple Python modules would still
be free to do so - it would just be up to the extension initialisation code
to call PyModule_Create to construct them and set __file__ based on the
__file__ of the passed in module.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121109/d54e8fc5/attachment.html>

From tseaver at palladion.com  Thu Nov  8 17:18:32 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 08 Nov 2012 11:18:32 -0500
Subject: [Python-Dev] Detecting tp_compare / tp_richcompare from Python
Message-ID: <k7gm2f$ujh$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

While porting the BTrees package (split out from ZODB) to Python3., my
first step is to get the pure-Python reference implementation working:
in order to do that, I need a way to check that objects used as keys are
*not* using the comparison semantics inherited from the base 'object'
class, because those semantics rely on properties which are not stable
across the lifetime of the (persisted / restored) key.

The existing C code does something like::

 static int
 check_argument_cmp(PyObject *arg)
 {
    if (arg->ob_type->tp_richcompare == NULL
       && arg->ob_type->tp_compare ==
        ((PyTypeObject *)object_)->ob_type->tp_compare)
     {
       PyErr_SetString(PyExc_TypeError, "Object has default comparison");
       return 0;
     }
   return 1;
 }

Unless I'm mistaken, there is no way to do the equivalent from pure
Python.. I tried a couple of approximations which relied on non-API
attributes (I"m looking at out, methodwrapper.__objclass__), but without
much success (and I need the code to work on PyPy / Jython, too).

Am I missing something?



Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCb21gACgkQ+gerLs4ltQ4jBACfV0lQaQ2eW2vhAtWunLUsPQWM
esEAoMYeeQvlJVnckaBg4HM19LoxPIWB
=+d+0
-----END PGP SIGNATURE-----


From guido at python.org  Thu Nov  8 18:38:46 2012
From: guido at python.org (Guido van Rossum)
Date: Thu, 8 Nov 2012 09:38:46 -0800
Subject: [Python-Dev] Detecting tp_compare / tp_richcompare from Python
In-Reply-To: <k7gm2f$ujh$1@ger.gmane.org>
References: <k7gm2f$ujh$1@ger.gmane.org>
Message-ID: <CAP7+vJJ3qDNDH=em2Q_dqE9C=Lu4fDwmNi1yFb+nRefaYeDdQg@mail.gmail.com>

Well, the default behavior has changed to raise an exception when
using <, <=, >, >=; i.e., inequalities do not have a default
implementation at all. Perhaps that is enough for your purpose? == and
!= still compare by pointer, but you're primarily interested in
ordering her, right?

On Thu, Nov 8, 2012 at 8:18 AM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> While porting the BTrees package (split out from ZODB) to Python3., my
> first step is to get the pure-Python reference implementation working:
> in order to do that, I need a way to check that objects used as keys are
> *not* using the comparison semantics inherited from the base 'object'
> class, because those semantics rely on properties which are not stable
> across the lifetime of the (persisted / restored) key.
>
> The existing C code does something like::
>
>  static int
>  check_argument_cmp(PyObject *arg)
>  {
>     if (arg->ob_type->tp_richcompare == NULL
>        && arg->ob_type->tp_compare ==
>         ((PyTypeObject *)object_)->ob_type->tp_compare)
>      {
>        PyErr_SetString(PyExc_TypeError, "Object has default comparison");
>        return 0;
>      }
>    return 1;
>  }
>
> Unless I'm mistaken, there is no way to do the equivalent from pure
> Python.. I tried a couple of approximations which relied on non-API
> attributes (I"m looking at out, methodwrapper.__objclass__), but without
> much success (and I need the code to work on PyPy / Jython, too).
>
> Am I missing something?
>
>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver          +1 540-429-0999          tseaver at palladion.com
> Palladion Software   "Excellence by Design"    http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
>
> iEYEARECAAYFAlCb21gACgkQ+gerLs4ltQ4jBACfV0lQaQ2eW2vhAtWunLUsPQWM
> esEAoMYeeQvlJVnckaBg4HM19LoxPIWB
> =+d+0
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org



-- 
--Guido van Rossum (python.org/~guido)

From storchaka at gmail.com  Thu Nov  8 18:45:28 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 08 Nov 2012 19:45:28 +0200
Subject: [Python-Dev] cpython: Issue #16218: skip test if filesystem
 doesn't support required encoding
In-Reply-To: <20121107230845.7A7D225006F@webabinitio.net>
References: <3Xv06h2mWtzMCY@mail.python.org>
	<20121103181343.27e690d7@pitrou.net>
	<CANcUUedKG+P4c+=SxxCG7_GVRJkNmSnnYXtuB=DYeQ2m+P5hLg@mail.gmail.com>
	<CAMpsgwYys2dWU0S+32TM1Kyq=u14ne50hZRg5myvezR=_5sRXQ@mail.gmail.com>
	<20121107230845.7A7D225006F@webabinitio.net>
Message-ID: <k7gr3v$fmq$1@ger.gmane.org>

On 08.11.12 01:08, R. David Murray wrote:
> Alexandre's point was that the string did not appear to be arbitrary,
> but rather appeared to specifically be a string containing surrogates.
> Is this not the case?

My intention was testing with filename which cannot be decoded as UTF-8 
in strict mode.  I agree that testing with name which is encodable in 
locale encoding can be useful too, but now the test has no effect on 
UTF-8 locale.



From tseaver at palladion.com  Thu Nov  8 18:55:47 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 08 Nov 2012 12:55:47 -0500
Subject: [Python-Dev] Detecting tp_compare / tp_richcompare from Python
In-Reply-To: <CAP7+vJJ3qDNDH=em2Q_dqE9C=Lu4fDwmNi1yFb+nRefaYeDdQg@mail.gmail.com>
References: <k7gm2f$ujh$1@ger.gmane.org>
	<CAP7+vJJ3qDNDH=em2Q_dqE9C=Lu4fDwmNi1yFb+nRefaYeDdQg@mail.gmail.com>
Message-ID: <509BF223.1040408@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/08/2012 12:38 PM, Guido van Rossum wrote:
> Well, the default behavior has changed to raise an exception when 
> using <, <=, >, >=; i.e., inequalities do not have a default 
> implementation at all. Perhaps that is enough for your purpose? ==
> and != still compare by pointer, but you're primarily interested in 
> ordering her, right?

The b-search algorithm needs ordering on keys, but also meaningful
(non-identity) equality.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCb8iMACgkQ+gerLs4ltQ6WOACgwERmICWr80qnEoOVVLQtFwTH
ttAAnAt4An0dPjaRuZyJlDAUGzH0CS7B
=u2mG
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Thu Nov  8 18:55:47 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 08 Nov 2012 12:55:47 -0500
Subject: [Python-Dev] Detecting tp_compare / tp_richcompare from Python
In-Reply-To: <CAP7+vJJ3qDNDH=em2Q_dqE9C=Lu4fDwmNi1yFb+nRefaYeDdQg@mail.gmail.com>
References: <k7gm2f$ujh$1@ger.gmane.org>
	<CAP7+vJJ3qDNDH=em2Q_dqE9C=Lu4fDwmNi1yFb+nRefaYeDdQg@mail.gmail.com>
Message-ID: <509BF223.1040408@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/08/2012 12:38 PM, Guido van Rossum wrote:
> Well, the default behavior has changed to raise an exception when 
> using <, <=, >, >=; i.e., inequalities do not have a default 
> implementation at all. Perhaps that is enough for your purpose? ==
> and != still compare by pointer, but you're primarily interested in 
> ordering her, right?

The b-search algorithm needs ordering on keys, but also meaningful
(non-identity) equality.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCb8iMACgkQ+gerLs4ltQ6WOACgwERmICWr80qnEoOVVLQtFwTH
ttAAnAt4An0dPjaRuZyJlDAUGzH0CS7B
=u2mG
-----END PGP SIGNATURE-----

From guido at python.org  Thu Nov  8 18:59:35 2012
From: guido at python.org (Guido van Rossum)
Date: Thu, 8 Nov 2012 09:59:35 -0800
Subject: [Python-Dev] Detecting tp_compare / tp_richcompare from Python
In-Reply-To: <509BF223.1040408@palladion.com>
References: <k7gm2f$ujh$1@ger.gmane.org>
	<CAP7+vJJ3qDNDH=em2Q_dqE9C=Lu4fDwmNi1yFb+nRefaYeDdQg@mail.gmail.com>
	<509BF223.1040408@palladion.com>
Message-ID: <CAP7+vJLnW9g7XtgeUUyv=+FiHwzaB_a=qRTa+Oj5mPF5SOH9jA@mail.gmail.com>

In Python, you can write

if C.__eq__ == object.__eq__:
    print('class C does not override __eq__')

Does that help?

On Thu, Nov 8, 2012 at 9:55 AM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 11/08/2012 12:38 PM, Guido van Rossum wrote:
>> Well, the default behavior has changed to raise an exception when
>> using <, <=, >, >=; i.e., inequalities do not have a default
>> implementation at all. Perhaps that is enough for your purpose? ==
>> and != still compare by pointer, but you're primarily interested in
>> ordering her, right?
>
> The b-search algorithm needs ordering on keys, but also meaningful
> (non-identity) equality.
>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver          +1 540-429-0999          tseaver at palladion.com
> Palladion Software   "Excellence by Design"    http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
>
> iEYEARECAAYFAlCb8iMACgkQ+gerLs4ltQ6WOACgwERmICWr80qnEoOVVLQtFwTH
> ttAAnAt4An0dPjaRuZyJlDAUGzH0CS7B
> =u2mG
> -----END PGP SIGNATURE-----



-- 
--Guido van Rossum (python.org/~guido)

From storchaka at gmail.com  Thu Nov  8 19:06:15 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 08 Nov 2012 20:06:15 +0200
Subject: [Python-Dev] chained assignment weirdity
In-Reply-To: <509B06B9.6000406@nedbatchelder.com>
References: <508EA6E3.5090201@simplistix.co.uk>
	<D13B9C12-0DEC-495F-9805-F5DFF964E524@voidspace.org.uk>
	<50937B1B.2080404@simplistix.co.uk>
	<5B9E75D2-5953-4772-BB44-E567AA3B2672@voidspace.org.uk>
	<5098AB9F.2010806@simplistix.co.uk> <k7al0r$iom$1@ger.gmane.org>
	<CADiSq7fhGORQF24EiPXnZUJJTP9WxfVDYWFkb1rJWW76aUqaow@mail.gmail.com>
	<k7bd1m$i5b$1@ger.gmane.org>
	<20121106162655.A858625006B@webabinitio.net>
	<50994FDD.8060906@nedbatchelder.com>
	<CAP7+vJKqW9UE338y=tjBAC93wTX7XoGA+VgN87j4CYJ1r9Htuw@mail.gmail.com>
	<509A504C.10807@nedbatchelder.com>
	<CAP7+vJLKAuWMxcym5oQEq_HJ8yLY-Xg_H_cqEoBnq2eD09FY5w@mail.gmail.com>
	<CADiSq7f=ZFB6ATMwzWa0V9+46YXc4Wzh7+jfs7Cv_mHv-bFruQ@mail.gmail.com>
	<k7e4ih$idp$1@ger.gmane.org> <509AD510.5060200@nedbatchelder.com>
	<k7emb0$okb$1@ger.gmane.org> <509B06B9.6000406@nedbatchelder.com>
Message-ID: <k7gsau$r7s$1@ger.gmane.org>

On 08.11.12 03:11, Ned Batchelder wrote:
> Sorry, I should have been clearer: I was asking about weird not to say, 
> "This is weird and should be changed!", but to get clarification from 
> Serhiy about his statement, " It will be weird if a dict comprehension 
> and a plain loop will be inconsistent."  I honestly didn't know which 
> behavior he considered inconsistent and therefore weird.

I was referring to two of the most popular idioms to dynamically create a dict.

  d = {}
  for x in a:
      d[k(x)] = v(x)

  d = {k(x): v(x) for x in a}

For now these methods are consistent.

I agree that the use of the side effects here is not a sensible idea, but when such effects occur by accident, it will cause a surprise.



From alexandre at peadrop.com  Thu Nov  8 19:10:42 2012
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Thu, 8 Nov 2012 10:10:42 -0800
Subject: [Python-Dev] cpython: Issue #16218: skip test if filesystem
 doesn't support required encoding
In-Reply-To: <k7gr3v$fmq$1@ger.gmane.org>
References: <3Xv06h2mWtzMCY@mail.python.org>
	<20121103181343.27e690d7@pitrou.net>
	<CANcUUedKG+P4c+=SxxCG7_GVRJkNmSnnYXtuB=DYeQ2m+P5hLg@mail.gmail.com>
	<CAMpsgwYys2dWU0S+32TM1Kyq=u14ne50hZRg5myvezR=_5sRXQ@mail.gmail.com>
	<20121107230845.7A7D225006F@webabinitio.net>
	<k7gr3v$fmq$1@ger.gmane.org>
Message-ID: <CANcUUee6-Tt=4gdPZa6Kq3H3sdGzCXzOp0eeDC452sL1cDbURQ@mail.gmail.com>

On Thu, Nov 8, 2012 at 9:45 AM, Serhiy Storchaka <storchaka at gmail.com>wrote:

> My intention was testing with filename which cannot be decoded as UTF-8 in
> strict mode.  I agree that testing with name which is encodable in locale
> encoding can be useful too, but now the test has no effect on UTF-8 locale.


So should we change the test back? Or just change the test name?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121108/10342bfa/attachment.html>

From storchaka at gmail.com  Thu Nov  8 19:21:19 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 08 Nov 2012 20:21:19 +0200
Subject: [Python-Dev] cpython: Issue #16218: skip test if filesystem
 doesn't support required encoding
In-Reply-To: <CANcUUee6-Tt=4gdPZa6Kq3H3sdGzCXzOp0eeDC452sL1cDbURQ@mail.gmail.com>
References: <3Xv06h2mWtzMCY@mail.python.org>
	<20121103181343.27e690d7@pitrou.net>
	<CANcUUedKG+P4c+=SxxCG7_GVRJkNmSnnYXtuB=DYeQ2m+P5hLg@mail.gmail.com>
	<CAMpsgwYys2dWU0S+32TM1Kyq=u14ne50hZRg5myvezR=_5sRXQ@mail.gmail.com>
	<20121107230845.7A7D225006F@webabinitio.net>
	<k7gr3v$fmq$1@ger.gmane.org>
	<CANcUUee6-Tt=4gdPZa6Kq3H3sdGzCXzOp0eeDC452sL1cDbURQ@mail.gmail.com>
Message-ID: <k7gt76$40n$1@ger.gmane.org>

On 08.11.12 20:10, Alexandre Vassalotti wrote:
> So should we change the test back? Or just change the test name?

No. I also missed some details. The issue is still not closed.


From tseaver at palladion.com  Thu Nov  8 20:51:54 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 08 Nov 2012 14:51:54 -0500
Subject: [Python-Dev] Detecting tp_compare / tp_richcompare from Python
In-Reply-To: <CAP7+vJLnW9g7XtgeUUyv=+FiHwzaB_a=qRTa+Oj5mPF5SOH9jA@mail.gmail.com>
References: <k7gm2f$ujh$1@ger.gmane.org>
	<CAP7+vJJ3qDNDH=em2Q_dqE9C=Lu4fDwmNi1yFb+nRefaYeDdQg@mail.gmail.com>
	<509BF223.1040408@palladion.com>
	<CAP7+vJLnW9g7XtgeUUyv=+FiHwzaB_a=qRTa+Oj5mPF5SOH9jA@mail.gmail.com>
Message-ID: <509C0D5A.5090403@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/08/2012 12:59 PM, Guido van Rossum wrote:
> In Python, you can write
> 
> if C.__eq__ == object.__eq__: print('class C does not override
> __eq__')
> 
> Does that help?

That works in Python3, but not in Python2 -- methodwrappers don't compare
equal the same way slotwrappers do :(.  I still need to straddle (back to
Python 2.6.



Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCcDVoACgkQ+gerLs4ltQ49kwCgtBJX1oyi5cudZeE4LBEdcAmO
aTAAn0jL56d3eK/sDUD3G7zzZ/ZdQH4W
=NG86
-----END PGP SIGNATURE-----


From tseaver at palladion.com  Thu Nov  8 21:31:51 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 08 Nov 2012 15:31:51 -0500
Subject: [Python-Dev] Detecting tp_compare / tp_richcompare from Python
In-Reply-To: <509C0D5A.5090403@palladion.com>
References: <k7gm2f$ujh$1@ger.gmane.org>
	<CAP7+vJJ3qDNDH=em2Q_dqE9C=Lu4fDwmNi1yFb+nRefaYeDdQg@mail.gmail.com>
	<509BF223.1040408@palladion.com>
	<CAP7+vJLnW9g7XtgeUUyv=+FiHwzaB_a=qRTa+Oj5mPF5SOH9jA@mail.gmail.com>
	<509C0D5A.5090403@palladion.com>
Message-ID: <509C16B7.20608@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/08/2012 02:51 PM, Tres Seaver wrote:
> On 11/08/2012 12:59 PM, Guido van Rossum wrote:
>> In Python, you can write
> 
>> if C.__eq__ == object.__eq__: print('class C does not override 
>> __eq__')
> 
>> Does that help?
> 
> That works in Python3, but not in Python2 -- methodwrappers don't
> compare equal the same way slotwrappers do :(.  I still need to
> straddle (back to Python 2.6.

I think I'm going to need two separate checkers:

- - In Py3k, use your check above for both '__eq__' and '__lt__', e.g:

   def check_comparable(key);
       klass = type(key)
       if klass.__eq__ == object.__eq__ or klass.__lt__ == object.__lt__:
           raise TypeError('Incomparable')

- - In Python2, I think I can just check that the instance (not its class)
  has either ('__lt__' and '__eq__') *or* '__cmp__', e.g.:

   def check_comparable(key);
       if not ((getattr(key, '__eq__', None) is not None and
                getattr(key, '__lt__', None) is not None) or
                getattr(key, '__cmp__', None) is not None):
          raise TypeError('Incomparable')



Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCcFrcACgkQ+gerLs4ltQ6JDgCeJpDmo70H9Xgz3preNHVvaTl5
dYkAoNS+fpSXwZiaD2J2ONyQZ2mPIcFC
=OiMY
-----END PGP SIGNATURE-----


From g.brandl at gmx.net  Thu Nov  8 21:43:19 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 08 Nov 2012 21:43:19 +0100
Subject: [Python-Dev] cpython: issue9584: Add {} list expansion to glob.
 Original patch by Mathieu Bridon
In-Reply-To: <3XwsjZ4sTWzLs9@mail.python.org>
References: <3XwsjZ4sTWzLs9@mail.python.org>
Message-ID: <k7h5h6$frq$1@ger.gmane.org>

On 11/06/2012 02:56 PM, tim.golden wrote:
> http://hg.python.org/cpython/rev/dafca4714298
> changeset:   80273:dafca4714298
> user:        Tim Golden <mail at timgolden.me.uk>
> date:        Tue Nov 06 13:50:42 2012 +0000
> summary:
>   issue9584: Add {} list expansion to glob. Original patch by Mathieu Bridon
> 
> files:
>   Doc/library/glob.rst  |  11 ++--
>   Lib/glob.py           |  65 ++++++++++++++++++++++--------
>   Lib/test/test_glob.py |  64 +++++++++++++++++++++++++++++-
>   Misc/NEWS             |   3 +
>   4 files changed, 118 insertions(+), 25 deletions(-)
> 
> 
> diff --git a/Doc/library/glob.rst b/Doc/library/glob.rst
> --- a/Doc/library/glob.rst
> +++ b/Doc/library/glob.rst
> @@ -13,10 +13,10 @@
>  
>  The :mod:`glob` module finds all the pathnames matching a specified pattern
>  according to the rules used by the Unix shell.  No tilde expansion is done, but
> -``*``, ``?``, and character ranges expressed with ``[]`` will be correctly
> -matched.  This is done by using the :func:`os.listdir` and
> -:func:`fnmatch.fnmatch` functions in concert, and not by actually invoking a
> -subshell.  (For tilde and shell variable expansion, use
> +``*``, ``?``, character ranges expressed with ``[]`` and list of options
> +expressed with ``{}`` will be correctly matched.  This is done by using the
> +:func:`os.listdir` and :func:`fnmatch.fnmatch` functions in concert, and not by
> +actually invoking a subshell.  (For tilde and shell variable expansion, use
>  :func:`os.path.expanduser` and :func:`os.path.expandvars`.)

Needs a versionchanged.

In any case, brace expansion is not part of globbing (see the bash or zsh
manuals) because it does not generate valid file names, and it is a non-POSIX
expansion of some shells.  Are you sure it should be put into the glob module?
(Not speaking of the backward incompatibility it creates.)

Georg


From g.brandl at gmx.net  Thu Nov  8 21:46:00 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 08 Nov 2012 21:46:00 +0100
Subject: [Python-Dev] cpython: issue9584: Add {} list expansion to glob.
 Original patch by Mathieu Bridon
In-Reply-To: <k7h5h6$frq$1@ger.gmane.org>
References: <3XwsjZ4sTWzLs9@mail.python.org> <k7h5h6$frq$1@ger.gmane.org>
Message-ID: <k7h5m7$frq$2@ger.gmane.org>

On 11/08/2012 09:43 PM, Georg Brandl wrote:
> 
> Needs a versionchanged.
> 
> In any case, brace expansion is not part of globbing (see the bash or zsh
> manuals) because it does not generate valid file names, and it is a non-POSIX
> expansion of some shells.  Are you sure it should be put into the glob module?
> (Not speaking of the backward incompatibility it creates.)

Sorry, just saw the reversion.  Never mind, my concerns have been addressed.

Georg


From mail at timgolden.me.uk  Thu Nov  8 21:52:25 2012
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 08 Nov 2012 20:52:25 +0000
Subject: [Python-Dev] cpython: issue9584: Add {} list expansion to glob.
 Original patch by Mathieu Bridon
In-Reply-To: <k7h5h6$frq$1@ger.gmane.org>
References: <3XwsjZ4sTWzLs9@mail.python.org> <k7h5h6$frq$1@ger.gmane.org>
Message-ID: <509C1B89.7080005@timgolden.me.uk>

On 08/11/2012 20:43, Georg Brandl wrote:
> On 11/06/2012 02:56 PM, tim.golden wrote:
>> http://hg.python.org/cpython/rev/dafca4714298
>> changeset:   80273:dafca4714298
>> user:        Tim Golden <mail at timgolden.me.uk>
>> date:        Tue Nov 06 13:50:42 2012 +0000
>> summary:
>>    issue9584: Add {} list expansion to glob. Original patch by Mathieu Bridon
>>
>> files:
>>    Doc/library/glob.rst  |  11 ++--
>>    Lib/glob.py           |  65 ++++++++++++++++++++++--------
>>    Lib/test/test_glob.py |  64 +++++++++++++++++++++++++++++-
>>    Misc/NEWS             |   3 +
>>    4 files changed, 118 insertions(+), 25 deletions(-)
>>
>>
>> diff --git a/Doc/library/glob.rst b/Doc/library/glob.rst
>> --- a/Doc/library/glob.rst
>> +++ b/Doc/library/glob.rst
>> @@ -13,10 +13,10 @@
>>
>>   The :mod:`glob` module finds all the pathnames matching a specified pattern
>>   according to the rules used by the Unix shell.  No tilde expansion is done, but
>> -``*``, ``?``, and character ranges expressed with ``[]`` will be correctly
>> -matched.  This is done by using the :func:`os.listdir` and
>> -:func:`fnmatch.fnmatch` functions in concert, and not by actually invoking a
>> -subshell.  (For tilde and shell variable expansion, use
>> +``*``, ``?``, character ranges expressed with ``[]`` and list of options
>> +expressed with ``{}`` will be correctly matched.  This is done by using the
>> +:func:`os.listdir` and :func:`fnmatch.fnmatch` functions in concert, and not by
>> +actually invoking a subshell.  (For tilde and shell variable expansion, use
>>   :func:`os.path.expanduser` and :func:`os.path.expandvars`.)
>
> Needs a versionchanged.
>
> In any case, brace expansion is not part of globbing (see the bash or zsh
> manuals) because it does not generate valid file names, and it is a non-POSIX
> expansion of some shells.  Are you sure it should be put into the glob module?
> (Not speaking of the backward incompatibility it creates.)

I backed it out very soon afterwards, Georg. It had had some (quite a 
bit of) discussion on the issue, but I'd messed up the patch somehow and 
the backwards compat issue was raised pretty much immediately by Serhiy. 
So I pulled the commit.

TJG

Insofar as

From chris at simplistix.co.uk  Fri Nov  9 10:57:22 2012
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Nov 2012 09:57:22 +0000
Subject: [Python-Dev] problems building python2.7
Message-ID: <509CD382.4080609@simplistix.co.uk>

Hi All,

I wanted to run the unit tests before checking in the patch for 
http://bugs.python.org/issue16441, even though it's a trivial change, so 
I was trying to follow the instructions at:

http://docs.python.org/devguide/

I'm on MacOS, so following the "unix" instructions did:

./configure --with-pydebug && make -j2

This appears to have worked, given the end of the output:

Python build finished, but the necessary bits to build these modules 
were not found:
_bsddb             dl                 gdbm
imageop            linuxaudiodev      ossaudiodev
readline           spwd               sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the 
module's name.

running build_scripts
creating build/scripts-2.7
copying and adjusting /Users/chris/LocalHG/cpython/Tools/scripts/pydoc 
-> build/scripts-2.7
copying and adjusting /Users/chris/LocalHG/cpython/Tools/scripts/idle -> 
build/scripts-2.7
copying and adjusting /Users/chris/LocalHG/cpython/Tools/scripts/2to3 -> 
build/scripts-2.7
copying and adjusting /Users/chris/LocalHG/cpython/Lib/smtpd.py -> 
build/scripts-2.7
changing mode of build/scripts-2.7/pydoc from 644 to 755
changing mode of build/scripts-2.7/idle from 644 to 755
changing mode of build/scripts-2.7/2to3 from 644 to 755
changing mode of build/scripts-2.7/smtpd.py from 644 to 755
[65930 refs]

However, I can't find the python it's built...
I thought I'd be clever and try:

buzzkill:cpython chris$ cat build/scripts-2.7/2to3
#!/usr/local/bin/python2.7
...

There is a python there, but it's a symlink put in place around a year ago.

So, where should I look for the built python?

Okay, so regardless, my change is only to the stdlib, so I thought I'd 
try the test instructions anyway:

/usr/local/bin/python2.7 -m test -j3 gives:

/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: 
No module named test.__main__; 'test' is a package and cannot be 
directly executed

Now, if this had worked, would it have discovered the right gzip tests, 
or is it going to run whatever came with the python2.7 distro the binary 
comes from?

Chris


-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk

From hansmu at xs4all.nl  Fri Nov  9 11:37:15 2012
From: hansmu at xs4all.nl (Hans Mulder)
Date: Fri, 09 Nov 2012 11:37:15 +0100
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <509CD382.4080609@simplistix.co.uk>
References: <509CD382.4080609@simplistix.co.uk>
Message-ID: <k7imcq$8f0$1@ger.gmane.org>

On 9/11/12 10:57:22, Chris Withers wrote:
> Hi All,
> 
> I wanted to run the unit tests before checking in the patch for
> http://bugs.python.org/issue16441, even though it's a trivial change, so
> I was trying to follow the instructions at:
> 
> http://docs.python.org/devguide/
> 
> I'm on MacOS, so following the "unix" instructions did:
> 
> ./configure --with-pydebug && make -j2
> 
> This appears to have worked, given the end of the output:
> 
> Python build finished, but the necessary bits to build these modules
> were not found:
> _bsddb             dl                 gdbm
> imageop            linuxaudiodev      ossaudiodev
> readline           spwd               sunaudiodev
> To find the necessary bits, look in setup.py in detect_modules() for the
> module's name.
> 
> running build_scripts
> creating build/scripts-2.7
> copying and adjusting /Users/chris/LocalHG/cpython/Tools/scripts/pydoc
> -> build/scripts-2.7
> copying and adjusting /Users/chris/LocalHG/cpython/Tools/scripts/idle ->
> build/scripts-2.7
> copying and adjusting /Users/chris/LocalHG/cpython/Tools/scripts/2to3 ->
> build/scripts-2.7
> copying and adjusting /Users/chris/LocalHG/cpython/Lib/smtpd.py ->
> build/scripts-2.7
> changing mode of build/scripts-2.7/pydoc from 644 to 755
> changing mode of build/scripts-2.7/idle from 644 to 755
> changing mode of build/scripts-2.7/2to3 from 644 to 755
> changing mode of build/scripts-2.7/smtpd.py from 644 to 755
> [65930 refs]
> 
> However, I can't find the python it's built...

On my system, it's in the current directory.  It's called "python.exe",
to avoid a name clash with the "Python" subdirectory.

> I thought I'd be clever and try:
> 
> buzzkill:cpython chris$ cat build/scripts-2.7/2to3
> #!/usr/local/bin/python2.7
> ...

Presumable, it would be installed in /usr/local/bin if I'd tried
"make install".

> There is a python there, but it's a symlink put in place around a year ago.

Same here.

> So, where should I look for the built python?

In the current directory:
$ ./python.exe
Python 2.7.3+ (2.7:8b181c75792f, Nov  9 2012, 11:26:59)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>


Hope this helps,

-- HansM



From fuzzyman at voidspace.org.uk  Fri Nov  9 11:52:06 2012
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 9 Nov 2012 10:52:06 +0000
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <509CD382.4080609@simplistix.co.uk>
References: <509CD382.4080609@simplistix.co.uk>
Message-ID: <999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>


On 9 Nov 2012, at 09:57, Chris Withers <chris at simplistix.co.uk> wrote:

> Hi All,
> 
> I wanted to run the unit tests before checking in the patch for http://bugs.python.org/issue16441, even though it's a trivial change, so I was trying to follow the instructions at:
> 
> http://docs.python.org/devguide/
> 
> I'm on MacOS, so following the "unix" instructions did:
> 
> ./configure --with-pydebug && make -j2
> 
> This appears to have worked, given the end of the output:
> 
> Python build finished, but the necessary bits to build these modules were not found:
> _bsddb             dl                 gdbm
> imageop            linuxaudiodev      ossaudiodev
> readline           spwd               sunaudiodev
> To find the necessary bits, look in setup.py in detect_modules() for the module's name.
> 
> running build_scripts
> creating build/scripts-2.7
> copying and adjusting /Users/chris/LocalHG/cpython/Tools/scripts/pydoc -> build/scripts-2.7
> copying and adjusting /Users/chris/LocalHG/cpython/Tools/scripts/idle -> build/scripts-2.7
> copying and adjusting /Users/chris/LocalHG/cpython/Tools/scripts/2to3 -> build/scripts-2.7
> copying and adjusting /Users/chris/LocalHG/cpython/Lib/smtpd.py -> build/scripts-2.7
> changing mode of build/scripts-2.7/pydoc from 644 to 755
> changing mode of build/scripts-2.7/idle from 644 to 755
> changing mode of build/scripts-2.7/2to3 from 644 to 755
> changing mode of build/scripts-2.7/smtpd.py from 644 to 755
> [65930 refs]
> 
> However, I can't find the python it's built...


It should be python.exe (yes really).

After the build you should be able to do:

	./python.exe

Michael


> I thought I'd be clever and try:
> 
> buzzkill:cpython chris$ cat build/scripts-2.7/2to3
> #!/usr/local/bin/python2.7
> ...
> 
> There is a python there, but it's a symlink put in place around a year ago.
> 
> So, where should I look for the built python?
> 
> Okay, so regardless, my change is only to the stdlib, so I thought I'd try the test instructions anyway:
> 
> /usr/local/bin/python2.7 -m test -j3 gives:
> 
> /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: No module named test.__main__; 'test' is a package and cannot be directly executed
> 
> Now, if this had worked, would it have discovered the right gzip tests, or is it going to run whatever came with the python2.7 distro the binary comes from?
> 
> Chris
> 
> 
> -- 
> Simplistix - Content Management, Batch Processing & Python Consulting
>            - http://www.simplistix.co.uk
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
> 


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From chris at simplistix.co.uk  Fri Nov  9 11:57:39 2012
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Nov 2012 10:57:39 +0000
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
Message-ID: <509CE1A3.2000202@simplistix.co.uk>

On 09/11/2012 10:52, Michael Foord wrote:
>
>> However, I can't find the python it's built...
>
> It should be python.exe (yes really).

Hah! Should http://docs.python.org/devguide/ be updated to reflect this 
or does this only affect Mac OS? (or should we correct the build so it 
doesn't spit out a .exe?)

> After the build you should be able to do:
>
> 	./python.exe

Unfortunately, still:

buzzkill:cpython chris$ ./python.exe -m test -j3
/Users/chris/LocalHG/cpython/python.exe: No module named test.__main__; 
'test' is a package and cannot be directly executed
[18856 refs]

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk

From hansmu at xs4all.nl  Fri Nov  9 12:09:26 2012
From: hansmu at xs4all.nl (Hans Mulder)
Date: Fri, 09 Nov 2012 12:09:26 +0100
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <509CE1A3.2000202@simplistix.co.uk>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
Message-ID: <k7io94$o92$1@ger.gmane.org>

On 9/11/12 11:57:39, Chris Withers wrote:
> On 09/11/2012 10:52, Michael Foord wrote:
>>
>>> However, I can't find the python it's built...
>>
>> It should be python.exe (yes really).
> 
> Hah! Should http://docs.python.org/devguide/ be updated to reflect this
> or does this only affect Mac OS? (or should we correct the build so it
> doesn't spit out a .exe?)

It only affects MacOS.  On other Unix system, the build process spits
out an executable named "python".  On MacOS that's not possible,
because that name would clash with the subdirectory named "Python".

>> After the build you should be able to do:
>>
>>     ./python.exe
> 
> Unfortunately, still:
> 
> buzzkill:cpython chris$ ./python.exe -m test -j3
> /Users/chris/LocalHG/cpython/python.exe: No module named test.__main__;
> 'test' is a package and cannot be directly executed
> [18856 refs]

How about "make test" ?


Hope this helps,

-- HansM



From petri at digip.org  Fri Nov  9 12:10:42 2012
From: petri at digip.org (Petri Lehtinen)
Date: Fri, 9 Nov 2012 13:10:42 +0200
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <509CE1A3.2000202@simplistix.co.uk>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
Message-ID: <20121109111042.GG4265@p29>

Chris Withers wrote:
> On 09/11/2012 10:52, Michael Foord wrote:
> >
> >>However, I can't find the python it's built...
> >
> >It should be python.exe (yes really).
> 
> Hah! Should http://docs.python.org/devguide/ be updated to reflect
> this or does this only affect Mac OS? (or should we correct the
> build so it doesn't spit out a .exe?)

The filesystem on Mac OS X is case-insensitive by default, so plain
"python" would clash with the "Python" directory.

> >After the build you should be able to do:
> >
> >	./python.exe
> 
> Unfortunately, still:
> 
> buzzkill:cpython chris$ ./python.exe -m test -j3
> /Users/chris/LocalHG/cpython/python.exe: No module named
> test.__main__; 'test' is a package and cannot be directly executed
> [18856 refs]

You're running the tests for Python 2, so the corret way to do it is:

  ./python.exe -m test.regrtest

See http://docs.python.org/devguide/runtests.html#running .

From hansmu at xs4all.nl  Fri Nov  9 12:54:24 2012
From: hansmu at xs4all.nl (Hans Mulder)
Date: Fri, 09 Nov 2012 12:54:24 +0100
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <k7io94$o92$1@ger.gmane.org>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk> <k7io94$o92$1@ger.gmane.org>
Message-ID: <k7iqte$eq5$1@ger.gmane.org>

On 9/11/12 12:09:26, Hans Mulder wrote:
> On 9/11/12 11:57:39, Chris Withers wrote:
>> On 09/11/2012 10:52, Michael Foord wrote:
>>>
>>>> However, I can't find the python it's built...
>>>
>>> It should be python.exe (yes really).
>>
>> Hah! Should http://docs.python.org/devguide/ be updated to reflect this
>> or does this only affect Mac OS? (or should we correct the build so it
>> doesn't spit out a .exe?)
> 
> It only affects MacOS.  On other Unix system, the build process spits
> out an executable named "python".  On MacOS that's not possible,
> because that name would clash with the subdirectory named "Python".
> 
>>> After the build you should be able to do:
>>>
>>>     ./python.exe
>>
>> Unfortunately, still:
>>
>> buzzkill:cpython chris$ ./python.exe -m test -j3
>> /Users/chris/LocalHG/cpython/python.exe: No module named test.__main__;
>> 'test' is a package and cannot be directly executed
>> [18856 refs]
> 
> How about "make test" ?
> 
> 
> Hope this helps,
> 
> -- HansM

I tried "make test", and I got:

test test_urllib failed -- Traceback (most recent call last):
  File "/Users/hans/python/cpython/cpython-2.7/Lib/test/test_urllib.py",
line 235, in test_missing_localfile
    fp.close()
UnboundLocalError: local variable 'fp' referenced before assignment

Is this a bug in the test suite, or a regression in 2.7.3+ ?

This is on MacOS 10.6.5, using a checkout from hg.python.org
from earlier today.

-- HansM




From ronaldoussoren at mac.com  Fri Nov  9 12:54:17 2012
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri, 09 Nov 2012 12:54:17 +0100
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <509CE1A3.2000202@simplistix.co.uk>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
Message-ID: <59B407E8-B1B7-464C-A0D3-5B692953763A@mac.com>


On 9 Nov, 2012, at 11:57, Chris Withers <chris at simplistix.co.uk> wrote:

> On 09/11/2012 10:52, Michael Foord wrote:
>> 
>>> However, I can't find the python it's built...
>> 
>> It should be python.exe (yes really).
> 
> Hah! Should http://docs.python.org/devguide/ be updated to reflect this or does this only affect Mac OS? (or should we correct the build so it doesn't spit out a .exe?)

How else should it be named, given that the default filesystem on OSX machines is not case sensitive. The binary cannot be named "python" because that clashes with the directory named "Python" in the source and build tres.

Ronald


From chris at simplistix.co.uk  Fri Nov  9 12:57:54 2012
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Nov 2012 11:57:54 +0000
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <59B407E8-B1B7-464C-A0D3-5B692953763A@mac.com>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
	<59B407E8-B1B7-464C-A0D3-5B692953763A@mac.com>
Message-ID: <509CEFC2.9020205@simplistix.co.uk>

On 09/11/2012 11:54, Ronald Oussoren wrote:
>
> On 9 Nov, 2012, at 11:57, Chris Withers<chris at simplistix.co.uk>  wrote:
>
>> On 09/11/2012 10:52, Michael Foord wrote:
>>>
>>>> However, I can't find the python it's built...
>>>
>>> It should be python.exe (yes really).
>>
>> Hah! Should http://docs.python.org/devguide/ be updated to reflect this or does this only affect Mac OS? (or should we correct the build so it doesn't spit out a .exe?)
>
> How else should it be named, given that the default filesystem on OSX machines is not case sensitive. The binary cannot be named "python" because that clashes with the directory named "Python" in the source and build tres.

Maybe python.bin, or put it in the ./bin folder?

I saw python.exe, but my brain went "oh, .exe, that's windows, I'll 
ignore that then, since I'm on Mac OS"

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Fri Nov  9 15:11:26 2012
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 09 Nov 2012 14:11:26 +0000
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <k7iqte$eq5$1@ger.gmane.org>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
	<k7io94$o92$1@ger.gmane.org> <k7iqte$eq5$1@ger.gmane.org>
Message-ID: <509D0F0E.7080306@simplistix.co.uk>

On 09/11/2012 11:54, Hans Mulder wrote:
> I tried "make test", and I got:
>
> test test_urllib failed -- Traceback (most recent call last):
>    File "/Users/hans/python/cpython/cpython-2.7/Lib/test/test_urllib.py",
> line 235, in test_missing_localfile
>      fp.close()
> UnboundLocalError: local variable 'fp' referenced before assignment
>
> Is this a bug in the test suite, or a regression in 2.7.3+ ?
>
> This is on MacOS 10.6.5, using a checkout from hg.python.org
> from earlier today.

I'm on MacOS 10.7.5 and got:

356 tests OK.
35 tests skipped:
     test_al test_bsddb test_bsddb3 test_cd test_cl test_codecmaps_cn
     test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr
     test_codecmaps_tw test_curses test_dl test_epoll test_gdb
     test_gdbm test_gl test_imageop test_imgfile test_largefile
     test_linuxaudiodev test_msilib test_ossaudiodev test_readline
     test_smtpnet test_socketserver test_startfile test_sunaudiodev
     test_timeout test_tk test_ttk_guionly test_urllib2net
     test_urllibnet test_winreg test_winsound test_zipfile64
4 skips unexpected on darwin:
     test_dl test_readline test_tk test_ttk_guionly

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk

From hansmu at xs4all.nl  Fri Nov  9 16:44:00 2012
From: hansmu at xs4all.nl (Hans Mulder)
Date: Fri, 09 Nov 2012 16:44:00 +0100
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <509D0F0E.7080306@simplistix.co.uk>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
	<k7io94$o92$1@ger.gmane.org> <k7iqte$eq5$1@ger.gmane.org>
	<509D0F0E.7080306@simplistix.co.uk>
Message-ID: <k7j8bv$98f$1@ger.gmane.org>

On 9/11/12 15:11:26, Chris Withers wrote:
> On 09/11/2012 11:54, Hans Mulder wrote:
>> I tried "make test", and I got:
>>
>> test test_urllib failed -- Traceback (most recent call last):
>>    File "/Users/hans/python/cpython/cpython-2.7/Lib/test/test_urllib.py",
>> line 235, in test_missing_localfile
>>      fp.close()
>> UnboundLocalError: local variable 'fp' referenced before assignment
>>
>> Is this a bug in the test suite, or a regression in 2.7.3+ ?
>>
>> This is on MacOS 10.6.5, using a checkout from hg.python.org
>> from earlier today.
> 
> I'm on MacOS 10.7.5 and got:
> 
> 356 tests OK.
> 35 tests skipped:
>     test_al test_bsddb test_bsddb3 test_cd test_cl test_codecmaps_cn
>     test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr
>     test_codecmaps_tw test_curses test_dl test_epoll test_gdb
>     test_gdbm test_gl test_imageop test_imgfile test_largefile
>     test_linuxaudiodev test_msilib test_ossaudiodev test_readline
>     test_smtpnet test_socketserver test_startfile test_sunaudiodev
>     test_timeout test_tk test_ttk_guionly test_urllib2net
>     test_urllibnet test_winreg test_winsound test_zipfile64
> 4 skips unexpected on darwin:
>     test_dl test_readline test_tk test_ttk_guionly

I looked into it, and the problem is this bit of code (line 230-235):

        try:
            self.assertTrue(os.path.exists(tmp_file))
            fp = urllib.urlopen(tmp_fileurl)
        finally:
            os.close(fd)
            fp.close()

Due to a misconfiguration, urllib.thishost() raises an IOError on my
laptop.  This causes urllib.urlopen to raise an exception, and the
name fp is never bound, so that the fp.close() in the finally clause
raises an UnboundLocalError, masking the problem in urlopen.

A quick fix would be:

        try:
            self.assertTrue(os.path.exists(tmp_file))
            fp = urllib.urlopen(tmp_fileurl)
            fp.close()
        finally:
            os.close(fd)

That way, the .close is only attempted if the open succeeds.

-- HansM



From tjreedy at udel.edu  Fri Nov  9 16:48:06 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 09 Nov 2012 10:48:06 -0500
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <509CE1A3.2000202@simplistix.co.uk>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
Message-ID: <k7j8jr$83t$1@ger.gmane.org>

On 11/9/2012 5:57 AM, Chris Withers wrote:
> On 09/11/2012 10:52, Michael Foord wrote:
>>
>>> However, I can't find the python it's built...
>>
>> It should be python.exe (yes really).
>
> Hah! Should http://docs.python.org/devguide/ be updated to reflect this
> or does this only affect Mac OS?

The devguide could be updated to briefly note the Mac specific variation.

-- 
Terry Jan Reedy


From rdmurray at bitdance.com  Fri Nov  9 17:07:13 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 09 Nov 2012 11:07:13 -0500
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <k7j8bv$98f$1@ger.gmane.org>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
	<k7io94$o92$1@ger.gmane.org> <k7iqte$eq5$1@ger.gmane.org>
	<509D0F0E.7080306@simplistix.co.uk> <k7j8bv$98f$1@ger.gmane.org>
Message-ID: <20121109160714.4327E25003E@webabinitio.net>

On Fri, 09 Nov 2012 16:44:00 +0100, Hans Mulder <hansmu at xs4all.nl> wrote:
> I looked into it, and the problem is this bit of code (line 230-235):
> 
>         try:
>             self.assertTrue(os.path.exists(tmp_file))
>             fp = urllib.urlopen(tmp_fileurl)
>         finally:
>             os.close(fd)
>             fp.close()
> 
> Due to a misconfiguration, urllib.thishost() raises an IOError on my
> laptop.  This causes urllib.urlopen to raise an exception, and the
> name fp is never bound, so that the fp.close() in the finally clause
> raises an UnboundLocalError, masking the problem in urlopen.
> 
> A quick fix would be:
> 
>         try:
>             self.assertTrue(os.path.exists(tmp_file))
>             fp = urllib.urlopen(tmp_fileurl)
>             fp.close()
>         finally:
>             os.close(fd)
> 
> That way, the .close is only attempted if the open succeeds.

Could you open an issue for this on the tracker, please?  That
way we won't forget to fix it.

--David

From status at bugs.python.org  Fri Nov  9 18:07:26 2012
From: status at bugs.python.org (Python tracker)
Date: Fri,  9 Nov 2012 18:07:26 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20121109170726.C1B0C1CBD9@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-11-02 - 2012-11-09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3806 ( +2)
  closed 24418 (+57)
  total  28224 (+59)

Open issues with patches: 1657 


Issues opened (42)
==================

#13424: Add examples for open???s new opener argument
http://bugs.python.org/issue13424  reopened by eric.araujo

#16350: zlib.Decompress.decompress() after EOF discards existing value
http://bugs.python.org/issue16350  reopened by serhiy.storchaka

#16389: re._compiled_typed's lru_cache causes significant degradation 
http://bugs.python.org/issue16389  opened by pjenvey

#16391: add "terminator" ctor argument to logging.StreamHandlers deriv
http://bugs.python.org/issue16391  opened by nikicat

#16392: import crashes on circular imports in ext modules
http://bugs.python.org/issue16392  opened by scoder

#16394: Reducing tee() memory footprint
http://bugs.python.org/issue16394  opened by cami

#16395: Documentation claims that PySequence_Fast returns a tuple, whe
http://bugs.python.org/issue16395  opened by sfllaw

#16396: Importing ctypes.wintypes on Linux gives a traceback
http://bugs.python.org/issue16396  opened by techtonik

#16397: UserString doesn't combine nicely with strings
http://bugs.python.org/issue16397  opened by Jorge.Cardona

#16398: deque.rotate() could be much faster
http://bugs.python.org/issue16398  opened by sfllaw

#16399: argparse: append action with default list adds to list instead
http://bugs.python.org/issue16399  opened by Markus.Amalthea.Magnuson

#16400: update default PyPI behavior in docs re: listing versions
http://bugs.python.org/issue16400  opened by chris.jerdonek

#16401: mention PKG-INFO in the documentation
http://bugs.python.org/issue16401  opened by chris.jerdonek

#16403: update distutils docs to say that maintainer replaces author
http://bugs.python.org/issue16403  opened by chris.jerdonek

#16404: Uses of PyLong_FromLong that don't check for errors
http://bugs.python.org/issue16404  opened by nedbat

#16405: Explain how to set up the whitespace commit hook locally
http://bugs.python.org/issue16405  opened by ncoghlan

#16406: move the "Uploading Packages" section to distutils/packageinde
http://bugs.python.org/issue16406  opened by chris.jerdonek

#16408: FD leaks in zipfile
http://bugs.python.org/issue16408  opened by Eric.Busboom

#16409: urlretrieve regression: first call returns block size as 0
http://bugs.python.org/issue16409  opened by techtonik

#16411: zlib.Decompress.decompress() retains pointer to input buffer w
http://bugs.python.org/issue16411  opened by nadeem.vawda

#16413: Non cross-platform behavior of os.path.split
http://bugs.python.org/issue16413  opened by techtonik

#16416: Mac OS X: don't use the locale encoding but UTF-8 to encode an
http://bugs.python.org/issue16416  opened by haypo

#16418: argparse with many choices can generate absurdly long usage me
http://bugs.python.org/issue16418  opened by Juraj.Variny

#16421: importlib.machinery.ExtensionFileLoader cannot load several mo
http://bugs.python.org/issue16421  opened by eudoxos

#16422: Decimal constants should be the same for py & c module version
http://bugs.python.org/issue16422  opened by yselivanov

#16423: urllib data URL recipe
http://bugs.python.org/issue16423  opened by panzi

#16424: regression: os.path.split('//hostname/foo/bar.txt')
http://bugs.python.org/issue16424  opened by techtonik

#16425: minidom replaceChild(new_child, old_child) removes new_child e
http://bugs.python.org/issue16425  opened by Martin.Kugler

#16427: Faster hash implementation
http://bugs.python.org/issue16427  opened by serhiy.storchaka

#16428: turtle with compound shape doesn't get clicks
http://bugs.python.org/issue16428  opened by pythonick

#16429: Emit SyntaxWarning for code that risks UnboundLocalError
http://bugs.python.org/issue16429  opened by ncoghlan

#16434: SocketServer call shutdown in the wrong way
http://bugs.python.org/issue16434  opened by Erik.G??nther

#16436: Link directly to set and frozenset in built-in function docs
http://bugs.python.org/issue16436  opened by fossilet

#16437: issubclass doc improvement
http://bugs.python.org/issue16437  opened by fossilet

#16438: Numeric operator predecence confusing
http://bugs.python.org/issue16438  opened by fossilet

#16442: PATH_MAX vs MAXPATHLEN vs pathconf(..., _PC_PATH_MAX).
http://bugs.python.org/issue16442  opened by trent

#16443: Add docstrings to regular expression match objects
http://bugs.python.org/issue16443  opened by rhettinger

#16444: Use support.TESTFN_UNDECODABLE on UNIX
http://bugs.python.org/issue16444  opened by haypo

#16445: SEGFAULT when deleting Exception.message
http://bugs.python.org/issue16445  opened by amaury.forgeotdarc

#16446: pdb raises BdbQuit on 'quit' when started with set_trace
http://bugs.python.org/issue16446  opened by xdegaye

#16447: SEGFAULT when setting type.__name__
http://bugs.python.org/issue16447  opened by amaury.forgeotdarc

#16432: Template strings documentation in Python 3 refers to % substit
http://bugs.python.org/issue16432  opened by andrewsg



Most recent 15 issues with no replies (15)
==========================================

#16446: pdb raises BdbQuit on 'quit' when started with set_trace
http://bugs.python.org/issue16446

#16442: PATH_MAX vs MAXPATHLEN vs pathconf(..., _PC_PATH_MAX).
http://bugs.python.org/issue16442

#16429: Emit SyntaxWarning for code that risks UnboundLocalError
http://bugs.python.org/issue16429

#16428: turtle with compound shape doesn't get clicks
http://bugs.python.org/issue16428

#16427: Faster hash implementation
http://bugs.python.org/issue16427

#16425: minidom replaceChild(new_child, old_child) removes new_child e
http://bugs.python.org/issue16425

#16423: urllib data URL recipe
http://bugs.python.org/issue16423

#16416: Mac OS X: don't use the locale encoding but UTF-8 to encode an
http://bugs.python.org/issue16416

#16406: move the "Uploading Packages" section to distutils/packageinde
http://bugs.python.org/issue16406

#16403: update distutils docs to say that maintainer replaces author
http://bugs.python.org/issue16403

#16400: update default PyPI behavior in docs re: listing versions
http://bugs.python.org/issue16400

#16398: deque.rotate() could be much faster
http://bugs.python.org/issue16398

#16386: imp.find_module does not specify registry key it searches on w
http://bugs.python.org/issue16386

#16382: Better warnings exception for bad category
http://bugs.python.org/issue16382

#16379: SQLite error code not exposed to python
http://bugs.python.org/issue16379



Most recent 15 issues waiting for review (15)
=============================================

#16447: SEGFAULT when setting type.__name__
http://bugs.python.org/issue16447

#16445: SEGFAULT when deleting Exception.message
http://bugs.python.org/issue16445

#16444: Use support.TESTFN_UNDECODABLE on UNIX
http://bugs.python.org/issue16444

#16427: Faster hash implementation
http://bugs.python.org/issue16427

#16423: urllib data URL recipe
http://bugs.python.org/issue16423

#16421: importlib.machinery.ExtensionFileLoader cannot load several mo
http://bugs.python.org/issue16421

#16416: Mac OS X: don't use the locale encoding but UTF-8 to encode an
http://bugs.python.org/issue16416

#16411: zlib.Decompress.decompress() retains pointer to input buffer w
http://bugs.python.org/issue16411

#16408: FD leaks in zipfile
http://bugs.python.org/issue16408

#16395: Documentation claims that PySequence_Fast returns a tuple, whe
http://bugs.python.org/issue16395

#16392: import crashes on circular imports in ext modules
http://bugs.python.org/issue16392

#16389: re._compiled_typed's lru_cache causes significant degradation 
http://bugs.python.org/issue16389

#16386: imp.find_module does not specify registry key it searches on w
http://bugs.python.org/issue16386

#16382: Better warnings exception for bad category
http://bugs.python.org/issue16382

#16381: Introduce option to force the interpreter to exit upon MemoryE
http://bugs.python.org/issue16381



Top 10 most discussed issues (10)
=================================

#14621: Hash function is not randomized properly
http://bugs.python.org/issue14621  36 msgs

#16218: Python launcher does not support unicode characters
http://bugs.python.org/issue16218  32 msgs

#16353: add function to os module for getting path to default shell
http://bugs.python.org/issue16353  32 msgs

#9584: Allow curly brace expansion
http://bugs.python.org/issue9584  21 msgs

#14794: slice.indices raises OverflowError
http://bugs.python.org/issue14794  18 msgs

#16392: import crashes on circular imports in ext modules
http://bugs.python.org/issue16392  17 msgs

#16389: re._compiled_typed's lru_cache causes significant degradation 
http://bugs.python.org/issue16389  16 msgs

#16421: importlib.machinery.ExtensionFileLoader cannot load several mo
http://bugs.python.org/issue16421  15 msgs

#10030: Patch for zip decryption speedup
http://bugs.python.org/issue10030  12 msgs

#13349: Non-informative error message in index() and remove() function
http://bugs.python.org/issue13349   8 msgs



Issues closed (54)
==================

#4318: optparse: formatting of help text/descriptions
http://bugs.python.org/issue4318  closed by eric.araujo

#4711: Wide literals in the table of contents overflow in documentati
http://bugs.python.org/issue4711  closed by ezio.melotti

#5765: stack overflow evaluating eval("()" * 30000)
http://bugs.python.org/issue5765  closed by ncoghlan

#5894: Lookup of localised language name by ISO 639 language code and
http://bugs.python.org/issue5894  closed by christian.heimes

#6709: It's possible to create TryExcept with no handlers
http://bugs.python.org/issue6709  closed by benjamin.peterson

#7317: Display full tracebacks when an error occurs asynchronously
http://bugs.python.org/issue7317  closed by asvetlov

#8271: str.decode('utf8', 'replace') -- conformance with Unicode 5.2.
http://bugs.python.org/issue8271  closed by ezio.melotti

#8401: Strange behavior of bytearray slice assignment
http://bugs.python.org/issue8401  closed by ezio.melotti

#9109: absolute import cleanups for Python 3
http://bugs.python.org/issue9109  closed by akuchling

#10358: Doc styles for print should only use dark colors
http://bugs.python.org/issue10358  closed by ezio.melotti

#10385: Mark up "subprocess" as module in its doc
http://bugs.python.org/issue10385  closed by ezio.melotti

#11166: No exit when daemon thread is running.
http://bugs.python.org/issue11166  closed by akuchling

#11313: Speed up default encode()/decode()
http://bugs.python.org/issue11313  closed by ezio.melotti

#11481: The copy module already uses copyreg
http://bugs.python.org/issue11481  closed by ezio.melotti

#11631: Python 2.7.1 64bit, Win7 64bit problem to read and write with 
http://bugs.python.org/issue11631  closed by christian.heimes

#11842: slice.indices with negative step and default stop
http://bugs.python.org/issue11842  closed by mark.dickinson

#12759: "(?P=)" input for Tools/scripts/redemo.py raises unnhandled ex
http://bugs.python.org/issue12759  closed by ezio.melotti

#13301: the script Tools/i18n/msgfmt.py allows arbitrary code executio
http://bugs.python.org/issue13301  closed by ezio.melotti

#15001: segmentation fault with del sys.modules['__main__']
http://bugs.python.org/issue15001  closed by hynek

#15076: Sometimes couldn't import os, shown 'import site' failed, use 
http://bugs.python.org/issue15076  closed by akuchling

#15165: test_email: failure on Windows
http://bugs.python.org/issue15165  closed by python-dev

#15641: Clean up importlib for Python 3.4
http://bugs.python.org/issue15641  closed by asvetlov

#15814: memoryview: equality-hash invariant
http://bugs.python.org/issue15814  closed by skrah

#15837: Added test to test_random.py testing Random.shuffle
http://bugs.python.org/issue15837  closed by pitrou

#16048: Tutorial-classes-remarks: replace paragraph
http://bugs.python.org/issue16048  closed by ezio.melotti

#16152: Trailing whitespace makes tokenize.generate_tokens pathologica
http://bugs.python.org/issue16152  closed by ezio.melotti

#16183: ZipExtFile object close without file handle closed
http://bugs.python.org/issue16183  closed by python-dev

#16284: concurrent.futures ThreadPoolExecutor keeps unnecessary refere
http://bugs.python.org/issue16284  closed by asvetlov

#16304: re: Match Objects always have a boolean value of  True
http://bugs.python.org/issue16304  closed by ezio.melotti

#16309: "PYTHONPATH=" different from no PYTHONPATH at all
http://bugs.python.org/issue16309  closed by asvetlov

#16311: Use _PyUnicodeWriter API in text decoders
http://bugs.python.org/issue16311  closed by python-dev

#16336: Check input in surrogatepass error handler
http://bugs.python.org/issue16336  closed by ezio.melotti

#16338: pysnmp/asyncore - timeout ineffective?
http://bugs.python.org/issue16338  closed by neologix

#16354: Remember python version choice on docs.python.org
http://bugs.python.org/issue16354  closed by georg.brandl

#16385: evaluating literal dict with repeated keys gives no warnings/e
http://bugs.python.org/issue16385  closed by benjamin.peterson

#16390: re compilation slow in Python 3.3 due to functools.lru_cache o
http://bugs.python.org/issue16390  closed by pjenvey

#16393: typo in pkgutil.py
http://bugs.python.org/issue16393  closed by ezio.melotti

#16402: range slicing swallows exceptions from __index__ methods.
http://bugs.python.org/issue16402  closed by mark.dickinson

#16407: Python 2.7.3: test_sys fails when building under 10.7.5 with d
http://bugs.python.org/issue16407  closed by ned.deily

#16410: rewrite winsound with ctypes
http://bugs.python.org/issue16410  closed by loewis

#16412: test_shutil: 2 failures related to symlinks
http://bugs.python.org/issue16412  closed by pitrou

#16414: Add support.NONASCII to test non-ASCII characters
http://bugs.python.org/issue16414  closed by haypo

#16415: eventlet.monkey_patch() breaks subprocess.Popen on Windows
http://bugs.python.org/issue16415  closed by r.david.murray

#16417: Integer converted in tuple without request
http://bugs.python.org/issue16417  closed by r.david.murray

#16419: email.message._headers is a list
http://bugs.python.org/issue16419  closed by ezio.melotti

#16420: PEP 249 (DB-API 2.0) converted to reStructuredText
http://bugs.python.org/issue16420  closed by lemburg

#16426: RoundUp signature removal test (please ignore)
http://bugs.python.org/issue16426  closed by lemburg

#16430: re.match blocking and taking 100% CPU
http://bugs.python.org/issue16430  closed by serhiy.storchaka

#16431: CDecimal disregards subclass passed into __new__ when value ar
http://bugs.python.org/issue16431  closed by skrah

#16433: unittest.TestCase.assertNotEqual has incorrect docstring.
http://bugs.python.org/issue16433  closed by ezio.melotti

#16435: Python 3 doc link to Python 2 FAQ
http://bugs.python.org/issue16435  closed by asvetlov

#16439: Code not collapsed correctly
http://bugs.python.org/issue16439  closed by asvetlov

#16440: Exception type
http://bugs.python.org/issue16440  closed by ezio.melotti

#16441: range usage in gzip module leads to excessive memory usage.
http://bugs.python.org/issue16441  closed by cjw296

From chris.jerdonek at gmail.com  Fri Nov  9 18:48:18 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Fri, 9 Nov 2012 09:48:18 -0800
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <509CE1A3.2000202@simplistix.co.uk>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
Message-ID: <CAOTb1wcOwCencss7EuPDSdfzM0EGdCHaaFr2fuO6OkQteLkZOA@mail.gmail.com>

On Fri, Nov 9, 2012 at 2:57 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> On 09/11/2012 10:52, Michael Foord wrote:
>>
>> It should be python.exe (yes really).
>
> Hah! Should http://docs.python.org/devguide/ be updated to reflect this or
> does this only affect Mac OS? (or should we correct the build so it doesn't
> spit out a .exe?)

That link already mentions this information in the top "Quick Start" section:

"3. Run the tests with ./python -m test -j3 (use ./python.exe on most
Mac OS X systems and PCbuild\python_d.exe on Windows; replace test
with test.regrtest for 2.7)."

But it would probably be good to repeat this information in certain
locations, for example in the more detailed sections on building and
running tests.

--Chris

From solipsis at pitrou.net  Fri Nov  9 20:20:50 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 9 Nov 2012 20:20:50 +0100
Subject: [Python-Dev] problems building python2.7
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
	<59B407E8-B1B7-464C-A0D3-5B692953763A@mac.com>
	<509CEFC2.9020205@simplistix.co.uk>
Message-ID: <20121109202050.346dadaa@pitrou.net>

On Fri, 09 Nov 2012 11:57:54 +0000
Chris Withers <chris at simplistix.co.uk> wrote:

> On 09/11/2012 11:54, Ronald Oussoren wrote:
> >
> > On 9 Nov, 2012, at 11:57, Chris Withers<chris at simplistix.co.uk>  wrote:
> >
> >> On 09/11/2012 10:52, Michael Foord wrote:
> >>>
> >>>> However, I can't find the python it's built...
> >>>
> >>> It should be python.exe (yes really).
> >>
> >> Hah! Should http://docs.python.org/devguide/ be updated to reflect this or does this only affect Mac OS? (or should we correct the build so it doesn't spit out a .exe?)
> >
> > How else should it be named, given that the default filesystem on OSX machines is not case sensitive. The binary cannot be named "python" because that clashes with the directory named "Python" in the source and build tres.
> 
> Maybe python.bin, or put it in the ./bin folder?
> 
> I saw python.exe, but my brain went "oh, .exe, that's windows, I'll 
> ignore that then, since I'm on Mac OS"

Or just re-wire your brain.

Regards

Antoine.



From larry at hastings.org  Fri Nov  9 22:05:50 2012
From: larry at hastings.org (Larry Hastings)
Date: Fri, 09 Nov 2012 19:05:50 -0200
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <59B407E8-B1B7-464C-A0D3-5B692953763A@mac.com>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
	<59B407E8-B1B7-464C-A0D3-5B692953763A@mac.com>
Message-ID: <509D702E.10605@hastings.org>

On 11/09/2012 09:54 AM, Ronald Oussoren wrote:
> On 9 Nov, 2012, at 11:57, Chris Withers <chris at simplistix.co.uk> wrote:
>> Hah! Should http://docs.python.org/devguide/ be updated to reflect this or does this only affect Mac OS? (or should we correct the build so it doesn't spit out a .exe?)
> How else should it be named, given that the default filesystem on OSX machines is not case sensitive. The binary cannot be named "python" because that clashes with the directory named "Python" in the source and build tres.

Have we considered renaming the directory instead?


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121109/d3c17846/attachment.html>

From nad at acm.org  Fri Nov  9 23:32:18 2012
From: nad at acm.org (Ned Deily)
Date: Fri, 09 Nov 2012 14:32:18 -0800
Subject: [Python-Dev] problems building python2.7
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
	<59B407E8-B1B7-464C-A0D3-5B692953763A@mac.com>
	<509D702E.10605@hastings.org>
Message-ID: <nad-D5F982.14321809112012@news.gmane.org>

In article <509D702E.10605 at hastings.org>,
 Larry Hastings <larry at hastings.org> wrote:
> On 11/09/2012 09:54 AM, Ronald Oussoren wrote:
> > On 9 Nov, 2012, at 11:57, Chris Withers <chris at simplistix.co.uk> wrote:
> >> Hah! Should http://docs.python.org/devguide/ be updated to reflect this or 
> >> does this only affect Mac OS? (or should we correct the build so it 
> >> doesn't spit out a .exe?)
> > How else should it be named, given that the default filesystem on OSX 
> > machines is not case sensitive. The binary cannot be named "python" because 
> > that clashes with the directory named "Python" in the source and build 
> > tres.
> 
> Have we considered renaming the directory instead?

That's another solution.  Another is to use the optional case-sensitive 
variant of HFS+ on OS X.  But, really, this is a very minor one-time 
issue facing people doing development on Python itself.  Normal users or 
Python don't run iit from a source directory, they run it from an 
installed location (make install) and then there is no such problem.  
It's not worth changing at this point.

-- 
 Ned Deily,
 nad at acm.org


From hansmu at xs4all.nl  Sat Nov 10 15:46:47 2012
From: hansmu at xs4all.nl (Hans Mulder)
Date: Sat, 10 Nov 2012 15:46:47 +0100
Subject: [Python-Dev] problems building python2.7
In-Reply-To: <20121109160714.4327E25003E@webabinitio.net>
References: <509CD382.4080609@simplistix.co.uk>
	<999BF3A1-7456-4194-ADF7-0C68D59A8D16@voidspace.org.uk>
	<509CE1A3.2000202@simplistix.co.uk>
	<k7io94$o92$1@ger.gmane.org> <k7iqte$eq5$1@ger.gmane.org>
	<509D0F0E.7080306@simplistix.co.uk> <k7j8bv$98f$1@ger.gmane.org>
	<20121109160714.4327E25003E@webabinitio.net>
Message-ID: <k7lpcm$3vc$1@ger.gmane.org>

On 9/11/12 17:07:13, R. David Murray wrote:
> On Fri, 09 Nov 2012 16:44:00 +0100, Hans Mulder <hansmu at xs4all.nl> wrote:
>> I looked into it, and the problem is this bit of code (line 230-235):
>>
>>         try:
>>             self.assertTrue(os.path.exists(tmp_file))
>>             fp = urllib.urlopen(tmp_fileurl)
>>         finally:
>>             os.close(fd)
>>             fp.close()
>>
>> Due to a misconfiguration, urllib.thishost() raises an IOError on my
>> laptop.  This causes urllib.urlopen to raise an exception, and the
>> name fp is never bound, so that the fp.close() in the finally clause
>> raises an UnboundLocalError, masking the problem in urlopen.
>>
>> A quick fix would be:
>>
>>         try:
>>             self.assertTrue(os.path.exists(tmp_file))
>>             fp = urllib.urlopen(tmp_fileurl)
>>             fp.close()
>>         finally:
>>             os.close(fd)
>>
>> That way, the .close is only attempted if the open succeeds.
> 
> Could you open an issue for this on the tracker, please?  That
> way we won't forget to fix it.

Done: issue 16450.

-- HansM



From nadeem.vawda at gmail.com  Sat Nov 10 22:54:23 2012
From: nadeem.vawda at gmail.com (Nadeem Vawda)
Date: Sat, 10 Nov 2012 22:54:23 +0100
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default):
 Fixes issue #16409: The reporthook callback made by the legacy
In-Reply-To: <3XzWwq3HNVzPt0@mail.python.org>
References: <3XzWwq3HNVzPt0@mail.python.org>
Message-ID: <CANF4RM=wYFEvHiGphj27bUxOFcXyYMw9DTFzuwKEeD5j3PmdDQ@mail.gmail.com>

This change appears to have broken test_urllib. See, for example:

http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/4774/steps/test/logs/stdio

- Nadeem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121110/74206ebe/attachment.html>

From tismer at stackless.com  Sun Nov 11 21:31:08 2012
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 11 Nov 2012 21:31:08 +0100
Subject: [Python-Dev] Setting project home path the best way
Message-ID: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>

Hi friends,

I have a project that has its root somewhere on my machine. 
This project has many folders and contains quite some modules. 

There is a common root of the module tree, and I want to use
- either absolute imports
- relative imports with '.'

Problem:

- I want to run any module inside the heirarchy from the command-line

- this should work, regardless what my 'cwd' is

- this should work with or without virtualenv. 

So far, things work fine with virtualenv, because sys.executable is in the project module tree. 

Without virtualenv, this is not so. But I hate to make settings like PYTHONPATH, because these are not permanent. .  

Question:

How should I define my project root dir in a unique way, without setting an environment variable?
What is the lest intrusive way to spell that?

Reason:

I'd like to make things work correctly and unambigously when I call a script
inside the module heirarchy. Things are not fixed: there exist many checkouts
In the file system, and each should know where to search its home/root in the tree. 

Is this elegantly possible to deduce from the actually executed script file?

Cheers - chris

Sent from my Ei4Steve

From tismer at stackless.com  Sun Nov 11 21:53:40 2012
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 11 Nov 2012 21:53:40 +0100
Subject: [Python-Dev] Relatwd: py3 import strategy (was: Setting project
	home path the best way(
In-Reply-To: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
Message-ID: <9A3375A4-D61D-4938-983C-553CFDDAFC29@stackless.com>

Once again on this:

With the introduction of module folders
without __init__, it has become even harder to deduce a sensible project root
Dir without relying on a global settings file. Abd as you probably agree, such files are a bad idea. Any global is bad, also in the file system. 

Guido, this is another reason why I dislike the absence of __init__. :

Rhere is no longer an indicator that pretty clearly defines the root of my module heirarchy. 

Cheers, hoping for enlightment - chris

Sent from my Ei4Steve

On Nov 11, 2012, at 21:31, Christian Tismer <tismer at stackless.com> wrote:

> Hi friends,
> 
> I have a project that has its root somewhere on my machine. 
> This project has many folders and contains quite some modules. 
> 
> There is a common root of the module tree, and I want to use
> - either absolute imports
> - relative imports with '.'
> 
> Problem:
> 
> - I want to run any module inside the heirarchy from the command-line
> 
> - this should work, regardless what my 'cwd' is
> 
> - this should work with or without virtualenv. 
> 
> So far, things work fine with virtualenv, because sys.executable is in the project module tree. 
> 
> Without virtualenv, this is not so. But I hate to make settings like PYTHONPATH, because these are not permanent. .  
> 
> Question:
> 
> How should I define my project root dir in a unique way, without setting an environment variable?
> What is the lest intrusive way to spell that?
> 
> Reason:
> 
> I'd like to make things work correctly and unambigously when I call a script
> inside the module heirarchy. Things are not fixed: there exist many checkouts
> In the file system, and each should know where to search its home/root in the tree. 
> 
> Is this elegantly possible to deduce from the actually executed script file?
> 
> Cheers - chris
> 
> Sent from my Ei4Steve
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/tismer%40stackless.com

From ericsnowcurrently at gmail.com  Sun Nov 11 22:45:23 2012
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Sun, 11 Nov 2012 14:45:23 -0700
Subject: [Python-Dev] Relatwd: py3 import strategy (was: Setting project
 home path the best way(
In-Reply-To: <9A3375A4-D61D-4938-983C-553CFDDAFC29@stackless.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<9A3375A4-D61D-4938-983C-553CFDDAFC29@stackless.com>
Message-ID: <CALFfu7AGJOaAu=z7x1e-GWwmaWo+G_HOFsAR=_G7x5X70t57MQ@mail.gmail.com>

On Sun, Nov 11, 2012 at 1:53 PM, Christian Tismer <tismer at stackless.com>wrote:

> Once again on this:
>
> With the introduction of module folders
> without __init__, it has become even harder to deduce a sensible project
> root
> Dir without relying on a global settings file. Abd as you probably agree,
> such files are a bad idea. Any global is bad, also in the file system.
>
> Guido, this is another reason why I dislike the absence of __init__. :
>
> Rhere is no longer an indicator that pretty clearly defines the root of my
> module heirarchy.
>

 This came up during the discussions leading up to PEP 420, particularly
regarding the impact on PEP 395.  My thoughts at the time on addressing the
problem:

http://mail.python.org/pipermail/import-sig/2012-March/000438.html

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121111/da19ec0f/attachment.html>

From tismer at stackless.com  Sun Nov 11 22:59:34 2012
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 11 Nov 2012 22:59:34 +0100
Subject: [Python-Dev] Relatwd: py3 import strategy
In-Reply-To: <CALFfu7AGJOaAu=z7x1e-GWwmaWo+G_HOFsAR=_G7x5X70t57MQ@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<9A3375A4-D61D-4938-983C-553CFDDAFC29@stackless.com>
	<CALFfu7AGJOaAu=z7x1e-GWwmaWo+G_HOFsAR=_G7x5X70t57MQ@mail.gmail.com>
Message-ID: <50A01FC6.7050803@stackless.com>

Hi Eric,

On 11.11.12 22:45, Eric Snow wrote:
> On Sun, Nov 11, 2012 at 1:53 PM, Christian Tismer 
> <tismer at stackless.com <mailto:tismer at stackless.com>> wrote:
>
>     Once again on this:
>
>     With the introduction of module folders
>     without __init__, it has become even harder to deduce a sensible
>     project root
>     Dir without relying on a global settings file. Abd as you probably
>     agree, such files are a bad idea. Any global is bad, also in the
>     file system.
>
>     Guido, this is another reason why I dislike the absence of __init__. :
>
>     Rhere is no longer an indicator that pretty clearly defines the
>     root of my module heirarchy.
>
>
>  This came up during the discussions leading up to PEP 420, 
> particularly regarding the impact on PEP 395.  My thoughts at the time 
> on addressing the problem:
>
> http://mail.python.org/pipermail/import-sig/2012-March/000438.html
>

I do not quite understand what you want to propose:

"""

* there is no __init__.py in the current directory (in the case where
there was one adjacent to the original module)
* current directory is on sys.path
* setup.py is in the current directory

"""

Are you saying that this is a way for me to get at the project root?
And can this be done with a one-liner, or would I need a module for that,
where the cat would bite its tail, because that module would not be found,
easily?

Please tell me in a simpler way what you think  ;-)

ciao - Chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121111/efc52972/attachment.html>

From ncoghlan at gmail.com  Mon Nov 12 01:38:40 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 12 Nov 2012 10:38:40 +1000
Subject: [Python-Dev] Setting project home path the best way
In-Reply-To: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
Message-ID: <CADiSq7c8XiH1sOT6q53H=vScMbgeq1L3x2-OVYEmj0X8Gvfx+A@mail.gmail.com>

The only way I know how to do it is to have my cwd set to the directory I
want on sys.path, then use -m for script execution (using a separate shell
session for anything where I want a different working directory).

I don't know of any way to handle a varying cwd without manipulating the
path directly through either virtualenv or PYTHONPATH.

Cheers,
Nick.

--
Sent from my phone, thus the relative brevity :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121112/06a15458/attachment.html>

From tismer at stackless.com  Mon Nov 12 13:48:57 2012
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 12 Nov 2012 13:48:57 +0100
Subject: [Python-Dev] Setting project home path the best way SOLVED
In-Reply-To: <CADiSq7c8XiH1sOT6q53H=vScMbgeq1L3x2-OVYEmj0X8Gvfx+A@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<CADiSq7c8XiH1sOT6q53H=vScMbgeq1L3x2-OVYEmj0X8Gvfx+A@mail.gmail.com>
Message-ID: <50A0F039.4010401@stackless.com>

Hi Nick, Holger,

this is a crazy fault of mine, see the end below.

I wrote:
> I have a project that has its root somewhere on my machine.
> This project has many folders and contains quite some modules.
>
> There is a common root of the module tree, and I want to use
> - either absolute imports
> - relative imports with '.'
>
> Problem:
>
> - I want to run any module inside the heirarchy from the command-line
>
> - this should work, regardless what my 'cwd' is
>
> - this should work with or without virtualenv.
>
> So far, things work fine with virtualenv, because sys.executable is in the project module tree.
>
> Without virtualenv, this is not so. But I hate to make settings like PYTHONPATH, because these are not permanent. .
>
> Question:
>
> How should I define my project root dir in a unique way, without setting an environment variable?
> What is the lest intrusive way to spell that?
>
> Reason:
>
> I'd like to make things work correctly and unambigously when I call a script
> inside the module heirarchy. Things are not fixed: there exist many checkouts
> In the file system, and each should know where to search its home/root in the tree.
>
> Is this elegantly possible to deduce from the actually executed script file?
>

Nick wrote:

On 12.11.12 01:38, Nick Coghlan wrote:
>
> The only way I know how to do it is to have my cwd set to the 
> directory I want on sys.path, then use -m for script execution (using 
> a separate shell session for anything where I want a different working 
> directory).
>
> I don't know of any way to handle a varying cwd without manipulating 
> the path directly through either virtualenv or PYTHONPATH.
>
> Cheers,
> Nick.
>
> --
> Sent from my phone, thus the relative brevity :)
>

Holger wrote (private message):
> ich selbst nutze seit langer Zeit setuptools, genauer:
>
>      python setup.py develop
>
> z.B. in pytest, execnet, tox.  Das ist wie "setup.py install" nur dass
> das paket "inplace" installiert wird.  Aber vermutlich kennst du das schon?!

So here is the crazy story:

I'm actually using setuptools for my tiffany project with

      setup.py develop

My real work is about the Pydica project which is a lot more.
There I'm playing sys.path tricks, using virtualenv and the fact
that I have full control over the local site-packages, and use some
tricky .pth scripts to set everything up. But it needs virtualenv.

Believe it or not:
*I was not able to realize that all my problems are already solved*
if I change Pydica and build a proper setup using setuptools.

I needed to bug python-dev with this, and finally got the right
hint from Holger Krekel who is absolutely right here - no point in
playing tricks, because this is all solved. And Pydica is about to
be prepared for PyPI, anyway.

My blocker was probably that Pydica is very much in development,
and I did not think right about it. The project does not need to be
ready in order to use setuptools!

My apologies, and many thanks again! This solved a Gordian Knot.

It is great when people help me to leave a dead-lock in my brain :-)

cheers - Chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/


From jmatejek at suse.cz  Mon Nov 12 16:30:41 2012
From: jmatejek at suse.cz (=?UTF-8?B?SmFuIE1hdMSbamVr?=)
Date: Mon, 12 Nov 2012 16:30:41 +0100
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <5091A31D.4000904@pearwood.info>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
	<5091A31D.4000904@pearwood.info>
Message-ID: <50A11621.6080201@suse.cz>

Dne 31.10.2012 23:15, Steven D'Aprano napsal(a):

>> I wonder if this message can be improved with a
>> pointer to the concept on when global variables become local?
>
> If you have a suggestion for an improved message, please tell us. Or raise
> an issue on the bug tracker.

I believe the "human problem" here is that the one tends to gloss over "local 
variable VARNAME", because it describes VARNAME and you already think you know 
what that is, so you don't stop to think about it.

The following would be better in this regard, IMO:

"Variable VARNAME is local in FUNCNAME, but doesn't have a value at line 123"

regards
m.

From solipsis at pitrou.net  Mon Nov 12 23:08:59 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 12 Nov 2012 23:08:59 +0100
Subject: [Python-Dev] cpython: Issue #16416: OS data are now always
 encoded/decoded to/from
References: <3Y0mFY3pjgzQ8T@mail.python.org>
Message-ID: <20121112230859.10cf9c98@pitrou.net>

On Mon, 12 Nov 2012 23:03:45 +0100 (CET)
victor.stinner <python-checkins at python.org> wrote:
>  
> +- Issue #16416: OS data are now always encoded/decoded to/from
> +  UTF-8/surrogateescape, instead of the locale encoding (which may be ASCII if
> +  no locale environment variable is set), to avoid inconsistencies with
> +  os.fsencode() and os.fsdecode() functions which are already using
> +  UTF-8/surrogateescape.

I believe you mean "OS X", not "OS".

Regards

Antoine.



From mike_mp at zzzcomputing.com  Mon Nov 12 23:15:07 2012
From: mike_mp at zzzcomputing.com (Michael Bayer)
Date: Mon, 12 Nov 2012 17:15:07 -0500
Subject: [Python-Dev] Okay to document the "exec tuple" form of the exec
 statement in Python 2.7?
In-Reply-To: <509834C5.8060302@canterbury.ac.nz>
References: <CAAu3qLXJNOgLSnHxcTdFPYrfc2eCkRvKsKotw7LH=nzU2Mixpw@mail.gmail.com>
	<CADiSq7fuYDPJawjKejJn6rn6aYpydZGfYyJv5SxoOUVom=i4hQ@mail.gmail.com>
	<509834C5.8060302@canterbury.ac.nz>
Message-ID: <FC5B9D06-CE61-4881-8B50-9D8BD0D912DB@zzzcomputing.com>


On Nov 5, 2012, at 4:51 PM, Greg Ewing wrote:

> Nick Coghlan wrote:
>> On Mon, Nov 5, 2012 at 10:08 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
>>> In Python 2, the 'exec' statement supports 'exec'-ing a (statement,
>>> globals, locals) tuple:
> 
> If this is a deliberate feature, I'd guess it's because exec
> is a statement rather than a function in Python 2, so you
> can't use * to pass a tuple of arguments to it.

any chance, if this function is documented,  someone can make it very clear what the cross-compatibility implications are with the Python 2.x form of exec() and the Python 3.x form ?   I'm having a hard time demonstrating the difference to myself, though the winning answer to his StackOverflow question http://stackoverflow.com/questions/6561482/why-did-python-3-changes-to-exec-break-this-code seems to suggest the behavior of symbol tables has changed.


> 
> -- 
> Greg
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/mike_mp%40zzzcomputing.com
> 


From v+python at g.nevcal.com  Tue Nov 13 00:52:15 2012
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 12 Nov 2012 15:52:15 -0800
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <50A11621.6080201@suse.cz>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
	<5091A31D.4000904@pearwood.info> <50A11621.6080201@suse.cz>
Message-ID: <50A18BAF.9000105@g.nevcal.com>

On 11/12/2012 7:30 AM, Jan Mat?jek wrote:
>
> I believe the "human problem" here is that the one tends to gloss over 
> "local variable VARNAME", because it describes VARNAME and you already 
> think you know what that is, so you don't stop to think about it.
>
> The following would be better in this regard, IMO:
>
> "Variable VARNAME is local in FUNCNAME, but doesn't have a value at 
> line 123" 

+1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121112/d30fa5d7/attachment.html>

From ncoghlan at gmail.com  Tue Nov 13 03:04:33 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 13 Nov 2012 12:04:33 +1000
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
Message-ID: <CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>

On Mon, Oct 29, 2012 at 4:47 AM, Daniel Holth <dholth at gmail.com> wrote:

> I think Metadata 1.3 is done. Who would like to czar?
>
(Apologies for the belated reply, it's been a busy few weeks)

I'm happy to be BDFL delegate for these. I'd like to see PEP 425 updated
with some additional rationale based on Ronald's comments later in this
thread, though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121113/287b4dca/attachment.html>

From martin at v.loewis.de  Tue Nov 13 10:51:04 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Nov 2012 10:51:04 +0100
Subject: [Python-Dev] [Catalog-sig]  accept the wheel PEPs 425, 426, 427
In-Reply-To: <CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
Message-ID: <50A21808.8040909@v.loewis.de>

Am 13.11.12 03:04, schrieb Nick Coghlan:
> On Mon, Oct 29, 2012 at 4:47 AM, Daniel Holth <dholth at gmail.com
> <mailto:dholth at gmail.com>> wrote:
>
>     I think Metadata 1.3 is done. Who would like to czar?
>
> (Apologies for the belated reply, it's been a busy few weeks)
>
> I'm happy to be BDFL delegate for these. I'd like to see PEP 425 updated
> with some additional rationale based on Ronald's comments later in this
> thread, though.

For the record, I'm still -1 on PEP 427, because of the signature issues.

The FAQ in the PEP is incorrect in claiming PGP or X.509 cannot
readily be used to verify the integrity of an archive - the whole
point of these technologies is to do exactly that.

The FAQ is entirely silent on why it is not using a more standard
signature algorithm such as ECDSA. It explains why it uses Ed25519,
but ignores that the very same rationale would apply to ECDSA as well;
plus that would be one of the standard JWS algorithms.

In addition, the FAQ claims that the format is designed to introduce
cryptopgraphy that is actually used, yet leaves the issue of key
distribution alone (except that pointing out that you can put them
into requires.txt - a file that doesn't seem to be specified anywhere).

Regards,
Martin



From mal at egenix.com  Tue Nov 13 11:26:50 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Nov 2012 11:26:50 +0100
Subject: [Python-Dev] [Catalog-sig]  accept the wheel PEPs 425, 426, 427
In-Reply-To: <50A21808.8040909@v.loewis.de>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de>
Message-ID: <50A2206A.40804@egenix.com>

On 13.11.2012 10:51, "Martin v. L?wis" wrote:
> Am 13.11.12 03:04, schrieb Nick Coghlan:
>> On Mon, Oct 29, 2012 at 4:47 AM, Daniel Holth <dholth at gmail.com
>> <mailto:dholth at gmail.com>> wrote:
>>
>>     I think Metadata 1.3 is done. Who would like to czar?
>>
>> (Apologies for the belated reply, it's been a busy few weeks)
>>
>> I'm happy to be BDFL delegate for these. I'd like to see PEP 425 updated
>> with some additional rationale based on Ronald's comments later in this
>> thread, though.
> 
> For the record, I'm still -1 on PEP 427, because of the signature issues.
> 
> The FAQ in the PEP is incorrect in claiming PGP or X.509 cannot
> readily be used to verify the integrity of an archive - the whole
> point of these technologies is to do exactly that.
> 
> The FAQ is entirely silent on why it is not using a more standard
> signature algorithm such as ECDSA. It explains why it uses Ed25519,
> but ignores that the very same rationale would apply to ECDSA as well;
> plus that would be one of the standard JWS algorithms.
> 
> In addition, the FAQ claims that the format is designed to introduce
> cryptopgraphy that is actually used, yet leaves the issue of key
> distribution alone (except that pointing out that you can put them
> into requires.txt - a file that doesn't seem to be specified anywhere).

I agree with Martin. If the point is to "to protect against cryptography
that is not used", then not using the de-facto standard in signing
open source distribution files, which today is PGP/GPG, misses that
point :-)

Note that signing such distribution files can be handled outside
of the wheel format PEP. It just way to complex and out of scope
for the wheel format itself. Also note that PGP/GPG and the other
signing tools work well on any distribution file. There's really no
need to build these into the format itself.

It's a good idea to check integrity, but that can be done using
hashes.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 13 2012)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From p.f.moore at gmail.com  Tue Nov 13 11:39:52 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 13 Nov 2012 10:39:52 +0000
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs
 425, 426, 427
In-Reply-To: <50A2206A.40804@egenix.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
Message-ID: <CACac1F_ja5ZFCegEiGeXH2=np06_v0uzoX+hZnsFcAfP9WVrXQ@mail.gmail.com>

On 13 November 2012 10:26, M.-A. Lemburg <mal at egenix.com> wrote:

> I agree with Martin. If the point is to "to protect against cryptography
> that is not used", then not using the de-facto standard in signing
> open source distribution files, which today is PGP/GPG, misses that
> point :-)
>

I agree as well. For me, the main reason for cryptography not being used is
key distribution. Sure, I have a signed file, but without a key what's the
point? And if I'm creating a file, why sign it if I don't know how to
securely publish my key? So inventing a new signing infrastructure without
a key distribution process doesn't encourage me to use crypto at all...


> It's a good idea to check integrity, but that can be done using
> hashes.
>

+1 hashing is fine, and I don't have any problem with the hashing aspects
of the PEP.

Maybe the signing aspects could be deferred to a subsequent PEP, to be
thrashed out separately? I know Daniel has a strong interest in the signing
aspect, so I'm reluctant to suggest just dropping it, but I'd rather it not
be a showstopper for the rest of the current proposal.

Paul.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121113/c3fbb4db/attachment.html>

From kristjan at ccpgames.com  Tue Nov 13 14:41:22 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 13 Nov 2012 13:41:22 +0000
Subject: [Python-Dev] externals?
Message-ID: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>

This may be a silly question, but haven't the python externals been moved to HG yet?
I usually work on cpython without bothering with the externals, but I found today that I needed them.  On Windows this is a bit of a bother.  And I've thrown away all my SVN stuff...
K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121113/30868ba3/attachment.html>

From dholth at gmail.com  Tue Nov 13 16:00:26 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 13 Nov 2012 10:00:26 -0500
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs
 425, 426, 427
In-Reply-To: <50A2206A.40804@egenix.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
Message-ID: <CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>

On Tue, Nov 13, 2012 at 5:26 AM, M.-A. Lemburg <mal at egenix.com> wrote:

> On 13.11.2012 10:51, "Martin v. L?wis" wrote:
> > Am 13.11.12 03:04, schrieb Nick Coghlan:
> >> On Mon, Oct 29, 2012 at 4:47 AM, Daniel Holth <dholth at gmail.com
> >> <mailto:dholth at gmail.com>> wrote:
> >>
> >>     I think Metadata 1.3 is done. Who would like to czar?
> >>
> >> (Apologies for the belated reply, it's been a busy few weeks)
> >>
> >> I'm happy to be BDFL delegate for these. I'd like to see PEP 425 updated
> >> with some additional rationale based on Ronald's comments later in this
> >> thread, though.
> >
> > For the record, I'm still -1 on PEP 427, because of the signature issues.
> >
> > The FAQ in the PEP is incorrect in claiming PGP or X.509 cannot
> > readily be used to verify the integrity of an archive - the whole
> > point of these technologies is to do exactly that.
> >
> > The FAQ is entirely silent on why it is not using a more standard
> > signature algorithm such as ECDSA. It explains why it uses Ed25519,
> > but ignores that the very same rationale would apply to ECDSA as well;
> > plus that would be one of the standard JWS algorithms.
> >
> > In addition, the FAQ claims that the format is designed to introduce
> > cryptopgraphy that is actually used, yet leaves the issue of key
> > distribution alone (except that pointing out that you can put them
> > into requires.txt - a file that doesn't seem to be specified anywhere).
>
> I agree with Martin. If the point is to "to protect against cryptography
> that is not used", then not using the de-facto standard in signing
> open source distribution files, which today is PGP/GPG, misses that
> point :-)
>
> Note that signing such distribution files can be handled outside
> of the wheel format PEP. It just way to complex and out of scope
> for the wheel format itself. Also note that PGP/GPG and the other
> signing tools work well on any distribution file. There's really no
> need to build these into the format itself.
>
> It's a good idea to check integrity, but that can be done using
> hashes.


PGP-on-pypi is the very definition of cryptography that isn't used.

I'm willing to go ahead and move any mention of signing algorithms into a
separate PEP, leaving only the basic manifest hash vs. file contents
verification under the auspices of this PEP.

Is the rest of the wheel specification, explaining how packages are
actually produced and installed, clear?


I want to remove distutils from the standard library. If that happens then
we might want a secure way to install it from pypi. One way would be to
include the public key used to sign distutils in Python's own
signature-verifying bootstrap wheel installer, never mind whether it used
ECDSA or RSA or Ed25519. Do you have a better idea? TUF?
https://www.updateframework.com/wiki/SecuringPythonPackageManagement
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121113/9b99dd27/attachment.html>

From benjamin at python.org  Tue Nov 13 16:04:17 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 13 Nov 2012 10:04:17 -0500
Subject: [Python-Dev] externals?
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>

Their still in svn as far I know.

2012/11/13 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> This may be a silly question, but haven?t the python externals been moved to
> HG yet?
>
> I usually work on cpython without bothering with the externals, but I found
> today that I needed them.  On Windows this is a bit of a bother.  And I?ve
> thrown away all my SVN stuff...
>
> K
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/benjamin%40python.org
>



-- 
Regards,
Benjamin

From dirkjan at ochtman.nl  Tue Nov 13 16:04:50 2012
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Tue, 13 Nov 2012 16:04:50 +0100
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs
 425, 426, 427
In-Reply-To: <CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
	<CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
Message-ID: <CAKmKYaBS+xKO+Ht+jYPRe0HRJHjw3D0iO_SjGhRtdfc5D5YdLA@mail.gmail.com>

On Tue, Nov 13, 2012 at 4:00 PM, Daniel Holth <dholth at gmail.com> wrote:
> I'm willing to go ahead and move any mention of signing algorithms into a
> separate PEP, leaving only the basic manifest hash vs. file contents
> verification under the auspices of this PEP.

>From the discussion so far, that seems like the smart thing to do.

Cheers,

Dirkjan

From benjamin at python.org  Tue Nov 13 16:05:23 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 13 Nov 2012 10:05:23 -0500
Subject: [Python-Dev] externals?
In-Reply-To: <CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
Message-ID: <CAPZV6o-dKAVM=T2KrActxhiwAaJ+eeZgVo1hd1aOGn4j_FzFGA@mail.gmail.com>

2012/11/13 Benjamin Peterson <benjamin at python.org>:
> Their still in svn as far I know.

s/Their/They're


-- 
Regards,
Benjamin

From ronaldoussoren at mac.com  Tue Nov 13 16:10:30 2012
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 13 Nov 2012 16:10:30 +0100
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs
 425, 426, 427
In-Reply-To: <CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
	<CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
Message-ID: <55555795-0D06-4358-B4CE-E6E5DBBCA296@mac.com>


On 13 Nov, 2012, at 16:00, Daniel Holth <dholth at gmail.com> wrote:
> 
> I want to remove distutils from the standard library.

Why? Distutils may not be perfect, but is usable for basic packages. It could even be enhanced to support these peps and be even more useable, although patches for that would run into the self-imposed freeze of distutils development.

Ronald

From dholth at gmail.com  Tue Nov 13 17:35:13 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 13 Nov 2012 11:35:13 -0500
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs425, 426,
 427
In-Reply-To: <44174.1352824265@cavallinux.eu>
References: <44174.1352824265@cavallinux.eu>
Message-ID: <CAG8k2+6i6WyBxDY5GWaBCHH0507-6jByZ775u5dRDZiMU_s1Cg@mail.gmail.com>

Setuptools! You would avoid 75% of pypi. It is nonsense to pretend that
setuptools is not a significant packaging innovation. Its main flaw is that
it is based on distutils, a non-extensible design. distutils2 is a lot of
setuptools and distutils code with the plug-ability taken out.

Perhaps I should say that I would like distutils to become as relevant to
packaging as the cgi module is to web development. It is not a short-term
goal.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121113/fbacf1bd/attachment.html>

From dholth at gmail.com  Tue Nov 13 17:37:13 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 13 Nov 2012 11:37:13 -0500
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs
 425, 426, 427
In-Reply-To: <CAKmKYaBS+xKO+Ht+jYPRe0HRJHjw3D0iO_SjGhRtdfc5D5YdLA@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
	<CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
	<CAKmKYaBS+xKO+Ht+jYPRe0HRJHjw3D0iO_SjGhRtdfc5D5YdLA@mail.gmail.com>
Message-ID: <CAG8k2+63xGz-p=3MOt=Gj8jT14_=raeR0S7E+L_Zp5hb_5YuYA@mail.gmail.com>

The signatures section is now just:

+If JSON web signatures are used, one or more JSON Web Signature JSON
+Serialization (JWS-JS) signatures may be stored in a file RECORD.jws
+adjacent to RECORD.  JWS is used to sign RECORD by including the SHA-256
+hash of RECORD as the JWS payload::

     { "hash": "sha256=ADD-r2urObZHcxBW3Cr-vDCu5RJwT4CaRTHiFmbcIYY" }

+If RECORD.p7s is used, it must contain a PKCS#7 format signature of
+RECORD.
+
+A wheel installer may assume that the signature has already been checked
+against RECORD, and only must verify the hashes in RECORD against the
+extracted file contents.

FAQ

+Why does wheel include attached signatures?
+    Attached signatures are more convenient than detached signatures
+    because they travel with the archive.  Since only the individual files
+    are signed, the archive can be recompressed without invalidating
+    the signature, or individual files can be verified without having
+    to download the whole archive.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121113/f91629d1/attachment.html>

From a.cavallo at cavallinux.eu  Tue Nov 13 17:31:05 2012
From: a.cavallo at cavallinux.eu (a.cavallo at cavallinux.eu)
Date: Tue, 13 Nov 2012 17:31:05 +0100
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs425,
	426, 427
Message-ID: <44174.1352824265@cavallinux.eu>


+1 
Distutils is good enough: it could be better but for what is required
(essentially copying files and creating packages installers) is fine. The only
corner case is an absolute pain in the neck is in the cross compile scenario.

Currently I don't have *any* need for "auto" tools (setuptools and descendants).
As an example I always advice to avoid like a plague anything that depends on
setuptools... with all the due respect I think is the poor's developer attempt to
play the sys admin game.

Even virtual env is a poor work around on the python interpreter not being
relocatable (as in a portable app fashion).

thanks


On Tue 13/11/12 16:10, "Ronald Oussoren" ronaldoussoren at mac.com wrote:
> 
> On 13 Nov, 2012, at 16:00, Daniel Holth <dholth at gmail
> .com> wrote:
> 
> > I want to remove distutils from the standard
> library.
> Why? Distutils may not be perfect, but is usable for basic packages. It
> could even be enhanced to support these peps and be even more useable,
> although patches for that would run into the self-imposed freeze of
> distutils development.
> Ronald
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
> 
> 



From a.cavallo at cavallinux.eu  Tue Nov 13 17:43:22 2012
From: a.cavallo at cavallinux.eu (a.cavallo at cavallinux.eu)
Date: Tue, 13 Nov 2012 17:43:22 +0100
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs425,
	426, 427
Message-ID: <29996.1352825002@cavallinux.eu>


I'll give you that number(?) but ...

mercurial, docutils, jinjia2 pygments, sphinx, lxml, nose, cherrypy, django, pyqt ...
all they don't need/use setuptools: that 25% left is quite an interesting field
to play in.

If setuptools was "significant packaging innovation" do you think people wouldn't
have embraced already?
Allow me to call *that* a nonsense.

thanks 

ps. my experience is on the field, so please give me the credit of many years of
experience if I'm say I'm not that keen on "auto" tools: in this kiss rules.


On Tue 13/11/12 17:35, "Daniel Holth" dholth at gmail.com wrote:
> Setuptools! You would avoid 75% of pypi. It is nonsense to pretend
> that setuptools is not a significant packaging innovation. Its main
> flaw is that it is based on distutils, a non-extensible design.
> distutils2 is a lot of setuptools and distutils code with the
> plug-ability taken out.
> 
> Perhaps I should say that I would like distutils to become as
> relevant to packaging as the cgi module is to web development. It is
> not a short-term goal.
> 
> 



From fijall at gmail.com  Tue Nov 13 17:45:52 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 13 Nov 2012 17:45:52 +0100
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs425, 426,
 427
In-Reply-To: <44174.1352824265@cavallinux.eu>
References: <44174.1352824265@cavallinux.eu>
Message-ID: <CAK5idxQvY9WJk3GN6Bzyp=Y+UHK9ataG2R0cK_UZtiuho0QWmw@mail.gmail.com>

On Tue, Nov 13, 2012 at 5:31 PM,  <a.cavallo at cavallinux.eu> wrote:
>
> +1
> Distutils is good enough: it could be better but for what is required
> (essentially copying files and creating packages installers) is fine. The only
> corner case is an absolute pain in the neck is in the cross compile scenario.

I think you should qualify all such statements like "distutils is good
enough for *me*" and ideally also describe why exactly it's good
enough for you. Such blank statement does not bring anything to the
discussion (for example I could reply with "distutils is clearly not
good enough" and we're both right and we did not get anywhere).

For example distutils is absolutely *untestable* which makes it very
far from good enough for me.

Cheers,
fijal

From donald.stufft at gmail.com  Tue Nov 13 17:49:45 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Tue, 13 Nov 2012 11:49:45 -0500
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs425, 426,
 427
In-Reply-To: <44174.1352824265@cavallinux.eu>
References: <44174.1352824265@cavallinux.eu>
Message-ID: <046C6552C46E4A8B907B9D5A1CC97EBD@gmail.com>

Distutils is not good enough for the vast majority of people. Even for what it 
does, it does not do it well. It is a library that is user hostile and buggy. It was
a fine first revision of packaging, but the Python community needs something
better.


On Tuesday, November 13, 2012 at 11:31 AM, a.cavallo at cavallinux.eu wrote:

> 
> +1 
> Distutils is good enough: it could be better but for what is required
> (essentially copying files and creating packages installers) is fine. The only
> corner case is an absolute pain in the neck is in the cross compile scenario.
> 
> Currently I don't have *any* need for "auto" tools (setuptools and descendants).
> As an example I always advice to avoid like a plague anything that depends on
> setuptools... with all the due respect I think is the poor's developer attempt to
> play the sys admin game.
> 
> Even virtual env is a poor work around on the python interpreter not being
> relocatable (as in a portable app fashion).
> 
> thanks
> 
> 
> On Tue 13/11/12 16:10, "Ronald Oussoren" ronaldoussoren at mac.com (mailto:ronaldoussoren at mac.com) wrote:
> > 
> > On 13 Nov, 2012, at 16:00, Daniel Holth <dholth at gmail
> > .com> wrote:
> > 
> > > I want to remove distutils from the standard
> > library.
> > Why? Distutils may not be perfect, but is usable for basic packages. It
> > could even be enhanced to support these peps and be even more useable,
> > although patches for that would run into the self-imposed freeze of
> > distutils development.
> > Ronald
> > _______________________________________________
> > Distutils-SIG maillist - Distutils-SIG at python.org (mailto:Distutils-SIG at python.org)
> > http://mail.python.org/mailman/listinfo/distutils-sig
> > 
> 
> 
> 
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG at python.org (mailto:Distutils-SIG at python.org)
> http://mail.python.org/mailman/listinfo/distutils-sig
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121113/cc37ae5e/attachment-0001.html>

From martin at v.loewis.de  Tue Nov 13 18:07:05 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Nov 2012 18:07:05 +0100
Subject: [Python-Dev] [Catalog-sig]  accept the wheel PEPs 425, 426, 427
In-Reply-To: <50A2206A.40804@egenix.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
Message-ID: <50A27E39.8080809@v.loewis.de>

Am 13.11.12 11:26, schrieb M.-A. Lemburg:
> Note that signing such distribution files can be handled outside
> of the wheel format PEP. It just way to complex and out of scope
> for the wheel format itself. Also note that PGP/GPG and the other
> signing tools work well on any distribution file. There's really no
> need to build these into the format itself.

And even if the desire is to include the signature in the distribution
(as is common for code-signing - you want the signature in the 
executable, the jar file, etc), then it's still possible to include,
say, a PGP signature inside the file, which may well be a signature
to the manuscript, preserving the rest of the signing procedure.

Regards,
Martin


From martin at v.loewis.de  Tue Nov 13 18:23:35 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Nov 2012 18:23:35 +0100
Subject: [Python-Dev] [Catalog-sig] [Distutils] accept the wheel PEPs
 425, 426, 427
In-Reply-To: <CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
	<CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
Message-ID: <50A28217.5060301@v.loewis.de>

> I want to remove distutils from the standard library. If that happens
> then we might want a secure way to install it from pypi. One way would
> be to include the public key used to sign distutils in Python's own
> signature-verifying bootstrap wheel installer, never mind whether it
> used ECDSA or RSA or Ed25519. Do you have a better idea? TUF?
> https://www.updateframework.com/wiki/SecuringPythonPackageManagement

It depends on the threat model - whose definition is key to any security
discussion.

I'd say that providing the CA certificate of the CA, and to use https
for downloading, should be enough.

Alternatively, if the threat is that somebody may have hacked PyPI,
then hard-code the hash (SHA-3 if you are paranoid) in the Python
distribution, and rely on downloading a specific version from PyPI.

OTOH, I'm -1 on removing the code from Python in a way that it may
come back through downloading. Instead, it is much easier to keep
it included.

Regards,
Martin



From martin at v.loewis.de  Tue Nov 13 18:32:11 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Nov 2012 18:32:11 +0100
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs425, 426,
 427
In-Reply-To: <CAK5idxQvY9WJk3GN6Bzyp=Y+UHK9ataG2R0cK_UZtiuho0QWmw@mail.gmail.com>
References: <44174.1352824265@cavallinux.eu>
	<CAK5idxQvY9WJk3GN6Bzyp=Y+UHK9ataG2R0cK_UZtiuho0QWmw@mail.gmail.com>
Message-ID: <50A2841B.5020006@v.loewis.de>

Am 13.11.12 17:45, schrieb Maciej Fijalkowski:
> For example distutils is absolutely *untestable* which makes it very
> far from good enough for me.

I never had issues with testing distutils applications. I use
"python setup.py sdist", then unpack the resulting source package,
install it, and run its test - if that passes, the packaging was
successful.

Regards,
Martin



From dholth at gmail.com  Tue Nov 13 18:41:22 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 13 Nov 2012 12:41:22 -0500
Subject: [Python-Dev] [Catalog-sig] [Distutils] accept the wheel PEPs
 425, 426, 427
In-Reply-To: <50A28217.5060301@v.loewis.de>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
	<CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
	<50A28217.5060301@v.loewis.de>
Message-ID: <CAG8k2+4zSEPPjymYcOs9WGVEdRLfL66AJew-MQ9rppONzMWi=w@mail.gmail.com>

On Tue, Nov 13, 2012 at 12:23 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> I want to remove distutils from the standard library. If that happens
>> then we might want a secure way to install it from pypi. One way would
>> be to include the public key used to sign distutils in Python's own
>> signature-verifying bootstrap wheel installer, never mind whether it
>> used ECDSA or RSA or Ed25519. Do you have a better idea? TUF?
>> https://www.updateframework.**com/wiki/**SecuringPythonPackageManagemen**
>> t <https://www.updateframework.com/wiki/SecuringPythonPackageManagement>
>>
>
> It depends on the threat model - whose definition is key to any security
> discussion.
>
> I'd say that providing the CA certificate of the CA, and to use https
> for downloading, should be enough.
>
> Alternatively, if the threat is that somebody may have hacked PyPI,
> then hard-code the hash (SHA-3 if you are paranoid) in the Python
> distribution, and rely on downloading a specific version from PyPI.
>
> OTOH, I'm -1 on removing the code from Python in a way that it may
> come back through downloading. Instead, it is much easier to keep
> it included.
>

It is a long term goal. It would be more practical to discourage distutils
and encourage users to look elsewhere (Bento) for a beautifully designed
build system.

The short term goal is just to standardize a way to install packages
without having a build system, which is what wheel is for apart from the
practical goal of simply installing lxml in a reasonable amount of time.

I think Metadata 1.2 (PEP 426) is ready to be accepted. The compatibility
tags (PEP 425) need some additional text in the discussion section, any
contributors for https://bitbucket.org/dholth/pep425tags/ ? Wheel (PEP 427)
might mention that version 1.0 of the spec is only concerned with
losslessly representing existing (setuptools & distutils) packages without
trying to add too many new features apart from a standardized .egg
substitute.

distutils itself is not testable.

Daniel Holth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121113/d49b03e7/attachment.html>

From barry at python.org  Tue Nov 13 21:43:30 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 13 Nov 2012 15:43:30 -0500
Subject: [Python-Dev] [Distutils]  accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+4zSEPPjymYcOs9WGVEdRLfL66AJew-MQ9rppONzMWi=w@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
	<CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
	<50A28217.5060301@v.loewis.de>
	<CAG8k2+4zSEPPjymYcOs9WGVEdRLfL66AJew-MQ9rppONzMWi=w@mail.gmail.com>
Message-ID: <20121113154330.0b1d0e2c@resist.wooz.org>

I'm just beginning to review these PEPs and the threads, with an OS vendor
packager's eye.  Let me start with one small suggestion for PEP 425.

From the FAQ:

Q. Who will maintain the registry of abbreviated implementations?

A. New two-letter abbreviations can be requested on the python-dev mailing
   list. As a rule of thumb, abbreviations are reserved for the current 4 most
   prominent implementations.

I think the PEP should explicitly say that it will be the definitive keeper of
the abbreviations.  The request can be made on python-dev, and once approved,
PEP 425 will be updated with the new abbreviation.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121113/43123fcd/attachment.pgp>

From solipsis at pitrou.net  Tue Nov 13 21:43:32 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 13 Nov 2012 21:43:32 +0100
Subject: [Python-Dev] cpython (3.2): add gc_collects to weakref tests
References: <3Y1L3H4Qv8zRNf@mail.python.org>
Message-ID: <20121113214332.575108be@pitrou.net>

On Tue, 13 Nov 2012 21:26:51 +0100 (CET)
philip.jenvey <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/a580cf4ab940
> changeset:   80418:a580cf4ab940
> branch:      3.2
> parent:      80397:8a28c974f903
> user:        Philip Jenvey <pjenvey at underboss.org>
> date:        Tue Nov 13 12:26:31 2012 -0800
> summary:
>   add gc_collects to weakref tests
> 
> files:
>   Lib/test/test_exceptions.py |  3 +++
>   1 files changed, 3 insertions(+), 0 deletions(-)

While this is necessary for non-refcounted implementations, it does
relax the requirement quite a bit for CPython (absence of reference
cycles can be a significant feature). I think perhaps gc_collect()
should be called only for non-CPython implementations.

Regards

Antoine.



From ronaldoussoren at mac.com  Wed Nov 14 08:23:43 2012
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 14 Nov 2012 08:23:43 +0100
Subject: [Python-Dev] [Catalog-sig] [Distutils] accept the wheel
 PEPs	425, 426, 427
In-Reply-To: <20121113172114.40b0ca4a@pitrou.net>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
	<CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
	<55555795-0D06-4358-B4CE-E6E5DBBCA296@mac.com>
	<20121113172114.40b0ca4a@pitrou.net>
Message-ID: <2A6F78E2-65CF-4925-8DE0-09EA570A7239@mac.com>


On 13 Nov, 2012, at 17:21, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le Tue, 13 Nov 2012 16:10:30 +0100,
> Ronald Oussoren <ronaldoussoren at mac.com> a ?crit :
>> 
>> On 13 Nov, 2012, at 16:00, Daniel Holth <dholth at gmail.com> wrote:
>>> 
>>> I want to remove distutils from the standard library.
>> 
>> Why? Distutils may not be perfect, but is usable for basic packages.
>> It could even be enhanced to support these peps and be even more
>> useable, although patches for that would run into the self-imposed
>> freeze of distutils development.
> 
> It wouldn't be totally unreasonable to start a project named
> "distutils2" to build the next-generation distutils library.
> 
> Oops :-)

Or carefully enhance distutils itself.  Rewriting distutils in one go seems
to be too ambitious with the limited resources available to do so. 

Ronald


From ronaldoussoren at mac.com  Wed Nov 14 08:32:34 2012
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 14 Nov 2012 08:32:34 +0100
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs425, 426,
 427
In-Reply-To: <50A2841B.5020006@v.loewis.de>
References: <44174.1352824265@cavallinux.eu>
	<CAK5idxQvY9WJk3GN6Bzyp=Y+UHK9ataG2R0cK_UZtiuho0QWmw@mail.gmail.com>
	<50A2841B.5020006@v.loewis.de>
Message-ID: <EE91BA7A-A77A-41B9-9554-F1E68814EB8F@mac.com>


On 13 Nov, 2012, at 18:32, Martin v. L?wis <martin at v.loewis.de> wrote:

> Am 13.11.12 17:45, schrieb Maciej Fijalkowski:
>> For example distutils is absolutely *untestable* which makes it very
>> far from good enough for me.
> 
> I never had issues with testing distutils applications. I use
> "python setup.py sdist", then unpack the resulting source package,
> install it, and run its test - if that passes, the packaging was
> successful.

I'm pretty sure he means testing distutils itself.  

I'm not convinced that distutils is untestable. A major problem with
distutils is that its API is barely documented, which effectly makes
all of it public API (simular to asyncore).  IIRC that's the main reason
why distutils is frozen right now: with a lot of changes to distutils people
started to complain that this could break there there distutils scripts.

The lack of a specification also makes testing harder, as it is unclear
what should be tested.

Ronald


From chris at simplistix.co.uk  Wed Nov 14 10:12:05 2012
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 14 Nov 2012 09:12:05 +0000
Subject: [Python-Dev] performance of {} versus dict()
Message-ID: <50A36065.9010704@simplistix.co.uk>

Hi All,

A colleague pointed me at Doug's excellent article here:

http://www.doughellmann.com/articles/misc/dict-performance/index.html

...which made me a little sad, I suspect I'm not the only one who finds:

a_dict = dict(
     x = 1,
     y = 2,
     z = 3,
     ...
     )

...easier to read than:

a_dict = {
     'x':1,
     'y':2,
     'z':3,
     ...
     }

What can we do to speed up the former case?

Here's comparison for different versions of CPython:

$ python2.5 -m timeit -n 1000000 -r 5 -v 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
raw times: 2.96 2.49 2.47 2.42 2.42
1000000 loops, best of 5: 2.42 usec per loop
$ python2.5 -m timeit -n 1000000 -r 5 -v 
"{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}"
raw times: 1.69 1.71 1.68 1.68 1.68
1000000 loops, best of 5: 1.68 usec per loop

$ python2.6 -m timeit -n 1000000 -r 5 -v 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
raw times: 2.41 2.41 2.42 2.44 2.41
1000000 loops, best of 5: 2.41 usec per loop
$ python2.6 -m timeit -n 1000000 -r 5 -v 
"{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}"
raw times: 1.51 1.51 1.52 1.51 1.51
1000000 loops, best of 5: 1.51 usec per loop

$ python2.7 -m timeit -n 1000000 -r 5 -v 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
raw times: 2.32 2.31 2.31 2.32 2.31
1000000 loops, best of 5: 2.31 usec per loop
$ python2.7 -m timeit -n 1000000 -r 5 -v 
"{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}"
raw times: 1.49 1.49 1.77 1.76 1.55
1000000 loops, best of 5: 1.49 usec per loop

So, not the 6 times headline figure that Doug quotes, but certainly a 
difference. Can someone with Python 3 handy compare there too?

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ulrich.eckhardt at dominolaser.com  Wed Nov 14 10:30:27 2012
From: ulrich.eckhardt at dominolaser.com (Ulrich Eckhardt)
Date: Wed, 14 Nov 2012 10:30:27 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A36065.9010704@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
Message-ID: <50A364B3.5070807@dominolaser.com>

Am 14.11.2012 10:12, schrieb Chris Withers:
> Can someone with Python 3 handy compare there too?

C:\Python27\python -m timeit -n 1000000 -r 5 -v
    "dict(a=1, b=2, c=3, d=4, e=5, f=6, g=7)"
raw times: 0.918 0.924 0.922 0.928 0.926
1000000 loops, best of 5: 0.918 usec per loop

C:\Python27\python -m timeit -n 1000000 -r 5 -v
    "{'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7}"
raw times: 0.48 0.49 0.482 0.496 0.497
1000000 loops, best of 5: 0.48 usec per loop

C:\Python32\python -m timeit -n 1000000 -r 5 -v
    "dict(a=1, b=2, c=3, d=4, e=5, f=6, g=7)"
raw times: 0.898 0.891 0.892 0.899 0.891
1000000 loops, best of 5: 0.891 usec per loop

C:\Python32\python -m timeit -n 1000000 -r 5 -v
    "{'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7}"
raw times: 0.444 0.463 0.461 0.464 0.461
1000000 loops, best of 5: 0.444 usec per loop

C:\Python32_64\python -m timeit -n 1000000 -r 5 -v
    "dict(a=1, b=2, c=3, d=4, e=5, f=6, g=7)"
raw times: 0.908 0.923 0.927 0.912 0.923
1000000 loops, best of 5: 0.908 usec per loop

C:\Python32_64\python -m timeit -n 1000000 -r 5 -v
    "{'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7}"
raw times: 0.484 0.446 0.501 0.45 0.442
1000000 loops, best of 5: 0.442 usec per loop

C:\Python33_64\python -m timeit -n 1000000 -r 5 -v
    "dict(a=1, b=2, c=3, d=4, e=5, f=6, g=7)"
raw times: 1.02 1.07 1.03 1.11 1.07
1000000 loops, best of 5: 1.02 usec per loop

C:\Python33_64\python -m timeit -n 1000000 -r 5 -v
    "{'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7}"
raw times: 0.444 0.449 0.455 0.452 0.46
1000000 loops, best of 5: 0.444 usec per loop


Tested on Win7/64 bit. Python 2.7 is the 32-bit version, 3.2 is 
installed as 32-bit and 64-bit versions and 3.3 only as 64-bit version. 
In any case, the difference is even a bit stronger than what you 
experience and it seems that all versions perform roughly similar.

Uli



**************************************************************************************
Domino Laser GmbH, Fangdieckstra???e 75a, 22547 Hamburg, Deutschland
Gesch???ftsf???hrer: Hans Robert Dapprich, Amtsgericht Hamburg HR B62 932
**************************************************************************************
Visit our website at http://www.dominolaser.com
**************************************************************************************
Diese E-Mail einschlie???lich s???mtlicher Anh???nge ist nur f???r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf???nger sein sollten. Die E-Mail ist in diesem Fall zu l???schen und darf weder gelesen, weitergeleitet, ver???ffentlicht oder anderweitig benutzt werden.
E-Mails k???nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ???nderungen enthalten. Domino Laser GmbH ist f???r diese Folgen nicht verantwortlich.
**************************************************************************************


From chris at simplistix.co.uk  Wed Nov 14 11:00:59 2012
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 14 Nov 2012 10:00:59 +0000
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CADJO0eLD0PXLFSK37w5xqe9bN-OuiM0kega_Z_yhWdxBJnb2NQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CADJO0eLD0PXLFSK37w5xqe9bN-OuiM0kega_Z_yhWdxBJnb2NQ@mail.gmail.com>
Message-ID: <50A36BDB.3010202@simplistix.co.uk>

On 14/11/2012 09:58, Merlijn van Deen wrote:
> On 14 November 2012 10:12, Chris Withers <chris at simplistix.co.uk> wrote:
>> ...which made me a little sad
>
> Why did it make you sad? dict() takes 0.2?s, {} takes 0.04?s. In other
> words: you can run dict() _five million_ times per second, and {}
> twenty-five million times per second. That is 'a lot' and 'a lot'. It
> also means you are unlikely to notice the difference in real-world
> code. Just use the one you feel is clearer in the situation, and don't
> worry about micro-optimalization.

I'm inclined to agree, but it makes me sad for two reasons:

- it's something that people get hung up on, for better or worse. (if it 
wasn't, Doug wouldn't have written his article)

- it can make a difference, for example setting up a dict with many keys 
at the core of a type loop.

Without looking at implementation, they should logically perform the same...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From martin at v.loewis.de  Wed Nov 14 11:11:39 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Wed, 14 Nov 2012 11:11:39 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A36065.9010704@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
Message-ID: <20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>


Zitat von Chris Withers <chris at simplistix.co.uk>:

> a_dict = dict(
>     x = 1,
>     y = 2,
>     z = 3,
>     ...
>     )

> What can we do to speed up the former case?

It should be possible to special-case it. Rather than creating
a new dictionary from scratch, one could try to have the new dictionary
the same size as the original one, and copy all entries.

I also wonder whether the PyArg_ValidateKeywordArguments call is really
necessary: if this is not a proper keyword dictionary, dict creation
could still proceed in a reasonable way.

I don't know how much this would gain, though. You still have to
create two dictionary objects. For a better speedup, try

def xdict(**kwds):
   return kwds

(possibly written in C for even more speed)

Regards,
Martin



From solipsis at pitrou.net  Wed Nov 14 11:48:09 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 14 Nov 2012 11:48:09 +0100
Subject: [Python-Dev] performance of {} versus dict()
References: <50A36065.9010704@simplistix.co.uk>
	<CADJO0eLD0PXLFSK37w5xqe9bN-OuiM0kega_Z_yhWdxBJnb2NQ@mail.gmail.com>
	<50A36BDB.3010202@simplistix.co.uk>
Message-ID: <20121114114809.71b76936@pitrou.net>

Le Wed, 14 Nov 2012 10:00:59 +0000,
Chris Withers <chris at simplistix.co.uk> a ?crit :
> On 14/11/2012 09:58, Merlijn van Deen wrote:
> > On 14 November 2012 10:12, Chris Withers <chris at simplistix.co.uk>
> > wrote:
> >> ...which made me a little sad
> >
> > Why did it make you sad? dict() takes 0.2?s, {} takes 0.04?s. In
> > other words: you can run dict() _five million_ times per second,
> > and {} twenty-five million times per second. That is 'a lot' and 'a
> > lot'. It also means you are unlikely to notice the difference in
> > real-world code. Just use the one you feel is clearer in the
> > situation, and don't worry about micro-optimalization.
> 
> I'm inclined to agree, but it makes me sad for two reasons:
> 
> - it's something that people get hung up on, for better or worse. (if
> it wasn't, Doug wouldn't have written his article)
> 
> - it can make a difference, for example setting up a dict with many
> keys at the core of a type loop.

Well, please post examples of *real-world* use cases where it makes a
difference.
Otherwise, you're asking us to add hacks to the implementation just to
make you feel good, which is quite unacceptable.

Regards

Antoine.



From pete.alex.harris at gmail.com  Wed Nov 14 12:00:46 2012
From: pete.alex.harris at gmail.com (Peter Harris)
Date: Wed, 14 Nov 2012 11:00:46 +0000
Subject: [Python-Dev] Python-Dev Digest, Vol 112, Issue 23
In-Reply-To: <mailman.14373.1352887265.27097.python-dev@python.org>
References: <mailman.14373.1352887265.27097.python-dev@python.org>
Message-ID: <CANYapPP9QS4fBMbh9vyc5z_k+sD8+p8vdBmXJQpB9-w-aHyRvg@mail.gmail.com>

Chris Withers wrote:

> On 14/11/2012 09:58, Merlijn van Deen wrote:
> > On 14 November 2012 10:12, Chris Withers <chris at simplistix.co.uk> wrote:
> >> ...which made me a little sad
> >
> > Why did it make you sad? dict() takes 0.2?s, {} takes 0.04?s. In other
> > words: you can run dict() _five million_ times per second, and {}
> > twenty-five million times per second. That is 'a lot' and 'a lot'. It
> > also means you are unlikely to notice the difference in real-world
> > code. Just use the one you feel is clearer in the situation, and don't
> > worry about micro-optimalization.
>
> I'm inclined to agree, but it makes me sad for two reasons:
>
> - it's something that people get hung up on, for better or worse. (if it
> wasn't, Doug wouldn't have written his article)
>
> - it can make a difference, for example setting up a dict with many keys
> at the core of a type loop.
>
> Without looking at implementation, they should logically perform the
> same...
>
> Well, without looking at the implementation, you could form any opinion you
like about how they should perform. Still you could speculate that dict()
will
require a builtins name lookup, and that the process of passing keyword
arguments
might itself involve constructing a dictionary, so must inherently take at
least
a little longer than compiling a {} literal.

Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121114/e458572c/attachment.html>

From steve at pearwood.info  Wed Nov 14 12:21:03 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 14 Nov 2012 22:21:03 +1100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A36BDB.3010202@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
	<CADJO0eLD0PXLFSK37w5xqe9bN-OuiM0kega_Z_yhWdxBJnb2NQ@mail.gmail.com>
	<50A36BDB.3010202@simplistix.co.uk>
Message-ID: <50A37E9F.2020407@pearwood.info>

On 14/11/12 21:00, Chris Withers wrote:
> On 14/11/2012 09:58, Merlijn van Deen wrote:
>> On 14 November 2012 10:12, Chris Withers <chris at simplistix.co.uk> wrote:
>>> ...which made me a little sad
>>
>> Why did it make you sad? dict() takes 0.2?s, {} takes 0.04?s. In other
>> words: you can run dict() _five million_ times per second, and {}
>> twenty-five million times per second. That is 'a lot' and 'a lot'. It
>> also means you are unlikely to notice the difference in real-world
>> code. Just use the one you feel is clearer in the situation, and don't
>> worry about micro-optimalization.
>
> I'm inclined to agree, but it makes me sad for two reasons:
>
> - it's something that people get hung up on, for better or worse. (if it
>  wasn't, Doug wouldn't have written his article)

People get hung up on all sorts of things. I would hate to think we would
complicate the implementation to pander to pointless micro-optimization.
I'm sure that there are many far more important things than this.


> - it can make a difference, for example setting up a dict with many keys
>at the core of a type loop.

Ah yes, the semi-mythical "tight loop".

I've never come across one of these tight loops that creates a dict with
many keys in a tight loop, and then apparently fails to actually use it
for anything useful. For if it did, surely the actual work done with the
dict is going to outweigh the setup cost for all but the most trivial
applications. I find it hard to get uptight about a small inefficiency
in trivial applications that don't do much.

Show me a non-toy use-case where creating dicts is an actual bottleneck,
and I'll revise my position.


> Without looking at implementation, they should logically perform the same...

I disagree. Calling dict() has to do a name lookup, and then a function
call. That alone is almost 2.5 times as expensive as creating a dict
literal on my machine:

[steve at ando ~]$ python3.3 -m timeit "d = {}"
10000000 loops, best of 3: 0.17 usec per loop
[steve at ando ~]$ python3.3 -m timeit "d = dict()"
1000000 loops, best of 3: 0.416 usec per loop

Then you have the function call itself, which engages the argument parsing
mechanism, which does more work than dict literal syntax. For example, it
checks for duplicate keyword arguments, while dict literals happily accept
duplicate keys.

It's hardly a surprise that dict() is slower than {}.



-- 
Steven

From dholth at gmail.com  Wed Nov 14 13:04:57 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 14 Nov 2012 07:04:57 -0500
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel
	PEPs	425, 426, 427
In-Reply-To: <2A6F78E2-65CF-4925-8DE0-09EA570A7239@mac.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
	<CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
	<55555795-0D06-4358-B4CE-E6E5DBBCA296@mac.com>
	<20121113172114.40b0ca4a@pitrou.net>
	<2A6F78E2-65CF-4925-8DE0-09EA570A7239@mac.com>
Message-ID: <404A3CEE-7F9B-4611-B12C-E45BD780BDC8@gmail.com>

On Nov 14, 2012, at 2:23 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:

> 
> On 13 Nov, 2012, at 17:21, Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
>> Le Tue, 13 Nov 2012 16:10:30 +0100,
>> Ronald Oussoren <ronaldoussoren at mac.com> a ?crit :
>>> 
>>> On 13 Nov, 2012, at 16:00, Daniel Holth <dholth at gmail.com> wrote:
>>>> 
>>>> I want to remove distutils from the standard library.
>>> 
>>> Why? Distutils may not be perfect, but is usable for basic packages.
>>> It could even be enhanced to support these peps and be even more
>>> useable, although patches for that would run into the self-imposed
>>> freeze of distutils development.
>> 
>> It wouldn't be totally unreasonable to start a project named
>> "distutils2" to build the next-generation distutils library.
>> 
>> Oops :-)
> 
> Or carefully enhance distutils itself.  Rewriting distutils in one go seems
> to be too ambitious with the limited resources available to do so. 

That has been tried already (setuptools, distribute, distutils2). Instead, try bento (http://cournape.github.com/Bento/). 

Hilariously everyone I've showed it to is immediately put off by the indentation based syntax (who would use such a thing?) but the project has a few years of effort behind it and is well designed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121114/659c2e4c/attachment.html>

From rosuav at gmail.com  Wed Nov 14 14:18:38 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 15 Nov 2012 00:18:38 +1100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A36065.9010704@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
Message-ID: <CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>

On Wed, Nov 14, 2012 at 8:12 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> I suspect I'm not the only one who finds:
>
> a_dict = dict(
>     x = 1,
>     y = 2,
>     z = 3,
>     ...
>     )
>
> ...easier to read than:
>
> a_dict = {
>     'x':1,
>     'y':2,
>     'z':3,
>     ...
>     }
>
> What can we do to speed up the former case?

Perhaps an alternative question: What can be done to make the latter
less unpalatable? I personally prefer dict literal syntax to a dict
constructor call, but no doubt there are a number of people who feel
as you do. In what way(s) do you find the literal syntax less
readable, and can some simple (and backward-compatible) enhancements
help that?

I've seen criticisms (though I don't recall where) of Python,
comparing it to JavaScript/ECMAScript, that complain of the need to
quote the keys. IMO this is a worthwhile downside, as it allows you to
use variables as the keys, rather than requiring (effectively) literal
strings. But it does make a dict literal that much more "noisy" than
the constructor.

ChrisA

From benjamin at python.org  Wed Nov 14 14:43:18 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 14 Nov 2012 08:43:18 -0500
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
Message-ID: <CAPZV6o90yUeyixQKnGzKvykd9xZ2M2Z3SY5k8bjgJ4teD27-vA@mail.gmail.com>

2012/11/14  <martin at v.loewis.de>:
>
> Zitat von Chris Withers <chris at simplistix.co.uk>:
>
>
>> a_dict = dict(
>>     x = 1,
>>     y = 2,
>>     z = 3,
>>     ...
>>     )
>
>
>> What can we do to speed up the former case?
>
>
> It should be possible to special-case it. Rather than creating
> a new dictionary from scratch, one could try to have the new dictionary
> the same size as the original one, and copy all entries.
>
> I also wonder whether the PyArg_ValidateKeywordArguments call is really
> necessary: if this is not a proper keyword dictionary, dict creation
> could still proceed in a reasonable way.

In the common case PyArg_ValidateKeywordArguments should be a simple check.



-- 
Regards,
Benjamin

From stefan_ml at behnel.de  Wed Nov 14 15:35:37 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 14 Nov 2012 15:35:37 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
Message-ID: <k80a7n$66f$1@ger.gmane.org>

Chris Angelico, 14.11.2012 14:18:
> On Wed, Nov 14, 2012 at 8:12 PM, Chris Withers wrote:
>> I suspect I'm not the only one who finds:
>>
>> a_dict = dict(
>>     x = 1,
>>     y = 2,
>>     z = 3,
>>     ...
>>     )
>>
>> ...easier to read than:
>>
>> a_dict = {
>>     'x':1,
>>     'y':2,
>>     'z':3,
>>     ...
>>     }
>>
>> What can we do to speed up the former case?
> 
> Perhaps an alternative question: What can be done to make the latter
> less unpalatable? I personally prefer dict literal syntax to a dict
> constructor call, but no doubt there are a number of people who feel
> as you do. In what way(s) do you find the literal syntax less
> readable, and can some simple (and backward-compatible) enhancements
> help that?
> 
> I've seen criticisms (though I don't recall where) of Python,
> comparing it to JavaScript/ECMAScript, that complain of the need to
> quote the keys. IMO this is a worthwhile downside, as it allows you to
> use variables as the keys, rather than requiring (effectively) literal
> strings. But it does make a dict literal that much more "noisy" than
> the constructor.

If that bothers you in a specific case, I recommend using the constructor
instead of a literal.

Stefan



From p.f.moore at gmail.com  Wed Nov 14 15:38:11 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 14 Nov 2012 14:38:11 +0000
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs
 425, 426, 427
In-Reply-To: <404A3CEE-7F9B-4611-B12C-E45BD780BDC8@gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
	<CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
	<55555795-0D06-4358-B4CE-E6E5DBBCA296@mac.com>
	<20121113172114.40b0ca4a@pitrou.net>
	<2A6F78E2-65CF-4925-8DE0-09EA570A7239@mac.com>
	<404A3CEE-7F9B-4611-B12C-E45BD780BDC8@gmail.com>
Message-ID: <CACac1F9js9=R8Wk_+4sX+NEj6GVEUU-HbRQ7VfO5iKJcewtd=g@mail.gmail.com>

On 14 November 2012 12:04, Daniel Holth <dholth at gmail.com> wrote:

> That has been tried already (setuptools, distribute, distutils2). Instead,
> try bento (http://cournape.github.com/Bento/).
>
> Hilariously everyone I've showed it to is immediately put off by the
> indentation based syntax (who would use such a thing?) but the project has
> a few years of effort behind it and is well designed.
>
> Ironically, given the thread, it doesn't support the metadata PEPs, so
packages installed with bento aren't visible to pip, etc.
Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121114/927aa213/attachment.html>

From dholth at gmail.com  Wed Nov 14 16:17:23 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 14 Nov 2012 10:17:23 -0500
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs
 425, 426, 427
In-Reply-To: <CACac1F9js9=R8Wk_+4sX+NEj6GVEUU-HbRQ7VfO5iKJcewtd=g@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
	<CADiSq7e7xxyoiZZNtJeH8Vafw5vG+EWDh3SC7htC916FaCrOhw@mail.gmail.com>
	<50A21808.8040909@v.loewis.de> <50A2206A.40804@egenix.com>
	<CAG8k2+5uBpq7FO99QbH4yvt-MLXSAH3uoDfjxTttUHe3CqsdWQ@mail.gmail.com>
	<55555795-0D06-4358-B4CE-E6E5DBBCA296@mac.com>
	<20121113172114.40b0ca4a@pitrou.net>
	<2A6F78E2-65CF-4925-8DE0-09EA570A7239@mac.com>
	<404A3CEE-7F9B-4611-B12C-E45BD780BDC8@gmail.com>
	<CACac1F9js9=R8Wk_+4sX+NEj6GVEUU-HbRQ7VfO5iKJcewtd=g@mail.gmail.com>
Message-ID: <CAG8k2+6iVW2GSnZkuS-r1f_TMPXAUg-LF+=V47kijG5T97p=WA@mail.gmail.com>

Well, you can build eggs with Bento, and I have a patch that allows it to
build wheels, in both cases it will produce pip-compatible metadata. The
Bento author has his own informed opinions about the  way packaging should
work which do not necessarily include the packaging PEPs.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121114/0ea1bae0/attachment.html>

From dreamingforward at gmail.com  Wed Nov 14 17:12:54 2012
From: dreamingforward at gmail.com (Mark Adam)
Date: Wed, 14 Nov 2012 10:12:54 -0600
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A36065.9010704@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
Message-ID: <CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>

On Wed, Nov 14, 2012 at 3:12 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> Hi All,
>
> A colleague pointed me at Doug's excellent article here:
> ...which made me a little sad, I suspect I'm not the only one who finds:
>
> a_dict = dict(
>     x = 1,
>     y = 2,
>     z = 3,
>     ...
>     )
>
> ...easier to read than:
>
> a_dict = {
>     'x':1,
>     'y':2,
>     'z':3,
>     ...
>     }

Hey, it makes me a little sad that dict breaks convention by allowing
the use of unquoted characters (which everywhere else looks like
variable names) just for a silly typing optimization.

mark

From phd at phdru.name  Wed Nov 14 17:20:55 2012
From: phd at phdru.name (Oleg Broytman)
Date: Wed, 14 Nov 2012 20:20:55 +0400
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
Message-ID: <20121114162055.GA18900@iskra.aviel.ru>

On Wed, Nov 14, 2012 at 10:12:54AM -0600, Mark Adam <dreamingforward at gmail.com> wrote:
> On Wed, Nov 14, 2012 at 3:12 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> > Hi All,
> >
> > A colleague pointed me at Doug's excellent article here:
> > ...which made me a little sad, I suspect I'm not the only one who finds:
> >
> > a_dict = dict(
> >     x = 1,
> >     y = 2,
> >     z = 3,
> >     ...
> >     )
> >
> > ...easier to read than:
> >
> > a_dict = {
> >     'x':1,
> >     'y':2,
> >     'z':3,
> >     ...
> >     }
> 
> Hey, it makes me a little sad that dict breaks convention by allowing
> the use of unquoted characters (which everywhere else looks like
> variable names) just for a silly typing optimization.

   It doesn't. It's a call (function call or or a class instantiation)
and it's not dict-specific: function(a=1, b=None)...

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From storchaka at gmail.com  Wed Nov 14 17:23:16 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 14 Nov 2012 18:23:16 +0200
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A36065.9010704@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
Message-ID: <k80ghr$6ic$1@ger.gmane.org>

On 14.11.12 11:12, Chris Withers wrote:
> ....which made me a little sad, I suspect I'm not the only one who finds:
>
> a_dict = dict(
>      x = 1,
>      y = 2,
>      z = 3,
>      ...
>      )
>
> ....easier to read than:
>
> a_dict = {
>      'x':1,
>      'y':2,
>      'z':3,
>      ...
>      }

PEP 8 recommends:

a_dict = dict(
     x=1,
     y=2,
     z=3,
     ...
)

and

a_dict = {
     'x': 1,
     'y': 2,
     'z': 3,
     ...
}



From shibturn at gmail.com  Wed Nov 14 17:42:39 2012
From: shibturn at gmail.com (Richard Oudkerk)
Date: Wed, 14 Nov 2012 16:42:39 +0000
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <k80ghr$6ic$1@ger.gmane.org>
References: <50A36065.9010704@simplistix.co.uk> <k80ghr$6ic$1@ger.gmane.org>
Message-ID: <k80hm2$fs8$1@ger.gmane.org>

On 14/11/2012 4:23pm, Serhiy Storchaka wrote:
> PEP 8 recommends:
>
> a_dict = dict(
>      x=1,
>      y=2,
>      z=3,
>      ...
> )
>
> and
>
> a_dict = {
>      'x': 1,
>      'y': 2,
>      'z': 3,
>      ...
> }

In which section?  I can't see such a recommendation.

--
Richard


From brian at python.org  Wed Nov 14 18:00:47 2012
From: brian at python.org (Brian Curtin)
Date: Wed, 14 Nov 2012 11:00:47 -0600
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
Message-ID: <CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>

On Wed, Nov 14, 2012 at 10:12 AM, Mark Adam <dreamingforward at gmail.com> wrote:
> On Wed, Nov 14, 2012 at 3:12 AM, Chris Withers <chris at simplistix.co.uk> wrote:
>> Hi All,
>>
>> A colleague pointed me at Doug's excellent article here:
>> ...which made me a little sad, I suspect I'm not the only one who finds:
>>
>> a_dict = dict(
>>     x = 1,
>>     y = 2,
>>     z = 3,
>>     ...
>>     )
>>
>> ...easier to read than:
>>
>> a_dict = {
>>     'x':1,
>>     'y':2,
>>     'z':3,
>>     ...
>>     }
>
> Hey, it makes me a little sad that dict breaks convention by allowing
> the use of unquoted characters (which everywhere else looks like
> variable names) just for a silly typing optimization.

What convention and typing optimization is this? I hope you aren't
suggesting it should be dict("x"=1) or dict("x":1)?

From python-dev at masklinn.net  Wed Nov 14 18:02:43 2012
From: python-dev at masklinn.net (Xavier Morel)
Date: Wed, 14 Nov 2012 18:02:43 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <k80hm2$fs8$1@ger.gmane.org>
References: <50A36065.9010704@simplistix.co.uk> <k80ghr$6ic$1@ger.gmane.org>
	<k80hm2$fs8$1@ger.gmane.org>
Message-ID: <2081DD91-D5E9-4C59-B495-C7E163436821@masklinn.net>


On 2012-11-14, at 17:42 , Richard Oudkerk wrote:

> On 14/11/2012 4:23pm, Serhiy Storchaka wrote:
>> PEP 8 recommends:
>> 
>> a_dict = dict(
>>     x=1,
>>     y=2,
>>     z=3,
>>     ...
>> )
>> 
>> and
>> 
>> a_dict = {
>>     'x': 1,
>>     'y': 2,
>>     'z': 3,
>>     ...
>> }
> 
> In which section?  I can't see such a recommendation.

Whitespace in Expressions and Statements > Other Recommendations

3rd bullet:

?
Don't use spaces around the = sign when used to indicate a keyword argument or a default parameter value.

Yes:

def complex(real, imag=0.0):
    return magic(r=real, i=imag)

No:

def complex(real, imag = 0.0):
    return magic(r = real, i = imag)
?


From dreamingforward at gmail.com  Wed Nov 14 18:08:32 2012
From: dreamingforward at gmail.com (Mark Adam)
Date: Wed, 14 Nov 2012 11:08:32 -0600
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <2081DD91-D5E9-4C59-B495-C7E163436821@masklinn.net>
References: <50A36065.9010704@simplistix.co.uk> <k80ghr$6ic$1@ger.gmane.org>
	<k80hm2$fs8$1@ger.gmane.org>
	<2081DD91-D5E9-4C59-B495-C7E163436821@masklinn.net>
Message-ID: <CAMjeLr-n2BssxMc6SsyKo2yu_yjbSk8z3_wYhHsuWLnmhKCeSw@mail.gmail.com>

On Wed, Nov 14, 2012 at 11:02 AM, Xavier Morel <python-dev at masklinn.net> wrote:
>
> On 2012-11-14, at 17:42 , Richard Oudkerk wrote:
>
>> On 14/11/2012 4:23pm, Serhiy Storchaka wrote:
>>> PEP 8 recommends:
>>>
>>> a_dict = dict(
>>>     x=1,
>>>     y=2,
>>>     z=3,
>>>     ...
>>> )
>>>
>>> and
>>>
>>> a_dict = {
>>>     'x': 1,
>>>     'y': 2,
>>>     'z': 3,
>>>     ...
>>> }
>>
>> In which section?  I can't see such a recommendation.
>
> Whitespace in Expressions and Statements > Other Recommendations
>
> 3rd bullet:
>
> ?
> Don't use spaces around the = sign when used to indicate a keyword argument or a default parameter value.
>
> Yes:
>
> def complex(real, imag=0.0):
>     return magic(r=real, i=imag)
>
> No:
>
> def complex(real, imag = 0.0):
>     return magic(r = real, i = imag)

That's not a recommendation to use the **kwargs style.

mark

From dreamingforward at gmail.com  Wed Nov 14 18:10:15 2012
From: dreamingforward at gmail.com (Mark Adam)
Date: Wed, 14 Nov 2012 11:10:15 -0600
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
Message-ID: <CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>

On Wed, Nov 14, 2012 at 11:00 AM, Brian Curtin <brian at python.org> wrote:
> On Wed, Nov 14, 2012 at 10:12 AM, Mark Adam <dreamingforward at gmail.com> wrote:
>> On Wed, Nov 14, 2012 at 3:12 AM, Chris Withers <chris at simplistix.co.uk> wrote:
>>> Hi All,
>>>
>>> A colleague pointed me at Doug's excellent article here:
>>> ...which made me a little sad, I suspect I'm not the only one who finds:
>>>
>>> a_dict = dict(
>>>     x = 1,
>>>     y = 2,
>>>     z = 3,
>>>     ...
>>>     )
>>>
>>> ...easier to read than:
>>>
>>> a_dict = {
>>>     'x':1,
>>>     'y':2,
>>>     'z':3,
>>>     ...
>>>     }
>>
>> Hey, it makes me a little sad that dict breaks convention by allowing
>> the use of unquoted characters (which everywhere else looks like
>> variable names) just for a silly typing optimization.
>
> What convention and typing optimization is this? I hope you aren't
> suggesting it should be dict("x"=1) or dict("x":1)?

Try the canonical {'x':1}.  Only dict allows the special
initialization above.  Other collections require an iterable.  I'm guessing
**kwargs initialization was only used because it is so simple to
implement, but that's not necessarily a heuristic for good language design.

mark

From rdmurray at bitdance.com  Wed Nov 14 18:27:46 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 14 Nov 2012 12:27:46 -0500
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
Message-ID: <20121114172746.B08492500F1@webabinitio.net>

On Wed, 14 Nov 2012 11:10:15 -0600, Mark Adam <dreamingforward at gmail.com> wrote:
> On Wed, Nov 14, 2012 at 11:00 AM, Brian Curtin <brian at python.org> wrote:
> > On Wed, Nov 14, 2012 at 10:12 AM, Mark Adam <dreamingforward at gmail.com> wrote:
> >> On Wed, Nov 14, 2012 at 3:12 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> >>> Hi All,
> >>>
> >>> A colleague pointed me at Doug's excellent article here:
> >>> ...which made me a little sad, I suspect I'm not the only one who finds:
> >>>
> >>> a_dict = dict(
> >>>     x = 1,
> >>>     y = 2,
> >>>     z = 3,
> >>>     ...
> >>>     )
> >>>
> >>> ...easier to read than:
> >>>
> >>> a_dict = {
> >>>     'x':1,
> >>>     'y':2,
> >>>     'z':3,
> >>>     ...
> >>>     }
> >>
> >> Hey, it makes me a little sad that dict breaks convention by allowing
> >> the use of unquoted characters (which everywhere else looks like
> >> variable names) just for a silly typing optimization.
> >
> > What convention and typing optimization is this? I hope you aren't
> > suggesting it should be dict("x"=1) or dict("x":1)?
> 
> Try the canonical {'x':1}.  Only dict allows the special
> initialization above.  Other collections require an iterable.  I'm guessing
> **kwargs initialization was only used because it is so simple to
> implement, but that's not necessarily a heuristic for good language design.

Maybe it's not good design, but I'll bet you that if it didn't do that,
there would be lots of instances of this scattered around various
codebases:

    def makedict(**kw):
        return kw

--David

From dreamingforward at gmail.com  Wed Nov 14 18:57:48 2012
From: dreamingforward at gmail.com (Mark Adam)
Date: Wed, 14 Nov 2012 11:57:48 -0600
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <20121114172746.B08492500F1@webabinitio.net>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<20121114172746.B08492500F1@webabinitio.net>
Message-ID: <CAMjeLr-eroPBXTsi1wWtKDYZ6Z9wCjwCqL4k7B4K015RuadU4A@mail.gmail.com>

On Wed, Nov 14, 2012 at 11:27 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> Maybe it's not good design, but I'll bet you that if it didn't do that,
> there would be lots of instances of this scattered around various
> codebases:
>
>     def makedict(**kw):
>         return kw

Now that's a good solution and probably solves the OP speed problem.

mark

From shibturn at gmail.com  Wed Nov 14 19:01:53 2012
From: shibturn at gmail.com (Richard Oudkerk)
Date: Wed, 14 Nov 2012 18:01:53 +0000
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <2081DD91-D5E9-4C59-B495-C7E163436821@masklinn.net>
References: <50A36065.9010704@simplistix.co.uk> <k80ghr$6ic$1@ger.gmane.org>
	<k80hm2$fs8$1@ger.gmane.org>
	<2081DD91-D5E9-4C59-B495-C7E163436821@masklinn.net>
Message-ID: <k80mam$u1n$1@ger.gmane.org>

On 14/11/2012 5:02pm, Xavier Morel wrote:
>> In which section?  I can't see such a recommendation.
>
> Whitespace in Expressions and Statements > Other Recommendations
>
> 3rd bullet:
>
> ?
> Don't use spaces around the = sign when used to indicate a keyword argument or a default parameter value.

Oops, I did not even notice that difference.

I thought Serhiy was talking about indenting the closing ')' and '}'.

-- 
Richard


From python-dev at masklinn.net  Wed Nov 14 19:08:44 2012
From: python-dev at masklinn.net (Xavier Morel)
Date: Wed, 14 Nov 2012 19:08:44 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAMjeLr-n2BssxMc6SsyKo2yu_yjbSk8z3_wYhHsuWLnmhKCeSw@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk> <k80ghr$6ic$1@ger.gmane.org>
	<k80hm2$fs8$1@ger.gmane.org>
	<2081DD91-D5E9-4C59-B495-C7E163436821@masklinn.net>
	<CAMjeLr-n2BssxMc6SsyKo2yu_yjbSk8z3_wYhHsuWLnmhKCeSw@mail.gmail.com>
Message-ID: <DA00908F-A304-4EB2-A9B0-F0D0C64CF920@masklinn.net>

On 2012-11-14, at 18:08 , Mark Adam wrote:
> 
> That's not a recommendation to use the **kwargs style.

And nobody said it was. It's a recommendation to not put spaces around
the equals sign when using keyword arguments which is the correction
Serhiy applied to the original code (along with adding a space after the
colon in the literal dict, also a PEP8 recommendation).

From catch-all at masklinn.net  Wed Nov 14 19:12:25 2012
From: catch-all at masklinn.net (Xavier Morel)
Date: Wed, 14 Nov 2012 19:12:25 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
Message-ID: <8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>

On 2012-11-14, at 18:10 , Mark Adam wrote:
> 
> Try the canonical {'x':1}.  Only dict allows the special
> initialization above.  Other collections require an iterable.

Other collections don't have a choice, because it would often be
ambiguous. Dicts do not have that issue.

>  I'm guessing
> **kwargs initialization was only used because it is so simple to
> implement, but that's not necessarily a heuristic for good language design.

In this case it very much is, it permits easily merging two dicts in a
single expression or cloning-with-replacement. It also mirrors the
signature of dict.update which I think is a Good Thing.

From dreamingforward at gmail.com  Wed Nov 14 19:54:32 2012
From: dreamingforward at gmail.com (Mark Adam)
Date: Wed, 14 Nov 2012 12:54:32 -0600
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
Message-ID: <CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>

On Wed, Nov 14, 2012 at 12:12 PM, Xavier Morel <catch-all at masklinn.net> wrote:
> On 2012-11-14, at 18:10 , Mark Adam wrote:
>>
>> Try the canonical {'x':1}.  Only dict allows the special
>> initialization above.  Other collections require an iterable.
>
> Other collections don't have a choice, because it would often be
> ambiguous. Dicts do not have that issue.

mkay....

>>  I'm guessing
>> **kwargs initialization was only used because it is so simple to
>> implement, but that's not necessarily a heuristic for good language design.
>
> In this case it very much is, it permits easily merging two dicts in a
> single expression or cloning-with-replacement. It also mirrors the
> signature of dict.update which I think is a Good Thing.

Merging of two dicts is done with dict.update.   How do you do it on
initialization?  This doesn't make sense.

mark

From a.cavallo at cavallinux.eu  Wed Nov 14 13:15:02 2012
From: a.cavallo at cavallinux.eu (a.cavallo at cavallinux.eu)
Date: Wed, 14 Nov 2012 13:15:02 +0100
Subject: [Python-Dev] [Distutils] [Catalog-sig] accept the wheel PEPs425,
	426, 427
Message-ID: <9326.1352895302@cavallinux.eu>

Mmmm, 
interesting point and worth a discussion about different roles (developer, system
admin, final user etc.) having different needs.

I believe distutils is used as tool primarily (setup.py bdist_rpm/msi to create
installable objects, setup.py bdist_sdist to manage the source code etc.): this
complicates the landscape.

Some developer has expectation (wrong IMHO) to replace a whole lot of tools with
it (downloaders/installers/package managers/etc.): this is an uphill struggle
against already widely deployed systems (dpkg/yum/even windows has something!).

I think Tarek did some work in the past and the result is visible in the "The
Hitchhiker?s Guide to Packaging" but I've no idea where it went in the end ...
only is left is a punch-in-the-eye page (http://www.python.org/doc) :D

These days I'm stuck with the old KISS approach and I write a setup.py to create
"packages" and a makefile to create doc, run tests etc. I'm fairly happy with that.

Thanks

On Wed 14/11/12 08:32, "Ronald Oussoren" ronaldoussoren at mac.com wrote:
> 
> I'm not convinced that distutils is untestable. A major problem with
> distutils is that its API is barely documented, which effectly makes
> all of it public API (simular to asyncore).  IIRC that's the main
> reason
why distutils is frozen right now: with a lot of changes to distutils
> people
started to complain that this could break there there distutils
> scripts.
> The lack of a specification also makes testing harder, as it is
> unclear
what should be tested.
> 
> Ronald
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
> 
> 



From catch-all at masklinn.net  Wed Nov 14 20:37:02 2012
From: catch-all at masklinn.net (Xavier Morel)
Date: Wed, 14 Nov 2012 20:37:02 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
Message-ID: <21EFD6BC-574D-425C-8F1F-54033D098530@masklinn.net>

On 2012-11-14, at 19:54 , Mark Adam wrote:
> 
> Merging of two dicts is done with dict.update.

No, dict.update merges one dict (or two) into a third one.

> How do you do it on
> initialization?  This doesn't make sense.

dict(d1, **d2)

From phd at phdru.name  Wed Nov 14 14:25:00 2012
From: phd at phdru.name (Oleg Broytman)
Date: Wed, 14 Nov 2012 17:25:00 +0400
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
Message-ID: <20121114132500.GA14633@iskra.aviel.ru>

On Thu, Nov 15, 2012 at 12:18:38AM +1100, Chris Angelico <rosuav at gmail.com> wrote:
> On Wed, Nov 14, 2012 at 8:12 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> > I suspect I'm not the only one who finds:
> >
> > a_dict = dict(
> >     x = 1,
> >     y = 2,
> >     z = 3,
> >     ...
> >     )
> >
> > ...easier to read than:
> >
> > a_dict = {
> >     'x':1,
> >     'y':2,
> >     'z':3,
> >     ...
> >     }
> >
> > What can we do to speed up the former case?
> 
> Perhaps an alternative question: What can be done to make the latter
> less unpalatable? I personally prefer dict literal syntax to a dict
> constructor call, but no doubt there are a number of people who feel
> as you do. In what way(s) do you find the literal syntax less
> readable, and can some simple (and backward-compatible) enhancements
> help that?
> 
> I've seen criticisms (though I don't recall where) of Python,
> comparing it to JavaScript/ECMAScript, that complain of the need to
> quote the keys. IMO this is a worthwhile downside, as it allows you to
> use variables as the keys, rather than requiring (effectively) literal
> strings. But it does make a dict literal that much more "noisy" than
> the constructor.

   On the other had it's more powerful. You can write {'class': 'foo'}
but cannot dict(class='bar'). {1: '1'} but not dict(1='1').

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From dreamingforward at gmail.com  Wed Nov 14 21:53:11 2012
From: dreamingforward at gmail.com (Mark Adam)
Date: Wed, 14 Nov 2012 14:53:11 -0600
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <21EFD6BC-574D-425C-8F1F-54033D098530@masklinn.net>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<21EFD6BC-574D-425C-8F1F-54033D098530@masklinn.net>
Message-ID: <CAMjeLr8AkBbhO-kgLNib9=73WdE2JFdkij76yPckGBB7NkGiyQ@mail.gmail.com>

On Wed, Nov 14, 2012 at 1:37 PM, Xavier Morel <catch-all at masklinn.net> wrote:
> On 2012-11-14, at 19:54 , Mark Adam wrote:
>>
>> Merging of two dicts is done with dict.update.
>
> No, dict.update merges one dict (or two) into a third one.

No.  I think you need to read the docs.

>> How do you do it on
>> initialization?  This doesn't make sense.
>
> dict(d1, **d2)

That's not valid syntax is it?

mark

From solipsis at pitrou.net  Wed Nov 14 21:58:30 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 14 Nov 2012 21:58:30 +0100
Subject: [Python-Dev] performance of {} versus dict()
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<21EFD6BC-574D-425C-8F1F-54033D098530@masklinn.net>
	<CAMjeLr8AkBbhO-kgLNib9=73WdE2JFdkij76yPckGBB7NkGiyQ@mail.gmail.com>
Message-ID: <20121114215830.674acb49@pitrou.net>

On Wed, 14 Nov 2012 14:53:11 -0600
Mark Adam <dreamingforward at gmail.com> wrote:
> On Wed, Nov 14, 2012 at 1:37 PM, Xavier Morel <catch-all at masklinn.net> wrote:
> > On 2012-11-14, at 19:54 , Mark Adam wrote:
> >>
> >> Merging of two dicts is done with dict.update.
> >
> > No, dict.update merges one dict (or two) into a third one.
> 
> No.  I think you need to read the docs.
> 
> >> How do you do it on
> >> initialization?  This doesn't make sense.
> >
> > dict(d1, **d2)
> 
> That's not valid syntax is it?

Why don't you try it for yourself:

>>> d1 = {1:2}
>>> d2 = {3:4}
>>> dict(d1, **d2)
{1: 2, 3: 4}




From python at mrabarnett.plus.com  Wed Nov 14 22:20:33 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 14 Nov 2012 21:20:33 +0000
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAMjeLr8AkBbhO-kgLNib9=73WdE2JFdkij76yPckGBB7NkGiyQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<21EFD6BC-574D-425C-8F1F-54033D098530@masklinn.net>
	<CAMjeLr8AkBbhO-kgLNib9=73WdE2JFdkij76yPckGBB7NkGiyQ@mail.gmail.com>
Message-ID: <50A40B21.2090206@mrabarnett.plus.com>

On 2012-11-14 20:53, Mark Adam wrote:
> On Wed, Nov 14, 2012 at 1:37 PM, Xavier Morel <catch-all at masklinn.net> wrote:
>> On 2012-11-14, at 19:54 , Mark Adam wrote:
>>>
>>> Merging of two dicts is done with dict.update.
>>
>> No, dict.update merges one dict (or two) into a third one.
>
> No.  I think you need to read the docs.
>
>>> How do you do it on
>>> initialization?  This doesn't make sense.
>>
>> dict(d1, **d2)
>
> That's not valid syntax is it?
>
No.

You can have dict(d1) and dict(**d2), but not dict(d1, **d2).


From python at mrabarnett.plus.com  Wed Nov 14 22:24:14 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 14 Nov 2012 21:24:14 +0000
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A40B21.2090206@mrabarnett.plus.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<21EFD6BC-574D-425C-8F1F-54033D098530@masklinn.net>
	<CAMjeLr8AkBbhO-kgLNib9=73WdE2JFdkij76yPckGBB7NkGiyQ@mail.gmail.com>
	<50A40B21.2090206@mrabarnett.plus.com>
Message-ID: <50A40BFE.1010500@mrabarnett.plus.com>

On 2012-11-14 21:20, MRAB wrote:
> On 2012-11-14 20:53, Mark Adam wrote:
>> On Wed, Nov 14, 2012 at 1:37 PM, Xavier Morel <catch-all at masklinn.net> wrote:
>>> On 2012-11-14, at 19:54 , Mark Adam wrote:
>>>>
>>>> Merging of two dicts is done with dict.update.
>>>
>>> No, dict.update merges one dict (or two) into a third one.
>>
>> No.  I think you need to read the docs.
>>
>>>> How do you do it on
>>>> initialization?  This doesn't make sense.
>>>
>>> dict(d1, **d2)
>>
>> That's not valid syntax is it?
>>
> No.
>
> You can have dict(d1) and dict(**d2), but not dict(d1, **d2).
>
Oops, wrong! :-( (I see now where I went wrong...)

From brian at python.org  Wed Nov 14 22:24:20 2012
From: brian at python.org (Brian Curtin)
Date: Wed, 14 Nov 2012 15:24:20 -0600
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A40B21.2090206@mrabarnett.plus.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<21EFD6BC-574D-425C-8F1F-54033D098530@masklinn.net>
	<CAMjeLr8AkBbhO-kgLNib9=73WdE2JFdkij76yPckGBB7NkGiyQ@mail.gmail.com>
	<50A40B21.2090206@mrabarnett.plus.com>
Message-ID: <CAD+XWwrG4GNAT8FYaF7Tmfojnqmfrj6i=qwmKKCVo6gqqZ_vmQ@mail.gmail.com>

On Wed, Nov 14, 2012 at 3:20 PM, MRAB <python at mrabarnett.plus.com> wrote:
> On 2012-11-14 20:53, Mark Adam wrote:
>>
>> On Wed, Nov 14, 2012 at 1:37 PM, Xavier Morel <catch-all at masklinn.net>
>> wrote:
>>>
>>> On 2012-11-14, at 19:54 , Mark Adam wrote:
>>>>
>>>>
>>>> Merging of two dicts is done with dict.update.
>>>
>>>
>>> No, dict.update merges one dict (or two) into a third one.
>>
>>
>> No.  I think you need to read the docs.
>>
>>>> How do you do it on
>>>> initialization?  This doesn't make sense.
>>>
>>>
>>> dict(d1, **d2)
>>
>>
>> That's not valid syntax is it?
>>
> No.
>
> You can have dict(d1) and dict(**d2), but not dict(d1, **d2).

Yes you can.

From ethan at stoneleaf.us  Wed Nov 14 22:27:15 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 14 Nov 2012 13:27:15 -0800
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A40B21.2090206@mrabarnett.plus.com>
References: <50A36065.9010704@simplistix.co.uk>	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>	<21EFD6BC-574D-425C-8F1F-54033D098530@masklinn.net>	<CAMjeLr8AkBbhO-kgLNib9=73WdE2JFdkij76yPckGBB7NkGiyQ@mail.gmail.com>
	<50A40B21.2090206@mrabarnett.plus.com>
Message-ID: <50A40CB3.7010801@stoneleaf.us>

MRAB wrote:
> On 2012-11-14 20:53, Mark Adam wrote:
>> On Wed, Nov 14, 2012 at 1:37 PM, Xavier Morel <catch-all at masklinn.net> 
>> wrote:
>>> On 2012-11-14, at 19:54 , Mark Adam wrote:
>>>>
>>>> Merging of two dicts is done with dict.update.
>>>
>>> No, dict.update merges one dict (or two) into a third one.
>>
>> No.  I think you need to read the docs.
>>
>>>> How do you do it on
>>>> initialization?  This doesn't make sense.
>>>
>>> dict(d1, **d2)
>>
>> That's not valid syntax is it?
>>
> No.
> 
> You can have dict(d1) and dict(**d2), but not dict(d1, **d2).

To (mis-)quote Antoine:
 >--> d1 = {1:2}
 >--> d2 = {'3':4}
 >--> dict(d1, **d2)
 > {1: 2, '3': 4}

Apparently it is valid syntax.  Just make sure you keys for the ** operator are valid strings.  :)

~Ethan~

From bwmaister at gmail.com  Wed Nov 14 22:40:37 2012
From: bwmaister at gmail.com (Brandon W Maister)
Date: Wed, 14 Nov 2012 16:40:37 -0500
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A40CB3.7010801@stoneleaf.us>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<21EFD6BC-574D-425C-8F1F-54033D098530@masklinn.net>
	<CAMjeLr8AkBbhO-kgLNib9=73WdE2JFdkij76yPckGBB7NkGiyQ@mail.gmail.com>
	<50A40B21.2090206@mrabarnett.plus.com> <50A40CB3.7010801@stoneleaf.us>
Message-ID: <CANNZFEns1i1s96USM8gVePJWFOU5eYkwVAQo-TYh1ug1QG=Yxg@mail.gmail.com>

>
> To (mis-)quote Antoine:
> >--> d1 = {1:2}
> >--> d2 = {'3':4}
> >--> dict(d1, **d2)
> > {1: 2, '3': 4}
>
> Apparently it is valid syntax.  Just make sure you keys for the **
> operator are valid strings.  :)
>
>
or not:

>>> dict(**{'not a valid identifier': True, 1: True})
{1: True, 'not a valid identifier': True}

brandon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121114/41b790a0/attachment.html>

From brian at python.org  Wed Nov 14 22:43:46 2012
From: brian at python.org (Brian Curtin)
Date: Wed, 14 Nov 2012 15:43:46 -0600
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CANNZFEns1i1s96USM8gVePJWFOU5eYkwVAQo-TYh1ug1QG=Yxg@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<21EFD6BC-574D-425C-8F1F-54033D098530@masklinn.net>
	<CAMjeLr8AkBbhO-kgLNib9=73WdE2JFdkij76yPckGBB7NkGiyQ@mail.gmail.com>
	<50A40B21.2090206@mrabarnett.plus.com>
	<50A40CB3.7010801@stoneleaf.us>
	<CANNZFEns1i1s96USM8gVePJWFOU5eYkwVAQo-TYh1ug1QG=Yxg@mail.gmail.com>
Message-ID: <CAD+XWwo5Dmw7jaKjXBFvivoBbq59tmSgw66oT_krx2jLCC9cQQ@mail.gmail.com>

On Wed, Nov 14, 2012 at 3:40 PM, Brandon W Maister <bwmaister at gmail.com> wrote:
>
>>
>> To (mis-)quote Antoine:
>> >--> d1 = {1:2}
>> >--> d2 = {'3':4}
>> >--> dict(d1, **d2)
>> > {1: 2, '3': 4}
>>
>> Apparently it is valid syntax.  Just make sure you keys for the **
>> operator are valid strings.  :)
>>
>
> or not:
>
>>>> dict(**{'not a valid identifier': True, 1: True})
> {1: True, 'not a valid identifier': True}

Just because the string says it's not valid doesn't mean it's not valid.

Anyway, can this thread go to python-ideas or python-list now?

From python-dev at masklinn.net  Wed Nov 14 23:01:32 2012
From: python-dev at masklinn.net (Xavier Morel)
Date: Wed, 14 Nov 2012 23:01:32 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAMjeLr8AkBbhO-kgLNib9=73WdE2JFdkij76yPckGBB7NkGiyQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<21EFD6BC-574D-425C-8F1F-54033D098530@masklinn.net>
	<CAMjeLr8AkBbhO-kgLNib9=73WdE2JFdkij76yPckGBB7NkGiyQ@mail.gmail.com>
Message-ID: <B96CBAA5-D55B-49F8-A848-D4F0DE38CA93@masklinn.net>

On 2012-11-14, at 21:53 , Mark Adam wrote:

> On Wed, Nov 14, 2012 at 1:37 PM, Xavier Morel <catch-all at masklinn.net> wrote:
>> On 2012-11-14, at 19:54 , Mark Adam wrote:
>>> 
>>> Merging of two dicts is done with dict.update.
>> 
>> No, dict.update merges one dict (or two) into a third one.
> 
> No.  I think you need to read the docs.

I know what the docs say. dict.update requires an existing dict and (as
mutator methods usually do in Python) doesn't return anything. Thus it
merges a dict (or two) into a third one (the subject of the call).

>>> How do you do it on
>>> initialization?  This doesn't make sense.
>> 
>> dict(d1, **d2)
> 
> That's not valid syntax is it?

Of course it is, why would it not be?

From greg.ewing at canterbury.ac.nz  Wed Nov 14 22:40:48 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 15 Nov 2012 10:40:48 +1300
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
Message-ID: <50A40FE0.9050109@canterbury.ac.nz>

Chris Angelico wrote:
> Perhaps an alternative question: What can be done to make the latter
> less unpalatable?

* We could introduce a new syntax such as {a = 1, b = 2}.

* If the compiler were allowed to recognise builtins, it could
turn dict(a = 1, b = 2) into {'a':1, 'b':2} automatically.

-- 
Greg

From chris at simplistix.co.uk  Wed Nov 14 23:37:46 2012
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 14 Nov 2012 22:37:46 +0000
Subject: [Python-Dev] performance of {} versus dict(), de fmd(**kw):
 return kw trumps all ; -)
In-Reply-To: <20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
Message-ID: <50A41D3A.9040904@simplistix.co.uk>

On 14/11/2012 10:11, martin at v.loewis.de wrote:
>
> Zitat von Chris Withers <chris at simplistix.co.uk>:
>
>> a_dict = dict(
>>     x = 1,
>>     y = 2,
>>     z = 3,
>>     ...
>>     )
>
>> What can we do to speed up the former case?
>
> It should be possible to special-case it. Rather than creating
> a new dictionary from scratch, one could try to have the new dictionary
> the same size as the original one, and copy all entries.

Indeed, Doug, what are your views on this? Also, did you have a 
real-world example where this speed difference was causing you a problem?

> I don't know how much this would gain, though. You still have to
> create two dictionary objects. For a better speedup, try
>
> def xdict(**kwds):
>    return kwds

Hah, good call, this trumps both of the other options:

$ python2.7 -m timeit -n 1000000 -r 5 -v 
"{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}"
raw times: 1.45 1.45 1.44 1.45 1.45
1000000 loops, best of 5: 1.44 usec per loop
$ python2.6 -m timeit -n 1000000 -r 5 -v 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
raw times: 2.37 2.36 2.36 2.37 2.37
1000000 loops, best of 5: 2.36 usec per loop$ python2.6 -m timeit -n 
1000000 -r 5 -v 'def md(**kw): return kw; md(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
raw times: 0.548 0.533 0.55 0.577 0.539
1000000 loops, best of 5: 0.533 usec per loop

For the naive observer (ie: me!), why is that?

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Wed Nov 14 23:41:28 2012
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 14 Nov 2012 22:41:28 +0000
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A40FE0.9050109@canterbury.ac.nz>
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
	<50A40FE0.9050109@canterbury.ac.nz>
Message-ID: <50A41E18.1050102@simplistix.co.uk>

On 14/11/2012 21:40, Greg Ewing wrote:
> * If the compiler were allowed to recognise builtins, it could
> turn dict(a = 1, b = 2) into {'a':1, 'b':2} automatically.

That would be my naive suggestion, I am prepared to be shot down in 
flames ;-)

Would be even more awesome if it could end up with the magical 
performance of "def md(**kw): return kw"...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Wed Nov 14 23:43:52 2012
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 14 Nov 2012 22:43:52 +0000
Subject: [Python-Dev] performance of {} versus dict(), de fmd(**kw):
 return kw trumps all ; -)
In-Reply-To: <50A41D3A.9040904@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
	<50A41D3A.9040904@simplistix.co.uk>
Message-ID: <50A41EA8.9040502@simplistix.co.uk>

On 14/11/2012 22:37, Chris Withers wrote:
> On 14/11/2012 10:11, martin at v.loewis.de wrote:
>> def xdict(**kwds):
>>    return kwds
>
> Hah, good call, this trumps both of the other options:
>
> $ python2.7 -m timeit -n 1000000 -r 5 -v
> "{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}"
> raw times: 1.45 1.45 1.44 1.45 1.45
> 1000000 loops, best of 5: 1.44 usec per loop
> $ python2.6 -m timeit -n 1000000 -r 5 -v
> 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
> raw times: 2.37 2.36 2.36 2.37 2.37
> 1000000 loops, best of 5: 2.36 usec per loop$ python2.6 -m timeit -n
> 1000000 -r 5 -v 'def md(**kw): return kw; md(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
> raw times: 0.548 0.533 0.55 0.577 0.539
> 1000000 loops, best of 5: 0.533 usec per loop

Before anyone shoots me, yes, wrong python for two of them:

$ python2.7 -m timeit -n 1000000 -r 5 -v 
"{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}"
raw times: 1.49 1.49 1.5 1.49 1.48
1000000 loops, best of 5: 1.48 usec per loop

$ python2.7 -m timeit -n 1000000 -r 5 -v 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
raw times: 2.35 2.36 2.41 2.42 2.35
1000000 loops, best of 5: 2.35 usec per loop

$ python2.7 -m timeit -n 1000000 -r 5 -v 'def md(**kw): return kw; 
md(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
raw times: 0.507 0.515 0.516 0.529 0.524
1000000 loops, best of 5: 0.507 usec per loop

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From python at mrabarnett.plus.com  Wed Nov 14 23:51:37 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 14 Nov 2012 22:51:37 +0000
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A40FE0.9050109@canterbury.ac.nz>
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
	<50A40FE0.9050109@canterbury.ac.nz>
Message-ID: <50A42079.4060908@mrabarnett.plus.com>

On 2012-11-14 21:40, Greg Ewing wrote:
> Chris Angelico wrote:
>> Perhaps an alternative question: What can be done to make the latter
>> less unpalatable?
>
> * We could introduce a new syntax such as {a = 1, b = 2}.
>
> * If the compiler were allowed to recognise builtins, it could
> turn dict(a = 1, b = 2) into {'a':1, 'b':2} automatically.
>
That would be a transformation of the AST, although it assumes that
'dict' hasn't been rebound.

Should there be the option of a warning if a builtin is rebound? Or the
option of the transformation plus a warning if the builtin is rebound?

From donald.stufft at gmail.com  Thu Nov 15 00:00:58 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Wed, 14 Nov 2012 18:00:58 -0500
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A42079.4060908@mrabarnett.plus.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
	<50A40FE0.9050109@canterbury.ac.nz>
	<50A42079.4060908@mrabarnett.plus.com>
Message-ID: <37EDB7B5B89649AC9E0AA19C89A8C87E@gmail.com>

$ pypy -m timeit 'dict()'
1000000000 loops, best of 3: 0.000811 usec per loop

$ pypy -m timeit '{}'    
1000000000 loops, best of 3: 0.000809 usec per loop

$ pypy -m timeit 'def md(**kw): return kw; md()'
100000000 loops, best of 3: 0.0182 usec per loop

$ pypy -m timeit -s 'def md(**kw): return kw' 'md()'
1000000000 loops, best of 3: 0.00136 usec per loop

If the difference between dict() and {} is hurting your code why are
you still using CPython.



On Wednesday, November 14, 2012 at 5:51 PM, MRAB wrote:

> On 2012-11-14 21:40, Greg Ewing wrote:
> > Chris Angelico wrote:
> > > Perhaps an alternative question: What can be done to make the latter
> > > less unpalatable?
> > > 
> > 
> > 
> > * We could introduce a new syntax such as {a = 1, b = 2}.
> > 
> > * If the compiler were allowed to recognise builtins, it could
> > turn dict(a = 1, b = 2) into {'a':1, 'b':2} automatically.
> > 
> 
> That would be a transformation of the AST, although it assumes that
> 'dict' hasn't been rebound.
> 
> Should there be the option of a warning if a builtin is rebound? Or the
> option of the transformation plus a warning if the builtin is rebound?
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org (mailto:Python-Dev at python.org)
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121114/4aed86cd/attachment-0001.html>

From python-dev at masklinn.net  Thu Nov 15 00:03:45 2012
From: python-dev at masklinn.net (Xavier Morel)
Date: Thu, 15 Nov 2012 00:03:45 +0100
Subject: [Python-Dev] performance of {} versus dict(), de fmd(**kw):
	return kw trumps all ; -)
In-Reply-To: <50A41EA8.9040502@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
	<50A41D3A.9040904@simplistix.co.uk>
	<50A41EA8.9040502@simplistix.co.uk>
Message-ID: <F70A48A3-C13A-4814-BE3C-6BE43CBDAAD9@masklinn.net>


On 2012-11-14, at 23:43 , Chris Withers wrote:

> On 14/11/2012 22:37, Chris Withers wrote:
>> On 14/11/2012 10:11, martin at v.loewis.de wrote:
>>> def xdict(**kwds):
>>>   return kwds
>> 
>> Hah, good call, this trumps both of the other options:
>> 
>> $ python2.7 -m timeit -n 1000000 -r 5 -v
>> "{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}"
>> raw times: 1.45 1.45 1.44 1.45 1.45
>> 1000000 loops, best of 5: 1.44 usec per loop
>> $ python2.6 -m timeit -n 1000000 -r 5 -v
>> 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
>> raw times: 2.37 2.36 2.36 2.37 2.37
>> 1000000 loops, best of 5: 2.36 usec per loop$ python2.6 -m timeit -n
>> 1000000 -r 5 -v 'def md(**kw): return kw; md(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
>> raw times: 0.548 0.533 0.55 0.577 0.539
>> 1000000 loops, best of 5: 0.533 usec per loop
> 
> Before anyone shoots me, yes, wrong python for two of them:
> 
> $ python2.7 -m timeit -n 1000000 -r 5 -v "{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}"
> raw times: 1.49 1.49 1.5 1.49 1.48
> 1000000 loops, best of 5: 1.48 usec per loop
> 
> $ python2.7 -m timeit -n 1000000 -r 5 -v 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
> raw times: 2.35 2.36 2.41 2.42 2.35
> 1000000 loops, best of 5: 2.35 usec per loop
> 
> $ python2.7 -m timeit -n 1000000 -r 5 -v 'def md(**kw): return kw; md(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
> raw times: 0.507 0.515 0.516 0.529 0.524
> 1000000 loops, best of 5: 0.507 usec per loop

The last one is kind-of weird, it seems to be greatly advantaged by the local lookup:

> python2.7 -m timeit -n 1000000 -r 5 -v "{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}"                 
raw times: 0.676 0.683 0.682 0.698 0.691
1000000 loops, best of 5: 0.676 usec per loop
> python2.7 -m timeit -n 1000000 -r 5 -v 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'                           
raw times: 1.64 1.66 1.4 1.44 1.44
1000000 loops, best of 5: 1.4 usec per loop
> python2.7 -m timeit -n 1000000 -r 5 -v 'def md(**kw): return kw; md(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'    
raw times: 0.188 0.203 0.201 0.195 0.202
1000000 loops, best of 5: 0.188 usec per loop
> python2.7 -m timeit -n 1000000 -r 5 -v -s 'def md(**kw): return kw' 'md(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
raw times: 0.871 0.864 0.863 0.889 0.871
1000000 loops, best of 5: 0.863 usec per loop


From steve at pearwood.info  Thu Nov 15 00:36:24 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 15 Nov 2012 10:36:24 +1100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
Message-ID: <50A42AF8.1070807@pearwood.info>

On 15/11/12 05:54, Mark Adam wrote:

> Merging of two dicts is done with dict.update.   How do you do it on
> initialization?  This doesn't make sense.

Frequently.

my_prefs = dict(default_prefs, setting=True, another_setting=False)


Notice that I'm not merging one dict into another, but merging two dicts
into a third.

(Well, technically, one of the two comes from keyword arguments rather
than an actual dict, but the principle is the same.)

The Python 1.5 alternative was:

my_prefs = {}
my_prefs.update(default_prefs)
my_prefs['setting'] = True
my_prefs['another_setting'] = False


Blah, I'm so glad I don't have to write Python 1.5 code any more. Even
using copy only saves a line:

my_prefs = default_prefs.copy()
my_prefs['setting'] = True
my_prefs['another_setting'] = False




-- 
Steven

From lukas.lueg at gmail.com  Thu Nov 15 00:38:38 2012
From: lukas.lueg at gmail.com (Lukas Lueg)
Date: Thu, 15 Nov 2012 00:38:38 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A42AF8.1070807@pearwood.info>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<50A42AF8.1070807@pearwood.info>
Message-ID: <CAJF-kY=3EUAt3U2PZTzcjWNEHYa4WVVg6Gkzo2kmC_gBYf1cvA@mail.gmail.com>

Notice that {'x':1} and dict(x=1) are different beasts: The first one
compiles directly to BUILD_MAP. The second one loads a reference to 'dict'
from globals() and calls the constructor. The two are not the same.



2012/11/15 Steven D'Aprano <steve at pearwood.info>

> On 15/11/12 05:54, Mark Adam wrote:
>
>  Merging of two dicts is done with dict.update.   How do you do it on
>> initialization?  This doesn't make sense.
>>
>
> Frequently.
>
> my_prefs = dict(default_prefs, setting=True, another_setting=False)
>
>
> Notice that I'm not merging one dict into another, but merging two dicts
> into a third.
>
> (Well, technically, one of the two comes from keyword arguments rather
> than an actual dict, but the principle is the same.)
>
> The Python 1.5 alternative was:
>
> my_prefs = {}
> my_prefs.update(default_prefs)
> my_prefs['setting'] = True
> my_prefs['another_setting'] = False
>
>
> Blah, I'm so glad I don't have to write Python 1.5 code any more. Even
> using copy only saves a line:
>
> my_prefs = default_prefs.copy()
> my_prefs['setting'] = True
> my_prefs['another_setting'] = False
>
>
>
>
> --
> Steven
>
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> lukas.lueg%40gmail.com<http://mail.python.org/mailman/options/python-dev/lukas.lueg%40gmail.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121115/8316bda0/attachment.html>

From rosuav at gmail.com  Thu Nov 15 00:40:20 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 15 Nov 2012 10:40:20 +1100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A42AF8.1070807@pearwood.info>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<50A42AF8.1070807@pearwood.info>
Message-ID: <CAPTjJmpJ66sndb4aZ52WiZWa2dv6iGg4Ozua26=YqjOQC113hQ@mail.gmail.com>

On Thu, Nov 15, 2012 at 10:36 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On 15/11/12 05:54, Mark Adam wrote:
>
>> Merging of two dicts is done with dict.update.   How do you do it on
>> initialization?  This doesn't make sense.
>
>
> Frequently.
>
> my_prefs = dict(default_prefs, setting=True, another_setting=False)
>
>
> Notice that I'm not merging one dict into another, but merging two dicts
> into a third.

Side point: Wouldn't it be quite logical to support dict addition?

>>> {"a":1}+{"b":2}
Traceback (most recent call last):
  File "<pyshell#59>", line 1, in <module>
    {"a":1}+{"b":2}
TypeError: unsupported operand type(s) for +: 'dict' and 'dict'

It would make sense for this to result in {"a":1,"b":2}.

ChrisA

From tjreedy at udel.edu  Thu Nov 15 00:47:50 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 14 Nov 2012 18:47:50 -0500
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A36065.9010704@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
Message-ID: <k81ajk$m1v$1@ger.gmane.org>

On 11/14/2012 4:12 AM, Chris Withers wrote:
To somewhat paraphrase: '''
I prefer 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)' to
"{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}".
I am sad that the former takes +-2 times as long to run (in 2.7).
Is the difference about the same in 3.x?
What can we do to speed up the former case?
'''

My responses, trying not to duplicate others.
1. Visual preference depends on the viewer. I prefer the dict display, 
perhaps because I am more accustomed to it.

2. The two types of expressions have overlapping but distinct use cases. 
This differences include that dict can be wrapped or replaced, while 
displays cannot.

3. a) 3.x has dict comprehensions. How do they stack up? b) If one were 
really initializing multiple dicts with the same starting items, and one 
were really concerned about speed, should one calculate the common base 
dict just once and then copy? Win7 64 with 3.3.0:

 >>> repeat("dict(a=1, b=2, c=3, d=4, e=5)")
[0.6200045004915467, 0.6212762582470646, 0.6114683222573376]
 >>> repeat("{'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5}")
[0.27170026972208233, 0.2594874604131968, 0.25977058559879584]
 >>> repeat("d.copy()", "d={'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5}")
[0.25768296004457625, 0.243041299387869, 0.2421860830290825]

 >>> repeat("{str(i):i for i in range(10)}")
[4.914327732926495, 4.874041570524014, 4.871596119002334]
 >>> repeat("{'0':0, '1':1, '2':2, '3':3, '4':4, '5':5, '6':6, '7':7, 
'8':8, '9':9}")
[0.5207065648769458, 0.5000415004344632, 0.49980294978922757]
 >>> repeat("d.copy()", "d={'0':0, '1':1, '2':2, '3':3, '4':4, '5':5, 
'6':6, '7':7, '8':8, '9':9}")
[0.571671864980317, 0.5516194699132484, 0.5514937389677925]


Assuming no overlooked errors in the above...
a) Dict comprehensions are much slower than calls, which makes the calls 
look good by comparison. b) Copying is not worthwhile.

4. There are about 3000 issues on the tracker. Nearly all are worth more 
attention than this ;-).

-- 
Terry Jan Reedy


From dreamingforward at gmail.com  Thu Nov 15 00:52:17 2012
From: dreamingforward at gmail.com (Mark Adam)
Date: Wed, 14 Nov 2012 17:52:17 -0600
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAPTjJmpJ66sndb4aZ52WiZWa2dv6iGg4Ozua26=YqjOQC113hQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<50A42AF8.1070807@pearwood.info>
	<CAPTjJmpJ66sndb4aZ52WiZWa2dv6iGg4Ozua26=YqjOQC113hQ@mail.gmail.com>
Message-ID: <CAMjeLr-J06yb_tQDTu=qt5SkRq+sHLTfZjzMY4dj3e7gVvtiCQ@mail.gmail.com>

On Wed, Nov 14, 2012 at 5:40 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Thu, Nov 15, 2012 at 10:36 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> On 15/11/12 05:54, Mark Adam wrote:
>> Notice that I'm not merging one dict into another, but merging two dicts
>> into a third.
>
> Side point: Wouldn't it be quite logical to support dict addition?
>

Yes, but then you'd be in my old argument that dict should inherit from set.

From phd at phdru.name  Wed Nov 14 14:25:00 2012
From: phd at phdru.name (Oleg Broytman)
Date: Wed, 14 Nov 2012 17:25:00 +0400
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
Message-ID: <20121114132500.GA14633@iskra.aviel.ru>

On Thu, Nov 15, 2012 at 12:18:38AM +1100, Chris Angelico <rosuav at gmail.com> wrote:
> On Wed, Nov 14, 2012 at 8:12 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> > I suspect I'm not the only one who finds:
> >
> > a_dict = dict(
> >     x = 1,
> >     y = 2,
> >     z = 3,
> >     ...
> >     )
> >
> > ...easier to read than:
> >
> > a_dict = {
> >     'x':1,
> >     'y':2,
> >     'z':3,
> >     ...
> >     }
> >
> > What can we do to speed up the former case?
> 
> Perhaps an alternative question: What can be done to make the latter
> less unpalatable? I personally prefer dict literal syntax to a dict
> constructor call, but no doubt there are a number of people who feel
> as you do. In what way(s) do you find the literal syntax less
> readable, and can some simple (and backward-compatible) enhancements
> help that?
> 
> I've seen criticisms (though I don't recall where) of Python,
> comparing it to JavaScript/ECMAScript, that complain of the need to
> quote the keys. IMO this is a worthwhile downside, as it allows you to
> use variables as the keys, rather than requiring (effectively) literal
> strings. But it does make a dict literal that much more "noisy" than
> the constructor.

   On the other had it's more powerful. You can write {'class': 'foo'}
but cannot dict(class='bar'). {1: '1'} but not dict(1='1').

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From stephen at xemacs.org  Thu Nov 15 03:28:45 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 15 Nov 2012 11:28:45 +0900
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAPTjJmpJ66sndb4aZ52WiZWa2dv6iGg4Ozua26=YqjOQC113hQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<50A42AF8.1070807@pearwood.info>
	<CAPTjJmpJ66sndb4aZ52WiZWa2dv6iGg4Ozua26=YqjOQC113hQ@mail.gmail.com>
Message-ID: <874nkr8uqq.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Angelico writes:

 > >>> {"a":1}+{"b":2}

 > It would make sense for this to result in {"a":1,"b":2}.

The test is not "does this sometimes make sense?"  It's "does this
ever result in nonsense, and if so, do we care?"

Here, addition is usually commutative.  Should {'a':1}+{'a':2} be the
same as, or different from, {'a':2}+{'a':1}, or should it be an error?

From rosuav at gmail.com  Thu Nov 15 03:35:46 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 15 Nov 2012 13:35:46 +1100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <874nkr8uqq.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<50A42AF8.1070807@pearwood.info>
	<CAPTjJmpJ66sndb4aZ52WiZWa2dv6iGg4Ozua26=YqjOQC113hQ@mail.gmail.com>
	<874nkr8uqq.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPTjJmrzq11TrkTfVvAJoVw_HOCKHC9o56Y4zsjAY-Z=9qRTUQ@mail.gmail.com>

On Thu, Nov 15, 2012 at 1:28 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Chris Angelico writes:
>
>  > >>> {"a":1}+{"b":2}
>
>  > It would make sense for this to result in {"a":1,"b":2}.
>
> The test is not "does this sometimes make sense?"  It's "does this
> ever result in nonsense, and if so, do we care?"
>
> Here, addition is usually commutative.  Should {'a':1}+{'a':2} be the
> same as, or different from, {'a':2}+{'a':1}, or should it be an error?

>>> "a"+"b"
'ab'
>>> "b"+"a"
'ba'

I would say that the two dictionary examples are equally allowed to
give different results - that they should be equivalent to (shallow)
copy followed by update(), but possibly more efficiently.

ChrisA

From dreamingforward at gmail.com  Thu Nov 15 04:24:07 2012
From: dreamingforward at gmail.com (Mark Adam)
Date: Wed, 14 Nov 2012 21:24:07 -0600
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <874nkr8uqq.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<50A42AF8.1070807@pearwood.info>
	<CAPTjJmpJ66sndb4aZ52WiZWa2dv6iGg4Ozua26=YqjOQC113hQ@mail.gmail.com>
	<874nkr8uqq.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAMjeLr_fftJbBtiWkezSuge7cEdzVFWQk2C9z2rL2-54OKOh=Q@mail.gmail.com>

On Wed, Nov 14, 2012 at 8:28 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Chris Angelico writes:
>
>  > >>> {"a":1}+{"b":2}
>
>  > It would make sense for this to result in {"a":1,"b":2}.
>
> The test is not "does this sometimes make sense?"  It's "does this
> ever result in nonsense, and if so, do we care?"
>
> Here, addition is usually commutative.  Should {'a':1}+{'a':2} be the
> same as, or different from, {'a':2}+{'a':1}, or should it be an error?

Easy:  dict should have a (user substitutable) collision function that
is called in these cases.   This would allow significant functionality
with practically no cost.  In addition, it could be implemented in
such a way as to offer significant speedups (when using dict.update
for example) over any possible hand-written substitutes (since it's
only run on key collisions and otherwise uses an underlying loop coded
in C).

mark

From stephen at xemacs.org  Thu Nov 15 04:48:22 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 15 Nov 2012 12:48:22 +0900
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAPTjJmrzq11TrkTfVvAJoVw_HOCKHC9o56Y4zsjAY-Z=9qRTUQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<50A42AF8.1070807@pearwood.info>
	<CAPTjJmpJ66sndb4aZ52WiZWa2dv6iGg4Ozua26=YqjOQC113hQ@mail.gmail.com>
	<874nkr8uqq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAPTjJmrzq11TrkTfVvAJoVw_HOCKHC9o56Y4zsjAY-Z=9qRTUQ@mail.gmail.com>
Message-ID: <87zk2j7chl.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Angelico writes:
 > On Thu, Nov 15, 2012 at 1:28 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
 > > Chris Angelico writes:
 > >
 > >  > >>> {"a":1}+{"b":2}
 > >
 > >  > It would make sense for this to result in {"a":1,"b":2}.
 > >
 > > The test is not "does this sometimes make sense?"  It's "does this
 > > ever result in nonsense, and if so, do we care?"
 > >
 > > Here, addition is usually commutative.  Should {'a':1}+{'a':2} be the
 > > same as, or different from, {'a':2}+{'a':1}, or should it be an error?
 > 
 > >>> "a"+"b"
 > 'ab'
 > >>> "b"+"a"
 > 'ba'
 > 
 > I would say that the two dictionary examples are equally allowed to
 > give different results - that they should be equivalent to (shallow)
 > copy followed by update(), but possibly more efficiently.

I wouldn't.  A string is a sequence of uninterpreted letters, and
necessarily ordered.  In fact, that's about all you can say about
strings in general.  I would prefer that concatenation be expressed by
juxtaposition, but that's troublesome for machine parsing (especially
error recovery).  My intuition is elastic enough to admit exceptional
cases where the essential ordered nature of the objects being "added"
is more important than the customary interpretation of the operator
symbol, so interpreting string addition as concatenation doesn't
bother me.  Furthermore, in string addition both operands affect the
result in proportion to their content, though differently.

Dictionaries aren't ordered, and their "elements" have structure
(key-value pairs).  It would definitely bother me if dictionary
addition weren't commutative, and it's worse that an operand affects
the outcome in an all-or-nothing way.

Also, "update" is more appropriately expressed by an extended
assignment operator.  Defining "+" in terms of "+=" as you propose
just doesn't seem right to me.


From stephen at xemacs.org  Thu Nov 15 05:11:24 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 15 Nov 2012 13:11:24 +0900
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAMjeLr_fftJbBtiWkezSuge7cEdzVFWQk2C9z2rL2-54OKOh=Q@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<50A42AF8.1070807@pearwood.info>
	<CAPTjJmpJ66sndb4aZ52WiZWa2dv6iGg4Ozua26=YqjOQC113hQ@mail.gmail.com>
	<874nkr8uqq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAMjeLr_fftJbBtiWkezSuge7cEdzVFWQk2C9z2rL2-54OKOh=Q@mail.gmail.com>
Message-ID: <87vcd77bf7.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark Adam writes:

 > Easy:  dict should have a (user substitutable) collision function that
 > is called in these cases.

"I smell overengineering."

 > This would allow significant functionality with practically no
 > cost.

We already have that functionality if we want it; just define an
appropriate mapping class.

I don't need or want it, so I can ignore it, but I suspect to get
anywhere with this proposal you're going to need to show that this
"significant functionality" needs to be in syntax.

From martin at v.loewis.de  Thu Nov 15 05:19:31 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Thu, 15 Nov 2012 05:19:31 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A41E18.1050102@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
	<50A40FE0.9050109@canterbury.ac.nz> <50A41E18.1050102@simplistix.co.uk>
Message-ID: <20121115051931.Horde.liS5Mdjz9kRQpG1TeJazG-A@webmail.df.eu>


Zitat von Chris Withers <chris at simplistix.co.uk>:

> On 14/11/2012 21:40, Greg Ewing wrote:
>> * If the compiler were allowed to recognise builtins, it could
>> turn dict(a = 1, b = 2) into {'a':1, 'b':2} automatically.
>
> That would be my naive suggestion, I am prepared to be shot down in  
> flames ;-)

In general, special-casing builtins in the compiler is not possible
in Python. You cannot know statically that 'dict' really refers to
the builtin. Something may shadow the name at run-time, making dict
refer to some other callable.

Regards,
Martin



From martin at v.loewis.de  Thu Nov 15 05:27:23 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Thu, 15 Nov 2012 05:27:23 +0100
Subject: [Python-Dev] performance of {} versus dict(), de fmd(**kw):
 return kw trumps all ; -)
In-Reply-To: <50A41EA8.9040502@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
	<50A41D3A.9040904@simplistix.co.uk> <50A41EA8.9040502@simplistix.co.uk>
Message-ID: <20121115052723.Horde.smTDANjz9kRQpG8rXpwzJKA@webmail.df.eu>


Zitat von Chris Withers <chris at simplistix.co.uk>:

> $ python2.7 -m timeit -n 1000000 -r 5 -v  
> "{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}"
> raw times: 1.49 1.49 1.5 1.49 1.48
> 1000000 loops, best of 5: 1.48 usec per loop
>
> $ python2.7 -m timeit -n 1000000 -r 5 -v 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
> raw times: 2.35 2.36 2.41 2.42 2.35
> 1000000 loops, best of 5: 2.35 usec per loop
>
> $ python2.7 -m timeit -n 1000000 -r 5 -v 'def md(**kw): return kw;  
> md(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
> raw times: 0.507 0.515 0.516 0.529 0.524
> 1000000 loops, best of 5: 0.507 usec per loop
>
> For the naive observer (ie: me!), why is that?

It's faster than calling dict() because the dict code will
create a second dictionary, and discard the keywords dictionary.

It's (probably) faster than the dictionary display, because
the {} byte code builds the dictionary one-by-one, whereas
the keywords dictionary is built in a single step (taking
all keys and values from the evaluation stack).

Regards,
Martin



From martin at v.loewis.de  Thu Nov 15 06:04:57 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Thu, 15 Nov 2012 06:04:57 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <CAPTjJmrzq11TrkTfVvAJoVw_HOCKHC9o56Y4zsjAY-Z=9qRTUQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAMjeLr8jGwCmYU9mokr-nOM06-RyFJNbuD=z_rdfhYa_Hg8gbA@mail.gmail.com>
	<CAD+XWwpv8DjVXK6uq_9dUPXMYzzYJtbtX-Y-QUMtqT47u2m4NQ@mail.gmail.com>
	<CAMjeLr-+cDgvC3fujn99G8VJ1XrnZq4FgGFmPfSEDCkwFOuSxw@mail.gmail.com>
	<8C81B6EB-05FE-4C0E-BC6C-23400BBEE403@masklinn.net>
	<CAMjeLr9R4jks6A1vmAnWtuZ6=Gq6hM287bW_X86Yxgp5VGfCXQ@mail.gmail.com>
	<50A42AF8.1070807@pearwood.info>
	<CAPTjJmpJ66sndb4aZ52WiZWa2dv6iGg4Ozua26=YqjOQC113hQ@mail.gmail.com>
	<874nkr8uqq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAPTjJmrzq11TrkTfVvAJoVw_HOCKHC9o56Y4zsjAY-Z=9qRTUQ@mail.gmail.com>
Message-ID: <20121115060457.Horde.ugXBbKGZi1VQpHf5nyWTVWA@webmail.df.eu>


Zitat von Chris Angelico <rosuav at gmail.com>:

> On Thu, Nov 15, 2012 at 1:28 PM, Stephen J. Turnbull  
> <stephen at xemacs.org> wrote:
>> Chris Angelico writes:
>>
>>  > >>> {"a":1}+{"b":2}
>>
>>  > It would make sense for this to result in {"a":1,"b":2}.
>>
>> The test is not "does this sometimes make sense?"  It's "does this
>> ever result in nonsense, and if so, do we care?"
>>
>> Here, addition is usually commutative.  Should {'a':1}+{'a':2} be the
>> same as, or different from, {'a':2}+{'a':1}, or should it be an error?
>
>>>> "a"+"b"
> 'ab'
>>>> "b"+"a"
> 'ba'
>
> I would say that the two dictionary examples are equally allowed to
> give different results - that they should be equivalent to (shallow)
> copy followed by update(), but possibly more efficiently.

Can this be moved to python-ideas, please?

Regards,
Martin




From stefan_ml at behnel.de  Thu Nov 15 07:32:41 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 15 Nov 2012 07:32:41 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <37EDB7B5B89649AC9E0AA19C89A8C87E@gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
	<50A40FE0.9050109@canterbury.ac.nz>
	<50A42079.4060908@mrabarnett.plus.com>
	<37EDB7B5B89649AC9E0AA19C89A8C87E@gmail.com>
Message-ID: <k822a7$mmd$1@ger.gmane.org>

Donald Stufft, 15.11.2012 00:00:
> $ pypy -m timeit 'dict()'
> 1000000000 loops, best of 3: 0.000811 usec per loop
> 
> $ pypy -m timeit '{}'    
> 1000000000 loops, best of 3: 0.000809 usec per loop
> 
> $ pypy -m timeit 'def md(**kw): return kw; md()'
> 100000000 loops, best of 3: 0.0182 usec per loop
> 
> $ pypy -m timeit -s 'def md(**kw): return kw' 'md()'
> 1000000000 loops, best of 3: 0.00136 usec per loop

Yep, I really like the fact that optimisers can fold stupid benchmarks into
no-ops. I wonder why it fails so badly in the latter two cases, though. You
should bring that to the attention of the PyPy developers, they might want
to fix it.

Stefan



From chris at simplistix.co.uk  Thu Nov 15 08:14:42 2012
From: chris at simplistix.co.uk (Chris Withers)
Date: Thu, 15 Nov 2012 07:14:42 +0000
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <k822a7$mmd$1@ger.gmane.org>
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
	<50A40FE0.9050109@canterbury.ac.nz>
	<50A42079.4060908@mrabarnett.plus.com>
	<37EDB7B5B89649AC9E0AA19C89A8C87E@gmail.com>
	<k822a7$mmd$1@ger.gmane.org>
Message-ID: <50A49662.6030308@simplistix.co.uk>

On 15/11/2012 06:32, Stefan Behnel wrote:
> Donald Stufft, 15.11.2012 00:00:
>> $ pypy -m timeit 'dict()'
>> 1000000000 loops, best of 3: 0.000811 usec per loop
>>
>> $ pypy -m timeit '{}'
>> 1000000000 loops, best of 3: 0.000809 usec per loop
>>
>> $ pypy -m timeit 'def md(**kw): return kw; md()'
>> 100000000 loops, best of 3: 0.0182 usec per loop
>>
>> $ pypy -m timeit -s 'def md(**kw): return kw' 'md()'
>> 1000000000 loops, best of 3: 0.00136 usec per loop
>
> Yep, I really like the fact that optimisers can fold stupid benchmarks into
> no-ops. I wonder why it fails so badly in the latter two cases, though. You
> should bring that to the attention of the PyPy developers, they might want
> to fix it.

Agreed, but Donald, please try with a bunch of keys rather than an empty 
dict...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From stefan_ml at behnel.de  Thu Nov 15 08:22:57 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 15 Nov 2012 08:22:57 +0100
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <50A49662.6030308@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
	<50A40FE0.9050109@canterbury.ac.nz>
	<50A42079.4060908@mrabarnett.plus.com>
	<37EDB7B5B89649AC9E0AA19C89A8C87E@gmail.com>
	<k822a7$mmd$1@ger.gmane.org> <50A49662.6030308@simplistix.co.uk>
Message-ID: <k8258f$fbv$1@ger.gmane.org>

Chris Withers, 15.11.2012 08:14:
> On 15/11/2012 06:32, Stefan Behnel wrote:
>> Donald Stufft, 15.11.2012 00:00:
>>> $ pypy -m timeit 'dict()'
>>> 1000000000 loops, best of 3: 0.000811 usec per loop
>>>
>>> $ pypy -m timeit '{}'
>>> 1000000000 loops, best of 3: 0.000809 usec per loop
>>>
>>> $ pypy -m timeit 'def md(**kw): return kw; md()'
>>> 100000000 loops, best of 3: 0.0182 usec per loop
>>>
>>> $ pypy -m timeit -s 'def md(**kw): return kw' 'md()'
>>> 1000000000 loops, best of 3: 0.00136 usec per loop
>>
>> Yep, I really like the fact that optimisers can fold stupid benchmarks into
>> no-ops. I wonder why it fails so badly in the latter two cases, though. You
>> should bring that to the attention of the PyPy developers, they might want
>> to fix it.
> 
> Agreed, but Donald, please try with a bunch of keys rather than an empty
> dict...

Right. If that makes a difference, it's another bug.

Stefan



From storchaka at gmail.com  Thu Nov 15 08:24:21 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 15 Nov 2012 09:24:21 +0200
Subject: [Python-Dev] performance of {} versus dict()
In-Reply-To: <k81ajk$m1v$1@ger.gmane.org>
References: <50A36065.9010704@simplistix.co.uk> <k81ajk$m1v$1@ger.gmane.org>
Message-ID: <k825bl$gou$1@ger.gmane.org>

On 15.11.12 01:47, Terry Reedy wrote:
> 4. There are about 3000 issues on the tracker. Nearly all are worth more
> attention than this ;-).

This is the best conclusion of this thread.


From lrekucki at gmail.com  Thu Nov 15 10:22:50 2012
From: lrekucki at gmail.com (=?UTF-8?Q?=C5=81ukasz_Rekucki?=)
Date: Thu, 15 Nov 2012 10:22:50 +0100
Subject: [Python-Dev] performance of {} versus dict(), de fmd(**kw): return
 kw trumps all ; -)
In-Reply-To: <CAEZs-EKsvkhLVj=sryNdxOQTKkPHvgZGARTt6Aps-nvNc2i_sQ@mail.gmail.com>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
	<50A41D3A.9040904@simplistix.co.uk>
	<50A41EA8.9040502@simplistix.co.uk>
	<F70A48A3-C13A-4814-BE3C-6BE43CBDAAD9@masklinn.net>
	<CAEZs-EKsvkhLVj=sryNdxOQTKkPHvgZGARTt6Aps-nvNc2i_sQ@mail.gmail.com>
Message-ID: <CAEZs-E+AMsB3ySFo4yJjrsz-feBTTkmZTFEnNQ8RHs8FV8zEzQ@mail.gmail.com>

Hi,

I posted this (by accident) off the list:

> On 2012-11-14, at 23:43 , Chris Withers wrote:
>
>> On 14/11/2012 22:37, Chris Withers wrote:
>>> On 14/11/2012 10:11, martin at v.loewis.de wrote:
>>>> def xdict(**kwds):
>>>>   return kwds
>>>
>>> Hah, good call, this trumps both of the other options:
>>>
>>> 1000000 -r 5 -v 'def md(**kw): return kw; md(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
>>> raw times: 0.548 0.533 0.55 0.577 0.539
>>> 1000000 loops, best of 5: 0.533 usec per loop

No, this just doesn't execute the right code:

>>> def md(**kw): return kw; md(a=1,b=2,c=3,d=4,e=5,f=6,g=7)
...
>>> import dis
>>> dis.dis(md)
  1           0 LOAD_FAST                0 (kw)
              3 RETURN_VALUE
              4 LOAD_GLOBAL              0 (md)
              7 LOAD_CONST               1 ('a')
             10 LOAD_CONST               2 (1)
             13 LOAD_CONST               3 ('b')
             16 LOAD_CONST               4 (2)
             19 LOAD_CONST               5 ('c')
             22 LOAD_CONST               6 (3)
             25 LOAD_CONST               7 ('d')
             28 LOAD_CONST               8 (4)
             31 LOAD_CONST               9 ('e')
             34 LOAD_CONST              10 (5)
             37 LOAD_CONST              11 ('f')
             40 LOAD_CONST              12 (6)
             43 LOAD_CONST              13 ('g')
             46 LOAD_CONST              14 (7)
             49 CALL_FUNCTION         1792
             52 POP_TOP

Also:

Python 3.2.3 (default, Apr 11 2012, 07:12:16) [MSC v.1500 64 bit
(AMD64)] on win 32
Type "help", "copyright", "credits" or "license" for more information.
>>> dict({1: "foo"}, **{frozenset([2]): "bar"})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: keyword arguments must be strings

While:

Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> dict({1: "foo"}, **{2: "bar"})
{1: 'foo', 2: 'bar'}
>>> dict({1: "foo"}, **{frozenset([2]): "bar"})
{1: 'foo', frozenset([2]): 'bar'}

If you're worrying about global lookup, you should stop (in this case):

$ py -3.3 -m timeit -n 1000000 -r 5 -v -s "def xdict(): return dict()" "xdict()"
raw times: 0.477 0.47 0.468 0.473 0.469
1000000 loops, best of 5: 0.468 usec per loop

$ py -3.3 -m timeit -n 1000000 -r 5 -v -s "def xdict(dict=dict):
return dict()" "xdict()"
raw times: 0.451 0.45 0.451 0.45 0.449
1000000 loops, best of 5: 0.449 usec per loop

$ py -3.3 -m timeit -n 1000000 -r 5 -v -s "def xdict(dict=lambda **kw:
kw): return dict()" "xdict()"
raw times: 0.433 0.434 0.435 0.435 0.431
1000000 loops, best of 5: 0.431 usec per loop

$ py -3.3 -m timeit -n 1000000 -r 5 -v -s "def xdict(dict=dict):
return {}" "xdict()"
raw times: 0.276 0.279 0.279 0.277 0.275
1000000 loops, best of 5: 0.275 usec per loop

And using non-empty dicts doesn't change much and the first one is
roughly the sum of the latter two (as expected):

C:\Users\lrekucki>py -3.3  -m timeit -n 1000000 -r 5 -v -s "def
xdict(dict=dict): return dict(a=1, b=2, c=3, d=4, e=5, f=6)" "xdict()"
raw times: 1.72 1.71 1.71 1.71 1.71
1000000 loops, best of 5: 1.71 usec per loop

C:\Users\lrekucki>py -3.3  -m timeit -n 1000000 -r 5 -v -s "def
xdict(dict=lambda **kw: kw): return dict(a=1, b=2, c=3, d=4, e=5,
f=6)" "xdict()"
raw times: 1.01 1.01 1.01 1.01 1.01
1000000 loops, best of 5: 1.01 usec per loop

C:\Users\lrekucki>py -3.3  -m timeit -n 1000000 -r 5 -v -s "def
xdict(dict=dict): return {'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5, 'f':
6}" "xdict()"
raw times: 0.744 0.736 0.735 0.733 0.733
1000000 loops, best of 5: 0.733 usec per loop


I hope that this helps move python-dev's focus to some more useful discussion.

--
?ukasz Rekucki

From kristjan at ccpgames.com  Thu Nov 15 10:17:39 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Thu, 15 Nov 2012 09:17:39 +0000
Subject: [Python-Dev] Setting project home path the best way
In-Reply-To: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
Message-ID: <EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>

When python is being run from a compile environment, it detects this by looking for "Lib" folders in directories above the one containing the executable. 
(I always thought that this "special" execution mode, hardwired in, was a bit odd, and suggested that this could be made a function of pep405)
Anyway, keeping your executable as part of the tree is the trick I use, and to make things nice I put  right next to it:
site.py
sitecustomize.py

sitecustomize.py is where you would put the logic to set sys.path by walking up the hierarchy and finding the proper root.
site.py is there to merely import sitecustomize.py, in case a site.py is not found in all the default places python looks.

K


> -----Original Message-----
> From: Python-Dev [mailto:python-dev-
> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Christian Tismer
> Sent: 11. n?vember 2012 20:31
> To: python-dev at python.org
> Subject: [Python-Dev] Setting project home path the best way
> 
> Hi friends,
> 
> I have a project that has its root somewhere on my machine.
> This project has many folders and contains quite some modules.
> 
> There is a common root of the module tree, and I want to use
> - either absolute imports
> - relative imports with '.'
> 
> Problem:
> 
> - I want to run any module inside the heirarchy from the command-line
> 
> - this should work, regardless what my 'cwd' is
> 
> - this should work with or without virtualenv.
> 
> So far, things work fine with virtualenv, because sys.executable is in the
> project module tree.
> 
> Without virtualenv, this is not so. But I hate to make settings like
> PYTHONPATH, because these are not permanent. .
> 
> Question:
> 
> How should I define my project root dir in a unique way, without setting an
> environment variable?
> What is the lest intrusive way to spell that?
> 
> Reason:
> 
> I'd like to make things work correctly and unambigously when I call a script
> inside the module heirarchy. Things are not fixed: there exist many
> checkouts In the file system, and each should know where to search its
> home/root in the tree.
> 
> Is this elegantly possible to deduce from the actually executed script file?
> 
> Cheers - chris
> 
> Sent from my Ei4Steve
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-
> dev/kristjan%40ccpgames.com



From kristjan at ccpgames.com  Thu Nov 15 10:20:09 2012
From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=)
Date: Thu, 15 Nov 2012 09:20:09 +0000
Subject: [Python-Dev] externals?
In-Reply-To: <CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>

Okay, that means I need to re-install svn, cool.
But I should mention that this needs to be mentioned in the core development FAQs, setting up and so forth.
There is no mention of it there.  Perhaps the unix makefiles get the proper version, but a windows developer has to fetch those externals manually.

Also, is there any reason to keep this in svn?  Why not check this in to HG, we need not worry about history, etc.

K

> -----Original Message-----
> From: Benjamin Peterson [mailto:benjamin at python.org]
> Sent: 13. n?vember 2012 15:04
> To: Kristj?n Valur J?nsson
> Cc: Python-Dev (python-dev at python.org)
> Subject: Re: [Python-Dev] externals?
> 
> Their still in svn as far I know.
> 
> 2012/11/13 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> > This may be a silly question, but haven?t the python externals been
> > moved to HG yet?
> >
> > I usually work on cpython without bothering with the externals, but I
> > found today that I needed them.  On Windows this is a bit of a bother.
> > And I?ve thrown away all my SVN stuff...
> >
> > K
> >
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> > http://mail.python.org/mailman/options/python-
> dev/benjamin%40python.or
> > g
> >
> 
> 
> 
> --
> Regards,
> Benjamin


From greg.ewing at canterbury.ac.nz  Thu Nov 15 11:48:27 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 15 Nov 2012 23:48:27 +1300
Subject: [Python-Dev] performance of {} versus dict(), de fmd(**kw):
 return kw trumps all ; -)
In-Reply-To: <20121115052723.Horde.smTDANjz9kRQpG8rXpwzJKA@webmail.df.eu>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
	<50A41D3A.9040904@simplistix.co.uk> <50A41EA8.9040502@simplistix.co.uk>
	<20121115052723.Horde.smTDANjz9kRQpG8rXpwzJKA@webmail.df.eu>
Message-ID: <50A4C87B.7010401@canterbury.ac.nz>

martin at v.loewis.de wrote:
> It's faster than calling dict() because the dict code will
> create a second dictionary, and discard the keywords dictionary.

Perhaps in the case where dict() is called with keyword
args only, it could just return the passed-in keyword
dictionary instead of creating another one?

-- 
Greg

From sdl.web at gmail.com  Thu Nov 15 12:16:39 2012
From: sdl.web at gmail.com (Leo)
Date: Thu, 15 Nov 2012 19:16:39 +0800
Subject: [Python-Dev] [BUG] Trailing spaces in pretty-printed JSON
References: <m2k3uuri19.fsf@gmail.com>
	<33E0EAD5-8BEE-4574-ABEC-A342BDADA34D@masklinn.net>
Message-ID: <m24nkrjeug.fsf@gmail.com>

On 2012-10-14 02:06 +0800, Xavier Morel wrote:
> 1. Why didn't you report that on the tracker?

Reported: http://bugs.python.org/issue16476


From roundup-admin at psf.upfronthosting.co.za  Thu Nov 15 15:30:56 2012
From: roundup-admin at psf.upfronthosting.co.za (Python tracker)
Date: Thu, 15 Nov 2012 14:30:56 +0000
Subject: [Python-Dev] Failed issue tracker submission
Message-ID: <20121115143056.404191CCF3@psf.upfronthosting.co.za>


An unexpected error occurred during the processing
of your message. The tracker administrator is being
notified.
-------------- next part --------------
Return-Path: <python-dev at python.org>
X-Original-To: report at bugs.python.org
Delivered-To: roundup+tracker at psf.upfronthosting.co.za
Received: from mail.python.org (mail.python.org [82.94.164.166])
	by psf.upfronthosting.co.za (Postfix) with ESMTPS id 8F7E41C98F
	for <report at bugs.python.org>; Thu, 15 Nov 2012 15:30:55 +0100 (CET)
Received: from albatross.python.org (localhost [127.0.0.1])
	by mail.python.org (Postfix) with ESMTP id 3Y2Q3g2PtxzRjv
	for <report at bugs.python.org>; Thu, 15 Nov 2012 15:30:55 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901;
	t=1352989855; bh=X+inBqLoRVfh7Gtof4lWQ3GKcOPuRIb4f6aeDmPB+Z0=;
	h=MIME-Version:Content-Type:Content-Transfer-Encoding:From:To:
	 Subject:Message-Id:Date;
	b=bj7svLDU57Chaz18mDOtCNtAs4y0V8RO1R/xoKH8KOELcUKhqVHFdJPfiOKurVwc3
	 ZvP/i9O0oKNrXvACnjFnTi9Z9x8sW7kWvMxNKpWo4wQPwrXPn1RDhtRoIFZ9l98sbt
	 P8zJS8xwXS592hm1od1ZadbKgEaSw2BERQ6c1zsI=
Received: from localhost (HELO mail.python.org) (127.0.0.1)
  by albatross.python.org with SMTP; 15 Nov 2012 15:30:55 +0100
Received: from virt-7yvsjn.psf.osuosl.org (virt-7yvsjn.psf.osuosl.org [140.211.10.72])
	by mail.python.org (Postfix) with ESMTP
	for <report at bugs.python.org>; Thu, 15 Nov 2012 15:30:54 +0100 (CET)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
From: python-dev at python.org
To: report at bugs.python.org
Subject: [issue16144]
Message-Id: <3Y2Q3g2PtxzRjv at mail.python.org>
Date: Thu, 15 Nov 2012 15:30:55 +0100 (CET)

TmV3IGNoYW5nZXNldCBhOGNhMTQ5ODNhYjEgYnkgQW5kcmV3IFN2ZXRsb3YgaW4gYnJhbmNoICcz
LjMnOgpJc3N1ZSAjMTYxNDQ6IEZpeCBtaXNsZWFkaW5nIHNlbnRlbmNlIGluIHJlZmVyZW5jZS9p
bXBvcnQuCmh0dHA6Ly9oZy5weXRob24ub3JnL2NweXRob24vcmV2L2E4Y2ExNDk4M2FiMQoKCk5l
dyBjaGFuZ2VzZXQgOTk2MWEwZGFmY2M3IGJ5IEFuZHJldyBTdmV0bG92IGluIGJyYW5jaCAnZGVm
YXVsdCc6Ck1lcmdlIGlzc3VlICMxNjE0NDogRml4IG1pc2xlYWRpbmcgc2VudGVuY2UgaW4gcmVm
ZXJlbmNlL2ltcG9ydC4KaHR0cDovL2hnLnB5dGhvbi5vcmcvY3B5dGhvbi9yZXYvOTk2MWEwZGFm
Y2M3Cg==

From stefan_ml at behnel.de  Thu Nov 15 15:58:46 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 15 Nov 2012 15:58:46 +0100
Subject: [Python-Dev] performance of {} versus dict(), de fmd(**kw):
 return kw trumps all ; -)
In-Reply-To: <50A4C87B.7010401@canterbury.ac.nz>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
	<50A41D3A.9040904@simplistix.co.uk>
	<50A41EA8.9040502@simplistix.co.uk>
	<20121115052723.Horde.smTDANjz9kRQpG8rXpwzJKA@webmail.df.eu>
	<50A4C87B.7010401@canterbury.ac.nz>
Message-ID: <k82vv5$naj$1@ger.gmane.org>

Greg Ewing, 15.11.2012 11:48:
> martin at v.loewis.de wrote:
>> It's faster than calling dict() because the dict code will
>> create a second dictionary, and discard the keywords dictionary.
> 
> Perhaps in the case where dict() is called with keyword
> args only, it could just return the passed-in keyword
> dictionary instead of creating another one?

This should work as long as this still creates a copy of d at some point:

    d = {...}
    dict(**d)

Stefan



From alex.gaynor at gmail.com  Thu Nov 15 17:18:42 2012
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Thu, 15 Nov 2012 16:18:42 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?performance_of_=7B=7D_versus_dict=28=29?=
References: <50A36065.9010704@simplistix.co.uk>
	<CAPTjJmqcvN14+DM743jF3Ut+sq4YJL=OKpXONNEnhzyYjr+2gQ@mail.gmail.com>
	<50A40FE0.9050109@canterbury.ac.nz>
	<50A42079.4060908@mrabarnett.plus.com>
	<37EDB7B5B89649AC9E0AA19C89A8C87E@gmail.com>
	<k822a7$mmd$1@ger.gmane.org> <50A49662.6030308@simplistix.co.uk>
	<k8258f$fbv$1@ger.gmane.org>
Message-ID: <loom.20121115T171734-858@post.gmane.org>

Stefan Behnel <stefan_ml <at> behnel.de> writes:
> Right. If that makes a difference, it's another bug.
> 
> Stefan
> 
> 


It's fixed, with, I will note, fewer lines of code than many messages in this
thread:
https://bitbucket.org/pypy/pypy/changeset/c30cb1dcb7a9adc32548fd14274e4995

Alex


From tjreedy at udel.edu  Thu Nov 15 17:21:53 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 15 Nov 2012 11:21:53 -0500
Subject: [Python-Dev] performance of {} versus dict(), de fmd(**kw):
 return kw trumps all ; -)
In-Reply-To: <k82vv5$naj$1@ger.gmane.org>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
	<50A41D3A.9040904@simplistix.co.uk>
	<50A41EA8.9040502@simplistix.co.uk>
	<20121115052723.Horde.smTDANjz9kRQpG8rXpwzJKA@webmail.df.eu>
	<50A4C87B.7010401@canterbury.ac.nz> <k82vv5$naj$1@ger.gmane.org>
Message-ID: <k834r0$6v8$1@ger.gmane.org>

On 11/15/2012 9:58 AM, Stefan Behnel wrote:
> Greg Ewing, 15.11.2012 11:48:
>> martin at v.loewis.de wrote:
>>> It's faster than calling dict() because the dict code will
>>> create a second dictionary, and discard the keywords dictionary.
>>
>> Perhaps in the case where dict() is called with keyword
>> args only, it could just return the passed-in keyword
>> dictionary instead of creating another one?
>
> This should work as long as this still creates a copy of d at some point:
>
>      d = {...}
>      dict(**d)

I was thinking that CPython could check the ref count of the input 
keyword dict to determine whether it is newly created and can be 
returned or is pre-existing and must be copied. But it seems not so.

 >>> def d(**x): return sys.getrefcount(x)

 >>> import sys
 >>> d(a = 3)
2
 >>> d(**{'a': 3})
2
 >>> b = {'a': 3}
 >>> d(**b)
2

I was expecting 3 for the last one.

-- 
Terry Jan Reedy


From shibturn at gmail.com  Thu Nov 15 17:45:42 2012
From: shibturn at gmail.com (Richard Oudkerk)
Date: Thu, 15 Nov 2012 16:45:42 +0000
Subject: [Python-Dev] performance of {} versus dict(), de fmd(**kw):
 return kw trumps all ; -)
In-Reply-To: <k834r0$6v8$1@ger.gmane.org>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
	<50A41D3A.9040904@simplistix.co.uk>
	<50A41EA8.9040502@simplistix.co.uk>
	<20121115052723.Horde.smTDANjz9kRQpG8rXpwzJKA@webmail.df.eu>
	<50A4C87B.7010401@canterbury.ac.nz>
	<k82vv5$naj$1@ger.gmane.org> <k834r0$6v8$1@ger.gmane.org>
Message-ID: <k8367r$itn$1@ger.gmane.org>

On 15/11/2012 4:21pm, Terry Reedy wrote:
> I was thinking that CPython could check the ref count of the input
> keyword dict to determine whether it is newly created and can be
> returned or is pre-existing and must be copied. But it seems not so.
>
>  >>> def d(**x): return sys.getrefcount(x)
>
>  >>> import sys
>  >>> d(a = 3)
> 2
>  >>> d(**{'a': 3})
> 2
>  >>> b = {'a': 3}
>  >>> d(**b)
> 2
>
> I was expecting 3 for the last one.

Isn't it always newly created?

     >>> def f(**x): return x
     ...
     >>> b = {'a':3}
     >>> b is f(**b)
     False

-- 
Richard


From roundup-admin at psf.upfronthosting.co.za  Thu Nov 15 19:24:19 2012
From: roundup-admin at psf.upfronthosting.co.za (Python tracker)
Date: Thu, 15 Nov 2012 18:24:19 +0000
Subject: [Python-Dev] Failed issue tracker submission
Message-ID: <20121115182419.950971CE11@psf.upfronthosting.co.za>


An unexpected error occurred during the processing
of your message. The tracker administrator is being
notified.
-------------- next part --------------
Return-Path: <python-dev at python.org>
X-Original-To: report at bugs.python.org
Delivered-To: roundup+tracker at psf.upfronthosting.co.za
Received: from mail.python.org (mail.python.org [IPv6:2001:888:2000:d::a6])
	by psf.upfronthosting.co.za (Postfix) with ESMTPS id 5B78D1CC04
	for <report at bugs.python.org>; Thu, 15 Nov 2012 19:24:19 +0100 (CET)
Received: from albatross.python.org (localhost [127.0.0.1])
	by mail.python.org (Postfix) with ESMTP id 3Y2WDz1BQtzRhX
	for <report at bugs.python.org>; Thu, 15 Nov 2012 19:24:19 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901;
	t=1353003859; bh=GDDVQWyvA249FMc1twsoY4VfCXH15D/J8oGHrHuD8Og=;
	h=MIME-Version:Content-Type:Content-Transfer-Encoding:From:To:
	 Subject:Message-Id:Date;
	b=sMId1I1AgUhjGpLVNulgs6JADkCmjcEvAyX34oy66KWe8KvbSTNxig9wD53MNGnso
	 vUA4DbpD2v2oRy/Udvmgn2Tqap/lZpo/RPxjT+eivdsvTWOpbFUE8401pNmG/f316p
	 4wis2s3IIGJv1jpw9e4fuBwbCi5iSDyB3HYYaKjI=
Received: from localhost (HELO mail.python.org) (127.0.0.1)
  by albatross.python.org with SMTP; 15 Nov 2012 19:24:19 +0100
Received: from virt-7yvsjn.psf.osuosl.org (virt-7yvsjn.psf.osuosl.org [140.211.10.72])
	by mail.python.org (Postfix) with ESMTP
	for <report at bugs.python.org>; Thu, 15 Nov 2012 19:24:18 +0100 (CET)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
From: python-dev at python.org
To: report at bugs.python.org
Subject: [issue16481]
Message-Id: <3Y2WDz1BQtzRhX at mail.python.org>
Date: Thu, 15 Nov 2012 19:24:19 +0100 (CET)

TmV3IGNoYW5nZXNldCBjNTc0Y2U3OGNkNjEgYnkgUmljaGFyZCBPdWRrZXJrIGluIGJyYW5jaCAn
My4zJzoKSXNzdWUgIzE2NDgxOiBtdWx0aXByb2Nlc3Npbmcgbm8gbG9uZ2VyIGxlYWtzIHByb2Nl
c3MgaGFuZGxlcyBvbiBXaW5kb3dzLgpodHRwOi8vaGcucHl0aG9uLm9yZy9jcHl0aG9uL3Jldi9j
NTc0Y2U3OGNkNjEKCgpOZXcgY2hhbmdlc2V0IGNiNjEyYzVmMzBjYiBieSBSaWNoYXJkIE91ZGtl
cmsgaW4gYnJhbmNoICdkZWZhdWx0JzoKSXNzdWUgIzE2NDgxOiBNZXJnZQpodHRwOi8vaGcucHl0
aG9uLm9yZy9jcHl0aG9uL3Jldi9jYjYxMmM1ZjMwY2IK

From roundup-admin at psf.upfronthosting.co.za  Thu Nov 15 21:58:51 2012
From: roundup-admin at psf.upfronthosting.co.za (Python tracker)
Date: Thu, 15 Nov 2012 20:58:51 +0000
Subject: [Python-Dev] Failed issue tracker submission
Message-ID: <20121115205851.ADB351CDAD@psf.upfronthosting.co.za>


An unexpected error occurred during the processing
of your message. The tracker administrator is being
notified.
-------------- next part --------------
Return-Path: <python-dev at python.org>
X-Original-To: report at bugs.python.org
Delivered-To: roundup+tracker at psf.upfronthosting.co.za
Received: from mail.python.org (mail.python.org [82.94.164.166])
	by psf.upfronthosting.co.za (Postfix) with ESMTPS id 728881C884
	for <report at bugs.python.org>; Thu, 15 Nov 2012 21:58:51 +0100 (CET)
Received: from albatross.python.org (localhost [127.0.0.1])
	by mail.python.org (Postfix) with ESMTP id 3Y2ZgH1WglzRlk
	for <report at bugs.python.org>; Thu, 15 Nov 2012 21:58:51 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901;
	t=1353013131; bh=OUB+ieZrhXLf00fS3ScDXJOs2fZdncpLHhVH8Mq4aG8=;
	h=MIME-Version:Content-Type:Content-Transfer-Encoding:From:To:
	 Subject:Message-Id:Date;
	b=bxgo73yXi3M9v/34ihpVbL175HI0avNqRJ9ZZPR8RJ/Ebf+s6CLSY6KSFvEqsey6o
	 k9VOuwgxEoJ6EPYG9b80rF0TfIkaPS/ly2PkD0oDeL2MONpJ/T+mCqEYegdK16I1mO
	 QP53t81gJuW1rACwgBViWQfoGh0NTebNRHzjE2zw=
Received: from localhost (HELO mail.python.org) (127.0.0.1)
  by albatross.python.org with SMTP; 15 Nov 2012 21:58:51 +0100
Received: from virt-7yvsjn.psf.osuosl.org (virt-7yvsjn.psf.osuosl.org [140.211.10.72])
	by mail.python.org (Postfix) with ESMTP
	for <report at bugs.python.org>; Thu, 15 Nov 2012 21:58:50 +0100 (CET)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
From: python-dev at python.org
To: report at bugs.python.org
Subject: [issue16469]
Message-Id: <3Y2ZgH1WglzRlk at mail.python.org>
Date: Thu, 15 Nov 2012 21:58:51 +0100 (CET)

TmV3IGNoYW5nZXNldCBhMmI1NGI2ZDk3NTkgYnkgTWFyayBEaWNraW5zb24gaW4gYnJhbmNoICdk
ZWZhdWx0JzoKSXNzdWUgIzE2NDY5OiBGcmFjdGlvbihmbG9hdCgnbmFuJykpIGFuZCBGcmFjdGlv
bihmbG9hdCgnaW5mJykpIG5vdyByYWlzZSBWYWx1ZUVycm9yIGFuZCBPdmVyZmxvd0Vycm9yIChy
ZXNwLiksIG5vdCBUeXBlRXJyb3IuCmh0dHA6Ly9oZy5weXRob24ub3JnL2NweXRob24vcmV2L2Ey
YjU0YjZkOTc1OQo=

From tismer at stackless.com  Thu Nov 15 22:43:31 2012
From: tismer at stackless.com (Christian Tismer)
Date: Thu, 15 Nov 2012 22:43:31 +0100
Subject: [Python-Dev] Setting project home path the best way
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <2D5A4C65-85AD-463C-9C6B-263C4C36EFAA@stackless.com>

Hi Kristjan,

does that mean that your scheme simply works, without any config step
necessary after I did my checkout?
This would in fact be an interesting alternative to 

    Python setup.py develop

but I'm not sure if this is the same scheme on windows and Os X. 

Getting this part right was rather tricky, and I fear this is still an issue. 

Right now I think to just force my users to run the install step, since it is quite
accepted in general. 

Still, I'd love to see a way with no action needed at all: write yout your structure,
and it works as-is. Seems to be impossible without tricks. 

Cheers - chris

Sent from my Ei4Steve

On Nov 15, 2012, at 10:17, Kristj?n Valur J?nsson <kristjan at ccpgames.com> wrote:

> When python is being run from a compile environment, it detects this by looking for "Lib" folders in directories above the one containing the executable. 
> (I always thought that this "special" execution mode, hardwired in, was a bit odd, and suggested that this could be made a function of pep405)
> Anyway, keeping your executable as part of the tree is the trick I use, and to make things nice I put  right next to it:
> site.py
> sitecustomize.py
> 
> sitecustomize.py is where you would put the logic to set sys.path by walking up the hierarchy and finding the proper root.
> site.py is there to merely import sitecustomize.py, in case a site.py is not found in all the default places python looks.
> 
> K
> 
> 
>> -----Original Message-----
>> From: Python-Dev [mailto:python-dev-
>> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Christian Tismer
>> Sent: 11. n?vember 2012 20:31
>> To: python-dev at python.org
>> Subject: [Python-Dev] Setting project home path the best way
>> 
>> Hi friends,
>> 
>> I have a project that has its root somewhere on my machine.
>> This project has many folders and contains quite some modules.
>> 
>> There is a common root of the module tree, and I want to use
>> - either absolute imports
>> - relative imports with '.'
>> 
>> Problem:
>> 
>> - I want to run any module inside the heirarchy from the command-line
>> 
>> - this should work, regardless what my 'cwd' is
>> 
>> - this should work with or without virtualenv.
>> 
>> So far, things work fine with virtualenv, because sys.executable is in the
>> project module tree.
>> 
>> Without virtualenv, this is not so. But I hate to make settings like
>> PYTHONPATH, because these are not permanent. .
>> 
>> Question:
>> 
>> How should I define my project root dir in a unique way, without setting an
>> environment variable?
>> What is the lest intrusive way to spell that?
>> 
>> Reason:
>> 
>> I'd like to make things work correctly and unambigously when I call a script
>> inside the module heirarchy. Things are not fixed: there exist many
>> checkouts In the file system, and each should know where to search its
>> home/root in the tree.
>> 
>> Is this elegantly possible to deduce from the actually executed script file?
>> 
>> Cheers - chris
>> 
>> Sent from my Ei4Steve
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-
>> dev/kristjan%40ccpgames.com
> 
> <winmail.dat>

From greg.ewing at canterbury.ac.nz  Thu Nov 15 23:39:44 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 16 Nov 2012 11:39:44 +1300
Subject: [Python-Dev] performance of {} versus dict(), de fmd(**kw):
 return kw trumps all ; -)
In-Reply-To: <k82vv5$naj$1@ger.gmane.org>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
	<50A41D3A.9040904@simplistix.co.uk> <50A41EA8.9040502@simplistix.co.uk>
	<20121115052723.Horde.smTDANjz9kRQpG8rXpwzJKA@webmail.df.eu>
	<50A4C87B.7010401@canterbury.ac.nz> <k82vv5$naj$1@ger.gmane.org>
Message-ID: <50A56F30.5000608@canterbury.ac.nz>

Stefan Behnel wrote:
> This should work as long as this still creates a copy of d at some point:
> 
>     d = {...}
>     dict(**d)

It will -- the implementation of the function call opcode always
creates a new keyword dict for passing to the called function.

-- 
Greg

From tismer at stackless.com  Fri Nov 16 00:10:09 2012
From: tismer at stackless.com (Christian Tismer)
Date: Fri, 16 Nov 2012 00:10:09 +0100
Subject: [Python-Dev] Generally boared by installation (Re: Setting project
	home path the best way)
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>

Hi guys,

I am bored of installing things. 
Bored of things that happen to not work for some minor reasons. 
Reasons that are not immediately obvious. 
Things that don't work because some special case was not handled. 
Things that compile for half an hour and then complain that something is not as expected. 
May it be a compiler, a library, a command like pip or easy-install, a system like macports or homebrew,
virtualenv, whatsoever. 

These things are all great if they work. 

When they do not work, the user is in some real trouble. And he reads hundreds
Of blogs and sites and emails, which all answer a bit of slightly related questions, but all in all - 

This is not how Python should work !!

I am really bored and exhausted and annoyed by those packages which
Pretend to make my life eadier, but they don't really. 

Something is really missing. I want something that is easy to use in all
cases, also if it fails. 

Son't get me wrong, I like things like pip or virtualenv or homebrew. 
I just think they have to be rewritten completely. They have the wrong assumption that things work!

The opposite should be the truth: by default, things go wrong. Correctness is very fragile. 

I am thinking of a better concept that is harder to break. I thin to design a setup tool that is much more checking itself and does not trust in any assumption. 

Scenario:
After hours and hours, I find how to modify setup.py to function almost correctly for PySide. 

This was ridiculously hard to do! Settings for certain directories, included and stuff are not checked when they could be, but after compiling a lot of things!

After a lot of tries and headaches, I find out that virtualenv barfs on a simple
link like ./.Python, the executable, when switching from stock Python to a different (homebrew) version!!

This was obviously never tested well, so it frustrates me quite a lot.  

I could fill a huge list full of complaints like that if I had time. But I don't. 

Instead, I think installation scripts are generally still wrong by concept today and need to
be written in a different way. 

I would like to start a task force and some sprints about improving this
situation. 
My goal is some unbreakable system of building blocks that are self-contained with no dependencies, that have a defined interface to talk to, and that know themselves completely by introspection. 

They should not work because they happen to work around all known defects, but by design and control. 

Whoever is interested to work with me on this is hereby honestly welcomed!

Cheers - chris

Sent from my Ei4Steve

On Nov 15, 2012, at 10:17, Kristj?n Valur J?nsson <kristjan at ccpgames.com> wrote:

> When python is being run from a compile environment, it detects this by looking for "Lib" folders in directories above the one containing the executable. 
> (I always thought that this "special" execution mode, hardwired in, was a bit odd, and suggested that this could be made a function of pep405)
> Anyway, keeping your executable as part of the tree is the trick I use, and to make things nice I put  right next to it:
> site.py
> sitecustomize.py
> 
> sitecustomize.py is where you would put the logic to set sys.path by walking up the hierarchy and finding the proper root.
> site.py is there to merely import sitecustomize.py, in case a site.py is not found in all the default places python looks.
> 
> K
> 
> 
>> -----Original Message-----
>> From: Python-Dev [mailto:python-dev-
>> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Christian Tismer
>> Sent: 11. n?vember 2012 20:31
>> To: python-dev at python.org
>> Subject: [Python-Dev] Setting project home path the best way
>> 
>> Hi friends,
>> 
>> I have a project that has its root somewhere on my machine.
>> This project has many folders and contains quite some modules.
>> 
>> There is a common root of the module tree, and I want to use
>> - either absolute imports
>> - relative imports with '.'
>> 
>> Problem:
>> 
>> - I want to run any module inside the heirarchy from the command-line
>> 
>> - this should work, regardless what my 'cwd' is
>> 
>> - this should work with or without virtualenv.
>> 
>> So far, things work fine with virtualenv, because sys.executable is in the
>> project module tree.
>> 
>> Without virtualenv, this is not so. But I hate to make settings like
>> PYTHONPATH, because these are not permanent. .
>> 
>> Question:
>> 
>> How should I define my project root dir in a unique way, without setting an
>> environment variable?
>> What is the lest intrusive way to spell that?
>> 
>> Reason:
>> 
>> I'd like to make things work correctly and unambigously when I call a script
>> inside the module heirarchy. Things are not fixed: there exist many
>> checkouts In the file system, and each should know where to search its
>> home/root in the tree.
>> 
>> Is this elegantly possible to deduce from the actually executed script file?
>> 
>> Cheers - chris
>> 
>> Sent from my Ei4Steve
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-
>> dev/kristjan%40ccpgames.com
> 
> <winmail.dat>

From doug.hellmann at gmail.com  Fri Nov 16 00:17:19 2012
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Thu, 15 Nov 2012 18:17:19 -0500
Subject: [Python-Dev] performance of {} versus dict(), de fmd(**kw):
	return kw trumps all ; -)
In-Reply-To: <50A41D3A.9040904@simplistix.co.uk>
References: <50A36065.9010704@simplistix.co.uk>
	<20121114111139.Horde.PXapOElCcOxQo25burhGAmA@webmail.df.eu>
	<50A41D3A.9040904@simplistix.co.uk>
Message-ID: <386E45A9-3B0A-43CE-9197-23F89FA70B98@gmail.com>


On Nov 14, 2012, at 5:37 PM, Chris Withers wrote:

> On 14/11/2012 10:11, martin at v.loewis.de wrote:
>> 
>> Zitat von Chris Withers <chris at simplistix.co.uk>:
>> 
>>> a_dict = dict(
>>>    x = 1,
>>>    y = 2,
>>>    z = 3,
>>>    ...
>>>    )
>> 
>>> What can we do to speed up the former case?
>> 
>> It should be possible to special-case it. Rather than creating
>> a new dictionary from scratch, one could try to have the new dictionary
>> the same size as the original one, and copy all entries.
> 
> Indeed, Doug, what are your views on this? Also, did you have a real-world example where this speed difference was causing you a problem?

No, not particularly. I noticed people using dict() and wondered what impact it might have in a general case.

> 
>> I don't know how much this would gain, though. You still have to
>> create two dictionary objects. For a better speedup, try
>> 
>> def xdict(**kwds):
>>   return kwds
> 
> Hah, good call, this trumps both of the other options:
> 
> $ python2.7 -m timeit -n 1000000 -r 5 -v "{'a':1,'b':2,'c':3,'d':4,'e':5,'f':6,'g':7}"
> raw times: 1.45 1.45 1.44 1.45 1.45
> 1000000 loops, best of 5: 1.44 usec per loop
> $ python2.6 -m timeit -n 1000000 -r 5 -v 'dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
> raw times: 2.37 2.36 2.36 2.37 2.37
> 1000000 loops, best of 5: 2.36 usec per loop$ python2.6 -m timeit -n 1000000 -r 5 -v 'def md(**kw): return kw; md(a=1,b=2,c=3,d=4,e=5,f=6,g=7)'
> raw times: 0.548 0.533 0.55 0.577 0.539
> 1000000 loops, best of 5: 0.533 usec per loop
> 
> For the naive observer (ie: me!), why is that?
> 
> Chris
> 
> -- 
> Simplistix - Content Management, Batch Processing & Python Consulting
>           - http://www.simplistix.co.uk


From roundup-admin at psf.upfronthosting.co.za  Fri Nov 16 03:39:50 2012
From: roundup-admin at psf.upfronthosting.co.za (Python tracker)
Date: Fri, 16 Nov 2012 02:39:50 +0000
Subject: [Python-Dev] Failed issue tracker submission
Message-ID: <20121116023950.2EC2B1CDAD@psf.upfronthosting.co.za>


An unexpected error occurred during the processing
of your message. The tracker administrator is being
notified.
-------------- next part --------------
Return-Path: <python-dev at python.org>
X-Original-To: report at bugs.python.org
Delivered-To: roundup+tracker at psf.upfronthosting.co.za
Received: from mail.python.org (mail.python.org [IPv6:2001:888:2000:d::a6])
	by psf.upfronthosting.co.za (Postfix) with ESMTPS id DFD211C98E
	for <report at bugs.python.org>; Fri, 16 Nov 2012 03:39:49 +0100 (CET)
Received: from albatross.python.org (localhost [127.0.0.1])
	by mail.python.org (Postfix) with ESMTP id 3Y2kDj4Tk8zRb6
	for <report at bugs.python.org>; Fri, 16 Nov 2012 03:39:49 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901;
	t=1353033589; bh=H41Haza/UPvj9B16qvXQtqjlb22jRmazur238MqAPOE=;
	h=MIME-Version:Content-Type:Content-Transfer-Encoding:From:To:
	 Subject:Message-Id:Date;
	b=yz1YIEtIYuM3H9V1Ok0YJ/SDvU8xtO+cFOApgDH8jPRdxRS6z/CSdZ96bSY1cNFM7
	 4xxKY3LiUXipe439DPwEjxZr0HcsfX+2JdR4Pzf3OZmdopV6PEl8/twaIFoTHQMy8u
	 dTwpZ7G4OJKCc2IeCq4e5Bl/TEvQvVT9NO8eoa3k=
Received: from localhost (HELO mail.python.org) (127.0.0.1)
  by albatross.python.org with SMTP; 16 Nov 2012 03:39:49 +0100
Received: from virt-7yvsjn.psf.osuosl.org (virt-7yvsjn.psf.osuosl.org [140.211.10.72])
	by mail.python.org (Postfix) with ESMTP
	for <report at bugs.python.org>; Fri, 16 Nov 2012 03:39:49 +0100 (CET)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
From: python-dev at python.org
To: report at bugs.python.org
Subject: [issue15894]
Message-Id: <3Y2kDj4Tk8zRb6 at mail.python.org>
Date: Fri, 16 Nov 2012 03:39:49 +0100 (CET)

TmV3IGNoYW5nZXNldCBiZDg1MzMxMWZmZTAgYnkgQnJldHQgQ2Fubm9uIGluIGJyYW5jaCAnZGVm
YXVsdCc6Cklzc3VlICMxNTg5NDogRG9jdW1lbnQgd2h5IHdlIGRvbid0IHdvcnJ5IGFib3V0IHJl
LWFjcXVpcmluZyB0aGUKaHR0cDovL2hnLnB5dGhvbi5vcmcvY3B5dGhvbi9yZXYvYmQ4NTMzMTFm
ZmUwCg==

From dholth at gmail.com  Fri Nov 16 04:22:27 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 15 Nov 2012 22:22:27 -0500
Subject: [Python-Dev] Setting project home path the best way
In-Reply-To: <2D5A4C65-85AD-463C-9C6B-263C4C36EFAA@stackless.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2D5A4C65-85AD-463C-9C6B-263C4C36EFAA@stackless.com>
Message-ID: <72466DBD-CFF8-4FAD-8878-46984675A705@gmail.com>

Are you familiar with executing directories having __main__.py as python scripts?

Daniel Holth

On Nov 15, 2012, at 4:43 PM, Christian Tismer <tismer at stackless.com> wrote:

> Hi Kristjan,
> 
> does that mean that your scheme simply works, without any config step
> necessary after I did my checkout?
> This would in fact be an interesting alternative to 
> 
>    Python setup.py develop
> 
> but I'm not sure if this is the same scheme on windows and Os X. 
> 
> Getting this part right was rather tricky, and I fear this is still an issue. 
> 
> Right now I think to just force my users to run the install step, since it is quite
> accepted in general. 
> 
> Still, I'd love to see a way with no action needed at all: write yout your structure,
> and it works as-is. Seems to be impossible without tricks. 
> 
> Cheers - chris
> 
> Sent from my Ei4Steve
> 
> On Nov 15, 2012, at 10:17, Kristj?n Valur J?nsson <kristjan at ccpgames.com> wrote:
> 
>> When python is being run from a compile environment, it detects this by looking for "Lib" folders in directories above the one containing the executable. 
>> (I always thought that this "special" execution mode, hardwired in, was a bit odd, and suggested that this could be made a function of pep405)
>> Anyway, keeping your executable as part of the tree is the trick I use, and to make things nice I put  right next to it:
>> site.py
>> sitecustomize.py
>> 
>> sitecustomize.py is where you would put the logic to set sys.path by walking up the hierarchy and finding the proper root.
>> site.py is there to merely import sitecustomize.py, in case a site.py is not found in all the default places python looks.
>> 
>> K
>> 
>> 
>>> -----Original Message-----
>>> From: Python-Dev [mailto:python-dev-
>>> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Christian Tismer
>>> Sent: 11. n?vember 2012 20:31
>>> To: python-dev at python.org
>>> Subject: [Python-Dev] Setting project home path the best way
>>> 
>>> Hi friends,
>>> 
>>> I have a project that has its root somewhere on my machine.
>>> This project has many folders and contains quite some modules.
>>> 
>>> There is a common root of the module tree, and I want to use
>>> - either absolute imports
>>> - relative imports with '.'
>>> 
>>> Problem:
>>> 
>>> - I want to run any module inside the heirarchy from the command-line
>>> 
>>> - this should work, regardless what my 'cwd' is
>>> 
>>> - this should work with or without virtualenv.
>>> 
>>> So far, things work fine with virtualenv, because sys.executable is in the
>>> project module tree.
>>> 
>>> Without virtualenv, this is not so. But I hate to make settings like
>>> PYTHONPATH, because these are not permanent. .
>>> 
>>> Question:
>>> 
>>> How should I define my project root dir in a unique way, without setting an
>>> environment variable?
>>> What is the lest intrusive way to spell that?
>>> 
>>> Reason:
>>> 
>>> I'd like to make things work correctly and unambigously when I call a script
>>> inside the module heirarchy. Things are not fixed: there exist many
>>> checkouts In the file system, and each should know where to search its
>>> home/root in the tree.
>>> 
>>> Is this elegantly possible to deduce from the actually executed script file?
>>> 
>>> Cheers - chris
>>> 
>>> Sent from my Ei4Steve
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: http://mail.python.org/mailman/options/python-
>>> dev/kristjan%40ccpgames.com
>> 
>> <winmail.dat>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com

From phd at phdru.name  Fri Nov 16 06:01:35 2012
From: phd at phdru.name (Oleg Broytman)
Date: Fri, 16 Nov 2012 09:01:35 +0400
Subject: [Python-Dev] Generally bored by installation
In-Reply-To: <2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
Message-ID: <20121116050135.GA7247@iskra.aviel.ru>

Hello, Christian!

On Fri, Nov 16, 2012 at 12:10:09AM +0100, Christian Tismer <tismer at stackless.com> wrote:
> I am bored of installing things. 
> Bored of things that happen to not work for some minor reasons. 
> Reasons that are not immediately obvious. 
> Things that don't work because some special case was not handled. 
[a hot rant skipped]

   Christian, I feel your pain. Everyone does, I am sure. Who among us
never cursed that dratted computers? Those dirty rotten things that
always hang due to bugs in drivers or whatnot, require constant
cleaning, mending, upgrading, repairing, debugging. Condemn that magical
mouse gestures, be cursed those magical incantations. Who in his normal
mind would ever think of writing things like

exec find . \( -type d \( -name CVS -o -name .svn -o -name .hg -o -name .git -o -path ./.mozilla/\*/Cache/\* -o -path ./tmp/\* \) -prune \) -o -type f -exec grep -I "$@" '{}' \+ 2>/dev/null

   ???!!!

   Unfortunately, there is no escape. Remember that classical book "The
Mythical Man-Month"? Quoting Brooks, "There is no silver bullet". Things
are becoming better but will never be absolutely perfect.

> I would like to start a task force and some sprints about improving this
> situation. 
> My goal is some unbreakable system of building blocks that are self-contained with no dependencies, that have a defined interface to talk to, and that know themselves completely by introspection. 
> 
> They should not work because they happen to work around all known defects, but by design and control. 
> 
> Whoever is interested to work with me on this is hereby honestly welcomed!

   I am sorry to disappoint you but I don't believe this could work.
Many single developers and teams -- small teams and huge companies,
commercial and non-commercial -- have tried the approach and failed.
Sometimes things work fine, some small projects look perfect... in
isolation... But every now and then they failed, especially when
combined. Try to connect a perfect project that measures distances in
inches with another perfect project that that measures distances in
centimeters -- and poof! your perfect satellite failed and dropped off
of the sky.
   This will always happen. There is always pain for both users and
developers. And still people are not afraid to create bigger and more
complex projects -- and often they succeed.

   The only sane approach in my humble opinion is evolution. Similar to
biological evolution. And biological evolution means huge birth rate,
adapting and huge death rate.
   So write a lot of code, adapt it to your user's need and suffer a lot
of pain -- code death and all that. Try to lessen user's pain, but never
expect a piece of software to become perfect once, for all and forever.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From tismer at stackless.com  Fri Nov 16 12:32:46 2012
From: tismer at stackless.com (Christian Tismer)
Date: Fri, 16 Nov 2012 12:32:46 +0100
Subject: [Python-Dev] Setting project home path the best way
In-Reply-To: <72466DBD-CFF8-4FAD-8878-46984675A705@gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2D5A4C65-85AD-463C-9C6B-263C4C36EFAA@stackless.com>
	<72466DBD-CFF8-4FAD-8878-46984675A705@gmail.com>
Message-ID: <50A6245E.2030205@stackless.com>

Hi Daniel,

no, I was not aware of this. I just read it up on

http://sayspy.blogspot.de/2010/03/various-ways-of-distributing-python.html

Yeah, thank you very much for this hint, very useful! ;-)

cheers - Chris

On 16.11.12 04:22, Daniel Holth wrote:
> Are you familiar with executing directories having __main__.py as python scripts?
>
> Daniel Holth
>
> On Nov 15, 2012, at 4:43 PM, Christian Tismer <tismer at stackless.com> wrote:
>
>> Hi Kristjan,
>>
>> does that mean that your scheme simply works, without any config step
>> necessary after I did my checkout?
>> This would in fact be an interesting alternative to
>>
>>     Python setup.py develop
>>
>> but I'm not sure if this is the same scheme on windows and Os X.
>>
>> Getting this part right was rather tricky, and I fear this is still an issue.
>>
>> Right now I think to just force my users to run the install step, since it is quite
>> accepted in general.
>>
>> Still, I'd love to see a way with no action needed at all: write yout your structure,
>> and it works as-is. Seems to be impossible without tricks.
>>
>> Cheers - chris
>>
>> Sent from my Ei4Steve
>>
>> On Nov 15, 2012, at 10:17, Kristj?n Valur J?nsson <kristjan at ccpgames.com> wrote:
>>
>>> When python is being run from a compile environment, it detects this by looking for "Lib" folders in directories above the one containing the executable.
>>> (I always thought that this "special" execution mode, hardwired in, was a bit odd, and suggested that this could be made a function of pep405)
>>> Anyway, keeping your executable as part of the tree is the trick I use, and to make things nice I put  right next to it:
>>> site.py
>>> sitecustomize.py
>>>
>>> sitecustomize.py is where you would put the logic to set sys.path by walking up the hierarchy and finding the proper root.
>>> site.py is there to merely import sitecustomize.py, in case a site.py is not found in all the default places python looks.
>>>
>>> K
>>>
>>>
>>>> -----Original Message-----
>>>> From: Python-Dev [mailto:python-dev-
>>>> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Christian Tismer
>>>> Sent: 11. n?vember 2012 20:31
>>>> To: python-dev at python.org
>>>> Subject: [Python-Dev] Setting project home path the best way
>>>>
>>>> Hi friends,
>>>>
>>>> I have a project that has its root somewhere on my machine.
>>>> This project has many folders and contains quite some modules.
>>>>
>>>> There is a common root of the module tree, and I want to use
>>>> - either absolute imports
>>>> - relative imports with '.'
>>>>
>>>> Problem:
>>>>
>>>> - I want to run any module inside the heirarchy from the command-line
>>>>
>>>> - this should work, regardless what my 'cwd' is
>>>>
>>>> - this should work with or without virtualenv.
>>>>
>>>> So far, things work fine with virtualenv, because sys.executable is in the
>>>> project module tree.
>>>>
>>>> Without virtualenv, this is not so. But I hate to make settings like
>>>> PYTHONPATH, because these are not permanent. .
>>>>
>>>> Question:
>>>>
>>>> How should I define my project root dir in a unique way, without setting an
>>>> environment variable?
>>>> What is the lest intrusive way to spell that?
>>>>
>>>> Reason:
>>>>
>>>> I'd like to make things work correctly and unambigously when I call a script
>>>> inside the module heirarchy. Things are not fixed: there exist many
>>>> checkouts In the file system, and each should know where to search its
>>>> home/root in the tree.
>>>>
>>>> Is this elegantly possible to deduce from the actually executed script file?
>>>>
>>>> Cheers - chris
>>>>
>>>> Sent from my Ei4Steve
>>>> _______________________________________________
>>>> Python-Dev mailing list
>>>> Python-Dev at python.org
>>>> http://mail.python.org/mailman/listinfo/python-dev
>>>> Unsubscribe: http://mail.python.org/mailman/options/python-
>>>> dev/kristjan%40ccpgames.com
>>> <winmail.dat>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com


-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/


From trent at snakebite.org  Fri Nov 16 13:12:52 2012
From: trent at snakebite.org (Trent Nelson)
Date: Fri, 16 Nov 2012 07:12:52 -0500
Subject: [Python-Dev] externals?
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <20121116121251.GA13953@snakebite.org>

On Thu, Nov 15, 2012 at 01:20:09AM -0800, Kristj?n Valur J?nsson wrote:
> Perhaps the unix makefiles get the proper version, but a windows developer has to fetch those externals manually.

    Pro-tip: if you're developing on Windows, you're mad if you don't
    prime your dev env with Tools\buildbot\externals.bat.  It takes
    care of *everything*.  I wish every proprietary UNIX system we
    support had something similar.

> Also, is there any reason to keep this in svn?

    I think it's more a case of there being no tangible benefit (and
    numerous drawbacks) to switching it to hg.  I personally have no
    need for a local hg repo with 30 different Tcl/Tk histories in
    it.

    Subversion's good for this sort of use case.  The externals repo
    gets committed to maybe, what, 3-4 times a year?

> Why not check this in to HG, we need not worry about history, etc.

    Who are these mystical people worried about history? ;-)

        Trent.

From eliben at gmail.com  Fri Nov 16 14:34:37 2012
From: eliben at gmail.com (Eli Bendersky)
Date: Fri, 16 Nov 2012 05:34:37 -0800
Subject: [Python-Dev] Generally bored by installation
In-Reply-To: <20121116050135.GA7247@iskra.aviel.ru>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<20121116050135.GA7247@iskra.aviel.ru>
Message-ID: <CAF-Rda92uB8vKtErhQx9T10D2MqQ-RNxNoAKDGH2B=+g51tV2A@mail.gmail.com>

>    Christian, I feel your pain. Everyone does, I am sure. Who among us
> never cursed that dratted computers? Those dirty rotten things that
> always hang due to bugs in drivers or whatnot, require constant
> cleaning, mending, upgrading, repairing, debugging. Condemn that magical
> mouse gestures, be cursed those magical incantations. Who in his normal
> mind would ever think of writing things like
>
> exec find . \( -type d \( -name CVS -o -name .svn -o -name .hg -o -name
> .git -o -path ./.mozilla/\*/Cache/\* -o -path ./tmp/\* \) -prune \) -o
> -type f -exec grep -I "$@" '{}' \+ 2>/dev/null
>
>    ???!!!
>
>
Use pss ;-) [http://pypi.python.org/pypi/pss]

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121116/3bc7dbf5/attachment.html>

From phd at phdru.name  Fri Nov 16 15:25:45 2012
From: phd at phdru.name (Oleg Broytman)
Date: Fri, 16 Nov 2012 18:25:45 +0400
Subject: [Python-Dev] pss (was: Generally bored by installation)
In-Reply-To: <CAF-Rda92uB8vKtErhQx9T10D2MqQ-RNxNoAKDGH2B=+g51tV2A@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<20121116050135.GA7247@iskra.aviel.ru>
	<CAF-Rda92uB8vKtErhQx9T10D2MqQ-RNxNoAKDGH2B=+g51tV2A@mail.gmail.com>
Message-ID: <20121116142545.GA19390@iskra.aviel.ru>

On Fri, Nov 16, 2012 at 05:34:37AM -0800, Eli Bendersky <eliben at gmail.com> wrote:
> > exec find . \( -type d \( -name CVS -o -name .svn -o -name .hg -o -name
> > .git -o -path ./.mozilla/\*/Cache/\* -o -path ./tmp/\* \) -prune \) -o
> > -type f -exec grep -I "$@" '{}' \+ 2>/dev/null
> >
> Use pss ;-) [http://pypi.python.org/pypi/pss]

   Hardly. I am proficient in find+grep. And pss is underdocumented.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From status at bugs.python.org  Fri Nov 16 18:07:26 2012
From: status at bugs.python.org (Python tracker)
Date: Fri, 16 Nov 2012 18:07:26 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20121116170726.825591CC04@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-11-09 - 2012-11-16)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3801 ( -5)
  closed 24464 (+46)
  total  28265 (+41)

Open issues with patches: 1660 


Issues opened (27)
==================

#13863: import.c sometimes generates incorrect timestamps on Windows +
http://bugs.python.org/issue13863  reopened by eric.snow

#16450: test_missing_localfile masks problems in urlopen
http://bugs.python.org/issue16450  opened by HansM

#16451: Remove duplication between slice_indices and compute_slice_ind
http://bugs.python.org/issue16451  opened by mark.dickinson

#16455: sys.getfilesystemencoding() is not the locale encoding on Free
http://bugs.python.org/issue16455  opened by haypo

#16458: subprocess.py throw "The handle is invalid" error on duplicati
http://bugs.python.org/issue16458  opened by kartlee05

#16461: wave module: wrong integer format
http://bugs.python.org/issue16461  opened by ckern

#16462: smtpd should return greeting
http://bugs.python.org/issue16462  opened by mike.a

#16463: test_timeout failure on the RHEL buildbot
http://bugs.python.org/issue16463  opened by pitrou

#16464: urllib.request: opener not resetting content-length
http://bugs.python.org/issue16464  opened by terry.reedy

#16465: dict creation performance regression
http://bugs.python.org/issue16465  opened by phihag

#16466: register command forgets password if no config file is created
http://bugs.python.org/issue16466  opened by techtonik

#16470: Backport set and dictionary comprehensions in tutorial to 2.7
http://bugs.python.org/issue16470  opened by fossilet

#16471: upgrade to sphinx 1.1
http://bugs.python.org/issue16471  opened by chris.jerdonek

#16472: Distutils+mingw links agains msvcr90, while python27.dll is li
http://bugs.python.org/issue16472  opened by eudoxos

#16473: Minor difference in decoding quoted-printable text
http://bugs.python.org/issue16473  opened by aleperalta

#16474: More code coverage for imp module
http://bugs.python.org/issue16474  opened by tenuki

#16475: Support object instancing and recursion in marshal
http://bugs.python.org/issue16475  opened by kristjan.jonsson

#16477: tarfile fails to close file handles in case of exception
http://bugs.python.org/issue16477  opened by ssam

#16480: pyvenv 3.3 fails to create symlinks for <virtualenv>/local/{bi
http://bugs.python.org/issue16480  opened by Marco.Amadori

#16482: pdb.set_trace() clobbering traceback on error
http://bugs.python.org/issue16482  opened by akaptur

#16483: Make int(float('inf')) raise ValueError rather than OverflowEr
http://bugs.python.org/issue16483  opened by mark.dickinson

#16484: Missing/broken documentation redirect for http://docs.python.o
http://bugs.python.org/issue16484  opened by mgedmin

#16485: FD leaks in aifc module
http://bugs.python.org/issue16485  opened by serhiy.storchaka

#16486: Add context manager support to aifc module
http://bugs.python.org/issue16486  opened by serhiy.storchaka

#16487: Allow ssl certificates to be speficfied from memory rather tha
http://bugs.python.org/issue16487  opened by kristjan.jonsson

#16488: Add context manager support to epoll object
http://bugs.python.org/issue16488  opened by serhiy.storchaka

#16468: argparse only supports iterable choices
http://bugs.python.org/issue16468  opened by chris.jerdonek



Most recent 15 issues with no replies (15)
==========================================

#16488: Add context manager support to epoll object
http://bugs.python.org/issue16488

#16486: Add context manager support to aifc module
http://bugs.python.org/issue16486

#16484: Missing/broken documentation redirect for http://docs.python.o
http://bugs.python.org/issue16484

#16482: pdb.set_trace() clobbering traceback on error
http://bugs.python.org/issue16482

#16480: pyvenv 3.3 fails to create symlinks for <virtualenv>/local/{bi
http://bugs.python.org/issue16480

#16472: Distutils+mingw links agains msvcr90, while python27.dll is li
http://bugs.python.org/issue16472

#16471: upgrade to sphinx 1.1
http://bugs.python.org/issue16471

#16464: urllib.request: opener not resetting content-length
http://bugs.python.org/issue16464

#16463: test_timeout failure on the RHEL buildbot
http://bugs.python.org/issue16463

#16450: test_missing_localfile masks problems in urlopen
http://bugs.python.org/issue16450

#16446: pdb raises BdbQuit on 'quit' when started with set_trace
http://bugs.python.org/issue16446

#16429: Emit SyntaxWarning for code that risks UnboundLocalError
http://bugs.python.org/issue16429

#16428: turtle with compound shape doesn't get clicks
http://bugs.python.org/issue16428

#16425: minidom replaceChild(new_child, old_child) removes new_child e
http://bugs.python.org/issue16425

#16406: move the "Uploading Packages" section to distutils/packageinde
http://bugs.python.org/issue16406



Most recent 15 issues waiting for review (15)
=============================================

#16488: Add context manager support to epoll object
http://bugs.python.org/issue16488

#16487: Allow ssl certificates to be speficfied from memory rather tha
http://bugs.python.org/issue16487

#16486: Add context manager support to aifc module
http://bugs.python.org/issue16486

#16485: FD leaks in aifc module
http://bugs.python.org/issue16485

#16483: Make int(float('inf')) raise ValueError rather than OverflowEr
http://bugs.python.org/issue16483

#16475: Support object instancing and recursion in marshal
http://bugs.python.org/issue16475

#16474: More code coverage for imp module
http://bugs.python.org/issue16474

#16473: Minor difference in decoding quoted-printable text
http://bugs.python.org/issue16473

#16470: Backport set and dictionary comprehensions in tutorial to 2.7
http://bugs.python.org/issue16470

#16466: register command forgets password if no config file is created
http://bugs.python.org/issue16466

#16465: dict creation performance regression
http://bugs.python.org/issue16465

#16455: sys.getfilesystemencoding() is not the locale encoding on Free
http://bugs.python.org/issue16455

#16451: Remove duplication between slice_indices and compute_slice_ind
http://bugs.python.org/issue16451

#16447: SEGFAULT when setting type.__name__
http://bugs.python.org/issue16447

#16445: SEGFAULT when deleting Exception.message
http://bugs.python.org/issue16445



Top 10 most discussed issues (10)
=================================

#16465: dict creation performance regression
http://bugs.python.org/issue16465  14 msgs

#8865: select.poll is not thread safe
http://bugs.python.org/issue8865  13 msgs

#16418: argparse with many choices can generate absurdly long usage me
http://bugs.python.org/issue16418  13 msgs

#16427: Faster hash implementation
http://bugs.python.org/issue16427  13 msgs

#16444: Use support.TESTFN_UNDECODABLE on UNIX
http://bugs.python.org/issue16444  11 msgs

#16218: Python launcher does not support unicode characters
http://bugs.python.org/issue16218  10 msgs

#14631: Instance methods and WeakRefs don't mix.
http://bugs.python.org/issue14631   8 msgs

#13349: Non-informative error message in index() and remove() function
http://bugs.python.org/issue13349   7 msgs

#16416: Mac OS X: don't use the locale encoding but UTF-8 to encode an
http://bugs.python.org/issue16416   7 msgs

#16458: subprocess.py throw "The handle is invalid" error on duplicati
http://bugs.python.org/issue16458   7 msgs



Issues closed (41)
==================

#7620: Vim syntax highlight
http://bugs.python.org/issue7620  closed by brett.cannon

#9084: vimrc: use matchall() instead of ":match" to allow multiple ma
http://bugs.python.org/issue9084  closed by brett.cannon

#9535: Pending signals are inherited by child processes
http://bugs.python.org/issue9535  closed by gregory.p.smith

#9893: Removing the Misc/Vim/ files
http://bugs.python.org/issue9893  closed by brett.cannon

#12428: functools test coverage
http://bugs.python.org/issue12428  closed by pitrou

#12907: Update test coverage devguide page
http://bugs.python.org/issue12907  closed by brett.cannon

#12908: Update dev-in-a-box for new coverage steps
http://bugs.python.org/issue12908  closed by brett.cannon

#14215: http://www.python.org/dev/peps/ title is python.org
http://bugs.python.org/issue14215  closed by georg.brandl

#14396: Popen wait() doesn't handle spurious wakeups
http://bugs.python.org/issue14396  closed by gregory.p.smith

#15677: Gzip/zlib allows for compression level=0
http://bugs.python.org/issue15677  closed by nadeem.vawda

#15835: HP-UX build needs to be tweaked to pick up PATH_MAX
http://bugs.python.org/issue15835  closed by skrah

#15880: os.path.split() and long UNC names
http://bugs.python.org/issue15880  closed by serhiy.storchaka

#15894: _PyImport_ReInitLock() doesn't check return value of PyThread_
http://bugs.python.org/issue15894  closed by brett.cannon

#16140: subprocess.Popen the os.close calls in _execute_child can rais
http://bugs.python.org/issue16140  closed by gregory.p.smith

#16144: misleading sentence in reference/import
http://bugs.python.org/issue16144  closed by asvetlov

#16290: PyComplex_AsCComplex should allow __complex__ to return float.
http://bugs.python.org/issue16290  closed by mark.dickinson

#16327: subprocess.Popen leaks file descriptors on os.fork() failure
http://bugs.python.org/issue16327  closed by gregory.p.smith

#16357: SSLSocket created from SSLContext.wrap_socket doesn't include 
http://bugs.python.org/issue16357  closed by pitrou

#16378: venv.EnvBuilder docstring inconsistencies
http://bugs.python.org/issue16378  closed by python-dev

#16391: add "terminator" ctor argument to logging.StreamHandlers deriv
http://bugs.python.org/issue16391  closed by vinay.sajip

#16397: UserString doesn't combine nicely with strings
http://bugs.python.org/issue16397  closed by Jorge.Cardona

#16400: update default PyPI behavior in docs re: listing versions
http://bugs.python.org/issue16400  closed by chris.jerdonek

#16409: urlretrieve regression: first call returns block size as 0
http://bugs.python.org/issue16409  closed by gregory.p.smith

#16411: zlib.Decompress.decompress() retains pointer to input buffer w
http://bugs.python.org/issue16411  closed by nadeem.vawda

#16424: regression: os.path.split('//hostname/foo/bar.txt')
http://bugs.python.org/issue16424  closed by jcea

#16436: Link directly to set and frozenset in built-in function docs
http://bugs.python.org/issue16436  closed by chris.jerdonek

#16448: 'module' object has no attribute 'font' when "import tkinter" 
http://bugs.python.org/issue16448  closed by r.david.murray

#16449: RotatingFileHandler rollover doesn't work on QNX shmem filesys
http://bugs.python.org/issue16449  closed by vinay.sajip

#16452: Add support for weak reference to bound methods
http://bugs.python.org/issue16452  closed by sfeltman

#16453: Inconsistent dead weakref equality
http://bugs.python.org/issue16453  closed by pitrou

#16454: Mostly for discussion: _winapi as builtin for bootstrapping di
http://bugs.python.org/issue16454  closed by sbt

#16456: UnicodeDecodeError thrown for 'encode' operation on string
http://bugs.python.org/issue16456  closed by haypo

#16457: Allow operator 'getter' methods to take a list and return a tu
http://bugs.python.org/issue16457  closed by r.david.murray

#16459: sys.stdout.write printing length of string
http://bugs.python.org/issue16459  closed by firatozgul

#16460: Strange results for floor division ("//") with non-integer div
http://bugs.python.org/issue16460  closed by mark.dickinson

#16469: Exceptions raised by Fraction() different from those raised by
http://bugs.python.org/issue16469  closed by mark.dickinson

#16476: Trailing spaces in pretty-printed JSON
http://bugs.python.org/issue16476  closed by ezio.melotti

#16478: Fix division in tabnanny
http://bugs.python.org/issue16478  closed by ezio.melotti

#16479: Can't install Python 3 on Redhat Linux, make failed
http://bugs.python.org/issue16479  closed by pitrou

#16481: process handle leak on windows in multiprocessing
http://bugs.python.org/issue16481  closed by sbt

#16467: frozen importlib required for extending Python interpreter not
http://bugs.python.org/issue16467  closed by atuining

From vinay_sajip at yahoo.co.uk  Fri Nov 16 18:18:02 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 16 Nov 2012 17:18:02 +0000 (UTC)
Subject: [Python-Dev] Generally bored by installation
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<20121116050135.GA7247@iskra.aviel.ru>
	<CAF-Rda92uB8vKtErhQx9T10D2MqQ-RNxNoAKDGH2B=+g51tV2A@mail.gmail.com>
Message-ID: <loom.20121116T181532-233@post.gmane.org>

Eli Bendersky <eliben <at> gmail.com> writes:

> Use pss  [http://pypi.python.org/pypi/pss]

How does it compare with grin? [http://pypi.python.org/pypi/grin]

On the face of it, they both do the same job.

Regards,

Vinay Sajip




From chris.jerdonek at gmail.com  Fri Nov 16 21:47:52 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Fri, 16 Nov 2012 12:47:52 -0800
Subject: [Python-Dev] Failed issue tracker submission
In-Reply-To: <20121116023950.2EC2B1CDAD@psf.upfronthosting.co.za>
References: <20121116023950.2EC2B1CDAD@psf.upfronthosting.co.za>
Message-ID: <CAOTb1wcaUsB8-ZOTGgAdSJHg519iMMxm8=PU-i3DVXhS43a_bQ@mail.gmail.com>

I filed an issue in the meta tracker about e-mails like the below that
are sent to python-dev.  The issue link is here:

http://psf.upfronthosting.co.za/roundup/meta/issue492

--Chris


On Thu, Nov 15, 2012 at 6:39 PM, Python tracker
<roundup-admin at psf.upfronthosting.co.za> wrote:
>
>
> An unexpected error occurred during the processing
> of your message. The tracker administrator is being
> notified.
>
> Return-Path: <python-dev at python.org>
> X-Original-To: report at bugs.python.org
> Delivered-To: roundup+tracker at psf.upfronthosting.co.za
> Received: from mail.python.org (mail.python.org [IPv6:2001:888:2000:d::a6])
>         by psf.upfronthosting.co.za (Postfix) with ESMTPS id DFD211C98E
>         for <report at bugs.python.org>; Fri, 16 Nov 2012 03:39:49 +0100 (CET)
> Received: from albatross.python.org (localhost [127.0.0.1])
>         by mail.python.org (Postfix) with ESMTP id 3Y2kDj4Tk8zRb6
>         for <report at bugs.python.org>; Fri, 16 Nov 2012 03:39:49 +0100 (CET)
> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901;
>         t=1353033589; bh=H41Haza/UPvj9B16qvXQtqjlb22jRmazur238MqAPOE=;
>         h=MIME-Version:Content-Type:Content-Transfer-Encoding:From:To:
>          Subject:Message-Id:Date;
>         b=yz1YIEtIYuM3H9V1Ok0YJ/SDvU8xtO+cFOApgDH8jPRdxRS6z/CSdZ96bSY1cNFM7
>          4xxKY3LiUXipe439DPwEjxZr0HcsfX+2JdR4Pzf3OZmdopV6PEl8/twaIFoTHQMy8u
>          dTwpZ7G4OJKCc2IeCq4e5Bl/TEvQvVT9NO8eoa3k=
> Received: from localhost (HELO mail.python.org) (127.0.0.1)
>   by albatross.python.org with SMTP; 16 Nov 2012 03:39:49 +0100
> Received: from virt-7yvsjn.psf.osuosl.org (virt-7yvsjn.psf.osuosl.org [140.211.10.72])
>         by mail.python.org (Postfix) with ESMTP
>         for <report at bugs.python.org>; Fri, 16 Nov 2012 03:39:49 +0100 (CET)
> MIME-Version: 1.0
> Content-Type: text/plain; charset="utf-8"
> Content-Transfer-Encoding: base64
> From: python-dev at python.org
> To: report at bugs.python.org
> Subject: [issue15894]
> Message-Id: <3Y2kDj4Tk8zRb6 at mail.python.org>
> Date: Fri, 16 Nov 2012 03:39:49 +0100 (CET)
>
> TmV3IGNoYW5nZXNldCBiZDg1MzMxMWZmZTAgYnkgQnJldHQgQ2Fubm9uIGluIGJyYW5jaCAnZGVm
> YXVsdCc6Cklzc3VlICMxNTg5NDogRG9jdW1lbnQgd2h5IHdlIGRvbid0IHdvcnJ5IGFib3V0IHJl
> LWFjcXVpcmluZyB0aGUKaHR0cDovL2hnLnB5dGhvbi5vcmcvY3B5dGhvbi9yZXYvYmQ4NTMzMTFm
> ZmUwCg==
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/chris.jerdonek%40gmail.com
>

From eliben at gmail.com  Fri Nov 16 22:15:26 2012
From: eliben at gmail.com (Eli Bendersky)
Date: Fri, 16 Nov 2012 13:15:26 -0800
Subject: [Python-Dev] Generally bored by installation
In-Reply-To: <loom.20121116T181532-233@post.gmane.org>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<20121116050135.GA7247@iskra.aviel.ru>
	<CAF-Rda92uB8vKtErhQx9T10D2MqQ-RNxNoAKDGH2B=+g51tV2A@mail.gmail.com>
	<loom.20121116T181532-233@post.gmane.org>
Message-ID: <CAF-Rda--Mi=J-L7MA1Ffo-1Jf3S=bBO9aoSVJfpmH+B+frQw9g@mail.gmail.com>

On Fri, Nov 16, 2012 at 9:18 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk>wrote:

> Eli Bendersky <eliben <at> gmail.com> writes:
>
> > Use pss  [http://pypi.python.org/pypi/pss]
>
> How does it compare with grin? [http://pypi.python.org/pypi/grin]
>
> On the face of it, they both do the same job.
>

They're quite similar. pss is more close to ack, in the sense that it knows
how to identify files belonging to various categories/languages and lets
you easily guide the search this way. And some others bells and whistles.
But generally, both are better than using the horrible "find | xargs grep"
incantation and all its variations.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121116/0c24a7dc/attachment.html>

From eliben at gmail.com  Fri Nov 16 22:20:17 2012
From: eliben at gmail.com (Eli Bendersky)
Date: Fri, 16 Nov 2012 13:20:17 -0800
Subject: [Python-Dev] pss (was: Generally bored by installation)
In-Reply-To: <20121116142545.GA19390@iskra.aviel.ru>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<20121116050135.GA7247@iskra.aviel.ru>
	<CAF-Rda92uB8vKtErhQx9T10D2MqQ-RNxNoAKDGH2B=+g51tV2A@mail.gmail.com>
	<20121116142545.GA19390@iskra.aviel.ru>
Message-ID: <CAF-Rda9K4ZFS6UsUH7tD6AxbtfubGcy=F7bkcemcQ4D7DY9ufQ@mail.gmail.com>

On Fri, Nov 16, 2012 at 6:25 AM, Oleg Broytman <phd at phdru.name> wrote:

> On Fri, Nov 16, 2012 at 05:34:37AM -0800, Eli Bendersky <eliben at gmail.com>
> wrote:
> > > exec find . \( -type d \( -name CVS -o -name .svn -o -name .hg -o -name
> > > .git -o -path ./.mozilla/\*/Cache/\* -o -path ./tmp/\* \) -prune \) -o
> > > -type f -exec grep -I "$@" '{}' \+ 2>/dev/null
> > >
> > Use pss ;-) [http://pypi.python.org/pypi/pss]
>
>    Hardly. I am proficient in find+grep. And pss is underdocumented.
>
>
Even reading https://bitbucket.org/eliben/pss/wiki/Usage#!usage-samplescombined
with the built-in help? If anything is missing, by all means let
me know and I'll gladly add it.

[Sorry for the offtopic, folks, promise it's short]

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121116/e256874c/attachment-0001.html>

From phd at phdru.name  Fri Nov 16 22:37:45 2012
From: phd at phdru.name (Oleg Broytman)
Date: Sat, 17 Nov 2012 01:37:45 +0400
Subject: [Python-Dev] Generally bored by installation
In-Reply-To: <CAF-Rda--Mi=J-L7MA1Ffo-1Jf3S=bBO9aoSVJfpmH+B+frQw9g@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<20121116050135.GA7247@iskra.aviel.ru>
	<CAF-Rda92uB8vKtErhQx9T10D2MqQ-RNxNoAKDGH2B=+g51tV2A@mail.gmail.com>
	<loom.20121116T181532-233@post.gmane.org>
	<CAF-Rda--Mi=J-L7MA1Ffo-1Jf3S=bBO9aoSVJfpmH+B+frQw9g@mail.gmail.com>
Message-ID: <20121116213745.GA28111@iskra.aviel.ru>

On Fri, Nov 16, 2012 at 01:15:26PM -0800, Eli Bendersky <eliben at gmail.com> wrote:
> But generally, both are better than using the horrible "find | xargs grep"
> incantation and all its variations.

   In what way they are better? I found 'find' to be quite capable, and
find -exec grep '{}' \+
   to be quite fast.

   In my opinion all those small utilities are pets of their authors and
they solve their problems quite good... but not mine. GNU utilities are
used by a huge number of people and solve much wider problems much
better. Though not necessary in a much more elegant way ;-)

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From phd at phdru.name  Fri Nov 16 22:40:04 2012
From: phd at phdru.name (Oleg Broytman)
Date: Sat, 17 Nov 2012 01:40:04 +0400
Subject: [Python-Dev] pss
In-Reply-To: <CAF-Rda9K4ZFS6UsUH7tD6AxbtfubGcy=F7bkcemcQ4D7DY9ufQ@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<20121116050135.GA7247@iskra.aviel.ru>
	<CAF-Rda92uB8vKtErhQx9T10D2MqQ-RNxNoAKDGH2B=+g51tV2A@mail.gmail.com>
	<20121116142545.GA19390@iskra.aviel.ru>
	<CAF-Rda9K4ZFS6UsUH7tD6AxbtfubGcy=F7bkcemcQ4D7DY9ufQ@mail.gmail.com>
Message-ID: <20121116214004.GB28111@iskra.aviel.ru>

On Fri, Nov 16, 2012 at 01:20:17PM -0800, Eli Bendersky <eliben at gmail.com> wrote:
> On Fri, Nov 16, 2012 at 6:25 AM, Oleg Broytman <phd at phdru.name> wrote:
> >    And pss is underdocumented.
> >
> Even reading https://bitbucket.org/eliben/pss/wiki/Usage#!usage-samplescombined

   Compare your page with

man find

   and

man grep

   Compare amount of text, levels of details, number of options,
examples...

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From guido at python.org  Fri Nov 16 23:35:13 2012
From: guido at python.org (Guido van Rossum)
Date: Fri, 16 Nov 2012 14:35:13 -0800
Subject: [Python-Dev] pss
In-Reply-To: <20121116214004.GB28111@iskra.aviel.ru>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<20121116050135.GA7247@iskra.aviel.ru>
	<CAF-Rda92uB8vKtErhQx9T10D2MqQ-RNxNoAKDGH2B=+g51tV2A@mail.gmail.com>
	<20121116142545.GA19390@iskra.aviel.ru>
	<CAF-Rda9K4ZFS6UsUH7tD6AxbtfubGcy=F7bkcemcQ4D7DY9ufQ@mail.gmail.com>
	<20121116214004.GB28111@iskra.aviel.ru>
Message-ID: <CAP7+vJJYbbRW_s1PZW=pWhgHiPn3F=tYvHCm4+kiHCaf7fJdig@mail.gmail.com>

Oleg, this is inappropriate for python-dev. It's also a bit
uncivilized and unkind.

On Fri, Nov 16, 2012 at 1:40 PM, Oleg Broytman <phd at phdru.name> wrote:
> On Fri, Nov 16, 2012 at 01:20:17PM -0800, Eli Bendersky <eliben at gmail.com> wrote:
>> On Fri, Nov 16, 2012 at 6:25 AM, Oleg Broytman <phd at phdru.name> wrote:
>> >    And pss is underdocumented.
>> >
>> Even reading https://bitbucket.org/eliben/pss/wiki/Usage#!usage-samplescombined
>
>    Compare your page with
>
> man find
>
>    and
>
> man grep
>
>    Compare amount of text, levels of details, number of options,
> examples...
>
> Oleg.
> --
>      Oleg Broytman            http://phdru.name/            phd at phdru.name
>            Programmers don't die, they just GOSUB without RETURN.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org



-- 
--Guido van Rossum (python.org/~guido)

From solipsis at pitrou.net  Fri Nov 16 23:41:11 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 16 Nov 2012 23:41:11 +0100
Subject: [Python-Dev] Failed issue tracker submission
References: <20121116023950.2EC2B1CDAD@psf.upfronthosting.co.za>
	<CAOTb1wcaUsB8-ZOTGgAdSJHg519iMMxm8=PU-i3DVXhS43a_bQ@mail.gmail.com>
Message-ID: <20121116234111.7ef55ee8@pitrou.net>

On Fri, 16 Nov 2012 12:47:52 -0800
Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
> I filed an issue in the meta tracker about e-mails like the below that
> are sent to python-dev.  The issue link is here:
> 
> http://psf.upfronthosting.co.za/roundup/meta/issue492

These e-mails are just because someone mentioned the wrong issue number
in a commit message, I think.

Regards

Antoine.



From guido at python.org  Sat Nov 17 00:36:03 2012
From: guido at python.org (Guido van Rossum)
Date: Fri, 16 Nov 2012 15:36:03 -0800
Subject: [Python-Dev] Failed issue tracker submission
In-Reply-To: <20121116234111.7ef55ee8@pitrou.net>
References: <20121116023950.2EC2B1CDAD@psf.upfronthosting.co.za>
	<CAOTb1wcaUsB8-ZOTGgAdSJHg519iMMxm8=PU-i3DVXhS43a_bQ@mail.gmail.com>
	<20121116234111.7ef55ee8@pitrou.net>
Message-ID: <CAP7+vJKG3a7T_gV4_kkJoi4WWtsi90BJQCS+XBy_MrA-DsMioA@mail.gmail.com>

But python-dev seems a poor place to spam with those errors.

On Fri, Nov 16, 2012 at 2:41 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 16 Nov 2012 12:47:52 -0800
> Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
>> I filed an issue in the meta tracker about e-mails like the below that
>> are sent to python-dev.  The issue link is here:
>>
>> http://psf.upfronthosting.co.za/roundup/meta/issue492
>
> These e-mails are just because someone mentioned the wrong issue number
> in a commit message, I think.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org



-- 
--Guido van Rossum (python.org/~guido)

From martin at v.loewis.de  Sat Nov 17 00:47:33 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Sat, 17 Nov 2012 00:47:33 +0100
Subject: [Python-Dev] Failed issue tracker submission
In-Reply-To: <CAP7+vJKG3a7T_gV4_kkJoi4WWtsi90BJQCS+XBy_MrA-DsMioA@mail.gmail.com>
References: <20121116023950.2EC2B1CDAD@psf.upfronthosting.co.za>
	<CAOTb1wcaUsB8-ZOTGgAdSJHg519iMMxm8=PU-i3DVXhS43a_bQ@mail.gmail.com>
	<20121116234111.7ef55ee8@pitrou.net>
	<CAP7+vJKG3a7T_gV4_kkJoi4WWtsi90BJQCS+XBy_MrA-DsMioA@mail.gmail.com>
Message-ID: <20121117004733.Horde.GGeAbLuWis5QptCVhIcgUVA@webmail.df.eu>


Zitat von Guido van Rossum <guido at python.org>:

> But python-dev seems a poor place to spam with those errors.

It's certainly not deliberate that they are sent. However, there
are too few people interested in working on fixing that so that
it remains unfixed. Even finding out why they are sent to python-dev
is tricky.

Regards,
Martin



From solipsis at pitrou.net  Sat Nov 17 01:01:52 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 17 Nov 2012 01:01:52 +0100
Subject: [Python-Dev] Failed issue tracker submission
References: <20121116023950.2EC2B1CDAD@psf.upfronthosting.co.za>
	<CAOTb1wcaUsB8-ZOTGgAdSJHg519iMMxm8=PU-i3DVXhS43a_bQ@mail.gmail.com>
	<20121116234111.7ef55ee8@pitrou.net>
	<CAP7+vJKG3a7T_gV4_kkJoi4WWtsi90BJQCS+XBy_MrA-DsMioA@mail.gmail.com>
	<20121117004733.Horde.GGeAbLuWis5QptCVhIcgUVA@webmail.df.eu>
Message-ID: <20121117010152.7ade2b38@pitrou.net>

On Sat, 17 Nov 2012 00:47:33 +0100
martin at v.loewis.de wrote:
> 
> Zitat von Guido van Rossum <guido at python.org>:
> 
> > But python-dev seems a poor place to spam with those errors.
> 
> It's certainly not deliberate that they are sent. However, there
> are too few people interested in working on fixing that so that
> it remains unfixed. Even finding out why they are sent to python-dev
> is tricky.

I think it's because I configured the dummy "python-dev" user that way:
http://bugs.python.org/user13902

Regards

Antoine.



From phd at phdru.name  Sat Nov 17 01:34:08 2012
From: phd at phdru.name (Oleg Broytman)
Date: Sat, 17 Nov 2012 04:34:08 +0400
Subject: [Python-Dev] pss
In-Reply-To: <CAP7+vJJYbbRW_s1PZW=pWhgHiPn3F=tYvHCm4+kiHCaf7fJdig@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<20121116050135.GA7247@iskra.aviel.ru>
	<CAF-Rda92uB8vKtErhQx9T10D2MqQ-RNxNoAKDGH2B=+g51tV2A@mail.gmail.com>
	<20121116142545.GA19390@iskra.aviel.ru>
	<CAF-Rda9K4ZFS6UsUH7tD6AxbtfubGcy=F7bkcemcQ4D7DY9ufQ@mail.gmail.com>
	<20121116214004.GB28111@iskra.aviel.ru>
	<CAP7+vJJYbbRW_s1PZW=pWhgHiPn3F=tYvHCm4+kiHCaf7fJdig@mail.gmail.com>
Message-ID: <20121117003408.GB2381@iskra.aviel.ru>

On Fri, Nov 16, 2012 at 02:35:13PM -0800, Guido van Rossum <guido at python.org> wrote:
> Oleg, this is inappropriate for python-dev. It's also a bit
> uncivilized and unkind.

   Shame on me! Please, everyone, take my sincerest apology! I'm
stopping now.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From victor.stinner at gmail.com  Sat Nov 17 02:13:25 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Sat, 17 Nov 2012 02:13:25 +0100
Subject: [Python-Dev] Register-based VM for CPython
Message-ID: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>

Hi,

I'm still looking for the best approach to speedup CPython. I found the
following article comparing a stack-based VM to a register-based VM which
announces a speed up between 20% and 40% (depending if the instruction
dispatch uses a dummy switch or computed goto):

http://static.usenix.org/events/vee05/full_papers/p153-yunhe.pdf
Yunhe Shi, David Gregg, Andrew Beatty, M. Anton Ertl, 2005

I tried to implement such register-based instructions for CPython in the
following fork of Python 3.4:
http://hg.python.org/sandbox/registervm/

Contact me if you are interested and/or if you would like to contribute!

--

My implementation is not finished and still experimental. It is only
enabled explictly by calling registervm.optimize_func(func) where
registervm is a new module. This function rewrites the bytecode of the
function inplace. I chose this approach because it was simple to implement,
it is simple to compare stack-based and register-based instructions, and it
is also the approach used in the paper :-)

The major drawback of the register approach (at least of my implementation)
is that it changes the lifetime of objects. Newly created objects are only
"destroyed" at the exit of the function, whereas the stack-based VM
destroys "immediatly" objects (thanks to the reference counter). PyPy has
similar issues with its different garbage collector.

On Linux (with computed goto in ceval.c), there are some interesting
speedups in pybench, up to 25% faster. Such speedup can be explained with
the bytecode. For example, "c = a + b" is compiled to:

   LOAD_FAST            'a'
   LOAD_FAST            'b'
   BINARY_ADD
   STORE_FAST           'c'

The optimized bytecode becomes:

  BINARY_ADD_REG       'c', 'a', 'b'

So 4 operations are replaced with only 1 operation. So instruction
dispatching has a cost, measurable in this microbenchmark.

My implementation is incomplete: if you see "PUSH_REG" or "POP_REG" in the
assembler, your code will be slower than the original bytecode. There is no
register allocator and the optimizer doesn't understand the control flow,
so I expect more performance improvment with a more complete
implementation. Some optimizations are also disabled by default because the
implementation is buggy.

The main changes of registervm are done in Python/ceval.c (implement new
instructions) and Lib/registervm.py (the new module rewriting the
bytecode). Objects/frameobject.c is also modified to allocate registers.

See registervm documentation for more information:
http://hg.python.org/sandbox/registervm/file/tip/REGISTERVM.txt

--

The WPython project is similar to my work (except that it does not use
registers). It tries also to reduce the overhead of instruction dispatch by
using more complex instructions.
http://code.google.com/p/wpython/

Using registers instead of a stack allow to implement more optimizations
(than WPython). For example, it's possible to load constants outside loops
and merge "duplicate constant loads".

I also implemented more aggressive and experimental optimizations (disabled
by default) which may break applications: move loads of attributes and
globals outside of loops, and replace binary operations with inplace
operations. For example, "x=[]; for ...: x.append(...)" is optimized to
something like "x=[]; x_append=x.append; for ...: x_append(...)", and "x =
x + 1" is replaced with "x += 1".

Victor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121117/3b6d4290/attachment.html>

From rdmurray at bitdance.com  Sat Nov 17 07:06:06 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 17 Nov 2012 01:06:06 -0500
Subject: [Python-Dev] Failed issue tracker submission
In-Reply-To: <20121117010152.7ade2b38@pitrou.net>
References: <20121116023950.2EC2B1CDAD@psf.upfronthosting.co.za>
	<CAOTb1wcaUsB8-ZOTGgAdSJHg519iMMxm8=PU-i3DVXhS43a_bQ@mail.gmail.com>
	<20121116234111.7ef55ee8@pitrou.net>
	<CAP7+vJKG3a7T_gV4_kkJoi4WWtsi90BJQCS+XBy_MrA-DsMioA@mail.gmail.com>
	<20121117004733.Horde.GGeAbLuWis5QptCVhIcgUVA@webmail.df.eu>
	<20121117010152.7ade2b38@pitrou.net>
Message-ID: <20121117060607.68142250129@webabinitio.net>

On Sat, 17 Nov 2012 01:01:52 +0100, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 17 Nov 2012 00:47:33 +0100
> martin at v.loewis.de wrote:
> > 
> > Zitat von Guido van Rossum <guido at python.org>:
> > 
> > > But python-dev seems a poor place to spam with those errors.
> > 
> > It's certainly not deliberate that they are sent. However, there
> > are too few people interested in working on fixing that so that
> > it remains unfixed. Even finding out why they are sent to python-dev
> > is tricky.
> 
> I think it's because I configured the dummy "python-dev" user that way:
> http://bugs.python.org/user13902

I'm pretty sure it's because python-dev is the 'from' address
used when the messages are sent...and the configuration of
that user is what allows them to be accepted.

I suggest changing the from address and the account configuration
to tracker-discuss at python.org instead.

--David

From chris.jerdonek at gmail.com  Sat Nov 17 07:35:38 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Fri, 16 Nov 2012 22:35:38 -0800
Subject: [Python-Dev] Failed issue tracker submission
In-Reply-To: <20121117060607.68142250129@webabinitio.net>
References: <20121116023950.2EC2B1CDAD@psf.upfronthosting.co.za>
	<CAOTb1wcaUsB8-ZOTGgAdSJHg519iMMxm8=PU-i3DVXhS43a_bQ@mail.gmail.com>
	<20121116234111.7ef55ee8@pitrou.net>
	<CAP7+vJKG3a7T_gV4_kkJoi4WWtsi90BJQCS+XBy_MrA-DsMioA@mail.gmail.com>
	<20121117004733.Horde.GGeAbLuWis5QptCVhIcgUVA@webmail.df.eu>
	<20121117010152.7ade2b38@pitrou.net>
	<20121117060607.68142250129@webabinitio.net>
Message-ID: <CAOTb1wdbaSWFnKkf2N5kzrddjbJgvCE1=5KgmNJ6v+eBqhESRA@mail.gmail.com>

On Fri, Nov 16, 2012 at 10:06 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Sat, 17 Nov 2012 01:01:52 +0100, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Sat, 17 Nov 2012 00:47:33 +0100
>> martin at v.loewis.de wrote:
>> >
>> > Zitat von Guido van Rossum <guido at python.org>:
>> >
>> > > But python-dev seems a poor place to spam with those errors.
>> >
>> > It's certainly not deliberate that they are sent. However, there
>> > are too few people interested in working on fixing that so that
>> > it remains unfixed. Even finding out why they are sent to python-dev
>> > is tricky.
>>
>> I think it's because I configured the dummy "python-dev" user that way:
>> http://bugs.python.org/user13902
>
> I'm pretty sure it's because python-dev is the 'from' address
> used when the messages are sent...and the configuration of
> that user is what allows them to be accepted.
>
> I suggest changing the from address and the account configuration
> to tracker-discuss at python.org instead.

Above and on the meta tracker issue I filed, it sounds like Martin is
saying that the e-mails should never be going to python-dev (or
tracker-discuss) in the first place -- due to a roundup bug and
because the e-mail is already being sent to the submitter and the
roundup admins.  Thus, changing the from address would only mitigate
the problem and not be fixing the root cause.

--Chris

From solipsis at pitrou.net  Sat Nov 17 11:00:03 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 17 Nov 2012 11:00:03 +0100
Subject: [Python-Dev] cpython (2.7): Closes #16461: Wave library should
 be able to deal with 4GB wav files, and
References: <3Y3LGW5lKLzRXK@mail.python.org>
Message-ID: <20121117110003.7086e779@pitrou.net>

On Sat, 17 Nov 2012 03:43:31 +0100 (CET)
jesus.cea <python-checkins at python.org> wrote:

> http://hg.python.org/cpython/rev/b8ece33ce27d
> changeset:   80455:b8ece33ce27d
> branch:      2.7
> parent:      80439:457c0c9c7893
> user:        Jesus Cea <jcea at jcea.es>
> date:        Sat Nov 17 03:38:17 2012 +0100
> summary:
>   Closes #16461: Wave library should be able to deal with 4GB wav files, and sample rate of 44100 Hz.

Any reason you didn't add any tests?

Regards

Antoine.



From arigo at tunes.org  Sat Nov 17 11:17:40 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 17 Nov 2012 11:17:40 +0100
Subject: [Python-Dev] Register-based VM for CPython
In-Reply-To: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>
References: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>
Message-ID: <CAMSv6X2hXLwB8NOSgONbVve2D1rMSXjEPmFWJVesrc4uaDxRYw@mail.gmail.com>

Hi Victor,

On Sat, Nov 17, 2012 at 2:13 AM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> The major drawback of the register approach (at least of my implementation)
> is that it changes the lifetime of objects. Newly created objects are only
> "destroyed" at the exit of the function, whereas the stack-based VM destroys
> "immediatly" objects (thanks to the reference counter). PyPy has similar
> issues with its different garbage collector.

That is not strictly correct.  PyPy, Jython and IronPython have
non-immediate destructors, but as far as I can tell they all avoid to
keep objects alive for an unbounded amount of time.  This important
difference is visible if the function calls other code that takes a
long while to run: in your approach, the objects created by the
function itself will stay alive for the whole duration, while the
other interpreters will all release them soon after they are not
referenced any more --- not instantly like CPython but still soon.


A bient?t,

Armin.

From chris.jerdonek at gmail.com  Sat Nov 17 19:47:02 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sat, 17 Nov 2012 10:47:02 -0800
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>
Message-ID: <CAOTb1wdbkGkK+pQ-qyVr3RBDxL-LBhs+Wp6D+yYcv9viH9JL_w@mail.gmail.com>

[Apologies for resurrecting a few-weeks old thread.]

On Thu, Oct 4, 2012 at 2:46 PM,  <martin at v.loewis.de> wrote:
>
> Zitat von Victor Stinner <victor.stinner at gmail.com>:
>
>> I only see one argument against such refactoring: it will be harder to
>> backport/forwardport bugfixes.
>
> I'm opposed for a different reason: I think it will be *harder* to maintain.
> The amount of code will not be reduced, but now you also need to guess what
> file some piece of functionality may be in. Instead of having my text editor
> (Emacs) search in one file, it will have to search across multiple files -
> but not across all open buffers, but only some of them (since I will have
> many other source files open as well).
>
> I really fail to see what problem people have with large source files.
> What is it that you want to do that can be done easier if it's multiple
> files?

One thing is browse or link to such code files on the web (e.g. from
within a tracker comment or from within our online documentation).
For example, today I was unable to open the following page from within
a browser to link to one of its lines on a tracker comment:

http://hg.python.org/cpython/file/27c20650aeab/Objects/unicodeobject.c

My laptop's fan simply turns on and the page hangs indefinitely while loading.

I don't think this point was ever mentioned.

--Chris

From rosuav at gmail.com  Sat Nov 17 19:55:02 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 18 Nov 2012 05:55:02 +1100
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAOTb1wdbkGkK+pQ-qyVr3RBDxL-LBhs+Wp6D+yYcv9viH9JL_w@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>
	<CAOTb1wdbkGkK+pQ-qyVr3RBDxL-LBhs+Wp6D+yYcv9viH9JL_w@mail.gmail.com>
Message-ID: <CAPTjJmot3xJtsVNHbZomWR9c=KiM_neUTH2OToHvH8TroFq+wg@mail.gmail.com>

On Sun, Nov 18, 2012 at 5:47 AM, Chris Jerdonek
<chris.jerdonek at gmail.com> wrote:
> On Thu, Oct 4, 2012 at 2:46 PM,  <martin at v.loewis.de> wrote:
>> I really fail to see what problem people have with large source files.
>> What is it that you want to do that can be done easier if it's multiple
>> files?
>
> One thing is browse or link to such code files on the web (e.g. from
> within a tracker comment or from within our online documentation).
> For example, today I was unable to open the following page from within
> a browser to link to one of its lines on a tracker comment:
>
> http://hg.python.org/cpython/file/27c20650aeab/Objects/unicodeobject.c
>
> My laptop's fan simply turns on and the page hangs indefinitely while loading.

Curious. This sounds like a web browser issue - I can pull it up in
either Chrome or Firefox on Windows on my 2GHz/2GB RAM laptop with a
visible pause, but not more than half a second. However, this page is
rather more significant, and is affected equally by the file size:

http://hg.python.org/cpython/annotate/27c20650aeab/Objects/unicodeobject.c

ChrisA

From chris.jerdonek at gmail.com  Sat Nov 17 20:16:21 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sat, 17 Nov 2012 11:16:21 -0800
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAPTjJmot3xJtsVNHbZomWR9c=KiM_neUTH2OToHvH8TroFq+wg@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>
	<CAOTb1wdbkGkK+pQ-qyVr3RBDxL-LBhs+Wp6D+yYcv9viH9JL_w@mail.gmail.com>
	<CAPTjJmot3xJtsVNHbZomWR9c=KiM_neUTH2OToHvH8TroFq+wg@mail.gmail.com>
Message-ID: <CAOTb1wd0q8LfGjwrsyHRZq9RGGNj_BxZFUyqydNFbBeKuacS0Q@mail.gmail.com>

On Sat, Nov 17, 2012 at 10:55 AM, Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, Nov 18, 2012 at 5:47 AM, Chris Jerdonek
> <chris.jerdonek at gmail.com> wrote:
>> On Thu, Oct 4, 2012 at 2:46 PM,  <martin at v.loewis.de> wrote:
>>> I really fail to see what problem people have with large source files.
>>> What is it that you want to do that can be done easier if it's multiple
>>> files?
>>
>> One thing is browse or link to such code files on the web (e.g. from
>> within a tracker comment or from within our online documentation).
>> For example, today I was unable to open the following page from within
>> a browser to link to one of its lines on a tracker comment:
>>
>> http://hg.python.org/cpython/file/27c20650aeab/Objects/unicodeobject.c
>>
>> My laptop's fan simply turns on and the page hangs indefinitely while loading.
>
> Curious. This sounds like a web browser issue - I can pull it up in
> either Chrome or Firefox on Windows on my 2GHz/2GB RAM laptop with a
> visible pause, but not more than half a second.

I'm also using Chrome and on a fairly new Mac.  Perhaps.  I tried
again and it froze up several open *.python.org tabs (mail.python.org,
bugs.python.org, etc).  However, later it worked as you describe.  The
problem seems sporadic.

--Chris

From kristjan at ccpgames.com  Sat Nov 17 22:52:44 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Sat, 17 Nov 2012 21:52:44 +0000
Subject: [Python-Dev] externals?
In-Reply-To: <20121116121251.GA13953@snakebite.org>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
	<20121116121251.GA13953@snakebite.org>
Message-ID: <EFE3877620384242A686D52278B7CCD329DE039E@RKV-IT-EXCH103.ccp.ad.local>

Thanks for your pro-tip.  Might I suggest that it ought to go into the dev FAQ?  Along with an explanation that a windows dev has to have SVN installed too, just for the laughs?
I think there might be a benefit to moving at least the current externals to a separate HG repository.  We could easily have multiple branches in that repo reflecting the required externals for each version under active HG development).
There is an inherent drawback in having to rely on two different RCS to fetch the necessary stuff, imho.
K

-----Original Message-----
From: Trent Nelson [mailto:trent at snakebite.org] 
Sent: 16. n?vember 2012 12:13
To: Kristj?n Valur J?nsson
Cc: Benjamin Peterson; Python-Dev (python-dev at python.org)
Subject: Re: [Python-Dev] externals?

On Thu, Nov 15, 2012 at 01:20:09AM -0800, Kristj?n Valur J?nsson wrote:
> Perhaps the unix makefiles get the proper version, but a windows developer has to fetch those externals manually.

    Pro-tip: if you're developing on Windows, you're mad if you don't
    prime your dev env with Tools\buildbot\externals.bat.  It takes
    care of *everything*.  I wish every proprietary UNIX system we
    support had something similar.

> Also, is there any reason to keep this in svn?

    I think it's more a case of there being no tangible benefit (and
    numerous drawbacks) to switching it to hg.  I personally have no
    need for a local hg repo with 30 different Tcl/Tk histories in
    it.

    Subversion's good for this sort of use case.  The externals repo
    gets committed to maybe, what, 3-4 times a year?

> Why not check this in to HG, we need not worry about history, etc.

    Who are these mystical people worried about history? ;-)

        Trent.



From kristjan at ccpgames.com  Sat Nov 17 22:48:03 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Sat, 17 Nov 2012 21:48:03 +0000
Subject: [Python-Dev] need reviewers for #16475 and #16487
Message-ID: <EFE3877620384242A686D52278B7CCD329DE035B@RKV-IT-EXCH103.ccp.ad.local>

Hello there.
I'd like to have some pair of eyes on a couple of patches i?ve submitted.

http://bugs.python.org/issue16475
This fixes a regression in marshal between 2.x and 3.x, reinstating string reuse and internment support.  In addition, it generalizes string reuse to all objects, allowing for data optimizations to be made on code objects before marshaling.  This straighforward extension considerably enhances the utility of the marshal module as a low-cost data serialization tool.

http://bugs.python.org/issue16487
This allows ssl contexts to be initialized with certificates from memory, rather than having to rely on the openssl performing its own file IO to read them.   This allows clients and servers that have their certificates deployed e.g. from a db or verbatim in a module, to use ssl without having to resort to temporary disk files and physical IO.

Both of these patches are bourne out of work performed at CCP.  The former comes from work on marshal in order to support our own code object optimizer, which helps save memory on the PS3.  The second comes from us supporting isolated embedded python servers and clients and not wanting to complicate things with unnecessary temporary files for storing credidentials that are obtained from elsewhere.

Both were, of course, 2.7 modifications, that I have now ported to 3.4 for the benefit of the python community.

K





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121117/e389ba20/attachment.html>

From martin at v.loewis.de  Sat Nov 17 23:47:57 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Sat, 17 Nov 2012 23:47:57 +0100
Subject: [Python-Dev] externals?
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DE039E@RKV-IT-EXCH103.ccp.ad.local>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
	<20121116121251.GA13953@snakebite.org>
	<EFE3877620384242A686D52278B7CCD329DE039E@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <20121117234757.Horde.9Jf8druWis5QqBQd8O5BnVA@webmail.df.eu>


Zitat von Kristj?n Valur J?nsson <kristjan at ccpgames.com>:

> Thanks for your pro-tip.  Might I suggest that it ought to go into  
> the dev FAQ?  Along with an explanation that a windows dev has to  
> have SVN installed too, just for the laughs?
> I think there might be a benefit to moving at least the current  
> externals to a separate HG repository.  We could easily have  
> multiple branches in that repo reflecting the required externals for  
> each version under active HG development).

Feel free to work on this. Consider that using hg may significantly
increase the amount of network traffic, since the repo(s) will contain
multiple versions, when only one specific version is needed.

When working on this, try to come up with other automated download
procedures, e.g. ones that do not involve revision control and have
support built into Windows.

Regards,
Martin



From solipsis at pitrou.net  Sun Nov 18 00:05:31 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 18 Nov 2012 00:05:31 +0100
Subject: [Python-Dev] Failed issue tracker submission
In-Reply-To: <CAOTb1wdbaSWFnKkf2N5kzrddjbJgvCE1=5KgmNJ6v+eBqhESRA@mail.gmail.com>
References: <20121116023950.2EC2B1CDAD@psf.upfronthosting.co.za>
	<CAOTb1wcaUsB8-ZOTGgAdSJHg519iMMxm8=PU-i3DVXhS43a_bQ@mail.gmail.com>
	<20121116234111.7ef55ee8@pitrou.net>
	<CAP7+vJKG3a7T_gV4_kkJoi4WWtsi90BJQCS+XBy_MrA-DsMioA@mail.gmail.com>
	<20121117004733.Horde.GGeAbLuWis5QptCVhIcgUVA@webmail.df.eu>
	<20121117010152.7ade2b38@pitrou.net>
	<20121117060607.68142250129@webabinitio.net>
	<CAOTb1wdbaSWFnKkf2N5kzrddjbJgvCE1=5KgmNJ6v+eBqhESRA@mail.gmail.com>
Message-ID: <20121118000531.6cf0b00b@pitrou.net>

On Fri, 16 Nov 2012 22:35:38 -0800
Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
> >
> > I'm pretty sure it's because python-dev is the 'from' address
> > used when the messages are sent...and the configuration of
> > that user is what allows them to be accepted.
> >
> > I suggest changing the from address and the account configuration
> > to tracker-discuss at python.org instead.
> 
> Above and on the meta tracker issue I filed, it sounds like Martin is
> saying that the e-mails should never be going to python-dev (or
> tracker-discuss) in the first place -- due to a roundup bug and
> because the e-mail is already being sent to the submitter and the
> roundup admins.  Thus, changing the from address would only mitigate
> the problem and not be fixing the root cause.

So, for the record, the "From" address has been changed to
tracker-discuss at python.org, so python-dev shouldn't be spammed
anymore.

Cheers

Antoine.

From guido at python.org  Sun Nov 18 01:05:04 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 17 Nov 2012 16:05:04 -0800
Subject: [Python-Dev] need reviewers for #16475 and #16487
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DE035B@RKV-IT-EXCH103.ccp.ad.local>
References: <EFE3877620384242A686D52278B7CCD329DE035B@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <CAP7+vJ+f+csADFCSpZFvUxhVV=X_pu5jGsHDc2yJaz1wYcuoZw@mail.gmail.com>

On Sat, Nov 17, 2012 at 1:48 PM, Kristj?n Valur J?nsson
<kristjan at ccpgames.com> wrote:
> Hello there.
>
> I?d like to have some pair of eyes on a couple of patches i?ve submitted.

Sorry, can't help you with these, but I've got a language nit...

> http://bugs.python.org/issue16475
>
> This fixes a regression in marshal between 2.x and 3.x, reinstating string
> reuse and internment support.  In addition, it generalizes string reuse to

It's not internment -- that means imprisonment. The term we use is
interning. (The dictionary will tell you that means imprisonment too
-- but it's long been used as the name for this particular technique.
Internment has not.)

> all objects, allowing for data optimizations to be made on code objects
> before marshaling.  This straighforward extension considerably enhances the
> utility of the marshal module as a low-cost data serialization tool.
>
>
>
> http://bugs.python.org/issue16487
>
> This allows ssl contexts to be initialized with certificates from memory,
> rather than having to rely on the openssl performing its own file IO to read
> them.   This allows clients and servers that have their certificates
> deployed e.g. from a db or verbatim in a module, to use ssl without having
> to resort to temporary disk files and physical IO.
>
>
>
> Both of these patches are bourne out of work performed at CCP.  The former
> comes from work on marshal in order to support our own code object
> optimizer, which helps save memory on the PS3.  The second comes from us
> supporting isolated embedded python servers and clients and not wanting to
> complicate things with unnecessary temporary files for storing credidentials
> that are obtained from elsewhere.
>
>
>
> Both were, of course, 2.7 modifications, that I have now ported to 3.4 for
> the benefit of the python community.
>
>
>
> K
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)

From g.brandl at gmx.net  Sun Nov 18 08:22:23 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 18 Nov 2012 08:22:23 +0100
Subject: [Python-Dev] externals?
In-Reply-To: <20121117234757.Horde.9Jf8druWis5QqBQd8O5BnVA@webmail.df.eu>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
	<20121116121251.GA13953@snakebite.org>
	<EFE3877620384242A686D52278B7CCD329DE039E@RKV-IT-EXCH103.ccp.ad.local>
	<20121117234757.Horde.9Jf8druWis5QqBQd8O5BnVA@webmail.df.eu>
Message-ID: <k8a2at$tg5$1@ger.gmane.org>

Am 17.11.2012 23:47, schrieb martin at v.loewis.de:
> 
> Zitat von Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> 
>> Thanks for your pro-tip.  Might I suggest that it ought to go into  
>> the dev FAQ?  Along with an explanation that a windows dev has to  
>> have SVN installed too, just for the laughs?
>> I think there might be a benefit to moving at least the current  
>> externals to a separate HG repository.  We could easily have  
>> multiple branches in that repo reflecting the required externals for  
>> each version under active HG development).
> 
> Feel free to work on this. Consider that using hg may significantly
> increase the amount of network traffic, since the repo(s) will contain
> multiple versions, when only one specific version is needed.
> 
> When working on this, try to come up with other automated download
> procedures, e.g. ones that do not involve revision control and have
> support built into Windows.

One way would be to use one hg repo per version, and (maybe, if needed)
a master repo that has them as subrepos.  Whoever needs externals can
download the repo as a zipfile and unpack it (both of which is possible
with batteries only).

Note that in this scenario, hg is used mostly in order to avoid another
service (such as rsync), and for developer convenience.

Georg


From arigo at tunes.org  Sun Nov 18 10:00:43 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 18 Nov 2012 10:00:43 +0100
Subject: [Python-Dev] externals?
In-Reply-To: <k8a2at$tg5$1@ger.gmane.org>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
	<20121116121251.GA13953@snakebite.org>
	<EFE3877620384242A686D52278B7CCD329DE039E@RKV-IT-EXCH103.ccp.ad.local>
	<20121117234757.Horde.9Jf8druWis5QqBQd8O5BnVA@webmail.df.eu>
	<k8a2at$tg5$1@ger.gmane.org>
Message-ID: <CAMSv6X1PvEvV-4zBzfiUKsVwpC+5LU7=Ug36vKsjkcpHj2C=cw@mail.gmail.com>

Hi,

On Sun, Nov 18, 2012 at 8:22 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> One way would be to use one hg repo per version, and (maybe, if needed)
> a master repo that has them as subrepos.

Or have all versions in the same repo as usual (with branches), but
have hg subrepos point to different repos: ones extracted from the
main repo by containing only the correct branch.  But it might be a
bit delicate to pull this off.  (hg clone takes a "-r" option and
copies only things needed for the given revision or branch, but
apparently we can't pass this option automatically to the cloning of
subrepos.  (Maybe it points out that subrepos are a hack best done
without altogether, which is what we did in pypy.))


A bient?t,

Armin.

From g.brandl at gmx.net  Sun Nov 18 10:04:24 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 18 Nov 2012 10:04:24 +0100
Subject: [Python-Dev] externals?
In-Reply-To: <CAMSv6X1PvEvV-4zBzfiUKsVwpC+5LU7=Ug36vKsjkcpHj2C=cw@mail.gmail.com>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
	<20121116121251.GA13953@snakebite.org>
	<EFE3877620384242A686D52278B7CCD329DE039E@RKV-IT-EXCH103.ccp.ad.local>
	<20121117234757.Horde.9Jf8druWis5QqBQd8O5BnVA@webmail.df.eu>
	<k8a2at$tg5$1@ger.gmane.org>
	<CAMSv6X1PvEvV-4zBzfiUKsVwpC+5LU7=Ug36vKsjkcpHj2C=cw@mail.gmail.com>
Message-ID: <k8a8a7$3bf$1@ger.gmane.org>

Am 18.11.2012 10:00, schrieb Armin Rigo:
> Hi,
> 
> On Sun, Nov 18, 2012 at 8:22 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>> One way would be to use one hg repo per version, and (maybe, if needed)
>> a master repo that has them as subrepos.
> 
> Or have all versions in the same repo as usual (with branches), but
> have hg subrepos point to different repos: ones extracted from the
> main repo by containing only the correct branch.  But it might be a
> bit delicate to pull this off.  (hg clone takes a "-r" option and
> copies only things needed for the given revision or branch, but
> apparently we can't pass this option automatically to the cloning of
> subrepos.  (Maybe it points out that subrepos are a hack best done
> without altogether, which is what we did in pypy.))

Yep.  Anyway, if every external version goes into a branch, then we don't
need subrepos anyway.  That is a better idea than mine.

Since you can use (e.g.) "hg clone -r tk-8.5" or download a tarball
specific to a branch, nobody should need to get the whole externals
history on clone.

Georg


From kristjan at ccpgames.com  Sun Nov 18 11:54:57 2012
From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=)
Date: Sun, 18 Nov 2012 10:54:57 +0000
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
Message-ID: <EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>

Yes!
For many years I have been very frustrated by the install-centric nature of python.  I am biased, of course, by the fact that I am developing an application where python is embedded, an application that needs to run out of the box.  A developer may have many many versions (branches) of the application on his drive, and each needs to work separately.
We have managed to isolate things, by patching python (and contributing that patch) to override the default library seach path (and ignore environment paths) when python is started up thogh the api.  All well and good.
But recently we have started in increasing amount to use external libraries and packages and we have been introduced to the dependency hell that is public python packages.  In this install-centric world, developers reference huge external packages without a second thought, which cause large dependency trees.  Using a simple tool may require whole HTTP frameworks to be downloaded.
What is worse is when there are versioning conflicts between those dependencies.

I don't have a well formed solution in mind, but I would see it desirable to have a way for someone to release his package with all its dependencies as a self-contained and isolated unit.  E.g. if package foo.py relies on functionality from version 1.7 of bar.py, then that functionality could be bottled up for foo?s exclusive usage.
Another package, baz.py, could then also make use of bar, but version 1.8.  The two bar versions would be isolated.

Perhaps this is just a pipedream.  Even unpossible.  But it doesn't harm to try to think about better ways to do things.
K


-----Original Message-----
From: Christian Tismer [mailto:tismer at stackless.com] 
Sent: 15. n?vember 2012 23:10
To: Kristj?n Valur J?nsson
Cc: python-dev at python.org
Subject: Generally boared by installation (Re: [Python-Dev] Setting project home path the best way)

Hi guys,

I am bored of installing things. 
Bored of things that happen to not work for some minor reasons. 
Reasons that are not immediately obvious. 
Things that don't work because some special case was not handled. 
Things that compile for half an hour and then complain that something is not as expected. 
May it be a compiler, a library, a command like pip or easy-install, a system like macports or homebrew, virtualenv, whatsoever. 

These things are all great if they work. 

When they do not work, the user is in some real trouble. And he reads hundreds Of blogs and sites and emails, which all answer a bit of slightly related questions, but all in all - 

This is not how Python should work !!

I am really bored and exhausted and annoyed by those packages which Pretend to make my life eadier, but they don't really. 

Something is really missing. I want something that is easy to use in all cases, also if it fails. 

Son't get me wrong, I like things like pip or virtualenv or homebrew. 
I just think they have to be rewritten completely. They have the wrong assumption that things work!

The opposite should be the truth: by default, things go wrong. Correctness is very fragile. 

I am thinking of a better concept that is harder to break. I thin to design a setup tool that is much more checking itself and does not trust in any assumption. 

Scenario:
After hours and hours, I find how to modify setup.py to function almost correctly for PySide. 

This was ridiculously hard to do! Settings for certain directories, included and stuff are not checked when they could be, but after compiling a lot of things!

After a lot of tries and headaches, I find out that virtualenv barfs on a simple link like ./.Python, the executable, when switching from stock Python to a different (homebrew) version!!

This was obviously never tested well, so it frustrates me quite a lot.  

I could fill a huge list full of complaints like that if I had time. But I don't. 

Instead, I think installation scripts are generally still wrong by concept today and need to be written in a different way. 

I would like to start a task force and some sprints about improving this situation. 
My goal is some unbreakable system of building blocks that are self-contained with no dependencies, that have a defined interface to talk to, and that know themselves completely by introspection. 

They should not work because they happen to work around all known defects, but by design and control. 

Whoever is interested to work with me on this is hereby honestly welcomed!

Cheers - chris

Sent from my Ei4Steve

On Nov 15, 2012, at 10:17, Kristj?n Valur J?nsson <kristjan at ccpgames.com> wrote:

> When python is being run from a compile environment, it detects this by looking for "Lib" folders in directories above the one containing the executable. 
> (I always thought that this "special" execution mode, hardwired in, 
> was a bit odd, and suggested that this could be made a function of pep405) Anyway, keeping your executable as part of the tree is the trick I use, and to make things nice I put  right next to it:
> site.py
> sitecustomize.py
> 
> sitecustomize.py is where you would put the logic to set sys.path by walking up the hierarchy and finding the proper root.
> site.py is there to merely import sitecustomize.py, in case a site.py is not found in all the default places python looks.
> 
> K
> 
> 
>> -----Original Message-----
>> From: Python-Dev [mailto:python-dev-
>> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Christian 
>> bounces+Tismer
>> Sent: 11. n?vember 2012 20:31
>> To: python-dev at python.org
>> Subject: [Python-Dev] Setting project home path the best way
>> 
>> Hi friends,
>> 
>> I have a project that has its root somewhere on my machine.
>> This project has many folders and contains quite some modules.
>> 
>> There is a common root of the module tree, and I want to use
>> - either absolute imports
>> - relative imports with '.'
>> 
>> Problem:
>> 
>> - I want to run any module inside the heirarchy from the command-line
>> 
>> - this should work, regardless what my 'cwd' is
>> 
>> - this should work with or without virtualenv.
>> 
>> So far, things work fine with virtualenv, because sys.executable is 
>> in the project module tree.
>> 
>> Without virtualenv, this is not so. But I hate to make settings like 
>> PYTHONPATH, because these are not permanent. .
>> 
>> Question:
>> 
>> How should I define my project root dir in a unique way, without 
>> setting an environment variable?
>> What is the lest intrusive way to spell that?
>> 
>> Reason:
>> 
>> I'd like to make things work correctly and unambigously when I call a 
>> script inside the module heirarchy. Things are not fixed: there exist 
>> many checkouts In the file system, and each should know where to 
>> search its home/root in the tree.
>> 
>> Is this elegantly possible to deduce from the actually executed script file?
>> 
>> Cheers - chris
>> 
>> Sent from my Ei4Steve
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-
>> dev/kristjan%40ccpgames.com
> 
> <winmail.dat>


From ncoghlan at gmail.com  Sun Nov 18 12:24:49 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 18 Nov 2012 21:24:49 +1000
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>

On Sun, Nov 18, 2012 at 8:54 PM, Kristj?n Valur J?nsson <
kristjan at ccpgames.com> wrote:

> I don't have a well formed solution in mind, but I would see it desirable
> to have a way for someone to release his package with all its dependencies
> as a self-contained and isolated unit.  E.g. if package foo.py relies on
> functionality from version 1.7 of bar.py, then that functionality could be
> bottled up for foo?s exclusive usage.
> Another package, baz.py, could then also make use of bar, but version 1.8.
>  The two bar versions would be isolated.
>
> Perhaps this is just a pipedream.  Even unpossible.  But it doesn't harm
> to try to think about better ways to do things.
>
>
Easily bundling dependencies is a key principle behind the ability to
execute directories and zipfiles that contain a top level __main__.py file
that was added back in 2.6 (although the zipfile version doesn't play
nicely with extension modules).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121118/e0da2c51/attachment.html>

From kristjan at ccpgames.com  Sun Nov 18 12:18:45 2012
From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=)
Date: Sun, 18 Nov 2012 11:18:45 +0000
Subject: [Python-Dev] Register-based VM for CPython
In-Reply-To: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>
References: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD329DE0CF6@RKV-IT-EXCH103.ccp.ad.local>

Interesting work indeed.
From profiling CPython it has long been clear to me that enormous gains can be made by making instruction dispatching faster.  A huge amount of time is spent in the evaluation loop.  I have also been making small inroads to offline bytecode optimization.  Identifying common patterns and introducing special opcodes to deal with them.  Obviously using register addressing makes such an approach more effective.

(Working with code objects is fun and exciting, btw, and the reason for my patch http://bugs.python.org/issue16475)

K

From: Python-Dev [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Victor Stinner
Sent: 17. n?vember 2012 01:13
To: Python Dev
Subject: [Python-Dev] Register-based VM for CPython


The WPython project is similar to my work (except that it does not use registers). It tries also to reduce the overhead of instruction dispatch by using more complex instructions.
http://code.google.com/p/wpython/

Using registers instead of a stack allow to implement more optimizations (than WPython). For example, it's possible to load constants outside loops and merge "duplicate constant loads".

I also implemented more aggressive and experimental optimizations (disabled by default) which may break applications: move loads of attributes and globals outside of loops, and replace binary operations with inplace operations. For example, "x=[]; for ...: x.append(...)" is optimized to something like "x=[]; x_append=x.append; for ...: x_append(...)", and "x = x + 1" is replaced with "x += 1".

Victor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121118/04c05c17/attachment.html>

From solipsis at pitrou.net  Sun Nov 18 12:41:56 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 18 Nov 2012 12:41:56 +0100
Subject: [Python-Dev] Register-based VM for CPython
References: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>
	<CAMSv6X2hXLwB8NOSgONbVve2D1rMSXjEPmFWJVesrc4uaDxRYw@mail.gmail.com>
Message-ID: <20121118124156.1e2651a0@pitrou.net>

On Sat, 17 Nov 2012 11:17:40 +0100
Armin Rigo <arigo at tunes.org> wrote:
> Hi Victor,
> 
> On Sat, Nov 17, 2012 at 2:13 AM, Victor Stinner
> <victor.stinner at gmail.com> wrote:
> > The major drawback of the register approach (at least of my implementation)
> > is that it changes the lifetime of objects. Newly created objects are only
> > "destroyed" at the exit of the function, whereas the stack-based VM destroys
> > "immediatly" objects (thanks to the reference counter). PyPy has similar
> > issues with its different garbage collector.
> 
> That is not strictly correct.  PyPy, Jython and IronPython have
> non-immediate destructors, but as far as I can tell they all avoid to
> keep objects alive for an unbounded amount of time.  This important
> difference is visible if the function calls other code that takes a
> long while to run: in your approach, the objects created by the
> function itself will stay alive for the whole duration, while the
> other interpreters will all release them soon after they are not
> referenced any more --- not instantly like CPython but still soon.

Agreed with Armin.
Also, I would point out that the reference counting behaviour is an
important feature of *C*Python (to the point that we have test cases
checking against reference cycles), so we can't break it nilly-willy.

Regards

Antoine.



From martin at v.loewis.de  Sun Nov 18 13:18:43 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Sun, 18 Nov 2012 13:18:43 +0100
Subject: [Python-Dev] externals?
In-Reply-To: <CAMSv6X1PvEvV-4zBzfiUKsVwpC+5LU7=Ug36vKsjkcpHj2C=cw@mail.gmail.com>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
	<20121116121251.GA13953@snakebite.org>
	<EFE3877620384242A686D52278B7CCD329DE039E@RKV-IT-EXCH103.ccp.ad.local>
	<20121117234757.Horde.9Jf8druWis5QqBQd8O5BnVA@webmail.df.eu>
	<k8a2at$tg5$1@ger.gmane.org>
	<CAMSv6X1PvEvV-4zBzfiUKsVwpC+5LU7=Ug36vKsjkcpHj2C=cw@mail.gmail.com>
Message-ID: <20121118131843.Horde.8WNlTtjz9kRQqNIjgKDTaVA@webmail.df.eu>


Zitat von Armin Rigo <arigo at tunes.org>:

> Hi,
>
> On Sun, Nov 18, 2012 at 8:22 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>> One way would be to use one hg repo per version, and (maybe, if needed)
>> a master repo that has them as subrepos.
>
> Or have all versions in the same repo as usual (with branches), but
> have hg subrepos point to different repos: ones extracted from the
> main repo by containing only the correct branch.  But it might be a
> bit delicate to pull this off.  (hg clone takes a "-r" option and
> copies only things needed for the given revision or branch, but
> apparently we can't pass this option automatically to the cloning of
> subrepos.  (Maybe it points out that subrepos are a hack best done
> without altogether, which is what we did in pypy.))

I'd like to stress that we don't need any versioning here. wget and
tar would be sufficient, except that it's Windows, so we have neither
wget nor tar. However, including a PowerShell script may be an option;
most developers will have PowerShell already on their system. AFAICT,
PowerShell can do HTTP downloads and extract zip files.

Regards,
Martin



From storchaka at gmail.com  Sun Nov 18 14:58:33 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 18 Nov 2012 15:58:33 +0200
Subject: [Python-Dev] Register-based VM for CPython
In-Reply-To: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>
References: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>
Message-ID: <k8apia$28k$1@ger.gmane.org>

On 17.11.12 03:13, Victor Stinner wrote:
> The major drawback of the register approach (at least of my
> implementation) is that it changes the lifetime of objects. Newly
> created objects are only "destroyed" at the exit of the function,
> whereas the stack-based VM destroys "immediatly" objects (thanks to the
> reference counter).

It should not be a problem. Just register instructions should clear 
input registers if they are not binded to named local variables. I.e. "a 
= b + c * d" should be compiled to:

    BINARY_MUL_REG       R1, 'c', 'd'
    BINARY_ADD_REG       'a', 'b', R1 # R1 cleared



From benjamin at python.org  Sun Nov 18 15:37:57 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 18 Nov 2012 09:37:57 -0500
Subject: [Python-Dev] Register-based VM for CPython
In-Reply-To: <20121118124156.1e2651a0@pitrou.net>
References: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>
	<CAMSv6X2hXLwB8NOSgONbVve2D1rMSXjEPmFWJVesrc4uaDxRYw@mail.gmail.com>
	<20121118124156.1e2651a0@pitrou.net>
Message-ID: <CAPZV6o-BFQmA_+O_s+10cbnzk5pocS2agAajf19AEi6d18pQ0A@mail.gmail.com>

2012/11/18 Antoine Pitrou <solipsis at pitrou.net>:
> Also, I would point out that the reference counting behaviour is an
> important feature of *C*Python (to the point that we have test cases
> checking against reference cycles), so we can't break it nilly-willy.

The tests about reference cycles are just tests that garbage is
collected, a Python language feature. Those aren't technically CPython
specific.


-- 
Regards,
Benjamin

From solipsis at pitrou.net  Sun Nov 18 16:11:34 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 18 Nov 2012 16:11:34 +0100
Subject: [Python-Dev] Register-based VM for CPython
In-Reply-To: <CAPZV6o-BFQmA_+O_s+10cbnzk5pocS2agAajf19AEi6d18pQ0A@mail.gmail.com>
References: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>
	<CAMSv6X2hXLwB8NOSgONbVve2D1rMSXjEPmFWJVesrc4uaDxRYw@mail.gmail.com>
	<20121118124156.1e2651a0@pitrou.net>
	<CAPZV6o-BFQmA_+O_s+10cbnzk5pocS2agAajf19AEi6d18pQ0A@mail.gmail.com>
Message-ID: <20121118161134.2e7de294@pitrou.net>

On Sun, 18 Nov 2012 09:37:57 -0500
Benjamin Peterson <benjamin at python.org> wrote:

> 2012/11/18 Antoine Pitrou <solipsis at pitrou.net>:
> > Also, I would point out that the reference counting behaviour is an
> > important feature of *C*Python (to the point that we have test cases
> > checking against reference cycles), so we can't break it nilly-willy.
> 
> The tests about reference cycles are just tests that garbage is
> collected, a Python language feature. Those aren't technically CPython
> specific.

We do have tests which check reference cycles are not created.

Regards

Antoine.

From benjamin at python.org  Sun Nov 18 17:27:32 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 18 Nov 2012 11:27:32 -0500
Subject: [Python-Dev] Register-based VM for CPython
In-Reply-To: <20121118161134.2e7de294@pitrou.net>
References: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>
	<CAMSv6X2hXLwB8NOSgONbVve2D1rMSXjEPmFWJVesrc4uaDxRYw@mail.gmail.com>
	<20121118124156.1e2651a0@pitrou.net>
	<CAPZV6o-BFQmA_+O_s+10cbnzk5pocS2agAajf19AEi6d18pQ0A@mail.gmail.com>
	<20121118161134.2e7de294@pitrou.net>
Message-ID: <CAPZV6o9u+EYHMJDYMWw3wF=f5w8sHC6qwDmNp21KkUyBYJbvRw@mail.gmail.com>

2012/11/18 Antoine Pitrou <solipsis at pitrou.net>:
> On Sun, 18 Nov 2012 09:37:57 -0500
> Benjamin Peterson <benjamin at python.org> wrote:
>
>> 2012/11/18 Antoine Pitrou <solipsis at pitrou.net>:
>> > Also, I would point out that the reference counting behaviour is an
>> > important feature of *C*Python (to the point that we have test cases
>> > checking against reference cycles), so we can't break it nilly-willy.
>>
>> The tests about reference cycles are just tests that garbage is
>> collected, a Python language feature. Those aren't technically CPython
>> specific.
>
> We do have tests which check reference cycles are not created.

Oh, those. Those are implementation details.



-- 
Regards,
Benjamin

From brian at python.org  Sun Nov 18 18:05:45 2012
From: brian at python.org (Brian Curtin)
Date: Sun, 18 Nov 2012 11:05:45 -0600
Subject: [Python-Dev] externals?
In-Reply-To: <20121118131843.Horde.8WNlTtjz9kRQqNIjgKDTaVA@webmail.df.eu>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
	<20121116121251.GA13953@snakebite.org>
	<EFE3877620384242A686D52278B7CCD329DE039E@RKV-IT-EXCH103.ccp.ad.local>
	<20121117234757.Horde.9Jf8druWis5QqBQd8O5BnVA@webmail.df.eu>
	<k8a2at$tg5$1@ger.gmane.org>
	<CAMSv6X1PvEvV-4zBzfiUKsVwpC+5LU7=Ug36vKsjkcpHj2C=cw@mail.gmail.com>
	<20121118131843.Horde.8WNlTtjz9kRQqNIjgKDTaVA@webmail.df.eu>
Message-ID: <CAD+XWwrf_Qedv+_+-T+Y=Wj9waY5ud0dFX-L8Nu9aZ+QCLvVQg@mail.gmail.com>

On Sun, Nov 18, 2012 at 6:18 AM,  <martin at v.loewis.de> wrote:
>
> Zitat von Armin Rigo <arigo at tunes.org>:
>
>
>> Hi,
>>
>> On Sun, Nov 18, 2012 at 8:22 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>>>
>>> One way would be to use one hg repo per version, and (maybe, if needed)
>>> a master repo that has them as subrepos.
>>
>>
>> Or have all versions in the same repo as usual (with branches), but
>> have hg subrepos point to different repos: ones extracted from the
>> main repo by containing only the correct branch.  But it might be a
>> bit delicate to pull this off.  (hg clone takes a "-r" option and
>> copies only things needed for the given revision or branch, but
>> apparently we can't pass this option automatically to the cloning of
>> subrepos.  (Maybe it points out that subrepos are a hack best done
>> without altogether, which is what we did in pypy.))
>
>
> I'd like to stress that we don't need any versioning here. wget and
> tar would be sufficient, except that it's Windows, so we have neither
> wget nor tar. However, including a PowerShell script may be an option;
> most developers will have PowerShell already on their system. AFAICT,
> PowerShell can do HTTP downloads and extract zip files.

I would hope we can just write a simple Python script to do this,
rather than require PowerShell.

I'm 99.99999% certain anyone building Python on Windows will already
have a version of Python installed. Plus, they're going to need it
anyway to build OpenSSL (see PCbuild/build_ssl.py and the references
to it in VS projects).

From solipsis at pitrou.net  Sun Nov 18 18:31:49 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 18 Nov 2012 18:31:49 +0100
Subject: [Python-Dev] Register-based VM for CPython
In-Reply-To: <CAPZV6o9u+EYHMJDYMWw3wF=f5w8sHC6qwDmNp21KkUyBYJbvRw@mail.gmail.com>
References: <CAMpsgwaONSm_BGAB3hjrSde=yJjGEWH2Cw5r-t01vaMnz6i=dQ@mail.gmail.com>
	<CAMSv6X2hXLwB8NOSgONbVve2D1rMSXjEPmFWJVesrc4uaDxRYw@mail.gmail.com>
	<20121118124156.1e2651a0@pitrou.net>
	<CAPZV6o-BFQmA_+O_s+10cbnzk5pocS2agAajf19AEi6d18pQ0A@mail.gmail.com>
	<20121118161134.2e7de294@pitrou.net>
	<CAPZV6o9u+EYHMJDYMWw3wF=f5w8sHC6qwDmNp21KkUyBYJbvRw@mail.gmail.com>
Message-ID: <20121118183149.7625db59@pitrou.net>

On Sun, 18 Nov 2012 11:27:32 -0500
Benjamin Peterson <benjamin at python.org> wrote:
> 2012/11/18 Antoine Pitrou <solipsis at pitrou.net>:
> > On Sun, 18 Nov 2012 09:37:57 -0500
> > Benjamin Peterson <benjamin at python.org> wrote:
> >
> >> 2012/11/18 Antoine Pitrou <solipsis at pitrou.net>:
> >> > Also, I would point out that the reference counting behaviour is an
> >> > important feature of *C*Python (to the point that we have test cases
> >> > checking against reference cycles), so we can't break it nilly-willy.
> >>
> >> The tests about reference cycles are just tests that garbage is
> >> collected, a Python language feature. Those aren't technically CPython
> >> specific.
> >
> > We do have tests which check reference cycles are not created.
> 
> Oh, those. Those are implementation details.

At this point I'm not sure which statement you are arguing against.

Regards

Antoine.

From tjreedy at udel.edu  Sun Nov 18 20:12:12 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 18 Nov 2012 14:12:12 -0500
Subject: [Python-Dev] externals?
In-Reply-To: <CAD+XWwrf_Qedv+_+-T+Y=Wj9waY5ud0dFX-L8Nu9aZ+QCLvVQg@mail.gmail.com>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
	<20121116121251.GA13953@snakebite.org>
	<EFE3877620384242A686D52278B7CCD329DE039E@RKV-IT-EXCH103.ccp.ad.local>
	<20121117234757.Horde.9Jf8druWis5QqBQd8O5BnVA@webmail.df.eu>
	<k8a2at$tg5$1@ger.gmane.org>
	<CAMSv6X1PvEvV-4zBzfiUKsVwpC+5LU7=Ug36vKsjkcpHj2C=cw@mail.gmail.com>
	<20121118131843.Horde.8WNlTtjz9kRQqNIjgKDTaVA@webmail.df.eu>
	<CAD+XWwrf_Qedv+_+-T+Y=Wj9waY5ud0dFX-L8Nu9aZ+QCLvVQg@mail.gmail.com>
Message-ID: <k8bbuf$j4b$1@ger.gmane.org>

On 11/18/2012 12:05 PM, Brian Curtin wrote:
> On Sun, Nov 18, 2012 at 6:18 AM,  <martin at v.loewis.de> wrote:
>>
>> Zitat von Armin Rigo <arigo at tunes.org>:

>>> Or have all versions in the same repo as usual (with branches), but
>>> have hg subrepos point to different repos: ones extracted from the
>>> main repo by containing only the correct branch.  But it might be a
>>> bit delicate to pull this off.  (hg clone takes a "-r" option and
>>> copies only things needed for the given revision or branch, but
>>> apparently we can't pass this option automatically to the cloning of
>>> subrepos.  (Maybe it points out that subrepos are a hack best done
>>> without altogether, which is what we did in pypy.))
>>
>>
>> I'd like to stress that we don't need any versioning here. wget and
>> tar would be sufficient, except that it's Windows, so we have neither
>> wget nor tar. However, including a PowerShell script may be an option;
>> most developers will have PowerShell already on their system. AFAICT,
>> PowerShell can do HTTP downloads and extract zip files.
>
> I would hope we can just write a simple Python script to do this,
> rather than require PowerShell.
>
> I'm 99.99999% certain anyone building Python on Windows will already
> have a version of Python installed. Plus, they're going to need it
> anyway to build OpenSSL (see PCbuild/build_ssl.py and the references
> to it in VS projects).

After reading the thread, I realize that I do not actually want externam 
dependency files moved to hg. I and others are not going to push changes 
back, so we do not need hg clones.

What would be good would to be able to access the files and use them to 
build python without svn installed. I don't know the best way to do 
that, but if tarred or zipped releases were made for each version that 
should be downloaded, our urllib, tarfile/ziplib, and any other modules 
needed should be sufficient to transfer and unpack.

-- 
Terry Jan Reedy


From kristjan at ccpgames.com  Mon Nov 19 11:29:38 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Mon, 19 Nov 2012 10:29:38 +0000
Subject: [Python-Dev] need reviewers for #16475 and #16487
In-Reply-To: <CAP7+vJ+f+csADFCSpZFvUxhVV=X_pu5jGsHDc2yJaz1wYcuoZw@mail.gmail.com>
References: <EFE3877620384242A686D52278B7CCD329DE035B@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJ+f+csADFCSpZFvUxhVV=X_pu5jGsHDc2yJaz1wYcuoZw@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD329DE1B76@RKV-IT-EXCH103.ccp.ad.local>

Thank you!  The sensitivity of this issue obviously is born out of our collective
bad conscience for this unjust incarceration.
K

> -----Original Message-----
> From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf
> Of Guido van Rossum
> >
> > This fixes a regression in marshal between 2.x and 3.x, reinstating
> > string reuse and internment support.  In addition, it generalizes
> > string reuse to
> 
> It's not internment -- that means imprisonment. The term we use is
> interning. (The dictionary will tell you that means imprisonment too
> -- but it's long been used as the name for this particular technique.
> Internment has not.)



From kristjan at ccpgames.com  Mon Nov 19 11:43:03 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Mon, 19 Nov 2012 10:43:03 +0000
Subject: [Python-Dev] externals?
In-Reply-To: <k8bbuf$j4b$1@ger.gmane.org>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
	<20121116121251.GA13953@snakebite.org>
	<EFE3877620384242A686D52278B7CCD329DE039E@RKV-IT-EXCH103.ccp.ad.local>
	<20121117234757.Horde.9Jf8druWis5QqBQd8O5BnVA@webmail.df.eu>
	<k8a2at$tg5$1@ger.gmane.org>
	<CAMSv6X1PvEvV-4zBzfiUKsVwpC+5LU7=Ug36vKsjkcpHj2C=cw@mail.gmail.com>
	<20121118131843.Horde.8WNlTtjz9kRQqNIjgKDTaVA@webmail.df.eu>
	<CAD+XWwrf_Qedv+_+-T+Y=Wj9waY5ud0dFX-L8Nu9aZ+QCLvVQg@mail.gmail.com>
	<k8bbuf$j4b$1@ger.gmane.org>
Message-ID: <EFE3877620384242A686D52278B7CCD329DE1BC0@RKV-IT-EXCH103.ccp.ad.local>

But that's what hg clone does.
You have a lorry for your work at the mine.  You don't need a Mini to go to the fishmongers.  You can use your lorry even if you are not going to dump a tonne of ore on the pavement.
K

> -----Original Message-----
> 
> What would be good would to be able to access the files and use them to
> build python without svn installed. I don't know the best way to do that, but
> if tarred or zipped releases were made for each version that should be
> downloaded, our urllib, tarfile/ziplib, and any other modules needed should
> be sufficient to transfer and unpack.



From trent at snakebite.org  Mon Nov 19 12:42:31 2012
From: trent at snakebite.org (Trent Nelson)
Date: Mon, 19 Nov 2012 06:42:31 -0500
Subject: [Python-Dev] externals?
In-Reply-To: <20121118131843.Horde.8WNlTtjz9kRQqNIjgKDTaVA@webmail.df.eu>
References: <EFE3877620384242A686D52278B7CCD329DD7EF4@RKV-IT-EXCH103.ccp.ad.local>
	<CAPZV6o8geJWfgDp3L-dOmt_6=pfgVayh6V7EJAEGr0R+=DPoyw@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DDC6E0@RKV-IT-EXCH103.ccp.ad.local>
	<20121116121251.GA13953@snakebite.org>
	<EFE3877620384242A686D52278B7CCD329DE039E@RKV-IT-EXCH103.ccp.ad.local>
	<20121117234757.Horde.9Jf8druWis5QqBQd8O5BnVA@webmail.df.eu>
	<k8a2at$tg5$1@ger.gmane.org>
	<CAMSv6X1PvEvV-4zBzfiUKsVwpC+5LU7=Ug36vKsjkcpHj2C=cw@mail.gmail.com>
	<20121118131843.Horde.8WNlTtjz9kRQqNIjgKDTaVA@webmail.df.eu>
Message-ID: <20121119114126.GA31524@snakebite.org>

On Sun, Nov 18, 2012 at 04:18:43AM -0800, martin at v.loewis.de wrote:
> I'd like to stress that we don't need any versioning here. wget and
> tar would be sufficient, except that it's Windows, so we have neither
> wget nor tar. However, including a PowerShell script may be an option;
> most developers will have PowerShell already on their system. AFAICT,
> PowerShell can do HTTP downloads and extract zip files.

    I'm disturbed that I subconsciously interpreted this as a challenge
    to do it via a standalone batch/.bat ;-)

        Trent.

From brian at python.org  Mon Nov 19 16:01:44 2012
From: brian at python.org (Brian Curtin)
Date: Mon, 19 Nov 2012 09:01:44 -0600
Subject: [Python-Dev] New Contributor Experience in Python and other FOSS
	Communities - A Survey
Message-ID: <CAD+XWwoVUSaP1C4qsMbn05ROApkA5fAo2Fr=vXssQy=iAcJSzg@mail.gmail.com>

Hi all,

Along with a number of other free and open communities, Python is
being included in a survey of new contributors since January 2010. The
survey is being done by Kevin Carillo, a PhD student at Victoria
University of Wellington who is studying various approaches that
projects use to gain and work with new contributors.

If you have written patches, reviewed issues, or done anything to
contribute to the development of Python and you started this after
January 2010, I hope you can spare around 20 minutes to complete this
survey: https://limesurvey.sim.vuw.ac.nz/index.php?sid=65151&lang=en

There's a longer blog post up at
http://blog.python.org/2012/11/new-contributor-experience-in-python.html
if you would like a bit more information.

On behalf of Kevin Carillo, I thank you for your time and
consideration of this survey.

Brian Curtin

From dholth at gmail.com  Tue Nov 20 00:53:45 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 19 Nov 2012 18:53:45 -0500
Subject: [Python-Dev] Accept just PEP-0426
Message-ID: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>

I think this PEP is a significant improvement from its predecessor. It
represents features like extras (provides-extra) and build requirements
(setup-requires-dist) that are in use in the Python community but cannot be
represented in older versions of the format, it finally specifies a UTF-8
encoding, removes RFC 822, provides an extension mechanism, and allows the
description to be placed in the document payload.

PEP 426 doesn't have anything to do with the Wheel PEPs 425 and 427, other
than that its features are necessary to usefully represent a large number
of existing Python packages. How about moving this one along to focus on
the other two.

I'm not sure what the Post-History should be. We have been talking about it
for a while.

Thanks,

Daniel Holth



PEP: 426
Title: Metadata for Python Software Packages 1.3
Version: $Revision$
Last-Modified: $Date$
Author: Daniel Holth <dholth at fastmail.fm>
Discussions-To: Distutils SIG
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 30 Aug 2012


Abstract
========

This PEP describes a mechanism for adding metadata to Python distributions.
It includes specifics of the field names, and their semantics and
usage.

This document specifies version 1.3 of the metadata format.
Version 1.0 is specified in PEP 241.
Version 1.1 is specified in PEP 314.
Version 1.2 is specified in PEP 345.

Version 1.3 of the metadata format adds fields designed to make
third-party packaging of Python Software easier and defines a
formal extension mechanism.  The fields are "Setup-Requires-Dist"
"Provides-Extra", and "Extension".  This version also adds the `extra`
variable to the `environment markers` specification and allows the
description to be placed into a payload section.

Metadata Files
==============

The syntax defined in this PEP is for use with Python distribution
metadata files. The file format is a simple UTF-8 encoded Key: value
format with case-insensitive keys and no maximum line length, followed by
a blank line and an arbitrary payload.  It is parseable by the ``email``
module with an appropriate ``email.policy.Policy()``.

When ``metadata`` is a Unicode string,
```email.parser.Parser().parsestr(metadata)`` is a serviceable parser.

There are two standard locations for these metadata files:

* the ``PKG-INFO`` file included in the base directory of Python
  source distribution archives (as created by the distutils ``sdist``
  command)
* the ``.dist-info/METADATA`` files in a Python installation database, as
  described in PEP 376.

Other tools involved in Python distribution may also use this format.

Encoding
========

Metadata 1.3 files are UTF-8 with the restriction that keys must be
ASCII. Parser implementations should be aware that older versions of
the Metadata specification do not specify an encoding.

Fields
======

This section specifies the names and semantics of each of the
supported metadata fields.

In a single Metadata 1.3 file, fields marked with "(optional)" may occur
0 or 1 times.  Fields marked with "(multiple use)" may be specified
0, 1 or more times.  Only "Metadata-Version", "Name", "Version", and
"Summary" must appear exactly once.  The fields may appear in any order
within the file.

Metadata-Version
::::::::::::::::

Version of the file format; "1.3" is the only legal value.

Example::

    Metadata-Version: 1.3


Name
::::

The name of the distribution.

Example::

    Name: BeagleVote


Version
:::::::

A string containing the distribution's version number.  This
field  must be in the format specified in PEP 386.

Example::

    Version: 1.0a2


Summary
:::::::

A one-line summary of what the distribution does.

Example::

    Summary: A module for collecting votes from beagles.


Platform (multiple use)
:::::::::::::::::::::::

A Platform specification describing an operating system supported by
the distribution which is not listed in the "Operating System" Trove
classifiers.
See "Classifier" below.

Examples::

    Platform: ObscureUnix
    Platform: RareDOS


Supported-Platform (multiple use)
:::::::::::::::::::::::::::::::::

Binary distributions containing a metadata file will use the
Supported-Platform field in their metadata to specify the OS and
CPU for which the binary distribution was compiled.  The semantics of
the Supported-Platform field are not specified in this PEP.

Example::

    Supported-Platform: RedHat 7.2
    Supported-Platform: i386-win32-2791


Description (optional, deprecated)
::::::::::::::::::::::::::::::::::

A longer description of the distribution that can run to several
paragraphs.  Software that deals with metadata should not assume
any maximum size for this field.

The contents of this field can be written using reStructuredText
markup [1]_.  For programs that work with the metadata, supporting
markup is optional; programs can also display the contents of the
field as-is.  This means that authors should be conservative in
the markup they use.

Since a line separator immediately followed by another line separator
indicates the end of the headers section, any line separators in the
description must be suffixed by whitespace to indicate continuation.

Since Metadata 1.3 the recommended place for the description is in the
payload section of the document, after the last header.  The description
does not need to be reformatted when it is included in the payload.


Keywords (optional)
:::::::::::::::::::

A list of additional keywords to be used to assist searching
for the distribution in a larger catalog.

Example::

    Keywords: dog puppy voting election


Home-page (optional)
::::::::::::::::::::

A string containing the URL for the distribution's home page.

Example::

    Home-page: http://www.example.com/~cschultz/bvote/


Download-URL (optional)
:::::::::::::::::::::::

A string containing the URL from which this version of the distribution
can be downloaded.  (This means that the URL can't be something like
".../BeagleVote-latest.tgz", but instead must be ".../BeagleVote-0.45.tgz".)


Author (optional)
:::::::::::::::::

A string containing the author's name at a minimum; additional
contact information may be provided.

Example::

    Author: C. Schultz, Universal Features Syndicate,
            Los Angeles, CA <cschultz at peanuts.example.com>


Author-email (optional)
:::::::::::::::::::::::

A string containing the author's e-mail address.  It contains a name
and e-mail address in the RFC 5322 recommended ``Address Specification``
format.

Example::

    Author-email: "C. Schultz" <cschultz at example.com>


Maintainer (optional)
:::::::::::::::::::::

A string containing the maintainer's name at a minimum; additional
contact information may be provided.

Note that this field is intended for use when a project is being
maintained by someone other than the original author:  it should be
omitted if it is identical to ``Author``.

Example::

    Maintainer: C. Schultz, Universal Features Syndicate,
            Los Angeles, CA <cschultz at peanuts.example.com>


Maintainer-email (optional)
:::::::::::::::::::::::::::

A string containing the maintainer's e-mail address.  It has the same
format as ``Author-email``.

Note that this field is intended for use when a project is being
maintained by someone other than the original author:  it should be
omitted if it is identical to ``Author-email``.

Example::

    Maintainer-email: "C. Schultz" <cschultz at example.com>


License (optional)
::::::::::::::::::

Text indicating the license covering the distribution where the license
is not a selection from the "License" Trove classifiers. See
"Classifier" below.  This field may also be used to specify a
particular version of a license which is named via the ``Classifier``
field, or to indicate a variation or exception to such a license.

Examples::

    License: This software may only be obtained by sending the
            author a postcard, and then the user promises not
            to redistribute it.

    License: GPL version 3, excluding DRM provisions

The full text of the license would normally be included in a separate
file.


Classifier (multiple use)
:::::::::::::::::::::::::

Each entry is a string giving a single classification value
for the distribution.  Classifiers are described in PEP 301 [2].

Examples::

    Classifier: Development Status :: 4 - Beta
    Classifier: Environment :: Console (Text Based)


Requires-Dist (multiple use)
::::::::::::::::::::::::::::

Each entry contains a string naming some other distutils
project required by this distribution.

The format of a requirement string is identical to that of a
distutils project name (e.g., as found in the ``Name:`` field.
optionally followed by a version declaration within parentheses.

The distutils project names should correspond to names as found
on the `Python Package Index`_.

Version declarations must follow the rules described in
`Version Specifiers`_

Examples::

    Requires-Dist: pkginfo
    Requires-Dist: PasteDeploy
    Requires-Dist: zope.interface (>3.5.0)


Setup-Requires-Dist (multiple use)
::::::::::::::::::::::::::::::::::

Like Requires-Dist, but names dependencies needed while the
distributions's distutils / packaging `setup.py` / `setup.cfg` is run.
Commonly used to generate a manifest from version control.

Examples::

    Setup-Requires-Dist: custom_setup_command

Dependencies mentioned in `Setup-Requires-Dist` may be installed exclusively
for setup and are not guaranteed to be available at run time.


Provides-Dist (multiple use)
::::::::::::::::::::::::::::

Each entry contains a string naming a Distutils project which
is contained within this distribution.  This field *must* include
the project identified in the ``Name`` field, followed by the
version : Name (Version).

A distribution may provide additional names, e.g. to indicate that
multiple projects have been bundled together.  For instance, source
distributions of the ``ZODB`` project have historically included
the ``transaction`` project, which is now available as a separate
distribution.  Installing such a source distribution satisfies
requirements for both ``ZODB`` and ``transaction``.

A distribution may also provide a "virtual" project name, which does
not correspond to any separately-distributed project:  such a name
might be used to indicate an abstract capability which could be supplied
by one of multiple projects.  E.g., multiple projects might supply
RDBMS bindings for use by a given ORM:  each project might declare
that it provides ``ORM-bindings``, allowing other projects to depend
only on having at most one of them installed.

A version declaration may be supplied and must follow the rules described
in `Version Specifiers`_. The distribution's version number will be implied
if none is specified.

Examples::

    Provides-Dist: OtherProject
    Provides-Dist: AnotherProject (3.4)
    Provides-Dist: virtual_package


Obsoletes-Dist (multiple use)
:::::::::::::::::::::::::::::

Each entry contains a string describing a distutils project's distribution
which this distribution renders obsolete, meaning that the two projects
should not be installed at the same time.

Version declarations can be supplied.  Version numbers must be in the
format specified in `Version Specifiers`_.

The most common use of this field will be in case a project name
changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0.
When you install Torqued Python, the Gorgon distribution should be
removed.

Examples::

    Obsoletes-Dist: Gorgon
    Obsoletes-Dist: OtherProject (<3.0)


Requires-Python (optional)
::::::::::::::::::::::::::

This field specifies the Python version(s) that the distribution is
guaranteed to be compatible with.

Version numbers must be in the format specified in `Version Specifiers`_.

Examples::

    Requires-Python: 2.5
    Requires-Python: >2.1
    Requires-Python: >=2.3.4
    Requires-Python: >=2.5,<2.7


Requires-External (multiple use)
::::::::::::::::::::::::::::::::

Each entry contains a string describing some dependency in the
system that the distribution is to be used.  This field is intended to
serve as a hint to downstream project maintainers, and has no
semantics which are meaningful to the ``distutils`` distribution.

The format of a requirement string is a name of an external
dependency, optionally followed by a version declaration within
parentheses.

Because they refer to non-Python software releases, version numbers
for this field are **not** required to conform to the format
specified in PEP 386:  they should correspond to the
version scheme used by the external dependency.

Notice that there's is no particular rule on the strings to be used.

Examples::

    Requires-External: C
    Requires-External: libpng (>=1.5)


Project-URL (multiple use)
::::::::::::::::::::::::::

A string containing a label and a browsable URL for the project, separated
by the last occurrence of comma and space ", ".

Example::

    Bug, Issue Tracker, http://bitbucket.org/tarek/distribute/issues/

The label is a free text.


Provides-Extra (multiple use)
:::::::::::::::::::::::::::::

A string containing the name of an optional feature. Must be printable
ASCII, not containing whitespace, comma (,), or square brackets [].
May be used to make a dependency conditional on whether the optional
feature has been requested.

Example::

    Name: beaglevote
    Provides-Extra: pdf
    Requires-Dist: reportlab; extra == 'pdf'
    Requires-Dist: nose; extra == 'test'
    Requires-Dist: sphinx; extra == 'doc'

A second distribution requires an optional dependency by placing it
inside square brackets and can request multiple features by separating
them with a comma (,).

The full set of requirements is the union of the `Requires-Dist` sets
evaluated with `extra` set to `None` and then to the name of each
requested feature.

Example::

    Requires-Dist: beaglevote[pdf]
        -> requires beaglevote, reportlab

    Requires-Dist: beaglevote[test, doc]
        -> requires beaglevote, sphinx, nose

Two feature names `test` and `doc` are reserved to mark dependencies that
are needed for running automated tests and generating documentation,
respectively.

It is legal to specify `Provides-Extra` without referencing it in any
`Requires-Dist`. It is an error to request a feature name that has
not been declared with `Provides-Extra`.


Extension (multiple use)
::::::::::::::::::::::::

An ASCII string, not containing whitespace or the / character, that
indicates the presence of extended metadata. Additional tags defined by
an `Extension: Chili` must be of the form `Chili/Name`::

    Extension: Chili
    Chili/Type: Poblano
    Chili/Heat: Mild

An implementation might iterate over all the declared `Extension:`
fields to invoke the processors for those extensions.  As the order of
the fields is not used, the `Extension: Chili` field may appear before
or after its declared tags `Chili/Type:` etc.


Version Specifiers
==================

Version specifiers are a series of conditional operators and
version numbers, separated by commas.  Conditional operators
must be one of "<", ">", "<=", ">=", "==" and "!=".

Any number of conditional operators can be specified, e.g.
the string ">1.0, !=1.3.4, <2.0" is a legal version declaration.
The comma (",") is equivalent to the **and** operator.

Each version number must be in the format specified in PEP 386.

When a version is provided, it always includes all versions that
starts with the same value. For example the "2.5" version of Python
will include versions like "2.5.2" or "2.5.3". Pre and post releases
in that case are excluded. So in our example, versions like "2.5a1" are
not included when "2.5" is used. If the first version of the range is
required, it has to be explicitly given. In our example, it will be
"2.5.0".

Notice that some projects might omit the ".0" prefix for the first release
of the "2.5.x" series:

- 2.5
- 2.5.1
- 2.5.2
- etc.

In that case, "2.5.0" will have to be explicitly used to avoid any confusion
between the "2.5" notation that represents the full range. It is a
recommended
practice to use schemes of the same length for a series to completely avoid
this problem.

Some Examples:

- ``Requires-Dist: zope.interface (3.1)``: any version that starts with 3.1,
  excluding post or pre-releases.
- ``Requires-Dist: zope.interface (==3.1)``: equivalent to ``Requires-Dist:
  zope.interface (3.1)``.
- ``Requires-Dist: zope.interface (3.1.0)``: any version that starts with
  3.1.0, excluding post or pre-releases. Since that particular project
doesn't
  use more than 3 digits, it also means "only the 3.1.0 release".
- ``Requires-Python: 3``: Any Python 3 version, no matter wich one,
excluding
  post or pre-releases.
- ``Requires-Python: >=2.6,<3``: Any version of Python 2.6 or 2.7, including
  post releases of 2.6, pre and post releases of 2.7. It excludes pre
releases
  of Python 3.
- ``Requires-Python: 2.6.2``: Equivalent to ">=2.6.2,<2.6.3". So this
includes
  only Python 2.6.2. Of course, if Python was numbered with 4 digits, it
would
  have include all versions of the 2.6.2 series.
- ``Requires-Python: 2.5.0``: Equivalent to ">=2.5.0,<2.5.1".
- ``Requires-Dist: zope.interface (3.1,!=3.1.3)``: any version that starts
with
  3.1, excluding post or pre-releases of 3.1 *and* excluding any version
that
  starts with "3.1.3". For this particular project, this means: "any version
  of the 3.1 series but not 3.1.3". This is equivalent to:
  ">=3.1,!=3.1.3,<3.2".

Environment markers
===================

An **environment marker** is a marker that can be added at the end of a
field after a semi-colon (";"), to add a condition about the execution
environment.

Here are some example of fields using such markers::

   Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
   Obsoletes-Dist: pywin31; sys.platform == 'win32'
   Requires-Dist: foo (1,!=1.3); platform.machine == 'i386'
   Requires-Dist: bar; python_version == '2.4' or python_version == '2.5'
   Requires-External: libxslt; 'linux' in sys.platform

The micro-language behind this is a simple subset of Python: it compares
only strings, with the ``==`` and ``in`` operators (and their opposites),
and with the ability to combine expressions. Parenthesis are supported
for grouping.

The pseudo-grammar is ::

    EXPR [in|==|!=|not in] EXPR [or|and] ...

where ``EXPR`` belongs to any of those:

- python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])
- python_full_version = sys.version.split()[0]
- os.name = os.name
- sys.platform = sys.platform
- platform.version = platform.version()
- platform.machine = platform.machine()
- platform.python_implementation = platform.python_implementation()
- a free string, like ``'2.4'``, or ``'win32'``
- extra = (name of requested feature) or None

Notice that ``in`` is restricted to strings, meaning that it is not possible
to use other sequences like tuples or lists on the right side.

The fields that benefit from this marker are:

- Requires-Python
- Requires-External
- Requires-Dist
- Setup-Requires-Dist
- Provides-Dist
- Obsoletes-Dist
- Classifier

(The `extra` variable is only meaningful for Requires-Dist.)

Summary of Differences From PEP 345
===================================

* Metadata-Version is now 1.3.

* Values are now expected to be UTF-8.

* A payload (containing the description) may appear after the headers.

* Added `extra` to environment markers.

* Most fields are now optional.

* Changed fields:

  - Description
  - Project-URL
  - Requires-Dist

* Added fields:

  - Extension
  - Provides-Extra
  - Setup-Requires-Dist

References
==========

This document specifies version 1.3 of the metadata format.
Version 1.0 is specified in PEP 241.
Version 1.1 is specified in PEP 314.
Version 1.2 is specified in PEP 345.

.. [1] reStructuredText markup:
   http://docutils.sourceforge.net/

.. _`Python Package Index`: http://pypi.python.org/pypi/

.. [2] PEP 301:
   http://www.python.org/dev/peps/pep-0301/

Appendix
========

Parsing and generating the Metadata 1.3 serialization format using
Python 3.3::

    # Metadata 1.3 demo
    from email.generator import Generator
    from email import header
    from email.parser import Parser
    from email.policy import Compat32
    from email.utils import _has_surrogates

    class MetadataPolicy(Compat32):
        max_line_length = 0
        continuation_whitespace = '\t'

        def _sanitize_header(self, name, value):
            if not isinstance(value, str):
                return value
            if _has_surrogates(value):
                raise NotImplementedError()
            else:
                return value

        def _fold(self, name, value, sanitize):
            body = ((self.linesep+self.continuation_whitespace)
                    .join(value.splitlines()))
            return ''.join((name, ': ', body, self.linesep))

    if __name__ == "__main__":
        import sys
        import textwrap

        pkg_info = """\
    Metadata-Version: 1.3
    Name: package
    Version: 0.1.0
    Summary: A package.
    Description: Description
        ===========


        A description of the package.

    """

        m = Parser(policy=MetadataPolicy()).parsestr(pkg_info)

        m['License'] = 'GPL'
        description = m['Description']
        description_lines = description.splitlines()
        m.set_payload(description_lines[0]
                + '\n'
                + textwrap.dedent('\n'.join(description_lines[1:]))
                + '\n')
        del m['Description']

        # Correct if sys.stdout.encoding == 'UTF-8':
        Generator(sys.stdout, maxheaderlen=0).flatten(m)

Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/bed97c73/attachment-0001.html>

From pje at telecommunity.com  Tue Nov 20 01:37:52 2012
From: pje at telecommunity.com (PJ Eby)
Date: Mon, 19 Nov 2012 19:37:52 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
Message-ID: <CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>

On Mon, Nov 19, 2012 at 6:53 PM, Daniel Holth <dholth at gmail.com> wrote:

> I think this PEP is a significant improvement from its predecessor. It
> represents features like extras (provides-extra) and build requirements
> (setup-requires-dist) that are in use in the Python community but cannot be
> represented in older versions of the format, it finally specifies a UTF-8
> encoding, removes RFC 822, provides an extension mechanism, and allows the
> description to be placed in the document payload.
>

Can we maybe kill Provides-Dist and its associated baggage first, though?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/41f4efdf/attachment.html>

From donald.stufft at gmail.com  Tue Nov 20 01:41:56 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 19 Nov 2012 19:41:56 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
Message-ID: <B8924F82A9A24F04A0B8DC4C28CAC9E6@gmail.com>

On Monday, November 19, 2012 at 7:37 PM, PJ Eby wrote:
> Can we maybe kill Provides-Dist and its associated baggage first, though?
> 
> 

I would love to kill Provides-Dist. The biggest question there is how do you handle it's
functionality? If someone needs setuptools but they have distribute installed they
both shouldn't get installed.

The need for it for the "2 packages being distributed together" I'm (personally)
less concerned about since with proper dependency data we should be
able to just depend on things instead of bundling them. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/8bd03665/attachment.html>

From dholth at gmail.com  Tue Nov 20 01:43:37 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 19 Nov 2012 19:43:37 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
Message-ID: <98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>

Does it really have baggage? I think it is necessary, although it doesn't do favors to the implementer (and has never been implemented). How else is anyone supposed to fork or merge projects?

Daniel Holth

On Nov 19, 2012, at 7:37 PM, PJ Eby <pje at telecommunity.com> wrote:

> On Mon, Nov 19, 2012 at 6:53 PM, Daniel Holth <dholth at gmail.com> wrote:
>> I think this PEP is a significant improvement from its predecessor. It represents features like extras (provides-extra) and build requirements (setup-requires-dist) that are in use in the Python community but cannot be represented in older versions of the format, it finally specifies a UTF-8 encoding, removes RFC 822, provides an extension mechanism, and allows the description to be placed in the document payload.
> 
> Can we maybe kill Provides-Dist and its associated baggage first, though?
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/61693074/attachment.html>

From dholth at gmail.com  Tue Nov 20 01:46:41 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 19 Nov 2012 19:46:41 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <B8924F82A9A24F04A0B8DC4C28CAC9E6@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<B8924F82A9A24F04A0B8DC4C28CAC9E6@gmail.com>
Message-ID: <D5207188-D525-4049-88B0-85E32BD32973@gmail.com>

The "I bundled a renamed copy of six" is a totally different case which would not invoke provides-dist. "I merged sqlalchemy with a previously separate but wildly popular declarative / database support / whatever extension" would invoke provides-dist.

Daniel Holth

On Nov 19, 2012, at 7:41 PM, Donald Stufft <donald.stufft at gmail.com> wrote:

> On Monday, November 19, 2012 at 7:37 PM, PJ Eby wrote:
>> Can we maybe kill Provides-Dist and its associated baggage first, though?
> I would love to kill Provides-Dist. The biggest question there is how do you handle it's
> functionality? If someone needs setuptools but they have distribute installed they
> both shouldn't get installed.
> 
> The need for it for the "2 packages being distributed together" I'm (personally)
> less concerned about since with proper dependency data we should be
> able to just depend on things instead of bundling them. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/82e91941/attachment.html>

From donald.stufft at gmail.com  Tue Nov 20 01:49:41 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 19 Nov 2012 19:49:41 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
Message-ID: <4AB62E69A53048589648EA9A94CAC02A@gmail.com>

Other languages seem to get along fine without it. Even the OS 
managers which have it don't allow it to be used to masquerade
as another project, only to make generic virtual packages (e.g. "email").


On Monday, November 19, 2012 at 7:43 PM, Daniel Holth wrote:

> Does it really have baggage? I think it is necessary, although it doesn't do favors to the implementer (and has never been implemented). How else is anyone supposed to fork or merge projects?
> 
> Daniel Holth
> 
> On Nov 19, 2012, at 7:37 PM, PJ Eby <pje at telecommunity.com (mailto:pje at telecommunity.com)> wrote:
> 
> > On Mon, Nov 19, 2012 at 6:53 PM, Daniel Holth <dholth at gmail.com (mailto:dholth at gmail.com)> wrote:
> > > I think this PEP is a significant improvement from its predecessor. It represents features like extras (provides-extra) and build requirements (setup-requires-dist) that are in use in the Python community but cannot be represented in older versions of the format, it finally specifies a UTF-8 encoding, removes RFC 822, provides an extension mechanism, and allows the description to be placed in the document payload.
> > 
> > Can we maybe kill Provides-Dist and its associated baggage first, though?
> > 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org (mailto:Python-Dev at python.org)
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/03ab0f9b/attachment.html>

From dholth at gmail.com  Tue Nov 20 02:08:31 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 19 Nov 2012 20:08:31 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <4AB62E69A53048589648EA9A94CAC02A@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
Message-ID: <9317AC70-4833-475C-924D-768CA0E7A069@gmail.com>

The distro repos are centrally managed, too. Try getting setuptools to provide a virtual package just because you want to fork.. and then update the dependent packages?

Daniel Holth

On Nov 19, 2012, at 7:49 PM, Donald Stufft <donald.stufft at gmail.com> wrote:

> Other languages seem to get along fine without it. Even the OS
> managers which have it don't allow it to be used to masquerade
> as another project, only to make generic virtual packages (e.g. "email").
> On Monday, November 19, 2012 at 7:43 PM, Daniel Holth wrote:
> 
>> Does it really have baggage? I think it is necessary, although it doesn't do favors to the implementer (and has never been implemented). How else is anyone supposed to fork or merge projects?
>> 
>> Daniel Holth
>> 
>> On Nov 19, 2012, at 7:37 PM, PJ Eby <pje at telecommunity.com> wrote:
>> 
>>> On Mon, Nov 19, 2012 at 6:53 PM, Daniel Holth <dholth at gmail.com> wrote:
>>>> I think this PEP is a significant improvement from its predecessor. It represents features like extras (provides-extra) and build requirements (setup-requires-dist) that are in use in the Python community but cannot be represented in older versions of the format, it finally specifies a UTF-8 encoding, removes RFC 822, provides an extension mechanism, and allows the description to be placed in the document payload.
>>> 
>>> Can we maybe kill Provides-Dist and its associated baggage first, though?
>>> 
>> 
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/560ea2e7/attachment.html>

From donald.stufft at gmail.com  Tue Nov 20 02:14:47 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 19 Nov 2012 20:14:47 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <9317AC70-4833-475C-924D-768CA0E7A069@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<9317AC70-4833-475C-924D-768CA0E7A069@gmail.com>
Message-ID: <DE944FC1D4AD4C419E450D2A275B3566@gmail.com>

On Monday, November 19, 2012 at 8:08 PM, Daniel Holth wrote:
> The distro repos are centrally managed, too. Try getting setuptools to provide a virtual package just because you want to fork.. and then update the dependent packages?
> 
> Daniel Holth
Sorry didn't mean to make it sound like I thought we should do it
like the OS packagers, just mentioning that very few (any?) other
languages do it like that and they seem to be getting along just
fine. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/b37c3ce7/attachment.html>

From dholth at gmail.com  Tue Nov 20 02:31:01 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 19 Nov 2012 20:31:01 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <4AB62E69A53048589648EA9A94CAC02A@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
Message-ID: <B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>

We are getting along fine too. No tool parses metadata 1.x for package management reasons and provides has existed forever with no implementation. So it is not inconveniencing anyone. I would prefer to leave it alone.

Daniel Holth

On Nov 19, 2012, at 7:49 PM, Donald Stufft <donald.stufft at gmail.com> wrote:

> Other languages seem to get along fine without it. Even the OS
> managers which have it don't allow it to be used to masquerade
> as another project, only to make generic virtual packages (e.g. "email").
> On Monday, November 19, 2012 at 7:43 PM, Daniel Holth wrote:
> 
>> Does it really have baggage? I think it is necessary, although it doesn't do favors to the implementer (and has never been implemented). How else is anyone supposed to fork or merge projects?
>> 
>> Daniel Holth
>> 
>> On Nov 19, 2012, at 7:37 PM, PJ Eby <pje at telecommunity.com> wrote:
>> 
>>> On Mon, Nov 19, 2012 at 6:53 PM, Daniel Holth <dholth at gmail.com> wrote:
>>>> I think this PEP is a significant improvement from its predecessor. It represents features like extras (provides-extra) and build requirements (setup-requires-dist) that are in use in the Python community but cannot be represented in older versions of the format, it finally specifies a UTF-8 encoding, removes RFC 822, provides an extension mechanism, and allows the description to be placed in the document payload.
>>> 
>>> Can we maybe kill Provides-Dist and its associated baggage first, though?
>>> 
>> 
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/287e5b65/attachment.html>

From donald.stufft at gmail.com  Tue Nov 20 02:33:19 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 19 Nov 2012 20:33:19 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
Message-ID: <DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>

So you want to leave metadata in that you think people shouldn't implement? 

Or you do think people should implement it and the point about it existing
forever without an implementation is?

At the very least there needs to be some sort of guidelines as to what
to do with the field in the various states it could be in.


On Monday, November 19, 2012 at 8:31 PM, Daniel Holth wrote:

> We are getting along fine too. No tool parses metadata 1.x for package management reasons and provides has existed forever with no implementation. So it is not inconveniencing anyone. I would prefer to leave it alone.
> 
> Daniel Holth
> 
> On Nov 19, 2012, at 7:49 PM, Donald Stufft <donald.stufft at gmail.com (mailto:donald.stufft at gmail.com)> wrote:
> 
> > Other languages seem to get along fine without it. Even the OS 
> > managers which have it don't allow it to be used to masquerade
> > as another project, only to make generic virtual packages (e.g. "email").
> > 
> > On Monday, November 19, 2012 at 7:43 PM, Daniel Holth wrote:
> > 
> > > Does it really have baggage? I think it is necessary, although it doesn't do favors to the implementer (and has never been implemented). How else is anyone supposed to fork or merge projects?
> > > 
> > > Daniel Holth
> > > 
> > > On Nov 19, 2012, at 7:37 PM, PJ Eby <pje at telecommunity.com (mailto:pje at telecommunity.com)> wrote:
> > > 
> > > > On Mon, Nov 19, 2012 at 6:53 PM, Daniel Holth <dholth at gmail.com (mailto:dholth at gmail.com)> wrote:
> > > > > I think this PEP is a significant improvement from its predecessor. It represents features like extras (provides-extra) and build requirements (setup-requires-dist) that are in use in the Python community but cannot be represented in older versions of the format, it finally specifies a UTF-8 encoding, removes RFC 822, provides an extension mechanism, and allows the description to be placed in the document payload.
> > > > 
> > > > Can we maybe kill Provides-Dist and its associated baggage first, though?
> > > > 
> > > _______________________________________________
> > > Python-Dev mailing list
> > > Python-Dev at python.org (mailto:Python-Dev at python.org)
> > > http://mail.python.org/mailman/listinfo/python-dev
> > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
> > > 
> > > 
> > > 
> > 
> > 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/5f931f92/attachment-0001.html>

From a.badger at gmail.com  Tue Nov 20 02:35:50 2012
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Mon, 19 Nov 2012 17:35:50 -0800
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <4AB62E69A53048589648EA9A94CAC02A@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
Message-ID: <20121120013550.GR2133@unaka.lan>

On Mon, Nov 19, 2012 at 07:49:41PM -0500, Donald Stufft wrote:
> Other languages seem to get along fine without it. Even the OS
> managers which have it don't allow it to be used to masquerade
> as another project, only to make generic virtual packages (e.g. "email").
> 
I'm not sure this assertion about OS package managers is correct.  I've only
just read:
http://www.python.org/dev/peps/pep-0426/#provides-dist-multiple-use

but the rough rpm analogue seems to be the Provides: tag.

Provides is given a string which is parsed into a name or a name and version
like this:

Provides: python
Provides: python = 3.1.0

rpm has no way at package build time to tell that a particular name given in
a provides in one package is the actual name of another package.

At installtime, rpm keeps package names and provides names separately but in
dependency comparisons either one can be used to satisfy a requirement.
What that means is that when asking about information on a package with name
"python", you'll get information about the python package with that name and
not about anything else that Provides: "python".  But if you are installing
something that has a requirement on "python" either the package with the
name python or any package that Provides: python can satisfy the requirement.

Package managers with builtin dep solvers can be built on top of rpm.  The
one that I am familiar with is yum.  Since yum is downloading the packages
that are being fed into rpm, yum could choose to prefer the package name
instead of things in Provides when it downloads.  It doesn't, though.  Just
like the underlying rpm, it treats package names and names specificed
through Provides: as equivalent.

-Toshio

> On Monday, November 19, 2012 at 7:43 PM, Daniel Holth wrote:
> 
>     Does it really have baggage? I think it is necessary, although it doesn't
>     do favors to the implementer (and has never been implemented). How else is
>     anyone supposed to fork or merge projects?
> 
>     Daniel Holth
> 
>     On Nov 19, 2012, at 7:37 PM, PJ Eby <pje at telecommunity.com> wrote:
> 
> 
>         On Mon, Nov 19, 2012 at 6:53 PM, Daniel Holth <dholth at gmail.com> wrote:
> 
>             I think this PEP is a significant improvement from its predecessor.
>             It represents features like extras (provides-extra) and build
>             requirements (setup-requires-dist) that are in use in the Python
>             community but cannot be represented in older versions of the
>             format, it finally specifies a UTF-8 encoding, removes RFC 822,
>             provides an extension mechanism, and allows the description to be
>             placed in the document payload.
> 
> 
>         Can we maybe kill Provides-Dist and its associated baggage first,
>         though?
> 
> 
>     _______________________________________________
>     Python-Dev mailing list
>     Python-Dev at python.org
>     http://mail.python.org/mailman/listinfo/python-dev
>     Unsubscribe: http://mail.python.org/mailman/options/python-dev/
>     donald.stufft%40gmail.com
> 
> 

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/a.badger%40gmail.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/d7bbc53a/attachment.pgp>

From donald.stufft at gmail.com  Tue Nov 20 02:40:14 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 19 Nov 2012 20:40:14 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <20121120013550.GR2133@unaka.lan>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<20121120013550.GR2133@unaka.lan>
Message-ID: <D187BCC00CDA42F7B03228AA208C7B7F@gmail.com>

On Monday, November 19, 2012 at 8:35 PM, Toshio Kuratomi wrote:
> On Mon, Nov 19, 2012 at 07:49:41PM -0500, Donald Stufft wrote:
> > Other languages seem to get along fine without it. Even the OS
> > managers which have it don't allow it to be used to masquerade
> > as another project, only to make generic virtual packages (e.g. "email").
> > 
> 
> I'm not sure this assertion about OS package managers is correct. I've only
> just read:
> http://www.python.org/dev/peps/pep-0426/#provides-dist-multiple-use
> 
> but the rough rpm analogue seems to be the Provides: tag.
> 
> Provides is given a string which is parsed into a name or a name and version
> like this:
> 
> Provides: python
> Provides: python = 3.1.0
> 
> rpm has no way at package build time to tell that a particular name given in
> a provides in one package is the actual name of another package.
> 
> At installtime, rpm keeps package names and provides names separately but in
> dependency comparisons either one can be used to satisfy a requirement.
> What that means is that when asking about information on a package with name
> "python", you'll get information about the python package with that name and
> not about anything else that Provides: "python". But if you are installing
> something that has a requirement on "python" either the package with the
> name python or any package that Provides: python can satisfy the requirement.
> 
Are you saying the RPM documentation is wrong? 

http://www.rpm.org/max-rpm/s1-rpm-inside-tags.html

The provides tag is used to specify a *virtual package* that the packaged
software makes available when it is installed. Normally, this tag would be
used when different packages provide equivalent services. For example,
any package that allows a user to read mail might provide the mail-reader
virtual package. Another package that depends on a mail reader of some
sort, could require the mail-reader virtual package. It would then install
without dependency problems, if any one of several mail programs were
installed.

It pretty clearly states that it is not to be used for masquerading as a
different package, which was my point. I wasn't making any claims about
wether it was technically possible to do so or not, just what it's intended
purpose was.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/03a38ab7/attachment.html>

From turnbull at sk.tsukuba.ac.jp  Tue Nov 20 02:58:12 2012
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Tue, 20 Nov 2012 10:58:12 +0900
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
Message-ID: <871ufp6nnv.fsf@uwakimon.sk.tsukuba.ac.jp>

Daniel Holth writes:

 > Does [Provides-Dist] really have baggage? I think it is necessary,
 > although it doesn't do favors to the implementer (and has never
 > been implemented). How else is anyone supposed to fork or merge
 > projects?

It doesn't bother me personally if this traffic is on python-dev, but
this looks rather technical.  Shouldn't the distutils-sig at least be
CC'd?

Steve

From ncoghlan at gmail.com  Tue Nov 20 03:01:58 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 20 Nov 2012 12:01:58 +1000
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <D187BCC00CDA42F7B03228AA208C7B7F@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<20121120013550.GR2133@unaka.lan>
	<D187BCC00CDA42F7B03228AA208C7B7F@gmail.com>
Message-ID: <CADiSq7cswEfqRtPX6Q_-mrWFCsdR8ykaociXEiFVjZYbcc=4_g@mail.gmail.com>

Look more closely at the docs for "Obsoletes" in RPM, not just those for
"Provides". Being able to transparently replace an existing package with a
renamed one that installs files with the same names is certainly part of
the purpose/capabilities of the RPM dependency machinery (i.e. precisely
the distribute vs setuptools situation).

We may want to clarify the wording to ensure it is clear that the provision
of the dist name (as posted on PyPI) is implied, though.

Cheers,
Nick.

--
Sent from my phone, thus the relative brevity :)
On Nov 20, 2012 11:45 AM, "Donald Stufft" <donald.stufft at gmail.com> wrote:

> On Monday, November 19, 2012 at 8:35 PM, Toshio Kuratomi wrote:
>
> On Mon, Nov 19, 2012 at 07:49:41PM -0500, Donald Stufft wrote:
>
> Other languages seem to get along fine without it. Even the OS
> managers which have it don't allow it to be used to masquerade
> as another project, only to make generic virtual packages (e.g. "email").
>
> I'm not sure this assertion about OS package managers is correct. I've only
> just read:
> http://www.python.org/dev/peps/pep-0426/#provides-dist-multiple-use
>
> but the rough rpm analogue seems to be the Provides: tag.
>
> Provides is given a string which is parsed into a name or a name and
> version
> like this:
>
> Provides: python
> Provides: python = 3.1.0
>
> rpm has no way at package build time to tell that a particular name given
> in
> a provides in one package is the actual name of another package.
>
> At installtime, rpm keeps package names and provides names separately but
> in
> dependency comparisons either one can be used to satisfy a requirement.
> What that means is that when asking about information on a package with
> name
> "python", you'll get information about the python package with that name
> and
> not about anything else that Provides: "python". But if you are installing
> something that has a requirement on "python" either the package with the
> name python or any package that Provides: python can satisfy the
> requirement.
>
> Are you saying the RPM documentation is wrong?
>
> http://www.rpm.org/max-rpm/s1-rpm-inside-tags.html
>
> The provides tag is used to specify a *virtual package* that the packaged
> software makes available when it is installed. Normally, this tag would be
> used when different packages provide equivalent services. For example,
> any package that allows a user to read mail might provide the mail-reader
> virtual package. Another package that depends on a mail reader of some
> sort, could require the mail-reader virtual package. It would then install
> without dependency problems, if any one of several mail programs were
> installed.
>
> It pretty clearly states that it is not to be used for masquerading as a
> different package, which was my point. I wasn't making any claims about
> wether it was technically possible to do so or not, just what it's intended
> purpose was.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/62834995/attachment.html>

From donald.stufft at gmail.com  Tue Nov 20 03:04:51 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 19 Nov 2012 21:04:51 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CADiSq7cswEfqRtPX6Q_-mrWFCsdR8ykaociXEiFVjZYbcc=4_g@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<20121120013550.GR2133@unaka.lan>
	<D187BCC00CDA42F7B03228AA208C7B7F@gmail.com>
	<CADiSq7cswEfqRtPX6Q_-mrWFCsdR8ykaociXEiFVjZYbcc=4_g@mail.gmail.com>
Message-ID: <31346DA2FFF7423BA7F7D21286F1DD65@gmail.com>

On Monday, November 19, 2012 at 9:01 PM, Nick Coghlan wrote:
> Look more closely at the docs for "Obsoletes" in RPM, not just those for "Provides". Being able to transparently replace an existing package with a renamed one that installs files with the same names is certainly part of the purpose/capabilities of the RPM dependency machinery (i.e. precisely the distribute vs setuptools situation). 
> We may want to clarify the wording to ensure it is clear that the provision of the dist name (as posted on PyPI) is implied, though.
> Cheers,
> Nick.
I'm not sure if you're responding to me or not, but we also have Obsoletes-Dist
in the metadata. (Which I don't like the name of but that's for another argument).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/634c75db/attachment.html>

From dholth at gmail.com  Tue Nov 20 03:24:31 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 19 Nov 2012 21:24:31 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
Message-ID: <CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>

Mostly it seems a bit silly to have so much conversations about parts of
the pep that remain unchanged from previously accepted versions...
On Nov 19, 2012 8:33 PM, "Donald Stufft" <donald.stufft at gmail.com> wrote:

>  So you want to leave metadata in that you think people shouldn't
> implement?
>
> Or you do think people should implement it and the point about it existing
> forever without an implementation is?
>
> At the very least there needs to be some sort of guidelines as to what
> to do with the field in the various states it could be in.
>
> On Monday, November 19, 2012 at 8:31 PM, Daniel Holth wrote:
>
> We are getting along fine too. No tool parses metadata 1.x for package
> management reasons and provides has existed forever with no implementation.
> So it is not inconveniencing anyone. I would prefer to leave it alone.
>
> Daniel Holth
>
> On Nov 19, 2012, at 7:49 PM, Donald Stufft <donald.stufft at gmail.com>
> wrote:
>
>  Other languages seem to get along fine without it. Even the OS
> managers which have it don't allow it to be used to masquerade
> as another project, only to make generic virtual packages (e.g. "email").
>
> On Monday, November 19, 2012 at 7:43 PM, Daniel Holth wrote:
>
> Does it really have baggage? I think it is necessary, although it doesn't
> do favors to the implementer (and has never been implemented). How else is
> anyone supposed to fork or merge projects?
>
> Daniel Holth
>
> On Nov 19, 2012, at 7:37 PM, PJ Eby <pje at telecommunity.com> wrote:
>
> On Mon, Nov 19, 2012 at 6:53 PM, Daniel Holth <dholth at gmail.com> wrote:
>
> I think this PEP is a significant improvement from its predecessor. It
> represents features like extras (provides-extra) and build requirements
> (setup-requires-dist) that are in use in the Python community but cannot be
> represented in older versions of the format, it finally specifies a UTF-8
> encoding, removes RFC 822, provides an extension mechanism, and allows the
> description to be placed in the document payload.
>
>
> Can we maybe kill Provides-Dist and its associated baggage first, though?
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/7e223ea1/attachment.html>

From donald.stufft at gmail.com  Tue Nov 20 03:32:43 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Mon, 19 Nov 2012 21:32:43 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
Message-ID: <E88CB816E0164340BFC8468448FE2974@gmail.com>

On Monday, November 19, 2012 at 9:24 PM, Daniel Holth wrote:
> Mostly it seems a bit silly to have so much conversations about parts of the pep that remain unchanged from previously accepted versions... 

Well, I think the PEP should describe what we expect to be implemented *shrug*. Either we should expect it to be implemented and it should be part of the spec, or we shouldn't expect people to implement it and it should be removed. 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/e2faabd6/attachment-0001.html>

From dholth at gmail.com  Tue Nov 20 04:30:59 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 19 Nov 2012 22:30:59 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <E88CB816E0164340BFC8468448FE2974@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<E88CB816E0164340BFC8468448FE2974@gmail.com>
Message-ID: <CAG8k2+5+xrCn-LTD7unoD-NwVfCk_W17D_1ujjVignBMyrYiRQ@mail.gmail.com>

The section could definitely be much clearer. How about:

Provides-Dist (multiple use)

Each entry contains a string naming a requirement that is satisfied by
installing this distribution.  This field *must* include the project
identified in the ``Name`` field, optionally followed by the version:
Name (Version).

A distribution may provide additional names, e.g. to indicate that
multiple projects have been merged into a single distribution or to
indicate that this project is a substitute for another.  For instance
distribute (a fork of setuptools) could ``Provides-Dist`` setuptools to
prevent the conflicting package from being downloaded and installed when
distribute is already installed.  A distribution that has been merged
with another might ``Provides-Dist`` the obsolete name(s) to satisfy
any projects that require the obsolete distribution's name.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121119/914f3f09/attachment.html>

From kristjan at ccpgames.com  Tue Nov 20 10:06:30 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 20 Nov 2012 09:06:30 +0000
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD329DE2A53@RKV-IT-EXCH103.ccp.ad.local>

I'm intrigued.  I thought this was merely so that one could do
python -m mypackage.mysubpackage
Can you refer me to the rationale and discussion about this feature?

K

From: Nick Coghlan [mailto:ncoghlan at gmail.com]
Sent: 18. n?vember 2012 11:25
To: Kristj?n Valur J?nsson
Cc: Christian Tismer; python-dev at python.org
Subject: Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

Easily bundling dependencies is a key principle behind the ability to execute directories and zipfiles that contain a top level __main__.py file that was added back in 2.6 (although the zipfile version doesn't play nicely with extension modules).

Cheers,
Nick.

--
Nick Coghlan   |   ncoghlan at gmail.com<mailto:ncoghlan at gmail.com>   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/66800b3c/attachment.html>

From ncoghlan at gmail.com  Tue Nov 20 12:39:22 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 20 Nov 2012 21:39:22 +1000
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DE2A53@RKV-IT-EXCH103.ccp.ad.local>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE2A53@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <CADiSq7ccgu9aMCXUqmSfmRCr3_ZJY5-A8hwemwYTFzUMXHdKzw@mail.gmail.com>

On Tue, Nov 20, 2012 at 7:06 PM, Kristj?n Valur J?nsson <
kristjan at ccpgames.com> wrote:

>  I?m intrigued.  I thought this was merely so that one could do****
>
> python ?m mypackage.mysubpackage****
>
> Can you refer me to the rationale and discussion about this feature?
>

It was part of a fairly long progression in changes to the command line
execution support :)

Top level package execution with -m came first in 2.4, arbitrary package
execution for -m arrived in 2.5 (along with the runpy module), directory
and zipfile execution (by supplying a valid sys.path entry as the "script"
command line argument) was added in 2.6/3.0, and finally officially
supported package execution via -m only arrived in 2.7 and 3.1 (a broken
version of the last accidentally existed in 2.5, but that was just a
mistake that arose when merging the import emulations in runpy and pkgutil,
and had a side effect that violated at least one of the import system
invariants).

In the specific case of directory and zipfile execution, discussion
happened on the tracker: http://bugs.python.org/issue1739468

It was never brought up on python-dev because Guido was participating
directly in the tracker discussion. Unfortunately, we then also forgot to
mention it in the original version of the 2.6 What's New doc, so the vast
majority of people missed the addition :(

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/29969587/attachment.html>

From tismer at stackless.com  Tue Nov 20 14:45:25 2012
From: tismer at stackless.com (Christian Tismer)
Date: Tue, 20 Nov 2012 14:45:25 +0100
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <CADiSq7ccgu9aMCXUqmSfmRCr3_ZJY5-A8hwemwYTFzUMXHdKzw@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE2A53@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7ccgu9aMCXUqmSfmRCr3_ZJY5-A8hwemwYTFzUMXHdKzw@mail.gmail.com>
Message-ID: <50AB8975.3020405@stackless.com>

On 20.11.12 12:39, Nick Coghlan wrote:
> On Tue, Nov 20, 2012 at 7:06 PM, Kristj?n Valur J?nsson 
> <kristjan at ccpgames.com <mailto:kristjan at ccpgames.com>> wrote:
>
>     I?m intrigued.  I thought this was merely so that one could do
>
>     python ?m mypackage.mysubpackage
>
>     Can you refer me to the rationale and discussion about this feature?
>
>
> It was part of a fairly long progression in changes to the command 
> line execution support :)
>
> Top level package execution with -m came first in 2.4, arbitrary 
> package execution for -m arrived in 2.5 (along with the runpy module), 
> directory and zipfile execution (by supplying a valid sys.path entry 
> as the "script" command line argument) was added in 2.6/3.0, and 
> finally officially supported package execution via -m only arrived in 
> 2.7 and 3.1 (a broken version of the last accidentally existed in 2.5, 
> but that was just a mistake that arose when merging the import 
> emulations in runpy and pkgutil, and had a side effect that violated 
> at least one of the import system invariants).
>
> In the specific case of directory and zipfile execution, discussion 
> happened on the tracker: http://bugs.python.org/issue1739468
>
> It was never brought up on python-dev because Guido was participating 
> directly in the tracker discussion. Unfortunately, we then also forgot 
> to mention it in the original version of the 2.6 What's New doc, so 
> the vast majority of people missed the addition :(

Hi Nick,

thank you very much for this story and the link to the issue tracker!
A very good move for Python which is really not mentioned enough
to make people more aware of a great feature.

I think part of this discussion should get a more prominent place
for gems that can be found without knowing what to search. ;-)

Is the issue tracker permanent enough to reference it?
Maybe there could be some auxiliary info page with proper keywords
that collects links to relevant discussions like this.
Do we have such a thing already?

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/a2715f50/attachment.html>

From vinay_sajip at yahoo.co.uk  Tue Nov 20 15:35:07 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 20 Nov 2012 14:35:07 +0000 (UTC)
Subject: [Python-Dev] Accept just PEP-0426
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
Message-ID: <loom.20121120T150418-29@post.gmane.org>

Daniel Holth <dholth <at> gmail.com> writes:

> Mostly it seems a bit silly to have so much conversations about parts of the
> pep that remain unchanged from previously accepted versions...

I don't agree with the suggestion that we shouldn't discuss it because it was
accepted in a previous version. Perhaps it didn't receive the right scrutiny at
that time, but since it hasn't been implemented, it's reasonable to discuss it.

ISTM that implementing it as suggested in the PEP can lead to certain problems,
since it is a multi-valued field. If it is left in, then something should be
said in the PEP about the potential difficulties and if/how they can be
resolved.

The difficulties I am talking about relate to dependency resolution. Given the
current definition of Provides-Dist, it is possible for a package A on PyPI to
"Provide" all of e.g. "A (1.0)", "B (1.2)" and "C (1.5)", and it is also
possible for packages B and C on PyPI to provide the same (or slightly
different) versions of logical packages of A, B, and C. This will likely lead
to the need for a sophisticated dependency resolver because the dependency
graph can get quite convoluted. (Remember, we might need to do this resolution
when removing packages as well as when installing them.) I know there are SAT
solvers and such, but I'm not sure we need that level of sophistication, or
whether its complexity cost is outweighed by any benefit. Remember, we are
managing fine without multi-valued Provides-Dist, and while a case has been
made for virtual packages and forks (which just require a single-valued field),
no compelling case has been made for bundling packages in general (I understand
that such requirements might sometimes arise in certain corporate environments,
but they don't seem to be a mainstream use case). Hence, no strong case has
been made for a multi-valued "Provides" field.

If we have a good index and packaging infrastructure, there is no general need
for packages to bundle other packages, unless those bundled packages are changed
in some way to suit the bundler's needs. In that case, I don't know how you
could be sure that a bundled "A (1.0)" hasn't diverged from the equivalent
package on PyPI.

The "Provides" seems essentially useless in a metadata index, since if, when
asked to install D which has a dependency on A, you would download and install
A to resolve it rather than B or C, and I can't see when you would want to
query the index to say "who provides A?" and then use some heuristic to pick
e.g. B or C, rather than A.

distlib currently contains support for the multi-valued "Provides", but I'm
not confident that will work as expected given pathological cases like the
example I suggested, without getting "complicated" in the Zen of Python sense.
I'm not convinced that the maintenance burden of a complicated solution is
worth the heretofore unnecessary ability to bundle stuff in arbitrary ways.

Regards,

Vinay Sajip


From ncoghlan at gmail.com  Tue Nov 20 15:44:53 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Nov 2012 00:44:53 +1000
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <50AB8975.3020405@stackless.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE2A53@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7ccgu9aMCXUqmSfmRCr3_ZJY5-A8hwemwYTFzUMXHdKzw@mail.gmail.com>
	<50AB8975.3020405@stackless.com>
Message-ID: <CADiSq7c=E867L7_1PT4dCPVh1UO+kc3xwoXmK3tXbeXBxBApJQ@mail.gmail.com>

On Tue, Nov 20, 2012 at 11:45 PM, Christian Tismer <tismer at stackless.com>wrote:

>  On 20.11.12 12:39, Nick Coghlan wrote:
>
> On Tue, Nov 20, 2012 at 7:06 PM, Kristj?n Valur J?nsson <
> kristjan at ccpgames.com> wrote:
>
>>  I?m intrigued.  I thought this was merely so that one could do
>>
>> python ?m mypackage.mysubpackage
>>
>> Can you refer me to the rationale and discussion about this feature?
>>
>
> It was part of a fairly long progression in changes to the command line
> execution support :)
>
> Top level package execution with -m came first in 2.4, arbitrary package
> execution for -m arrived in 2.5 (along with the runpy module), directory
> and zipfile execution (by supplying a valid sys.path entry as the "script"
> command line argument) was added in 2.6/3.0, and finally officially
> supported package execution via -m only arrived in 2.7 and 3.1 (a broken
> version of the last accidentally existed in 2.5, but that was just a
> mistake that arose when merging the import emulations in runpy and pkgutil,
> and had a side effect that violated at least one of the import system
> invariants).
>
> In the specific case of directory and zipfile execution, discussion
> happened on the tracker: http://bugs.python.org/issue1739468
>
> It was never brought up on python-dev because Guido was participating
> directly in the tracker discussion. Unfortunately, we then also forgot to
> mention it in the original version of the 2.6 What's New doc, so the vast
> majority of people missed the addition :(
>
>
> Hi Nick,
>
> thank you very much for this story and the link to the issue tracker!
> A very good move for Python which is really not mentioned enough
> to make people more aware of a great feature.
>
> I think part of this discussion should get a more prominent place
> for gems that can be found without knowing what to search. ;-)
>

Technically, that's the "using" guide:
http://docs.python.org/2/using/cmdline.html#interface-options (see the
explanation of what's executable under the "<script>" tag)

I also wrote up a summary last year on my blog:
http://www.boredomandlaziness.org/2011/03/what-is-python-script.html

(The main change since that post is that the Python launcher brings shebang
line support to Windows, although I haven't checked if it works properly
with prefixed zip files)

Maybe I should plan to sign up to present an updated version of my PyCon AU
2010 "What is a Python script?" lightning talk at PyCon US 2013 :)


> Is the issue tracker permanent enough to reference it?
>

I've been referencing that particular issue for years now, so I certainly
think so :)


> Maybe there could be some auxiliary info page with proper keywords
> that collects links to relevant discussions like this.
> Do we have such a thing already?
>

I've sometimes thought it may be a good idea to split out a separate "What
is a Python script?" section in the using guide, separate from the existing
descriptions under the interpreter options. I've never tried to figure out
the details of how that would actually work, though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121121/642d32f1/attachment.html>

From dholth at gmail.com  Tue Nov 20 15:56:14 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 20 Nov 2012 09:56:14 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <loom.20121120T150418-29@post.gmane.org>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
Message-ID: <CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>

On Tue, Nov 20, 2012 at 9:35 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk>wrote:

> Daniel Holth <dholth <at> gmail.com> writes:
>
> > Mostly it seems a bit silly to have so much conversations about parts of
> the
> > pep that remain unchanged from previously accepted versions...
>
> I don't agree with the suggestion that we shouldn't discuss it because it
> was
> accepted in a previous version. Perhaps it didn't receive the right
> scrutiny at
> that time, but since it hasn't been implemented, it's reasonable to
> discuss it.
>
> ISTM that implementing it as suggested in the PEP can lead to certain
> problems,
> since it is a multi-valued field. If it is left in, then something should
> be
> said in the PEP about the potential difficulties and if/how they can be
> resolved.
>
> The difficulties I am talking about relate to dependency resolution. Given
> the
> current definition of Provides-Dist, it is possible for a package A on
> PyPI to
> "Provide" all of e.g. "A (1.0)", "B (1.2)" and "C (1.5)", and it is also
> possible for packages B and C on PyPI to provide the same (or slightly
> different) versions of logical packages of A, B, and C. This will likely
> lead
> to the need for a sophisticated dependency resolver because the dependency
> graph can get quite convoluted. (Remember, we might need to do this
> resolution
> when removing packages as well as when installing them.) I know there are
> SAT
> solvers and such, but I'm not sure we need that level of sophistication, or
> whether its complexity cost is outweighed by any benefit. Remember, we are
> managing fine without multi-valued Provides-Dist, and while a case has been
> made for virtual packages and forks (which just require a single-valued
> field),
> no compelling case has been made for bundling packages in general (I
> understand
> that such requirements might sometimes arise in certain corporate
> environments,
> but they don't seem to be a mainstream use case). Hence, no strong case has
> been made for a multi-valued "Provides" field.
>
> If we have a good index and packaging infrastructure, there is no general
> need
> for packages to bundle other packages, unless those bundled packages are
> changed
> in some way to suit the bundler's needs. In that case, I don't know how you
> could be sure that a bundled "A (1.0)" hasn't diverged from the equivalent
> package on PyPI.
>
> The "Provides" seems essentially useless in a metadata index, since if,
> when
> asked to install D which has a dependency on A, you would download and
> install
> A to resolve it rather than B or C, and I can't see when you would want to
> query the index to say "who provides A?" and then use some heuristic to
> pick
> e.g. B or C, rather than A.
>
> distlib currently contains support for the multi-valued "Provides", but I'm
> not confident that will work as expected given pathological cases like the
> example I suggested, without getting "complicated" in the Zen of Python
> sense.
> I'm not convinced that the maintenance burden of a complicated solution is
> worth the heretofore unnecessary ability to bundle stuff in arbitrary ways.
>

If you don't have Provides-Dist, then distribute must continue to bundle an
extra .egg-info directory to emulate the feature. This is more than enough
justification for me. Name: is essentially an alias for Provides-Dist: (or
vice-versa) so there is no such thing as a single-valued Provides-Dist.
Having two names for a package is just as complicated as having twenty.

You should not implement Provides-Dist by searching for every
Provides-Dist: name on PyPI. You should only use it when deciding whether
to download setuptools when distribute is already installed and a package
depends on setuptools.

The bundling term was bad wording on the part of the PEP. No one should
ever include non-renamed copies of other dists in their dists "import six"
vs. "import django.util.six". I've suggested a new wording in this thread.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/85189559/attachment.html>

From vinay_sajip at yahoo.co.uk  Tue Nov 20 16:16:25 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 20 Nov 2012 15:16:25 +0000 (UTC)
Subject: [Python-Dev] Accept just PEP-0426
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
Message-ID: <loom.20121120T160434-5@post.gmane.org>

Daniel Holth <dholth <at> gmail.com> writes:

> If you don't have Provides-Dist, then distribute must continue to bundle an
> extra .egg-info directory to emulate the feature. This is more than enough
> justification for me. Name: is essentially an alias for Provides-Dist: (or
> vice-versa) so there is no such thing as a single-valued Provides-Dist. Having
> two names for a package is just as complicated as having twenty.

I'm not so sure. In the case of two names, it could be assumed that one was a
fork of the other (as in the specific cases of distribute/setuptools, or PIL/
Pillow). You cannot reasonably make this assumption if you have twenty entries
in your Provides-Dist.

> You should not implement Provides-Dist by searching for every Provides-Dist:
> name on PyPI.

I wasn't seriously suggesting that this approach be taken - merely pointing out
that Provides-Dist isn't of much use in a metadata index.

> should only use it when deciding whether to download setuptools when distribute
> is already installed and a package depends on setuptools.The bundling term was
> bad wording on the part of the PEP. No one should ever include non-renamed
> copies of other dists in their dists "import six" vs. "import django.util.six".
> I've suggested a new wording in this thread.

So apart from the setuptools/distribute and PIL/Pillow scenarios, what are the
scenarios where you would have 3 or more values in Provides-Dist? If they are
e.g. a bundled SQLAlchemy, why would that be preferable to an entry in
Requires-Dist?

Regards,

Vinay Sajip






From ncoghlan at gmail.com  Tue Nov 20 16:49:28 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Nov 2012 01:49:28 +1000
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <loom.20121120T160434-5@post.gmane.org>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
Message-ID: <CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>

On Wed, Nov 21, 2012 at 1:16 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk>wrote:

> Daniel Holth <dholth <at> gmail.com> writes:
>
> > If you don't have Provides-Dist, then distribute must continue to bundle
> an
> > extra .egg-info directory to emulate the feature. This is more than
> enough
> > justification for me. Name: is essentially an alias for Provides-Dist:
> (or
> > vice-versa) so there is no such thing as a single-valued Provides-Dist.
> Having
> > two names for a package is just as complicated as having twenty.
>
> I'm not so sure. In the case of two names, it could be assumed that one
> was a
> fork of the other (as in the specific cases of distribute/setuptools, or
> PIL/
> Pillow). You cannot reasonably make this assumption if you have twenty
> entries
> in your Provides-Dist.
>
> > You should not implement Provides-Dist by searching for every
> Provides-Dist:
> > name on PyPI.
>
> I wasn't seriously suggesting that this approach be taken - merely
> pointing out
> that Provides-Dist isn't of much use in a metadata index.
>
> > should only use it when deciding whether to download setuptools when
> distribute
> > is already installed and a package depends on setuptools.The bundling
> term was
> > bad wording on the part of the PEP. No one should ever include
> non-renamed
> > copies of other dists in their dists "import six" vs. "import
> django.util.six".
> > I've suggested a new wording in this thread.
>
> So apart from the setuptools/distribute and PIL/Pillow scenarios, what are
> the
> scenarios where you would have 3 or more values in Provides-Dist? If they
> are
> e.g. a bundled SQLAlchemy, why would that be preferable to an entry in
> Requires-Dist?
>

Provides/Requires/Obsoletes are *not* for bundling. Publishing bundled
packages on the index is bad, and people shouldn't do it. What they're for
is tracking name changes over time, so that you can fork and rename and
merge projects without breaking the world for people that depend on your
projects (one example used in the Fedora RPM docs is the apache package
being renamed to httpd:
https://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html-single/RPM_Guide/index.html#ch-dependencies
).

The fact distribute can provide setuptools and Pillow can provide PIL are
examples of the simple fork/rename case - they're designed to be drop in
replacements for the projects they forked, so it's appropriate for them to
advertise that fact in a way the deployment tools can understand. The
multi-value support is then needed if you have multiple name changes over
time (e.g. if someone were to create a distribute2 that provided both
distribute and setuptools), or if you merge two projects together (e.g. if
a popular extension to a project was folded into the main distribution for
that project).

It's likely fine if an installer doesn't use sophisticated graph analysis
to find the "best" way to satisfy a set of requirements - you can just as
easily use it in the simple way Daniel describes of only using these fields
to check for existing locally installed packages with the necessary
capabilities, before going out to get whatever is missing from the package
index based purely on the distribution names.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121121/04b5f179/attachment.html>

From vinay_sajip at yahoo.co.uk  Tue Nov 20 17:04:06 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 20 Nov 2012 16:04:06 +0000 (UTC)
Subject: [Python-Dev] Accept just PEP-0426
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
Message-ID: <loom.20121120T165425-38@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Provides/Requires/Obsoletes are *not* for bundling. Publishing bundled packages
> on the index is bad, and people shouldn't do it.
[detail snipped]
> It's likely fine if an installer doesn't use sophisticated graph
> analysis to find the "best" way to satisfy a set of requirements - you can
> just as easily use it in the simple way Daniel describes of only using these
> fields to check for existing locally installed packages with the necessary
> capabilities, before going out to get whatever is missing from the package
> index based purely on the distribution names.

In which case, it seems sensible to put these constraints into the PEP,
so that both PEP implementers and users of those implementations have these
guidelines clarified.

Regards,

Vinay Sajip



From dholth at gmail.com  Tue Nov 20 17:18:00 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 20 Nov 2012 11:18:00 -0500
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <CADiSq7c=E867L7_1PT4dCPVh1UO+kc3xwoXmK3tXbeXBxBApJQ@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE2A53@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7ccgu9aMCXUqmSfmRCr3_ZJY5-A8hwemwYTFzUMXHdKzw@mail.gmail.com>
	<50AB8975.3020405@stackless.com>
	<CADiSq7c=E867L7_1PT4dCPVh1UO+kc3xwoXmK3tXbeXBxBApJQ@mail.gmail.com>
Message-ID: <CAG8k2+6+rRwyTndvf87gGrSGH9jh0h=5L8k+m_nHQyA59DxEJg@mail.gmail.com>

On Tue, Nov 20, 2012 at 9:44 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Tue, Nov 20, 2012 at 11:45 PM, Christian Tismer <tismer at stackless.com>wrote:
>
>>  On 20.11.12 12:39, Nick Coghlan wrote:
>>
>> On Tue, Nov 20, 2012 at 7:06 PM, Kristj?n Valur J?nsson <
>> kristjan at ccpgames.com> wrote:
>>
>>>  I?m intrigued.  I thought this was merely so that one could do
>>>
>>> python ?m mypackage.mysubpackage
>>>
>>> Can you refer me to the rationale and discussion about this feature?
>>>
>>
>> It was part of a fairly long progression in changes to the command line
>> execution support :)
>>
>> Top level package execution with -m came first in 2.4, arbitrary package
>> execution for -m arrived in 2.5 (along with the runpy module), directory
>> and zipfile execution (by supplying a valid sys.path entry as the "script"
>> command line argument) was added in 2.6/3.0, and finally officially
>> supported package execution via -m only arrived in 2.7 and 3.1 (a broken
>> version of the last accidentally existed in 2.5, but that was just a
>> mistake that arose when merging the import emulations in runpy and pkgutil,
>> and had a side effect that violated at least one of the import system
>> invariants).
>>
>> In the specific case of directory and zipfile execution, discussion
>> happened on the tracker: http://bugs.python.org/issue1739468
>>
>> It was never brought up on python-dev because Guido was participating
>> directly in the tracker discussion. Unfortunately, we then also forgot to
>> mention it in the original version of the 2.6 What's New doc, so the vast
>> majority of people missed the addition :(
>>
>>
>> Hi Nick,
>>
>> thank you very much for this story and the link to the issue tracker!
>> A very good move for Python which is really not mentioned enough
>> to make people more aware of a great feature.
>>
>> I think part of this discussion should get a more prominent place
>> for gems that can be found without knowing what to search. ;-)
>>
>
> Technically, that's the "using" guide:
> http://docs.python.org/2/using/cmdline.html#interface-options (see the
> explanation of what's executable under the "<script>" tag)
>
> I also wrote up a summary last year on my blog:
> http://www.boredomandlaziness.org/2011/03/what-is-python-script.html
>
> (The main change since that post is that the Python launcher brings
> shebang line support to Windows, although I haven't checked if it works
> properly with prefixed zip files)
>
> Maybe I should plan to sign up to present an updated version of my PyCon
> AU 2010 "What is a Python script?" lightning talk at PyCon US 2013 :)
>
>
>>  Is the issue tracker permanent enough to reference it?
>>
>
> I've been referencing that particular issue for years now, so I certainly
> think so :)
>
>
>>  Maybe there could be some auxiliary info page with proper keywords
>> that collects links to relevant discussions like this.
>> Do we have such a thing already?
>>
>
> I've sometimes thought it may be a good idea to split out a separate "What
> is a Python script?" section in the using guide, separate from the existing
> descriptions under the interpreter options. I've never tried to figure out
> the details of how that would actually work, though.
>

The wheel projects' own wheel file (a zip file) takes advantage of this
feature in its own way:

python wheel-0.14.0-py2.py3-none-any.whl/wheel

Python looks inside the /wheel directory of the zip file, finds
__main__.py, and executes it; the archive can be used to install itself or
any other wheel file. (There is no __main__.py at the root because the
wheel design would install that __main__.py into the root of site-packages).


This underutilized feature of executing directories with __main__.py is
very useful for implementing Python applications instead of just libraries.

It might be interesting to define a "wheels" format which would be a bunch
of unpacked wheel files and a __main__.py like:

__main__.py
package1-x86.whl/...
package1-armel.whl/...
package2-noarch.whl/

__main__.py would add the appropriate packages to PYTHONPATH based on the
architecture, unpack dll's pkg_resources style, and run the program.


There is some activity in the tracker about adding the missing "add this to
PYTHONPATH" / "isolate self from the environment" command line arguments to
Python.

Daniel Holth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/d9b97357/attachment.html>

From dholth at gmail.com  Tue Nov 20 17:19:32 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 20 Nov 2012 11:19:32 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <loom.20121120T165425-38@post.gmane.org>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
Message-ID: <CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>

Edit the following text:


Provides-Dist (multiple use)

Each entry contains a string naming a requirement that is satisfied by
installing this distribution.  This field *must* include the project
identified in the ``Name`` field, optionally followed by the version Name
(Version).

A distribution may provide additional names, e.g. to indicate that
multiple projects have been merged into a single distribution or to
indicate that this project is a substitute for another.  For instance
distribute (a fork of setuptools) could ``Provides-Dist`` setuptools to
prevent the conflicting package from being downloaded and installed when
distribute is already installed.  A distribution that has been merged
with another might ``Provides-Dist`` the obsolete name(s) to satisfy
any projects that require the obsolete distribution's name.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/a61380f2/attachment.html>

From ncoghlan at gmail.com  Tue Nov 20 17:36:00 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Nov 2012 02:36:00 +1000
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <loom.20121120T165425-38@post.gmane.org>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
Message-ID: <CADiSq7eaBHcFcNZC7V3tcS3TZ9RiPm4cj7i2B-sazxotEeprFw@mail.gmail.com>

On Wed, Nov 21, 2012 at 2:04 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk>wrote:

> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
> > Provides/Requires/Obsoletes are *not* for bundling. Publishing bundled
> packages
> > on the index is bad, and people shouldn't do it.
> [detail snipped]
> > It's likely fine if an installer doesn't use sophisticated graph
> > analysis to find the "best" way to satisfy a set of requirements - you
> can
> > just as easily use it in the simple way Daniel describes of only using
> these
> > fields to check for existing locally installed packages with the
> necessary
> > capabilities, before going out to get whatever is missing from the
> package
> > index based purely on the distribution names.
>
> In which case, it seems sensible to put these constraints into the PEP,
> so that both PEP implementers and users of those implementations have these
> guidelines clarified.
>

Yes, I thought Daniel's rewording looked pretty reasonable on that front.
However, the details of how an installer uses this information is really up
to the installer developers and what their users expect/demand. It
certainly isn't *practical* to do a full dependency analysis when PyPI
doesn't provide the same kind of precalculated metadata that a yum repo
does, but that's not something that should be spelled out in the
distribution metadata PEP, any more than it is spelled out in the RPM
format spec.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121121/e8c1bbd4/attachment.html>

From vinay_sajip at yahoo.co.uk  Tue Nov 20 17:49:56 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 20 Nov 2012 16:49:56 +0000 (UTC)
Subject: [Python-Dev] Accept just PEP-0426
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
Message-ID: <loom.20121120T172255-830@post.gmane.org>

Daniel Holth <dholth <at> gmail.com> writes:

 
> Edit the following text:

Okay, here is a possible version:
---------------------------------
Provides-Dist (multiple use)

Each entry contains a string naming a requirement that is satisfied by
installing this distribution.  The entry must consist of a name and version.

This name of the project identified in the ``Name`` field is implicitly
considered as provided, with the version specified in the ``Version`` field.

The use of multiple names in this field *must not* be used for bundling
distributions together. It is intended for use when projects are forked and
merged over time, while providing essentially the same function. Multiple
names reflect the evolution of the project over time and not the bringing
together of different packages, already distributed elsewhere, in a bundle.

Thus, the 'distribute' distribution, a fork of setuptools, could say that it
``Provides-Dist`` a particular version of setuptools, to prevent setuptools
from being downloaded and installed when distribute is already installed.

If, over time, distribute evolved into a new package called 'distribute2' (for
argument's sake), then that could say that it ``Provides-Dist`` a specific
version of distribute and a specific version of setuptools.
-----------------------------------

Some comments on the above: I'm not entirely comfortable with a Provides-Dist
entry which does not specify a version, since it does not allow to you to test
that a requirement is actually met. So, I've removed the "optional"
qualification from the version.

Also: what happens when a requirement is for setuptools (>= X.Y), but the
distribute fork hasn't kept pace, and so only supports setuptools at a lower
version than X.Y? I take it we're entirely comfortable with installing
setuptools X.Y in that case? How would you ensure the right setuptools is
always loaded, since presumably both are on sys.path?

Regards,

Vinay Sajip


From pje at telecommunity.com  Tue Nov 20 21:46:27 2012
From: pje at telecommunity.com (PJ Eby)
Date: Tue, 20 Nov 2012 15:46:27 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <loom.20121120T172255-830@post.gmane.org>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
Message-ID: <CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>

On Tue, Nov 20, 2012 at 11:49 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk>wrote:

> Also: what happens when a requirement is for setuptools (>= X.Y), but the
> distribute fork hasn't kept pace, and so only supports setuptools at a
> lower
> version than X.Y? I take it we're entirely comfortable with installing
> setuptools X.Y in that case? How would you ensure the right setuptools is
> always loaded, since presumably both are on sys.path?
>

Egg-based tools don't have any problem with this, since they set sys.path
to include the eggs needed for the running program.  Other tools will have
to tell the user and let them work it out, e.g. by using a different
virtualenv.

I personally don't think that forks claiming to "provide" something is
really a good thing to encourage; ISTM that saying a package *conflicts*
with another is more accurate, e.g. distribute Conflicts-Dist setuptools.
I also think distributions should say they are obsoleted, rather than
allowing other distributions to obsolete them.

That is, centralized packaging systems rely on a central authority to
resolve issues of who provides what and obsoletes what; there's an implicit
"x obsoletes y [by decree of semi-independent third-party z]".

However, in Python package metadata, it's "x obsoletes y [by decree of
x]".  IMO, this should be reversed to, "Y is obsoleted by x [by decree of
y]", and "installing Y will conflict with X [by decree of X]", so that in
each case the scope of authority for the statement is clear.

That is, in each case (conflict or obsolescence), the project's developers
are declaring under what conditions they will not be supporting an
installation.  In the case of obsolescence, the developer is saying, "this
is being phased out, you should use that other thing instead".  In the case
of forks, the developer is saying, "If you install both versions,
something's gonna break."

Note that installation conflict is a more conservative claim anyway: a
conflict between forked "foobar" packages is permanent, in the sense that
it doesn't matter what versions of both packages you're interested in: they
both want to install a foobar/__init__.py.  (Of course, installers can and
should detect that condition automatically, but not until they download the
package first.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/9952bd9c/attachment-0001.html>

From jimjjewett at gmail.com  Tue Nov 20 21:58:54 2012
From: jimjjewett at gmail.com (Jim J. Jewett)
Date: Tue, 20 Nov 2012 12:58:54 -0800 (PST)
Subject: [Python-Dev]  Keyword meanings [was: Accept just PEP-0426]
In-Reply-To: <loom.20121120T172255-830@post.gmane.org>
Message-ID: <50abef0e.ab96320a.6675.ffffb439@mx.google.com>


 
Vinay Sajip reworded the 'Provides-Dist' definition to explicitly say:

> The use of multiple names in this field *must not* be used for
> bundling distributions together. It is intended for use when
> projects are forked and merged over time ...

(1)  Then how *should* the "bundle-of-several-components" case be
represented?

(2)  How is 'Provides-Dist' different from 'Obsoletes-Dist'?
The only difference I can see is that it may be a bit more polite
to people who do want to install multiple versions of a (possibly
abstract) package.


-jJ

-- 

If there are still threading problems with my replies, please 
email me with details, so that I can try to resolve them.  -jJ


From v+python at g.nevcal.com  Tue Nov 20 22:07:52 2012
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 20 Nov 2012 13:07:52 -0800
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
Message-ID: <50ABF128.4020605@g.nevcal.com>

On 11/20/2012 12:46 PM, PJ Eby wrote:
> On Tue, Nov 20, 2012 at 11:49 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk 
> <mailto:vinay_sajip at yahoo.co.uk>> wrote:
>
>     Also: what happens when a requirement is for setuptools (>= X.Y),
>     but the
>     distribute fork hasn't kept pace, and so only supports setuptools
>     at a lower
>     version than X.Y? I take it we're entirely comfortable with installing
>     setuptools X.Y in that case? How would you ensure the right
>     setuptools is
>     always loaded, since presumably both are on sys.path?
>
>
> Egg-based tools don't have any problem with this, since they set 
> sys.path to include the eggs needed for the running program.  Other 
> tools will have to tell the user and let them work it out, e.g. by 
> using a different virtualenv.
>
> I personally don't think that forks claiming to "provide" something is 
> really a good thing to encourage; ISTM that saying a package 
> *conflicts* with another is more accurate, e.g. distribute 
> Conflicts-Dist setuptools.  I also think distributions should say they 
> are obsoleted, rather than allowing other distributions to obsolete them.

Obsolete distributions won't say they are obsoleted, unless they receive 
further maintenance. However, if the distribution is obsolete because 
the maintainer has lost interest, they won't receive further maintenance.


> That is, centralized packaging systems rely on a central authority to 
> resolve issues of who provides what and obsoletes what; there's an 
> implicit "x obsoletes y [by decree of semi-independent third-party z]".
>
> However, in Python package metadata, it's "x obsoletes y [by decree of 
> x]".  IMO, this should be reversed to, "Y is obsoleted by x [by decree 
> of y]", and "installing Y will conflict with X [by decree of X]", so 
> that in each case the scope of authority for the statement is clear.
>
> That is, in each case (conflict or obsolescence), the project's 
> developers are declaring under what conditions they will not be 
> supporting an installation.  In the case of obsolescence, the 
> developer is saying, "this is being phased out, you should use that 
> other thing instead".  In the case of forks, the developer is saying, 
> "If you install both versions, something's gonna break."
>
> Note that installation conflict is a more conservative claim anyway: a 
> conflict between forked "foobar" packages is permanent, in the sense 
> that it doesn't matter what versions of both packages you're 
> interested in: they both want to install a foobar/__init__.py.  (Of 
> course, installers can and should detect that condition automatically, 
> but not until they download the package first.)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/70c7ae5f/attachment.html>

From dholth at gmail.com  Tue Nov 20 22:31:24 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 20 Nov 2012 16:31:24 -0500
Subject: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
In-Reply-To: <50abef0e.ab96320a.6675.ffffb439@mx.google.com>
References: <loom.20121120T172255-830@post.gmane.org>
	<50abef0e.ab96320a.6675.ffffb439@mx.google.com>
Message-ID: <CAG8k2+6Q1r+i__Ea29yawxBJztY1F7iGGKYj78TEVJatVYFeXg@mail.gmail.com>

On Tue, Nov 20, 2012 at 3:58 PM, Jim J. Jewett <jimjjewett at gmail.com> wrote:

>
>
> Vinay Sajip reworded the 'Provides-Dist' definition to explicitly say:
>
> > The use of multiple names in this field *must not* be used for
> > bundling distributions together. It is intended for use when
> > projects are forked and merged over time ...
>
> (1)  Then how *should* the "bundle-of-several-components" case be
> represented?
>
> (2)  How is 'Provides-Dist' different from 'Obsoletes-Dist'?
> The only difference I can see is that it may be a bit more polite
> to people who do want to install multiple versions of a (possibly
> abstract) package.
>

The useful way to bundle a bunch of things would be to just include them
all in an executable folder or zipfile with __main__.py. PEP 426 and the
package database would not get involved. The bundle would be distributed as
an application you can download and use, not as an sdist on PyPI.

The intent of Provides and Obsoletes is different. Obsoletes would not
satisfy a requirement during dependency resolution.

The RPM guide explains a similar system:
This brings the total to four types of dependencies that the RPM system
tracks:

   - Requires, which tracks the capabilities a package requires
   - Provides, which tracks the capabilities a package provides for other
   packages
   - Conflicts, which describes the capabilities that if installed,
   conflict with capabilities in a package
   - Obsoletes, which describes the capabilities that this package will
   make obsolete

Packages advertise this dependency information. Each dependency holds the
type, such as requires, a capability, such as a shared library or a package
name, and optionally a version number, such as requiring the python package
at a version number greater than or equal to 2.2 (python >= 2.2).

http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-dependencies.html#RPM_Guide-Dependencies-obsoletes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/da409fec/attachment.html>

From pje at telecommunity.com  Tue Nov 20 22:48:51 2012
From: pje at telecommunity.com (PJ Eby)
Date: Tue, 20 Nov 2012 16:48:51 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <50ABF128.4020605@g.nevcal.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
Message-ID: <CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>

On Tue, Nov 20, 2012 at 4:07 PM, Glenn Linderman <v+python at g.nevcal.com>wrote:

>  On 11/20/2012 12:46 PM, PJ Eby wrote:
>
>  I personally don't think that forks claiming to "provide" something is
> really a good thing to encourage; ISTM that saying a package *conflicts*
> with another is more accurate, e.g. distribute Conflicts-Dist setuptools.
> I also think distributions should say they are obsoleted, rather than
> allowing other distributions to obsolete them.
>
>  Obsolete distributions won't say they are obsoleted, unless they receive
> further maintenance. However, if the distribution is obsolete because the
> maintainer has lost interest, they won't receive further maintenance.
>

(We've been over this before, the last time this discussion came up on the
Distutils-SIG for a previous Metadata PEP a year or two back, but here
goes....)

Obsoleting a package is for handling renames and support transitions.  For
example, if it actually did anything to do so, I'd mark RuleDispatch as
obsoleted-by PEAK, the Pylons folks might mark some version of that as
obsoleted-by Pyramid, etc.  To put it another way, marking a package
obsolete is part of deprecation and replacement, not an unsubstantiated
third-party claim about the maintenance status of an unrelated project.

If a package is *actually* dead, there's no real point to declaring that
something else obsoletes it, and certainly no reason to put it in metadata
form.  Otherwise, we could have Twisted claiming to obsolete GEvent and
vice-versa at the same time.  Which one should an installer believe?  It
makes no sense in a standard where the project's maintainers can say
whatever they want about somebody else's project.  The scope of authority
for automatically-consumed metadata should *only* encompass the project
that provided the metadata.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/9b613f22/attachment.html>

From donald.stufft at gmail.com  Tue Nov 20 22:52:11 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Tue, 20 Nov 2012 16:52:11 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
Message-ID: <C616B797714D4BB397ACAB9EC81B1C51@gmail.com>

On Tuesday, November 20, 2012 at 4:48 PM, PJ Eby wrote:
> Words
I agree that obsoletes is terrible, it's very specific and not something we particularly require. I'd much rather just have a generic "conflicts". 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/0deed046/attachment.html>

From dholth at gmail.com  Tue Nov 20 23:01:02 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 20 Nov 2012 17:01:02 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <C616B797714D4BB397ACAB9EC81B1C51@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
	<C616B797714D4BB397ACAB9EC81B1C51@gmail.com>
Message-ID: <CAG8k2+7DOcYxDo1vj_Fh5RE3Jbrmb3iQU_Tr_T59JaB5d_uNwA@mail.gmail.com>

I think the Metadata 1.1 treatment of these concepts is in some ways
better. (Metadata 1.2 added the -Dist suffix to the fields in an attempt to
make it clear that dependency names are PyPI names and not "import x"
names.)

http://www.python.org/dev/peps/pep-0314/ says:

Provides (multiple use)

      Each entry contains a string describing a package or module that
      will be provided by this package once it is installed.  These
      strings should match the ones used in Requirements fields.  A
      version declaration may be supplied (without a comparison
      operator); the package's version number will be implied if none
      is specified.

      Example:

          Provides: xml
          Provides: xml.utils
          Provides: xml.utils.iso8601
          Provides: xml.dom
          Provides: xmltools (1.3)

    Obsoletes (multiple use)

      Each entry contains a string describing a package or module
      that this package renders obsolete, meaning that the two packages
      should not be installed at the same time.  Version declarations
      can be supplied.

      The most common use of this field will be in case a package name
      changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0.
      When you install Torqued Python, the Gorgon package should be
      removed.

      Example:

          Obsoletes: Gorgon


They mean pretty much what the same words mean in RPM and do not need
further bikeshedding.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/bbc8a131/attachment.html>

From brett at python.org  Tue Nov 20 23:12:28 2012
From: brett at python.org (Brett Cannon)
Date: Tue, 20 Nov 2012 17:12:28 -0500
Subject: [Python-Dev] [Python-checkins] cpython (3.3): - Issue #16514:
 Fix regression causing a traceback when sys.path[0] is None
In-Reply-To: <3Y5dwB3ZCMzMlG@mail.python.org>
References: <3Y5dwB3ZCMzMlG@mail.python.org>
Message-ID: <CAP1=2W4KTyoDYW6jO_1oQd+rYDE_cboKgsOk=05R5FzjPxEyqQ@mail.gmail.com>

Should you also insert None into sys.path_importer_cache to signify there
is no finder for the path entry? I guess the real problem with that is
there is no guarantee the path entry is hashable, so that probably won't
work. So nevermind. =)


On Tue, Nov 20, 2012 at 3:35 PM, barry.warsaw <python-checkins at python.org>wrote:

> http://hg.python.org/cpython/rev/291406748217
> changeset:   80529:291406748217
> branch:      3.3
> parent:      80527:4537dd27b2dc
> user:        Barry Warsaw <barry at python.org>
> date:        Tue Nov 20 15:22:51 2012 -0500
> summary:
>   - Issue #16514: Fix regression causing a traceback when sys.path[0] is
> None
>   (actually, any non-string or non-bytes type).
>
> files:
>   Doc/library/sys.rst                          |    4 +-
>   Doc/reference/import.rst                     |   24 +-
>   Lib/importlib/_bootstrap.py                  |    2 +
>   Lib/test/test_importlib/import_/test_path.py |   25 +-
>   Misc/NEWS                                    |    3 +
>   Python/importlib.h                           |  130 +++++----
>   6 files changed, 111 insertions(+), 77 deletions(-)
>
>
> diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
> --- a/Doc/library/sys.rst
> +++ b/Doc/library/sys.rst
> @@ -783,7 +783,9 @@
>     current directory first.  Notice that the script directory is inserted
> *before*
>     the entries inserted as a result of :envvar:`PYTHONPATH`.
>
> -   A program is free to modify this list for its own purposes.
> +   A program is free to modify this list for its own purposes.  Only
> strings
> +   and bytes should be added to :data:`sys.path`; all other data types are
> +   ignored during import.
>
>
>     .. seealso::
> diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst
> --- a/Doc/reference/import.rst
> +++ b/Doc/reference/import.rst
> @@ -540,7 +540,10 @@
>  implementation-specific defaults.  Entries in :data:`sys.path` can name
>  directories on the file system, zip files, and potentially other
> "locations"
>  (see the :mod:`site` module) that should be searched for modules, such as
> -URLs, or database queries.
> +URLs, or database queries.  Only strings and bytes should be present on
> +:data:`sys.path`; all other data types are ignored.  The encoding of bytes
> +entries is determined by the individual :term:`path entry finders <path
> entry
> +finder>`.
>
>  The :term:`path based finder` is a :term:`meta path finder`, so the import
>  machinery begins the :term:`import path` search by calling the path
> @@ -563,14 +566,17 @@
>  the path based finder to perform the path entry search again [#fnpic]_.
>
>  If the path entry is not present in the cache, the path based finder
> iterates
> -over every callable in :data:`sys.path_hooks`.  Each of the
> -:term:`path entry hooks <path entry hook>` in this list is called with a
> -single argument, the path entry to be searched.  This callable may either
> -return a :term:`path entry finder` that can handle the path entry, or it
> may
> -raise :exc:`ImportError`.
> -An :exc:`ImportError` is used by the path based finder to signal that the
> hook
> -cannot find a :term:`path entry finder` for that :term:`path entry`.  The
> -exception is ignored and :term:`import path` iteration continues.
> +over every callable in :data:`sys.path_hooks`.  Each of the :term:`path
> entry
> +hooks <path entry hook>` in this list is called with a single argument,
> the
> +path entry to be searched.  This callable may either return a :term:`path
> +entry finder` that can handle the path entry, or it may raise
> +:exc:`ImportError`.  An :exc:`ImportError` is used by the path based
> finder to
> +signal that the hook cannot find a :term:`path entry finder` for that
> +:term:`path entry`.  The exception is ignored and :term:`import path`
> +iteration continues.  The hook should expect either a string or bytes
> object;
> +the encoding of bytes objects is up to the hook (e.g. it may be a file
> system
> +encoding, UTF-8, or something else), and if the hook cannot decode the
> +argument, it should raise :exc:`ImportError`.
>
>  If :data:`sys.path_hooks` iteration ends with no :term:`path entry finder`
>  being returned, then the path based finder's :meth:`find_module()` method
> diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
> --- a/Lib/importlib/_bootstrap.py
> +++ b/Lib/importlib/_bootstrap.py
> @@ -1281,6 +1281,8 @@
>          #  the list of paths that will become its __path__
>          namespace_path = []
>          for entry in path:
> +            if not isinstance(entry, (str, bytes)):
> +                continue
>              finder = cls._path_importer_cache(entry)
>              if finder is not None:
>                  if hasattr(finder, 'find_loader'):
> diff --git a/Lib/test/test_importlib/import_/test_path.py
> b/Lib/test/test_importlib/import_/test_path.py
> --- a/Lib/test/test_importlib/import_/test_path.py
> +++ b/Lib/test/test_importlib/import_/test_path.py
> @@ -1,15 +1,14 @@
>  from importlib import _bootstrap
>  from importlib import machinery
> +from importlib import import_module
>  from .. import util
>  from . import util as import_util
> -import imp
>  import os
>  import sys
> -import tempfile
> -from test import support
> -from types import MethodType
> +from types import ModuleType
>  import unittest
>  import warnings
> +import zipimport
>
>
>  class FinderTests(unittest.TestCase):
> @@ -89,6 +88,24 @@
>              self.assertIs(loader, importer)
>              self.assertIn(os.curdir, sys.path_importer_cache)
>
> +    def test_None_on_sys_path(self):
> +        # Putting None in sys.path[0] caused an import regression from
> Python
> +        # 3.2: http://bugs.python.org/issue16514
> +        new_path = sys.path[:]
> +        new_path.insert(0, None)
> +        new_path_importer_cache = sys.path_importer_cache.copy()
> +        new_path_importer_cache.pop(None, None)
> +        new_path_hooks = [zipimport.zipimporter,
> +                          _bootstrap.FileFinder.path_hook(
> +                              *_bootstrap._get_supported_file_loaders())]
> +        with util.uncache('email'):
> +            with util.import_state(meta_path=sys.meta_path[:],
> +                                   path=new_path,
> +
> path_importer_cache=new_path_importer_cache,
> +                                   path_hooks=new_path_hooks):
> +                module = import_module('email')
> +                self.assertIsInstance(module, ModuleType)
> +
>
>  def test_main():
>      from test.support import run_unittest
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -12,6 +12,9 @@
>  Core and Builtins
>  -----------------
>
> +- Issue #16514: Fix regression causing a traceback when sys.path[0] is
> None
> +  (actually, any non-string or non-bytes type).
> +
>  - Issue #16306: Fix multiple error messages when unknown command line
>    parameters where passed to the interpreter.  Patch by Hieu Nguyen.
>
> diff --git a/Python/importlib.h b/Python/importlib.h
> --- a/Python/importlib.h
> +++ b/Python/importlib.h
> [stripped]
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/0c7139ce/attachment-0001.html>

From vinay_sajip at yahoo.co.uk  Tue Nov 20 23:26:01 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 20 Nov 2012 22:26:01 +0000 (UTC)
Subject: [Python-Dev] Accept just PEP-0426
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
	<C616B797714D4BB397ACAB9EC81B1C51@gmail.com>
	<CAG8k2+7DOcYxDo1vj_Fh5RE3Jbrmb3iQU_Tr_T59JaB5d_uNwA@mail.gmail.com>
Message-ID: <loom.20121120T231553-707@post.gmane.org>

Daniel Holth <dholth <at> gmail.com> writes:

> They mean pretty much what the same words mean in RPM and do not need further
> bikeshedding.

But isn't it the case that the scenarios are different because in the case of
RPMs, we have a presumed authority which can determine e.g. what obsoletes what,
whereas with Python distributions, there's no central authority that has this
function?

Regards,

Vinay Sajip



From barry at python.org  Tue Nov 20 23:35:49 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 20 Nov 2012 17:35:49 -0500
Subject: [Python-Dev] [Python-checkins] cpython (3.3): - Issue #16514:
 Fix regression causing a traceback when sys.path[0] is None
In-Reply-To: <CAP1=2W4KTyoDYW6jO_1oQd+rYDE_cboKgsOk=05R5FzjPxEyqQ@mail.gmail.com>
References: <3Y5dwB3ZCMzMlG@mail.python.org>
	<CAP1=2W4KTyoDYW6jO_1oQd+rYDE_cboKgsOk=05R5FzjPxEyqQ@mail.gmail.com>
Message-ID: <20121120173549.01e9107e@resist.wooz.org>

On Nov 20, 2012, at 05:12 PM, Brett Cannon wrote:

>Should you also insert None into sys.path_importer_cache to signify there
>is no finder for the path entry? I guess the real problem with that is
>there is no guarantee the path entry is hashable, so that probably won't
>work. So nevermind. =)

I explicitly popped None from path_importer_cache, to try to force it to
ignore the cache.  Maybe it's not totally correct (and in practice probably
doesn't matter), but it WFM.

-Barry

From barry at python.org  Wed Nov 21 00:41:07 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 20 Nov 2012 18:41:07 -0500
Subject: [Python-Dev] [Python-checkins] cpython (3.3): - Issue #16514:
 Fix regression causing a traceback when sys.path[0] is None
In-Reply-To: <50AC05BD.3080402@udel.edu>
References: <3Y5dwB3ZCMzMlG@mail.python.org>
	<50AC05BD.3080402@udel.edu>
Message-ID: <20121120184107.16713a99@limelight.wooz.org>

On Nov 20, 2012, at 05:35 PM, Terry Reedy wrote:

>On 11/20/2012 3:35 PM, barry.warsaw wrote:
>
>>           for entry in path:
>> +            if not isinstance(entry, (str, bytes)):
>> +                continue
>
>Given that a non-(str,bytes) entry could indicate a programming error, should
>a warning be emitted before continuing?

That's not what happens in Python 3.2.  In fact, this bug report was triggered
by someone who was already inserting None to sys.path[0].  It is silently
ignored in Python 3.2, but tracebacked in 3.3.

Cheers,
-Barry

From dholth at gmail.com  Wed Nov 21 00:43:32 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 20 Nov 2012 18:43:32 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <loom.20121120T231553-707@post.gmane.org>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
	<C616B797714D4BB397ACAB9EC81B1C51@gmail.com>
	<CAG8k2+7DOcYxDo1vj_Fh5RE3Jbrmb3iQU_Tr_T59JaB5d_uNwA@mail.gmail.com>
	<loom.20121120T231553-707@post.gmane.org>
Message-ID: <B3C5A7B3-CF5F-44C3-863D-A7B4402F06CE@gmail.com>

No. We trust the packages we install, including the way they decide to use the metadata. A bad package could delete all our files or cause dependency resolution to fail. Mostly they won't.

Daniel Holth

On Nov 20, 2012, at 5:26 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:

> Daniel Holth <dholth <at> gmail.com> writes:
> 
>> They mean pretty much what the same words mean in RPM and do not need further
>> bikeshedding.
> 
> But isn't it the case that the scenarios are different because in the case of
> RPMs, we have a presumed authority which can determine e.g. what obsoletes what,
> whereas with Python distributions, there's no central authority that has this
> function?
> 
> Regards,
> 
> Vinay Sajip
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com

From a.badger at gmail.com  Wed Nov 21 01:12:08 2012
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Tue, 20 Nov 2012 16:12:08 -0800
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <B3C5A7B3-CF5F-44C3-863D-A7B4402F06CE@gmail.com>
References: <loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
	<C616B797714D4BB397ACAB9EC81B1C51@gmail.com>
	<CAG8k2+7DOcYxDo1vj_Fh5RE3Jbrmb3iQU_Tr_T59JaB5d_uNwA@mail.gmail.com>
	<loom.20121120T231553-707@post.gmane.org>
	<B3C5A7B3-CF5F-44C3-863D-A7B4402F06CE@gmail.com>
Message-ID: <20121121001208.GW2133@unaka.lan>

On Tue, Nov 20, 2012 at 06:43:32PM -0500, Daniel Holth wrote:
> No. We trust the packages we install, including the way they decide to use
> the metadata. A bad package could delete all our files or cause dependency
> resolution to fail. Mostly they won't.
> 
Agreed.  And this is closer to the way that distributions' tools have to
operate than they'd want to :-(  Within the distribution we like to pretend
that we only need to care about the packages that we generate.  But we also
know that whether or not we support it, ordinary users will install pacakges
from outside of our walls.  That means that the packaging tools that we
create will need to deal with things that we might not condone within our
"presumed authority".

We trust that people are going to do more or less the right thing with
the tools we offer.  Once in a while they don't but by and large they do.

-Toshio

> Daniel Holth
> 
> On Nov 20, 2012, at 5:26 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> 
> > Daniel Holth <dholth <at> gmail.com> writes:
> > 
> >> They mean pretty much what the same words mean in RPM and do not need further
> >> bikeshedding.
> > 
> > But isn't it the case that the scenarios are different because in the case of
> > RPMs, we have a presumed authority which can determine e.g. what obsoletes what,
> > whereas with Python distributions, there's no central authority that has this
> > function?
> > 
> > Regards,
> > 
> > Vinay Sajip
> > 
> > 
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/a.badger%40gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/dee0fea7/attachment.pgp>

From v+python at g.nevcal.com  Wed Nov 21 01:13:58 2012
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 20 Nov 2012 16:13:58 -0800
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
Message-ID: <50AC1CC6.1010100@g.nevcal.com>

On 11/20/2012 1:48 PM, PJ Eby wrote:
> (We've been over this before, the last time this discussion came up on 
> the Distutils-SIG for a previous Metadata PEP a year or two back, but 
> here goes....)

Thanks. I wasn't over there. Makes it clear that clarifying PEPs to 
reflect discussions is a good idea.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/529d3790/attachment.html>

From jimjjewett at gmail.com  Wed Nov 21 01:18:24 2012
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 20 Nov 2012 19:18:24 -0500
Subject: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
In-Reply-To: <CAG8k2+6Q1r+i__Ea29yawxBJztY1F7iGGKYj78TEVJatVYFeXg@mail.gmail.com>
References: <loom.20121120T172255-830@post.gmane.org>
	<50abef0e.ab96320a.6675.ffffb439@mx.google.com>
	<CAG8k2+6Q1r+i__Ea29yawxBJztY1F7iGGKYj78TEVJatVYFeXg@mail.gmail.com>
Message-ID: <CA+OGgf7nh5Zc1eKSE9YwF5Gz0iP7PJBi7swJczcUkbD7gKbA6w@mail.gmail.com>

On 11/20/12, Daniel Holth <dholth at gmail.com> wrote:
> On Tue, Nov 20, 2012 at 3:58 PM, Jim J. Jewett <jimjjewett at gmail.com>
> wrote:

>> Vinay Sajip reworded the 'Provides-Dist' definition to explicitly say:

>> > The use of multiple names in this field *must not* be used for
>> > bundling distributions together. It is intended for use when
>> > projects are forked and merged over time ...

>> (1)  Then how *should* the "bundle-of-several-components" case be
>> represented?

> The useful way to bundle a bunch of things would be to just include them
> all in an executable folder or zipfile with __main__.py. PEP 426 and the
> package database would not get involved. The bundle would be distributed as
> an application you can download and use, not as an sdist on PyPI.

When I look at, for example, twisted, there are some fairly fine distinctions.

I can imagine some people wanting to handle each little piece
differently, since that is the level at which they would be replaced
by a more efficient implementation.  That doesn't mean that someone
using the default should have to manage 47 separate little packages
individually.

Also note that ZODB is mentioned as a bundling example in the current
(2012-11-14) PEP.  What does the PEP recommend that they do?  Stop
including transaction?  Keep including it but stop 'Provides-Dist'-ing
it?

The current PEP also specifies that "This field must include the
project identified in the Name field, followed by the version : Name
(Version)." but the examples do not always include version.  Why is
the MUST there?

Is there some way to distinguish between concrete and abstract
provisions?  For example, if  MyMail (2012.11.10)  includes
'Provides-Dist: email', does that really get parsed as 'Provides-Dist:
email (2012.11.10)'?



>> (2)  How is 'Provides-Dist' different from 'Obsoletes-Dist'?
>> The only difference I can see is that it may be a bit more polite
>> to people who do want to install multiple versions of a (possibly
>> abstract) package.

> The intent of Provides and Obsoletes is different. Obsoletes would not
> satisfy a requirement during dependency resolution.

> The RPM guide explains a similar system:

As best I can understand, Obsoletes means "Go ahead and uninstall that
other package."
Saying that *without* providing the same functionality seems like a
sneaky spelling of "Please break whatever relies on that other
package."

I'm willing to believe that there is a more useful meaning.  I'm also
willing to believe that they are logically redundant but express
different intentions.  The current wording doesn't tell me which is
true.  (Admittedly, that is arguably an upstream bug with other
package systems, but you should still either fix it or explicitly
delegate the definitions.)

And as long as I'm asking for clarification, can foopkg-3.4 obsolete
foopgk3.2?  If not, is it a semantics problem, or just not idiomatic?
If so, does it have a precise meaning, such as "no longer
interoperates with"?

And now that I've looked more carefully ...

Can a "Key: Value" pair be continued onto another line?  The syntax
description under "Metadata Files" does not say so, but later text
suggests that either leading whitespace or a leading tab specifically
(from the example code) will work.  (And is description a special
case?)

Is the payload assumed to be utf8 text?  Can it be itself a mime message?

Are there any restrictions on 'Name'?  e.g., Can the name include
spaces?  line breaks?  Must it be a valid python identifier?  A valid
python qualname?

'Version' says that it must be in the format specified in PEP 386.
Unfortunately, it doesn't say which part of 386.  Do you mean that it
must be acceptable to verlib.NormalizedVersion without first having to
call suggest_normalized_version?

'Summary' specifies that it must be one line.  Is there a character
limit, or do you just mean "no line breaks"?  Do you want to add a
"Should be less than 80 characters" or some such, based on typical
tool presentation?  Would it be worth repeating the advice that longer
descriptions should go in the payload, after all headers?  (Otherwise,
they have to find 'Description' *and* notice that it is deprecated and
figure out what to do instead.)

Under 'Description', it isn't entirely clear whether what terminates
the field.  "Multiple paragraphs" suggests that there can be multiple
lines, but I'm guessing that -- in practice -- they have to be a
single logical line, with all but the first starting with whitespace.

Under 'Classifier', is PEP 301 really the current authority for
classifiers?  I would prefer at least a reference to
http://pypi.python.org/pypi?%3Aaction=list_classifiers demonstrating
which classifiers are currently meaningful.

Under 'Requires-Dist', there is an unclosed parenthesis.

Does the 'Setup-Requires-Dist' set implicitly include the
'Requires-Dist' set, or should a package be listed both ways if it is
required at both setup and runtime?

The Summary of Differences from PEP 345 mentions changes to
Requires-Dist, but I don't know what they were -- even the unclosed
parentheses seemed the same.

The appendix gives code for generating and parsing continuation lines
that suggests the continuation whitespace is exactly one tab -- is
other whitespace OK too?


-jJ

From pje at telecommunity.com  Wed Nov 21 03:44:54 2012
From: pje at telecommunity.com (PJ Eby)
Date: Tue, 20 Nov 2012 21:44:54 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CAG8k2+7DOcYxDo1vj_Fh5RE3Jbrmb3iQU_Tr_T59JaB5d_uNwA@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
	<C616B797714D4BB397ACAB9EC81B1C51@gmail.com>
	<CAG8k2+7DOcYxDo1vj_Fh5RE3Jbrmb3iQU_Tr_T59JaB5d_uNwA@mail.gmail.com>
Message-ID: <CALeMXf7JzAFqocJbjt_VUYnrn5KT_hKQ6iryOYFkv8ZUsVz-Jg@mail.gmail.com>

On Tue, Nov 20, 2012 at 5:01 PM, Daniel Holth <dholth at gmail.com> wrote:

> http://www.python.org/dev/peps/pep-0314/ says:
>
>       The most common use of this field will be in case a package name
>       changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0.
>       When you install Torqued Python, the Gorgon package should be
>       removed.
>
>       Example:
>
>           Obsoletes: Gorgon
>
>
> They mean pretty much what the same words mean in RPM and do not need
> further bikeshedding.
>

The problem is that the above *makes no sense*.  "Torqued Python" and
"Gorgon" are veiled pseudonyms for Twisted and Medusa....  and Twisted is
not actually a plug-and-play substitute for Medusa, AFAIK.

Can anybody suggest an *actual* use case for "Obsoletes", and explain how
it is supposed to work in software?  The last time this discussion came up,
nobody had any use cases that stood up to the "how's that actually going to
work and/or help?" test.  Here's a post of mine summarizing this and
related points in the previous thread:

  http://mail.python.org/pipermail/catalog-sig/2010-October/003364.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/d84d27cf/attachment.html>

From dholth at gmail.com  Wed Nov 21 04:01:48 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 20 Nov 2012 22:01:48 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CALeMXf7JzAFqocJbjt_VUYnrn5KT_hKQ6iryOYFkv8ZUsVz-Jg@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
	<C616B797714D4BB397ACAB9EC81B1C51@gmail.com>
	<CAG8k2+7DOcYxDo1vj_Fh5RE3Jbrmb3iQU_Tr_T59JaB5d_uNwA@mail.gmail.com>
	<CALeMXf7JzAFqocJbjt_VUYnrn5KT_hKQ6iryOYFkv8ZUsVz-Jg@mail.gmail.com>
Message-ID: <CAG8k2+5AayaYpWgLAS=qMOnmvJFYUdKm4tHWeTvKn_EGLfW2fQ@mail.gmail.com>

On Tue, Nov 20, 2012 at 9:44 PM, PJ Eby <pje at telecommunity.com> wrote:

> On Tue, Nov 20, 2012 at 5:01 PM, Daniel Holth <dholth at gmail.com> wrote:
>
> http://www.python.org/dev/peps/pep-0314/ says:
>>
>>       The most common use of this field will be in case a package name
>>       changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0.
>>       When you install Torqued Python, the Gorgon package should be
>>       removed.
>>
>>       Example:
>>
>>           Obsoletes: Gorgon
>>
>>
>> They mean pretty much what the same words mean in RPM and do not need
>> further bikeshedding.
>>
>
> The problem is that the above *makes no sense*.  "Torqued Python" and
> "Gorgon" are veiled pseudonyms for Twisted and Medusa....  and Twisted is
> not actually a plug-and-play substitute for Medusa, AFAIK.
>
> Can anybody suggest an *actual* use case for "Obsoletes", and explain how
> it is supposed to work in software?  The last time this discussion came up,
> nobody had any use cases that stood up to the "how's that actually going to
> work and/or help?" test.  Here's a post of mine summarizing this and
> related points in the previous thread:
>
>

Again I didn't write any of this.

Someone mentioned ZODB + transaction. The PEP should have used the word
"merged" instead of "bundled". When two packages become one, and the
redundant package is no longer being developed, Provides-Dist can be used.

I re-named a package once just because I did not like the name. I used
"Obsoletes" for that. It is documentation.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/19c6484d/attachment.html>

From pje at telecommunity.com  Wed Nov 21 04:10:58 2012
From: pje at telecommunity.com (PJ Eby)
Date: Tue, 20 Nov 2012 22:10:58 -0500
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <B3C5A7B3-CF5F-44C3-863D-A7B4402F06CE@gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
	<C616B797714D4BB397ACAB9EC81B1C51@gmail.com>
	<CAG8k2+7DOcYxDo1vj_Fh5RE3Jbrmb3iQU_Tr_T59JaB5d_uNwA@mail.gmail.com>
	<loom.20121120T231553-707@post.gmane.org>
	<B3C5A7B3-CF5F-44C3-863D-A7B4402F06CE@gmail.com>
Message-ID: <CALeMXf4m6_oFWXTdHffLmFdzkreSoCtrrvDxTTx0OtR3g7QaCg@mail.gmail.com>

On Tue, Nov 20, 2012 at 6:43 PM, Daniel Holth <dholth at gmail.com> wrote:

> No. We trust the packages we install, including the way they decide to use
> the metadata. A bad package could delete all our files or cause dependency
> resolution to fail. Mostly they won't.
>

That's sort of beside the point.  The *only* use case which Obsoletes
provides over Obsoleted-By is that it allows third parties to unilaterally
advertise their forked project as a substitute for the original, and maybe
block users from switching back to the un-forked project -- regardless of
the status of the original project or the consent of the original project's
maintainer.

This use case, however, benefits nobody besides the forkers.  There are
many other legitimate channels by which the forkers can advertise
themselves as a replacement for their parent project, and no reason for the
installing end user to be bothered with the subject, except in case of a
conflict.

For somebody obsoleting their own package, on the other hand, it's likely
well worth the effort to at least update their PyPI metadata to reflect the
change in status -- especially if this can be done through the web
interface.   It's likely they would wish to update their description as
well, to notify human beings of the change.

But here's the thing that kills "Obsoletes" dead in the first place as a
practical tool: unless installers use a PyPI search before installing
*every single project*, there is no way for them to realize that the
obsoleting package exists!

By contrast, if a package is "Obsoleted-By", then installing that package
(or declaring a dependency on it) provides an opportunity to inform the
user of the need to make a transition.  This can't be done with an
"Obsoletes" field.

Conversely, if you have already installed a package that says it
"Obsoletes" another package, this does *not* tell you that the obsolete
package shouldn't still be installed!  A replacement project doesn't
necessarily share the same API, and may exist in a different package
namespace altogether.

In short, "Obsoletes" is virtually *useless* as a machine-consumed metadata
field, because there is nothing you can actually do with it in a practical
installer.

I'm against adding more fields to the metadata which do not have a
specification for how they should be used in practice; the presence of such
fields has been a problem with most of the preceding metadata specs, IMO.


> I re-named a package once just because I did not like the name. I used
"Obsoletes" for that. It is documentation.

Note that "Obsoleted-By" would also serve that use case, and have the
additional benefit of being able to notify people who install new copies of
the replaced project.

(By the way Daniel, I'm sorry I didn't comment on this PEP sooner; I'd
forgotten about the previous PEP 345 rehashing in 2010, or rather, I just
sort of assumed that the results of that discussion had been incorporated
into the newer PEP, and didn't notice the reappearance of the noise fields
until your call for approval just now.  Sorry!)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/51defd4e/attachment-0001.html>

From ncoghlan at gmail.com  Wed Nov 21 04:12:18 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Nov 2012 13:12:18 +1000
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CALeMXf7JzAFqocJbjt_VUYnrn5KT_hKQ6iryOYFkv8ZUsVz-Jg@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
	<C616B797714D4BB397ACAB9EC81B1C51@gmail.com>
	<CAG8k2+7DOcYxDo1vj_Fh5RE3Jbrmb3iQU_Tr_T59JaB5d_uNwA@mail.gmail.com>
	<CALeMXf7JzAFqocJbjt_VUYnrn5KT_hKQ6iryOYFkv8ZUsVz-Jg@mail.gmail.com>
Message-ID: <CADiSq7d4SP09O6y72u5MN9_K0VHkEMYTB60XvYK_4P0A6xBGdA@mail.gmail.com>

On Wed, Nov 21, 2012 at 12:44 PM, PJ Eby <pje at telecommunity.com> wrote:

>
> Can anybody suggest an *actual* use case for "Obsoletes", and explain how
> it is supposed to work in software?  The last time this discussion came up,
> nobody had any use cases that stood up to the "how's that actually going to
> work and/or help?" test.  Here's a post of mine summarizing this and
> related points in the previous thread:
>
>   http://mail.python.org/pipermail/catalog-sig/2010-October/003364.html
> The problem is that the above *makes no sense*.  "Torqued Python" and
> "Gorgon" are veiled pseudonyms for Twisted and Medusa....  and Twisted is
> not actually a plug-and-play substitute for Medusa, AFAIK.
>

Sure. This is an RPM example, but exactly the same thing applies at the
Python level.

One of the dependencies of PulpDist (a directory mirroring tool I wrote),
is the Pulp project (originally just an RPM mirroring tool, but now with
plugin-based support for mirroring other things). The upstream version of
Pulp that I currently use is missing Kerberos login support, so I have
patched that in via RPMs patching features.

To avoid messing up others sharing the internal yum repo where this is
published, I actually use the Provides/Conflict/Obsoletes features of RPM
to make sure my patched and renamed copy and the upstream version don't
interfere with each other (and certainly can't be installed on the same
system, as they would trample all over each other by attempting to install
the same files).

Mostly though, these labelling tools are especially useful for internal
forks and mergers - the ones you *don't* share with the wider internet,
except perhaps in the form of upstream patches (For example:
https://bugzilla.redhat.com/show_bug.cgi?id=831937). On a public index,
drop-in replacements are *always* going to be controversial from a social
point of view, which is why there are only two current examples on PyPI I
am aware of (i.e. distribute vs setuptools and Pillow vs PIL). The first
was essentially a hostile fork, while the latter started as an attempt to
provide decent packaging support when the current maintainer didn't show
any interest in doing so. In such cases, it is absolutely essential that
the *forking* project is able to declare that it is a replacement for the
original project. It is then up to the community to decide whether or not
the claims of being a suitable replacement are valid, which will be shown
most clearly in relative uptake numbers between the original project and
the forked one.

I do consider it unfortunate that Python has only copied 3 of the 4 RPM
dependency management fields (i.e. only Provides, Requires, and Obsoletes,
without copying the more value neutral Conflicts) and I also prefer the
"capability" terminology in the Fedora RPM guide that makes it clear that
these are really arbitrary strings from a tooling point of view that only
match the package name by convention.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121121/5a18c4b9/attachment.html>

From dholth at gmail.com  Wed Nov 21 04:20:01 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 20 Nov 2012 22:20:01 -0500
Subject: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
In-Reply-To: <CA+OGgf7nh5Zc1eKSE9YwF5Gz0iP7PJBi7swJczcUkbD7gKbA6w@mail.gmail.com>
References: <loom.20121120T172255-830@post.gmane.org>
	<50abef0e.ab96320a.6675.ffffb439@mx.google.com>
	<CAG8k2+6Q1r+i__Ea29yawxBJztY1F7iGGKYj78TEVJatVYFeXg@mail.gmail.com>
	<CA+OGgf7nh5Zc1eKSE9YwF5Gz0iP7PJBi7swJczcUkbD7gKbA6w@mail.gmail.com>
Message-ID: <CAG8k2+60zucMTDVVDL0pW2thZh=rYEpMu_BFPDJG+MMd79xArg@mail.gmail.com>

On Tue, Nov 20, 2012 at 7:18 PM, Jim Jewett <jimjjewett at gmail.com> wrote:

> On 11/20/12, Daniel Holth <dholth at gmail.com> wrote:
> > On Tue, Nov 20, 2012 at 3:58 PM, Jim J. Jewett <jimjjewett at gmail.com>
> > wrote:
>
> >> Vinay Sajip reworded the 'Provides-Dist' definition to explicitly say:
>
> >> > The use of multiple names in this field *must not* be used for
> >> > bundling distributions together. It is intended for use when
> >> > projects are forked and merged over time ...
>
> >> (1)  Then how *should* the "bundle-of-several-components" case be
> >> represented?
>
> > The useful way to bundle a bunch of things would be to just include them
> > all in an executable folder or zipfile with __main__.py. PEP 426 and the
> > package database would not get involved. The bundle would be distributed
> as
> > an application you can download and use, not as an sdist on PyPI.
>
> When I look at, for example, twisted, there are some fairly fine
> distinctions.
>
> I can imagine some people wanting to handle each little piece
> differently, since that is the level at which they would be replaced
> by a more efficient implementation.  That doesn't mean that someone
> using the default should have to manage 47 separate little packages
> individually.
>
> Also note that ZODB is mentioned as a bundling example in the current
> (2012-11-14) PEP.  What does the PEP recommend that they do?  Stop
> including transaction?  Keep including it but stop 'Provides-Dist'-ing
> it?
>
> The current PEP also specifies that "This field must include the
> project identified in the Name field, followed by the version : Name
> (Version)." but the examples do not always include version.  Why is
> the MUST there?
>

ZODB is a bad example. The word "bundling" will be struck from the PEP
entirely. Two sdists should not be combined into one sdist when both
packages are still being developed.

If A and B are merged into a single PyPI package C, and A and B will no
longer be developed, then C may Provides-Dist A and B.

http://www.python.org/dev/peps/pep-0426/#requires-dist-multiple-use

No MUST on the Requires-Dist version. If no version is there, it should
satisfy any version requirement.

Is there some way to distinguish between concrete and abstract
> provisions?  For example, if  MyMail (2012.11.10)  includes
> 'Provides-Dist: email', does that really get parsed as 'Provides-Dist:
> email (2012.11.10)'?
>

No.

>> (2)  How is 'Provides-Dist' different from 'Obsoletes-Dist'?
>> The only difference I can see is that it may be a bit more polite
>> to people who do want to install multiple versions of a (possibly
>> abstract) package.

 > The intent of Provides and Obsoletes is different. Obsoletes would not
> > satisfy a requirement during dependency resolution.
>
> > The RPM guide explains a similar system:
>
> As best I can understand, Obsoletes means "Go ahead and uninstall that
> other package."
> Saying that *without* providing the same functionality seems like a
> sneaky spelling of "Please break whatever relies on that other
> package."
>
> I'm willing to believe that there is a more useful meaning.  I'm also
> willing to believe that they are logically redundant but express
> different intentions.  The current wording doesn't tell me which is
> true.  (Admittedly, that is arguably an upstream bug with other
> package systems, but you should still either fix it or explicitly
> delegate the definitions.)
>
> And as long as I'm asking for clarification, can foopkg-3.4 obsolete
> foopgk3.2?  If not, is it a semantics problem, or just not idiomatic?
> If so, does it have a precise meaning, such as "no longer
> interoperates with"?
>

When I used Obsoletes, it meant "I am no longer developing this other
package that is identical to this re-named package". The system of
requirements/conflicts (as the RPM system) does not appear to be entirely
orthogonal.

And now that I've looked more carefully ...
>
> Can a "Key: Value" pair be continued onto another line?  The syntax
> description under "Metadata Files" does not say so, but later text
> suggests that either leading whitespace or a leading tab specifically
> (from the example code) will work.  (And is description a special
> case?)
>

Description (now in the payload, please) is the only field that is commonly
multi-line.

Any field could continue onto the next line as far as the parser is
concerned. It probably would not make sense.

Is the payload assumed to be utf8 text?  Can it be itself a mime message?
>

The entire file needs to be utf-8. The payload is assumed to be utf-8 text
in this version. Wouldn't a mime message also be utf-8 text? (we wouldn't
know what to do with it)


> Are there any restrictions on 'Name'?  e.g., Can the name include
> spaces?  line breaks?  Must it be a valid python identifier?  A valid
> python qualname?
>

setuptools constrains it to alphanumeric characters. Metadata 1.3 doesn't
say.

'Version' says that it must be in the format specified in PEP 386.
> Unfortunately, it doesn't say which part of 386.  Do you mean that it
> must be acceptable to verlib.NormalizedVersion without first having to
> call suggest_normalized_version?
>

It means it is expected to match:

http://www.python.org/dev/peps/pep-0386/#the-new-versioning-algorithm

expr = r"""^
(?P<version>\d+\.\d+)         # minimum 'N.N'
(?P<extraversion>(?:\.\d+)*)  # any number of extra '.N' segments
(?:
    (?P<prerel>[abc]|rc)         # 'a' = alpha, 'b' = beta
                                 # 'c' or 'rc' = release candidate
    (?P<prerelversion>\d+(?:\.\d+)*)
)?
(?P<postdev>(\.post(?P<post>\d+))?(\.dev(?P<dev>\d+))?)?
$"""

Please do not enforce this regexp in your Metadata parser.



> 'Summary' specifies that it must be one line.  Is there a character
> limit, or do you just mean "no line breaks"?  Do you want to add a
> "Should be less than 80 characters" or some such, based on typical
> tool presentation?  Would it be worth repeating the advice that longer
> descriptions should go in the payload, after all headers?  (Otherwise,
> they have to find 'Description' *and* notice that it is deprecated and
> figure out what to do instead.)
>
> Under 'Description', it isn't entirely clear whether what terminates
> the field.  "Multiple paragraphs" suggests that there can be multiple
> lines, but I'm guessing that -- in practice -- they have to be a
> single logical line, with all but the first starting with whitespace.
>

Non-blank lines that don't start with whitespace are keys.
email.parser.Parser() takes care of this in an e-mail-inspired (but not any
literal RFC) way.

The distutils documentation has guidelines on the short description /
summary.


> Under 'Classifier', is PEP 301 really the current authority for
> classifiers?  I would prefer at least a reference to
> http://pypi.python.org/pypi?%3Aaction=list_classifiers demonstrating
> which classifiers are currently meaningful.
>
> Under 'Requires-Dist', there is an unclosed parenthesis.
>
> Does the 'Setup-Requires-Dist' set implicitly include the
> 'Requires-Dist' set, or should a package be listed both ways if it is
> required at both setup and runtime?
>
> The Summary of Differences from PEP 345 mentions changes to
> Requires-Dist, but I don't know what they were -- even the unclosed
> parentheses seemed the same.
>

The addition of the allowed "extra" variable in the ; condition is the most
significant change.


> The appendix gives code for generating and parsing continuation lines
> that suggests the continuation whitespace is exactly one tab -- is
> other whitespace OK too?
>

Any whitespace would work.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121120/2726fab8/attachment.html>

From ncoghlan at gmail.com  Wed Nov 21 04:20:41 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Nov 2012 13:20:41 +1000
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CALeMXf4m6_oFWXTdHffLmFdzkreSoCtrrvDxTTx0OtR3g7QaCg@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
	<C616B797714D4BB397ACAB9EC81B1C51@gmail.com>
	<CAG8k2+7DOcYxDo1vj_Fh5RE3Jbrmb3iQU_Tr_T59JaB5d_uNwA@mail.gmail.com>
	<loom.20121120T231553-707@post.gmane.org>
	<B3C5A7B3-CF5F-44C3-863D-A7B4402F06CE@gmail.com>
	<CALeMXf4m6_oFWXTdHffLmFdzkreSoCtrrvDxTTx0OtR3g7QaCg@mail.gmail.com>
Message-ID: <CADiSq7ehwkeu7AzVfR1Gy=m0BQQTkgHXs6yyHoTBCds11i5SZg@mail.gmail.com>

On Wed, Nov 21, 2012 at 1:10 PM, PJ Eby <pje at telecommunity.com> wrote:

> Conversely, if you have already installed a package that says it
> "Obsoletes" another package, this does *not* tell you that the obsolete
> package shouldn't still be installed!  A replacement project doesn't
> necessarily share the same API, and may exist in a different package
> namespace altogether.
>

Then that's a bug in the metadata of the project misusing "Obsoletes", and
should be reported as such. If the new package is not a drop-in
replacement, then it has no business claiming to obsolete the other package.

I think one of the big reasons this kind of use is rare in the Python
community is that project name changes are almost always accompanied by
*package* name changes, and as soon as you change the package name, you're
changing the public API, and thus it is no longer appropriate to use
Provides or Obsoletes, as the renamed project is no longer a drop-in
replacement for the original.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121121/5db6a035/attachment.html>

From sdl.web at gmail.com  Wed Nov 21 04:57:20 2012
From: sdl.web at gmail.com (Leo)
Date: Wed, 21 Nov 2012 11:57:20 +0800
Subject: [Python-Dev] operator.attrgetter(attr[, args...]) etc.
Message-ID: <m2ehjn4nhb.fsf@gmail.com>

Sorry the python issue tracker seems broken (I cannot log in). So I am
posting it here.

In the doc:

  operator.attrgetter(attr[, args...])
  operator.itemgetter(item[, args...])
  operator.methodcaller(name[, args...])

The signatures of these functions seem confusing. ARGS is not documented
and does not match the python implementation in the doc.

Leo


From ncoghlan at gmail.com  Wed Nov 21 05:01:24 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 21 Nov 2012 14:01:24 +1000
Subject: [Python-Dev] Accept just PEP-0426
In-Reply-To: <CADiSq7ehwkeu7AzVfR1Gy=m0BQQTkgHXs6yyHoTBCds11i5SZg@mail.gmail.com>
References: <CAG8k2+4SyyD+Ht73CDbYf54PEL79R1jpxSqwg+c56sGg_-4pNw@mail.gmail.com>
	<CALeMXf6ZWPuoQ0AJ7oEmngNVdKbRZch29UOneg+eHuzUWnBA9g@mail.gmail.com>
	<98BC16BE-3507-4D54-9AFC-8B1A0983B6F0@gmail.com>
	<4AB62E69A53048589648EA9A94CAC02A@gmail.com>
	<B7D9A3F9-29DF-4389-ADCF-50E8291EF6F7@gmail.com>
	<DBA510759A4D450A9A73C7E70EFE42CD@gmail.com>
	<CAG8k2+6F5G+=M5RUF8qpo-Kxyy2hKL0xumsOy7WXLttHS3LEjA@mail.gmail.com>
	<loom.20121120T150418-29@post.gmane.org>
	<CAG8k2+5ALzzCspea2NtjQW5KPqnzUOade33x0s_cPw1KxeUnWQ@mail.gmail.com>
	<loom.20121120T160434-5@post.gmane.org>
	<CADiSq7eNeh+AU2yyfE=QD1MVAsQkaOb6iFiitRNpgH1ukmtgew@mail.gmail.com>
	<loom.20121120T165425-38@post.gmane.org>
	<CAG8k2+7yB+h-wEK3abKiJq_vLN2uuHGKFwVh_YKCjnTSweD3sQ@mail.gmail.com>
	<loom.20121120T172255-830@post.gmane.org>
	<CALeMXf7EtrORHEq8GYQ0nXo52n5b2L0-KQRJAgJSiX101egiCw@mail.gmail.com>
	<50ABF128.4020605@g.nevcal.com>
	<CALeMXf6nK+0k091=BjmZuX+7LmPHrA1ibF5dru_P_k85fHwCDg@mail.gmail.com>
	<C616B797714D4BB397ACAB9EC81B1C51@gmail.com>
	<CAG8k2+7DOcYxDo1vj_Fh5RE3Jbrmb3iQU_Tr_T59JaB5d_uNwA@mail.gmail.com>
	<loom.20121120T231553-707@post.gmane.org>
	<B3C5A7B3-CF5F-44C3-863D-A7B4402F06CE@gmail.com>
	<CALeMXf4m6_oFWXTdHffLmFdzkreSoCtrrvDxTTx0OtR3g7QaCg@mail.gmail.com>
	<CADiSq7ehwkeu7AzVfR1Gy=m0BQQTkgHXs6yyHoTBCds11i5SZg@mail.gmail.com>
Message-ID: <CADiSq7epB_dkia-HzVjGXNk5p0_6PjE3Y6v1kW18+=SCZrW8Pw@mail.gmail.com>

On Wed, Nov 21, 2012 at 1:20 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Wed, Nov 21, 2012 at 1:10 PM, PJ Eby <pje at telecommunity.com> wrote:
>
>> Conversely, if you have already installed a package that says it
>> "Obsoletes" another package, this does *not* tell you that the obsolete
>> package shouldn't still be installed!  A replacement project doesn't
>> necessarily share the same API, and may exist in a different package
>> namespace altogether.
>>
>
> Then that's a bug in the metadata of the project misusing "Obsoletes", and
> should be reported as such. If the new package is not a drop-in
> replacement, then it has no business claiming to obsolete the other package.
>
> I think one of the big reasons this kind of use is rare in the Python
> community is that project name changes are almost always accompanied by
> *package* name changes, and as soon as you change the package name, you're
> changing the public API, and thus it is no longer appropriate to use
> Provides or Obsoletes, as the renamed project is no longer a drop-in
> replacement for the original.
>

I realised that my comments above are more about the appropriate use of
"Provides", rather than "Obsoletes". For a practically useful "Obsoletes",
I think I'm inclined to agree with you, as "Obsoleted-By" provides a way
for a maintainer to explicitly declare that a project is no longer
receiving updates, and users should migrate to the replacement project if
they want to continue to receive fixes and improvements. The current
version of "Obsoletes" is, as Daniel describes, really only useful as
documentation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121121/71924996/attachment.html>

From stephen at xemacs.org  Wed Nov 21 06:00:43 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 21 Nov 2012 14:00:43 +0900
Subject: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
In-Reply-To: <CAG8k2+60zucMTDVVDL0pW2thZh=rYEpMu_BFPDJG+MMd79xArg@mail.gmail.com>
References: <loom.20121120T172255-830@post.gmane.org>
	<50abef0e.ab96320a.6675.ffffb439@mx.google.com>
	<CAG8k2+6Q1r+i__Ea29yawxBJztY1F7iGGKYj78TEVJatVYFeXg@mail.gmail.com>
	<CA+OGgf7nh5Zc1eKSE9YwF5Gz0iP7PJBi7swJczcUkbD7gKbA6w@mail.gmail.com>
	<CAG8k2+60zucMTDVVDL0pW2thZh=rYEpMu_BFPDJG+MMd79xArg@mail.gmail.com>
Message-ID: <87pq375z44.fsf@uwakimon.sk.tsukuba.ac.jp>

Daniel Holth writes:

 > When I used Obsoletes, it meant "I am no longer developing this other
 > package that is identical to this re-named package".

But as a user I could care less!  The authors may care, but I don't
care if Torqued "obsoletes" Gorgon, because in using Torqued I'm
DTRT'ing even though I don't know it.

What I care about is when I'm using Gorgon, and there's something
"better" (or worse, "correct") to use in my application.

It might be a good idea to have a just-like-Amazon

    While-This-Package-Is-Great-You-Might-Also-Consider:

field.

Tongue-in-cheeki-ly y'rs,

From pje at telecommunity.com  Wed Nov 21 07:33:19 2012
From: pje at telecommunity.com (PJ Eby)
Date: Wed, 21 Nov 2012 01:33:19 -0500
Subject: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
In-Reply-To: <87pq375z44.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <loom.20121120T172255-830@post.gmane.org>
	<50abef0e.ab96320a.6675.ffffb439@mx.google.com>
	<CAG8k2+6Q1r+i__Ea29yawxBJztY1F7iGGKYj78TEVJatVYFeXg@mail.gmail.com>
	<CA+OGgf7nh5Zc1eKSE9YwF5Gz0iP7PJBi7swJczcUkbD7gKbA6w@mail.gmail.com>
	<CAG8k2+60zucMTDVVDL0pW2thZh=rYEpMu_BFPDJG+MMd79xArg@mail.gmail.com>
	<87pq375z44.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CALeMXf6cGSycaCwos5=Pk8M9qd22OqrxTcvkBFKLVC8axSHvCw@mail.gmail.com>

On Wed, Nov 21, 2012 at 12:00 AM, Stephen J. Turnbull <stephen at xemacs.org>wrote:

> Daniel Holth writes:
>
>  > When I used Obsoletes, it meant "I am no longer developing this other
>  > package that is identical to this re-named package".
>
> But as a user I could care less!  The authors may care, but I don't
> care if Torqued "obsoletes" Gorgon, because in using Torqued I'm
> DTRT'ing even though I don't know it.
>
> What I care about is when I'm using Gorgon, and there's something
> "better" (or worse, "correct") to use in my application.
>

Hence my suggestion for an Obsoleted-By field, in which Gorgon would be
able to suggest alternatives.



> It might be a good idea to have a just-like-Amazon
>
>     While-This-Package-Is-Great-You-Might-Also-Consider:
>
> field.
>

Yeah, that's basically what Obsoleted-By is for.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121121/183b4d96/attachment.html>

From stephen at xemacs.org  Wed Nov 21 08:54:17 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 21 Nov 2012 16:54:17 +0900
Subject: [Python-Dev] Keyword meanings [was: Accept just PEP-0426]
In-Reply-To: <CALeMXf6cGSycaCwos5=Pk8M9qd22OqrxTcvkBFKLVC8axSHvCw@mail.gmail.com>
References: <loom.20121120T172255-830@post.gmane.org>
	<50abef0e.ab96320a.6675.ffffb439@mx.google.com>
	<CAG8k2+6Q1r+i__Ea29yawxBJztY1F7iGGKYj78TEVJatVYFeXg@mail.gmail.com>
	<CA+OGgf7nh5Zc1eKSE9YwF5Gz0iP7PJBi7swJczcUkbD7gKbA6w@mail.gmail.com>
	<CAG8k2+60zucMTDVVDL0pW2thZh=rYEpMu_BFPDJG+MMd79xArg@mail.gmail.com>
	<87pq375z44.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CALeMXf6cGSycaCwos5=Pk8M9qd22OqrxTcvkBFKLVC8axSHvCw@mail.gmail.com>
Message-ID: <87obir5r2u.fsf@uwakimon.sk.tsukuba.ac.jp>

PJ Eby writes:
 > On Wed, Nov 21, 2012 at 12:00 AM, Stephen J. Turnbull <stephen at xemacs.org>wrote:

 > > What I care about is when I'm using Gorgon, and there's something
 > > "better" (or worse, "correct") to use in my application.
 > 
 > Hence my suggestion for an Obsoleted-By field, in which Gorgon would be
 > able to suggest alternatives.

My bad, my precise intention was to follow up on your idea (which,
credit where credit is due, I had *not* hit upon independently).  I
should have made that clear.

(I really shouldn't be answering English email at a Japanese-speaking
conference, my brain thinks it knows what it's doing but shirazuni ?
????????....)

 > > It might be a good idea to have a just-like-Amazon
 > >
 > >     While-This-Package-Is-Great-You-Might-Also-Consider:
 > >
 > > field.
 > 
 > Yeah, that's basically what Obsoleted-By is for.

Well, Obsoleted-By is pretty strong language for suggesting possible
alternatives.  But I suspect that few projects would really want to be
suggesting competitors' products *or* their own oldie-but-still-goodie
that they'd really like to obsolete ASAP (put an Obsoleted-By line in
every Python 2 distribution, anyone? :-)

From oscar.j.benjamin at gmail.com  Wed Nov 21 15:49:05 2012
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 21 Nov 2012 14:49:05 +0000
Subject: [Python-Dev] operator.attrgetter(attr[, args...]) etc.
In-Reply-To: <m2ehjn4nhb.fsf@gmail.com>
References: <m2ehjn4nhb.fsf@gmail.com>
Message-ID: <CAHVvXxSO1o_-5tzeQpccP9VbkYOuLExRmyf-_F-NmcPwVq4qSw@mail.gmail.com>

On 21 November 2012 03:57, Leo <sdl.web at gmail.com> wrote:
> Sorry the python issue tracker seems broken (I cannot log in). So I am
> posting it here.
>
> In the doc:
>
>   operator.attrgetter(attr[, args...])
>   operator.itemgetter(item[, args...])
>   operator.methodcaller(name[, args...])
>
> The signatures of these functions seem confusing. ARGS is not documented
> and does not match the python implementation in the doc.

What documentation are you reading?

$ python
Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import operator
>>> help(operator.attrgetter)
Help on class attrgetter in module operator:

class attrgetter(__builtin__.object)
 |  attrgetter(attr, ...) --> attrgetter object
 |
 |  Return a callable object that fetches the given attribute(s) from
its operand.
 |  After, f=attrgetter('name'), the call f(r) returns r.name.
 |  After, g=attrgetter('name', 'date'), the call g(r) returns (r.name, r.date).
 |  After, h=attrgetter('name.first', 'name.last'), the call h(r) returns
 |  (r.name.first, r.name.last).
[snip]

I think the above explains how attrgetter works with multiple
arguments. Here's a demo:

>>> class X:
...     a = 2
...     b = 3
...     c = 1
...
>>> getter = operator.attrgetter('a', 'b', 'c')
>>> getter(X)
(2, 3, 1)


Oscar

From rdmurray at bitdance.com  Wed Nov 21 16:16:55 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 21 Nov 2012 10:16:55 -0500
Subject: [Python-Dev] operator.attrgetter(attr[, args...]) etc.
In-Reply-To: <CAHVvXxSO1o_-5tzeQpccP9VbkYOuLExRmyf-_F-NmcPwVq4qSw@mail.gmail.com>
References: <m2ehjn4nhb.fsf@gmail.com>
	<CAHVvXxSO1o_-5tzeQpccP9VbkYOuLExRmyf-_F-NmcPwVq4qSw@mail.gmail.com>
Message-ID: <20121121151655.BE4F32500E1@webabinitio.net>

On Wed, 21 Nov 2012 14:49:05 +0000, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
> On 21 November 2012 03:57, Leo <sdl.web at gmail.com> wrote:
> > Sorry the python issue tracker seems broken (I cannot log in). So I am
> > posting it here.
> >
> > In the doc:
> >
> >   operator.attrgetter(attr[, args...])
> >   operator.itemgetter(item[, args...])
> >   operator.methodcaller(name[, args...])
> >
> > The signatures of these functions seem confusing. ARGS is not documented
> > and does not match the python implementation in the doc.
> 
> What documentation are you reading?

He's reading the operator module documentation.  I just ran into that
same thing the other day, but didn't think to file a bug report.  It
looks like the ``args`` stuff was incorrectly copy and pasted from the
methodcaller docs, where it is actually correct.

I've opened an issue:

    http://bugs.python.org/issue16523

--David

From brett at python.org  Wed Nov 21 17:33:40 2012
From: brett at python.org (Brett Cannon)
Date: Wed, 21 Nov 2012 11:33:40 -0500
Subject: [Python-Dev] [Python-checkins] cpython (3.3): - Issue #16514:
 Fix regression causing a traceback when sys.path[0] is None
In-Reply-To: <20121120184107.16713a99@limelight.wooz.org>
References: <3Y5dwB3ZCMzMlG@mail.python.org> <50AC05BD.3080402@udel.edu>
	<20121120184107.16713a99@limelight.wooz.org>
Message-ID: <CAP1=2W5Gj-q-UJb6Lp08a2ov4=fiHNPYoTkOy0Y_YCQ4e47N_w@mail.gmail.com>

On Tue, Nov 20, 2012 at 6:41 PM, Barry Warsaw <barry at python.org> wrote:

> On Nov 20, 2012, at 05:35 PM, Terry Reedy wrote:
>
> >On 11/20/2012 3:35 PM, barry.warsaw wrote:
> >
> >>           for entry in path:
> >> +            if not isinstance(entry, (str, bytes)):
> >> +                continue
> >
> >Given that a non-(str,bytes) entry could indicate a programming error,
> should
> >a warning be emitted before continuing?
>
> That's not what happens in Python 3.2.


Fine, but warnings are off by default and this is simply wrong behaviour to
do. We should not paper over it just because import.c did.


>  In fact, this bug report was triggered
> by someone who was already inserting None to sys.path[0].  It is silently
> ignored in Python 3.2, but tracebacked in 3.3.


But why were they doing that? What did they hope to get out of it?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121121/9c3a1dc1/attachment.html>

From chris.jerdonek at gmail.com  Thu Nov 22 03:12:49 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Wed, 21 Nov 2012 18:12:49 -0800
Subject: [Python-Dev] [Python-checkins] devguide: Add instructions for
 handling merge conflicts during null merges.
In-Reply-To: <3Y6PDh5mRgzMvQ@mail.python.org>
References: <3Y6PDh5mRgzMvQ@mail.python.org>
Message-ID: <CAOTb1wc+JqbaxMh7AoSyiNhKddfVsCrB2Q4DeiX=wsDiWx=rqw@mail.gmail.com>

On Wed, Nov 21, 2012 at 6:07 PM, chris.jerdonek
<python-checkins at python.org> wrote:
> http://hg.python.org/devguide/rev/78a69b929ab7
> changeset:   573:78a69b929ab7
> user:        Chris Jerdonek <chris.jerdonek at gmail.com>
> date:        Wed Nov 21 18:04:35 2012 -0800
> summary:
>   Add instructions for handling merge conflicts during null merges.

This was for issue #16517:

http://bugs.python.org/issue16517

--Chris


>
> files:
>   committing.rst |  11 +++++++++--
>   1 files changed, 9 insertions(+), 2 deletions(-)
>
>
> diff --git a/committing.rst b/committing.rst
> --- a/committing.rst
> +++ b/committing.rst
> @@ -306,17 +306,24 @@
>     # Fix any conflicts; compile; run the test suite
>     hg commit
>
> +.. index:: null merging
> +
>  .. note::
> -   *If the patch shouldn't be ported* from Python 3.3 to Python 3.4, you must
> -   also make it explicit: merge the changes but revert them before committing::
> +   If the patch should *not* be ported from Python 3.3 to Python 3.4, you must
> +   also make this explicit by doing a *null merge*: merge the changes but
> +   revert them before committing::
>
>        hg update default
>        hg merge 3.3
>        hg revert -ar default
> +      hg resolve -am  # needed only if the merge created conflicts
>        hg commit
>
>     This is necessary so that the merge gets recorded; otherwise, somebody
>     else will have to make a decision about your patch when they try to merge.
> +   (Using a three-way merge tool generally makes the ``hg resolve`` step
> +   in the above unnecessary; also see `this bug report
> +   <http://bz.selenic.com/show_bug.cgi?id=2706>`__.)
>
>  When you have finished your porting work (you can port several patches one
>  after another in your local repository), you can push **all** outstanding
>
> --
> Repository URL: http://hg.python.org/devguide
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From kristjan at ccpgames.com  Thu Nov 22 12:44:26 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Thu, 22 Nov 2012 11:44:26 +0000
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <CAG8k2+6+rRwyTndvf87gGrSGH9jh0h=5L8k+m_nHQyA59DxEJg@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE2A53@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7ccgu9aMCXUqmSfmRCr3_ZJY5-A8hwemwYTFzUMXHdKzw@mail.gmail.com>
	<50AB8975.3020405@stackless.com>
	<CADiSq7c=E867L7_1PT4dCPVh1UO+kc3xwoXmK3tXbeXBxBApJQ@mail.gmail.com>
	<CAG8k2+6+rRwyTndvf87gGrSGH9jh0h=5L8k+m_nHQyA59DxEJg@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD329DE76CF@RKV-IT-EXCH103.ccp.ad.local>

Where in the tracker?  I tried searching but didn't find it.

I contributed to the pep405 discussions with similar concerns back in march:
http://mail.python.org/pipermail/python-dev/2012-March/117894.html


From: Python-Dev [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Daniel Holth


There is some activity in the tracker about adding the missing "add this to PYTHONPATH" / "isolate self from the environment" command line arguments to Python.

Daniel Holth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121122/407890da/attachment.html>

From ncoghlan at gmail.com  Thu Nov 22 13:09:10 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 22 Nov 2012 22:09:10 +1000
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DE76CF@RKV-IT-EXCH103.ccp.ad.local>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE2A53@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7ccgu9aMCXUqmSfmRCr3_ZJY5-A8hwemwYTFzUMXHdKzw@mail.gmail.com>
	<50AB8975.3020405@stackless.com>
	<CADiSq7c=E867L7_1PT4dCPVh1UO+kc3xwoXmK3tXbeXBxBApJQ@mail.gmail.com>
	<CAG8k2+6+rRwyTndvf87gGrSGH9jh0h=5L8k+m_nHQyA59DxEJg@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE76CF@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <CADiSq7e0PhtkozwD0-4EP=hRs4V9M7GK8PoaWqz1moRvEeKJ6w@mail.gmail.com>

On Thu, Nov 22, 2012 at 9:44 PM, Kristj?n Valur J?nsson <
kristjan at ccpgames.com> wrote:

>  Where in the tracker?  I tried searching but didn?t find it.
>

This one: http://bugs.python.org/issue13475

This and the issue about being able to configure coverage.py cleanly in
subprocesses (http://bugs.python.org/issue14803) are the two issues which
have got me thinking about the interpreter startup and configuration
process in general lately. Both would add *more* complexity to an already
complicated and hard to follow initialisation sequence, so I now think we
should be look at opportunities for rationalisation *first*, before we make
things even messier.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121122/b00a17de/attachment.html>

From status at bugs.python.org  Fri Nov 23 18:07:15 2012
From: status at bugs.python.org (Python tracker)
Date: Fri, 23 Nov 2012 18:07:15 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20121123170715.8D2531CAB2@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-11-16 - 2012-11-23)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3799 ( -2)
  closed 24518 (+54)
  total  28317 (+52)

Open issues with patches: 1656 


Issues opened (37)
==================

#8627: Unchecked PyErr_WarnPy3k return value in Objects/typeobject.c
http://bugs.python.org/issue8627  reopened by mark.dickinson

#16306: Multiple error line for unknown command line parameter
http://bugs.python.org/issue16306  reopened by pitrou

#16490: "inspect.getargspec()" and "inspect.getcallargs()" don't work 
http://bugs.python.org/issue16490  opened by jcea

#16491: try-except-raise-bug
http://bugs.python.org/issue16491  opened by Jocjo

#16492: Add a load_parents argument to importlib.find_loader()
http://bugs.python.org/issue16492  opened by brett.cannon

#16494: Add a method on importlib.SourceLoader for creating bytecode f
http://bugs.python.org/issue16494  opened by brett.cannon

#16495: bytes_decode() unnecessarily examines encoding
http://bugs.python.org/issue16495  opened by chris.jerdonek

#16496: Simplify and optimize random_seed()
http://bugs.python.org/issue16496  opened by serhiy.storchaka

#16499: CLI option for isolated mode
http://bugs.python.org/issue16499  opened by christian.heimes

#16500: Add an 'afterfork' module
http://bugs.python.org/issue16500  opened by christian.heimes

#16501: deprecate RISCOS "support"
http://bugs.python.org/issue16501  opened by pitrou

#16504: IDLE - fatal error when opening a file with certain tokenizing
http://bugs.python.org/issue16504  opened by serwy

#16507: Patch selectmodule.c to support WSAPoll on Windows
http://bugs.python.org/issue16507  opened by trent

#16508: include the "object" type in the lists of documented types
http://bugs.python.org/issue16508  opened by chris.jerdonek

#16509: sqlite3 docs do not explain check_same_thread
http://bugs.python.org/issue16509  opened by strcat

#16510: Using appropriate checks in tests
http://bugs.python.org/issue16510  opened by serhiy.storchaka

#16511: IDLE configuration file: blank height and width fields trip up
http://bugs.python.org/issue16511  opened by doanviettrung

#16512: imghdr doesn't support jpegs with an ICC profile
http://bugs.python.org/issue16512  opened by joril

#16515: TypeError message incorrect for max() with one keyword arg
http://bugs.python.org/issue16515  opened by chris.jerdonek

#16516: argparse types (and actions) must be hashable
http://bugs.python.org/issue16516  opened by jnothman

#16518: add "buffer protocol" to glossary
http://bugs.python.org/issue16518  opened by chris.jerdonek

#16519: site.venv() should use abspath(executable)
http://bugs.python.org/issue16519  opened by christian.heimes

#16520: subprocess.Popen() TypeError message incorrect without args ar
http://bugs.python.org/issue16520  opened by chris.jerdonek

#16521: logging.basicConfig creates empty file when using handlers
http://bugs.python.org/issue16521  opened by Jovik

#16523: attrgetter and itemgetter signatures in docs need cleanup
http://bugs.python.org/issue16523  opened by r.david.murray

#16524: File access not always working with Python for Windows 32 bits
http://bugs.python.org/issue16524  opened by Antoine.Brodin

#16525: wave file module does not support 32bit float format
http://bugs.python.org/issue16525  opened by Sebastian.Kraft

#16526: Python does not cross compile properly
http://bugs.python.org/issue16526  opened by Lothsahn

#16529: Compiler error when trying to compile ceval.c on OpenSUSE 11.3
http://bugs.python.org/issue16529  opened by lemburg

#16530: documentation of os.wait3
http://bugs.python.org/issue16530  opened by quiver

#16531: Allow IPNetwork to take a tuple
http://bugs.python.org/issue16531  opened by pitrou

#16532: AMD64 Windows 7 build failures
http://bugs.python.org/issue16532  opened by pitrou

#16533: HPUX: Unable to fork() in thread
http://bugs.python.org/issue16533  opened by skrah

#16534: test_float failure on IA64 (HPUX)
http://bugs.python.org/issue16534  opened by skrah

#16535: json encoder unable to handle decimal
http://bugs.python.org/issue16535  opened by eric.araujo

#16537: setup.py throws a ValueError when self.extensions is empty
http://bugs.python.org/issue16537  opened by jhosmer

#16540: Make itertools count, cycle, and repeat objects subscriptable 
http://bugs.python.org/issue16540  opened by neil.g



Most recent 15 issues with no replies (15)
==========================================

#16533: HPUX: Unable to fork() in thread
http://bugs.python.org/issue16533

#16530: documentation of os.wait3
http://bugs.python.org/issue16530

#16523: attrgetter and itemgetter signatures in docs need cleanup
http://bugs.python.org/issue16523

#16521: logging.basicConfig creates empty file when using handlers
http://bugs.python.org/issue16521

#16519: site.venv() should use abspath(executable)
http://bugs.python.org/issue16519

#16518: add "buffer protocol" to glossary
http://bugs.python.org/issue16518

#16516: argparse types (and actions) must be hashable
http://bugs.python.org/issue16516

#16509: sqlite3 docs do not explain check_same_thread
http://bugs.python.org/issue16509

#16508: include the "object" type in the lists of documented types
http://bugs.python.org/issue16508

#16494: Add a method on importlib.SourceLoader for creating bytecode f
http://bugs.python.org/issue16494

#16492: Add a load_parents argument to importlib.find_loader()
http://bugs.python.org/issue16492

#16486: Add context manager support to aifc module
http://bugs.python.org/issue16486

#16472: Distutils+mingw links agains msvcr90, while python27.dll is li
http://bugs.python.org/issue16472

#16463: test_timeout failure on the RHEL buildbot
http://bugs.python.org/issue16463

#16450: test_missing_localfile masks problems in urlopen
http://bugs.python.org/issue16450



Most recent 15 issues waiting for review (15)
=============================================

#16537: setup.py throws a ValueError when self.extensions is empty
http://bugs.python.org/issue16537

#16515: TypeError message incorrect for max() with one keyword arg
http://bugs.python.org/issue16515

#16512: imghdr doesn't support jpegs with an ICC profile
http://bugs.python.org/issue16512

#16511: IDLE configuration file: blank height and width fields trip up
http://bugs.python.org/issue16511

#16510: Using appropriate checks in tests
http://bugs.python.org/issue16510

#16507: Patch selectmodule.c to support WSAPoll on Windows
http://bugs.python.org/issue16507

#16504: IDLE - fatal error when opening a file with certain tokenizing
http://bugs.python.org/issue16504

#16500: Add an 'afterfork' module
http://bugs.python.org/issue16500

#16499: CLI option for isolated mode
http://bugs.python.org/issue16499

#16496: Simplify and optimize random_seed()
http://bugs.python.org/issue16496

#16495: bytes_decode() unnecessarily examines encoding
http://bugs.python.org/issue16495

#16488: Add context manager support to epoll object
http://bugs.python.org/issue16488

#16487: Allow ssl certificates to be speficfied from memory rather tha
http://bugs.python.org/issue16487

#16486: Add context manager support to aifc module
http://bugs.python.org/issue16486

#16485: FD leaks in aifc module
http://bugs.python.org/issue16485



Top 10 most discussed issues (10)
=================================

#16475: Support object instancing and recursion in marshal
http://bugs.python.org/issue16475  25 msgs

#16496: Simplify and optimize random_seed()
http://bugs.python.org/issue16496  18 msgs

#15031: Split .pyc parsing from module loading
http://bugs.python.org/issue15031  13 msgs

#16500: Add an 'afterfork' module
http://bugs.python.org/issue16500  13 msgs

#16306: Multiple error line for unknown command line parameter
http://bugs.python.org/issue16306  10 msgs

#16499: CLI option for isolated mode
http://bugs.python.org/issue16499  10 msgs

#16480: pyvenv 3.3 fails to create symlinks for <virtualenv>/local/{bi
http://bugs.python.org/issue16480   9 msgs

#16511: IDLE configuration file: blank height and width fields trip up
http://bugs.python.org/issue16511   8 msgs

#12848: pickle.py treats 32bit lengths as signed, but _pickle.c as uns
http://bugs.python.org/issue12848   7 msgs

#8627: Unchecked PyErr_WarnPy3k return value in Objects/typeobject.c
http://bugs.python.org/issue8627   6 msgs



Issues closed (44)
==================

#1160: Medium size regexp crashes python
http://bugs.python.org/issue1160  closed by pitrou

#3580: failures in test_os
http://bugs.python.org/issue3580  closed by pitrou

#4182: warnings.warn shows the wrong filename and line number for sta
http://bugs.python.org/issue4182  closed by brett.cannon

#6308: termios fix for QNX breaks HP-UX
http://bugs.python.org/issue6308  closed by skrah

#6526: importlib.import_module affects permissions of .pyc files subs
http://bugs.python.org/issue6526  closed by brett.cannon

#7782: new test for test_iter.py
http://bugs.python.org/issue7782  closed by ezio.melotti

#9761: stale .pyc files aren't cleaned out
http://bugs.python.org/issue9761  closed by brett.cannon

#10966: eliminate use of ImportError implicitly representing SkipTest
http://bugs.python.org/issue10966  closed by brett.cannon

#11679: readline interferes with characters beginning with byte \xe9
http://bugs.python.org/issue11679  closed by takluyver

#11981: dupe self.fp.tell() in zipfile.ZipFile.writestr
http://bugs.python.org/issue11981  closed by ezio.melotti

#12005: modulo result of Decimal differs from float/int
http://bugs.python.org/issue12005  closed by mark.dickinson

#12614: Allow to explicitly set the method of urllib.request.Request
http://bugs.python.org/issue12614  closed by orsenthil

#13538: Improve doc for str(bytesobject)
http://bugs.python.org/issue13538  closed by chris.jerdonek

#14313: zipfile should raise an exception for unsupported compression 
http://bugs.python.org/issue14313  closed by ezio.melotti

#14525: ia64-hp-hpux11.31 won't compile 2.7 without -D_TERMIOS_INCLUDE
http://bugs.python.org/issue14525  closed by skrah

#14631: Instance methods and WeakRefs don't mix.
http://bugs.python.org/issue14631  closed by pitrou

#15627: Add a method to importlib.abc.SourceLoader for converting sour
http://bugs.python.org/issue15627  closed by brett.cannon

#16053: "strict" parameter is not documented in csv module
http://bugs.python.org/issue16053  closed by ezio.melotti

#16157: Irrelevant references to Misc/News
http://bugs.python.org/issue16157  closed by ezio.melotti

#16213: Expose private functions in marshal used by importlib
http://bugs.python.org/issue16213  closed by brett.cannon

#16215: Possible double memory free in str.replace
http://bugs.python.org/issue16215  closed by pitrou

#16240: Document a way to escape metacharacters in glob/fnmatch
http://bugs.python.org/issue16240  closed by ezio.melotti

#16408: FD leaks in zipfile
http://bugs.python.org/issue16408  closed by pitrou

#16451: Remove duplication between slice_indices and compute_slice_ind
http://bugs.python.org/issue16451  closed by mark.dickinson

#16461: wave module: wrong integer format
http://bugs.python.org/issue16461  closed by python-dev

#16470: Backport set and dictionary comprehensions in tutorial to 2.7
http://bugs.python.org/issue16470  closed by ezio.melotti

#16489: importlib find_loader returns a loader for a non existent modu
http://bugs.python.org/issue16489  closed by brett.cannon

#16493: Document the 'optimize' argument to compile()
http://bugs.python.org/issue16493  closed by ezio.melotti

#16497: zipimport.zipimporter.find_module() does not work with dotted 
http://bugs.python.org/issue16497  closed by brett.cannon

#16498: Unwanted link between volatile and shelve storage
http://bugs.python.org/issue16498  closed by serhiy.storchaka

#16502: PEP 305: eaten backslashes
http://bugs.python.org/issue16502  closed by ezio.melotti

#16503: Unclear documentation regarding apply(), 'extended call syntax
http://bugs.python.org/issue16503  closed by ezio.melotti

#16505: Drop Py_TPFLAGS_INT_SUBCLASS
http://bugs.python.org/issue16505  closed by python-dev

#16506: devguide should have table of contents
http://bugs.python.org/issue16506  closed by chris.jerdonek

#16513: SGMLParser processing <tr> which include two <a> will have pro
http://bugs.python.org/issue16513  closed by ezio.melotti

#16514: Cryptic traceback when sys.path[0] is None
http://bugs.python.org/issue16514  closed by barry

#16517: address merge conflicts in devguide null-merge instructions
http://bugs.python.org/issue16517  closed by chris.jerdonek

#16522: Add 'FAIL_FAST' flag to doctest
http://bugs.python.org/issue16522  closed by r.david.murray

#16527: (very) long list of elif causes segfault
http://bugs.python.org/issue16527  closed by benjamin.peterson

#16528: 3.2 docs not updating on docs.python.org
http://bugs.python.org/issue16528  closed by r.david.murray

#16536: test_cmd_line: failure on Ubuntu Shared
http://bugs.python.org/issue16536  closed by ezio.melotti

#16538: The docs doesn't describe MAKE_CLOSURE correctly
http://bugs.python.org/issue16538  closed by asvetlov

#667770: import C API mess
http://bugs.python.org/issue667770  closed by brett.cannon

#16539: Turn off 'No handlers could be found for logger' message
http://bugs.python.org/issue16539  closed by vinay.sajip

From MatatTHC at gmx.de  Sun Nov 25 13:14:11 2012
From: MatatTHC at gmx.de (Matthias Bernt)
Date: Sun, 25 Nov 2012 13:14:11 +0100
Subject: [Python-Dev] logging and rotation
Message-ID: <CAFwXb28UYBwWtd5RDJcOSg5pypLyNGU+LtbATkSHp4wz0zniqg@mail.gmail.com>

Dear mailing list,

I'm using the logging module and write my log messages via the FileHandler.
I just realized that using an external log rotation mechanism does not
work. That is, new messages are not added to the file after
rotation.

In my opinion external log rotate mechanisms should work with the standard
file handler.

websearch pointed me to TimedRotatingFileHandler . But this only seems like
a workaround. For instance I would like to get my log file mailed once a
week. But it seems difficult to sync the cron job doing this and the
logging-rotation mechanism.

I think TimedRotatingFileHandler is important, e.g. for windows users. But
for *nux users this seems to be unnecessary.

Just want to gather some opinions before filing a bug.

Regards,
Matthias
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121125/00953511/attachment.html>

From phd at phdru.name  Sun Nov 25 18:02:12 2012
From: phd at phdru.name (Oleg Broytman)
Date: Sun, 25 Nov 2012 21:02:12 +0400
Subject: [Python-Dev] logging and rotation
In-Reply-To: <CAFwXb28UYBwWtd5RDJcOSg5pypLyNGU+LtbATkSHp4wz0zniqg@mail.gmail.com>
References: <CAFwXb28UYBwWtd5RDJcOSg5pypLyNGU+LtbATkSHp4wz0zniqg@mail.gmail.com>
Message-ID: <20121125170212.GA29778@iskra.aviel.ru>

On Sun, Nov 25, 2012 at 01:14:11PM +0100, Matthias Bernt <MatatTHC at gmx.de> wrote:
> I'm using the logging module and write my log messages via the FileHandler.
> I just realized that using an external log rotation mechanism does not
> work. That is, new messages are not added to the file after
> rotation.

   An external log rotation mechanism ought to send a signal to the
application and the application ought to close and reopen logs. That is,
this is an application-level problem, not logging module-level.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From python-dev at masklinn.net  Sun Nov 25 18:34:07 2012
From: python-dev at masklinn.net (Xavier Morel)
Date: Sun, 25 Nov 2012 18:34:07 +0100
Subject: [Python-Dev] logging and rotation
In-Reply-To: <20121125170212.GA29778@iskra.aviel.ru>
References: <CAFwXb28UYBwWtd5RDJcOSg5pypLyNGU+LtbATkSHp4wz0zniqg@mail.gmail.com>
	<20121125170212.GA29778@iskra.aviel.ru>
Message-ID: <836590DB-1933-4CB5-825F-B9731EAA45CC@masklinn.net>

On 2012-11-25, at 18:02 , Oleg Broytman wrote:
> On Sun, Nov 25, 2012 at 01:14:11PM +0100, Matthias Bernt <MatatTHC at gmx.de> wrote:
>> I'm using the logging module and write my log messages via the FileHandler.
>> I just realized that using an external log rotation mechanism does not
>> work. That is, new messages are not added to the file after
>> rotation.
> 
>   An external log rotation mechanism ought to send a signal to the
> application and the application ought to close and reopen logs. That is,
> this is an application-level problem, not logging module-level.

I don't know that FileHandler officially supports reopening its
underlying file. On the other hand, WatchedFileHandler[0] does exactly
that and is specifically advertised for use with external log rotators:

> WatchedFileHandler [?] watches the file it is logging to. If the file
> changes, it is closed and reopened using the file name.

> A file change can happen because of usage of programs such as newsyslog
> and logrotate which perform log file rotation. [?] If the file has changed,
> the old file stream is closed, and the file opened to get a new stream.

[0] http://docs.python.org/2/library/logging.handlers.html#watchedfilehandler

From guido at python.org  Sun Nov 25 18:33:58 2012
From: guido at python.org (Guido van Rossum)
Date: Sun, 25 Nov 2012 09:33:58 -0800
Subject: [Python-Dev] logging and rotation
In-Reply-To: <20121125170212.GA29778@iskra.aviel.ru>
References: <CAFwXb28UYBwWtd5RDJcOSg5pypLyNGU+LtbATkSHp4wz0zniqg@mail.gmail.com>
	<20121125170212.GA29778@iskra.aviel.ru>
Message-ID: <CAP7+vJLE2hFD3P1BhfXLH1iJuGqbLvewFt5cSETiU-aXayWDAg@mail.gmail.com>

On Sun, Nov 25, 2012 at 9:02 AM, Oleg Broytman <phd at phdru.name> wrote:
> On Sun, Nov 25, 2012 at 01:14:11PM +0100, Matthias Bernt <MatatTHC at gmx.de> wrote:
>> I'm using the logging module and write my log messages via the FileHandler.
>> I just realized that using an external log rotation mechanism does not
>> work. That is, new messages are not added to the file after
>> rotation.
>
>    An external log rotation mechanism ought to send a signal to the
> application and the application ought to close and reopen logs. That is,
> this is an application-level problem, not logging module-level.

I'm not so sure. Logging ought to "just work" without the application
having to do much about it the details. Log rotation is definitely a
useful feature; I've run into this very issue before and never found a
satisfactory solution. Surely it should be possible for the file
handler to occasionally stat() its output file by name and compare
that to fstat() for the file descriptor and if they differ, reopen?

-- 
--Guido van Rossum (python.org/~guido)

From chris.jerdonek at gmail.com  Mon Nov 26 06:01:18 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sun, 25 Nov 2012 21:01:18 -0800
Subject: [Python-Dev] type vs. class terminology
Message-ID: <CAOTb1wdDvta+DFBeGHaOtU4Ftsq2xLZrfrL2i-Sdk22sZoYReQ@mail.gmail.com>

I would like to know when we should use "class" in the Python 3
documentation, and when we should use "type."  Are these terms
synonymous in Python 3, and do we have a preference for which to use
and when?

I'm sure this has been discussed before.  But if this terminology
issue has already been resolved, the resolution doesn't seem to be
reflected in the docs.  For example, the glossary entries for type and
class don't reference each other.

Thoughts?

--Chris

From ncoghlan at gmail.com  Mon Nov 26 07:54:20 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 26 Nov 2012 16:54:20 +1000
Subject: [Python-Dev] type vs. class terminology
In-Reply-To: <CAOTb1wdDvta+DFBeGHaOtU4Ftsq2xLZrfrL2i-Sdk22sZoYReQ@mail.gmail.com>
References: <CAOTb1wdDvta+DFBeGHaOtU4Ftsq2xLZrfrL2i-Sdk22sZoYReQ@mail.gmail.com>
Message-ID: <CADiSq7e-gkVrrtegLuzfR5nn33fbhxvZYW+E1V_KMGeOs+7xuA@mail.gmail.com>

On Mon, Nov 26, 2012 at 3:01 PM, Chris Jerdonek <chris.jerdonek at gmail.com>wrote:

> I would like to know when we should use "class" in the Python 3
> documentation, and when we should use "type."  Are these terms
> synonymous in Python 3, and do we have a preference for which to use
> and when?
>
> I'm sure this has been discussed before.  But if this terminology
> issue has already been resolved, the resolution doesn't seem to be
> reflected in the docs.  For example, the glossary entries for type and
> class don't reference each other.
>

The historical distinction between "builtin types" and "user-defined
classes" predates new-style classes (which unified the type system) and
Python 3 (which eliminated the "instance" type that was provided to
preserve the legacy user-defined class semantics in Python 2). The glossary
unfortunately still reflects this distinction, which no longer exists in
Python 3.

A slightly more useful distinction would be if type was used consistently
to refer to type(x), while class was used to refer to x.__class__, since
they can and do differ in the case of proxy types (like weakref.proxy).
However, it's probably too late for that kind of fine distinction - in
reality, the two terms are now used pretty much interchangeably.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121126/7ee8f491/attachment.html>

From python-dev at masklinn.net  Mon Nov 26 08:50:52 2012
From: python-dev at masklinn.net (Xavier Morel)
Date: Mon, 26 Nov 2012 08:50:52 +0100
Subject: [Python-Dev] type vs. class terminology
In-Reply-To: <CADiSq7e-gkVrrtegLuzfR5nn33fbhxvZYW+E1V_KMGeOs+7xuA@mail.gmail.com>
References: <CAOTb1wdDvta+DFBeGHaOtU4Ftsq2xLZrfrL2i-Sdk22sZoYReQ@mail.gmail.com>
	<CADiSq7e-gkVrrtegLuzfR5nn33fbhxvZYW+E1V_KMGeOs+7xuA@mail.gmail.com>
Message-ID: <C917503C-DA5C-4010-B527-4E01853267D7@masklinn.net>


On 2012-11-26, at 07:54 , Nick Coghlan wrote:

> On Mon, Nov 26, 2012 at 3:01 PM, Chris Jerdonek <chris.jerdonek at gmail.com>wrote:
> 
>> I would like to know when we should use "class" in the Python 3
>> documentation, and when we should use "type."  Are these terms
>> synonymous in Python 3, and do we have a preference for which to use
>> and when?
>> 
>> I'm sure this has been discussed before.  But if this terminology
>> issue has already been resolved, the resolution doesn't seem to be
>> reflected in the docs.  For example, the glossary entries for type and
>> class don't reference each other.
>> 
> 
> The historical distinction between "builtin types" and "user-defined
> classes" predates new-style classes (which unified the type system) and
> Python 3 (which eliminated the "instance" type that was provided to
> preserve the legacy user-defined class semantics in Python 2). The glossary
> unfortunately still reflects this distinction, which no longer exists in
> Python 3.
> 
> A slightly more useful distinction would be if type was used consistently
> to refer to type(x), while class was used to refer to x.__class__, since
> they can and do differ in the case of proxy types (like weakref.proxy).
> However, it's probably too late for that kind of fine distinction - in
> reality, the two terms are now used pretty much interchangeably.

There's an other possible usage which is between `type` subclasses and
`type` instances (`type` essentially becomes a synonym for `metaclass`),
I've seen that kind of usage at least once in the docs:
http://docs.python.org/2/c-api/object.html#PyObject_IsInstance

> If cls is a type object rather than a class object, PyObject_IsInstance()
> returns 1 if inst is of type cls.

From hrvoje.niksic at avl.com  Mon Nov 26 09:04:21 2012
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Mon, 26 Nov 2012 09:04:21 +0100
Subject: [Python-Dev] type vs. class terminology
In-Reply-To: <CAOTb1wdDvta+DFBeGHaOtU4Ftsq2xLZrfrL2i-Sdk22sZoYReQ@mail.gmail.com>
References: <CAOTb1wdDvta+DFBeGHaOtU4Ftsq2xLZrfrL2i-Sdk22sZoYReQ@mail.gmail.com>
Message-ID: <50B32285.2010806@avl.com>

On 11/26/2012 06:01 AM, Chris Jerdonek wrote:
> I would like to know when we should use "class" in the Python 3
> documentation, and when we should use "type."  Are these terms
> synonymous in Python 3, and do we have a preference for which to use
> and when?

Some people like to use "class" for the subset of types created by 
Python's "class" statement or its moral equivalent (explicit invocation 
of the metaclass). It makes sense that "class" is used to create 
classes. The word "type" then refers to both classes and built-in and 
extension types, such as "list" or "array.array".

From kristjan at ccpgames.com  Mon Nov 26 12:49:39 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Mon, 26 Nov 2012 11:49:39 +0000
Subject: [Python-Dev] Socket timeout and completion based sockets
Message-ID: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>

Regarding the recent discussion on python-ideas about asyncronous IO, I'd like to ask a question about python socket's Timeout feature.
Specifically this:  Is it a documented or a guaranteed feature that a send/receive operation that times out with a socket.timeout error is re-startable on that socket?

The reason I ask is that depending on the implementation, a timeout may leave a socket in an undefined state.
As it is implemented in the standard cPython implementation, the timeout feature is done with an internal select() call.  Thus, if the select() call times out, socket send/receive is not even called, so a retry is possible without issue.
However, other implementations of python sockets, e.g. ones that rely on IO completion, may not have the luxury of using select.  For example, on Windows, there is no way to abort an IOCP socket call, so a timeout must be implemented by aborting the wait.  Dealing with the resulting race can be an interesting challenge.

K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121126/d2a661ba/attachment.html>

From guido at python.org  Mon Nov 26 16:52:41 2012
From: guido at python.org (Guido van Rossum)
Date: Mon, 26 Nov 2012 07:52:41 -0800
Subject: [Python-Dev] type vs. class terminology
In-Reply-To: <50B32285.2010806@avl.com>
References: <CAOTb1wdDvta+DFBeGHaOtU4Ftsq2xLZrfrL2i-Sdk22sZoYReQ@mail.gmail.com>
	<50B32285.2010806@avl.com>
Message-ID: <CAP7+vJKgOui+xC_fGenAzGNXrpEs5ufqJ8xGtHjAkfatjnqMLA@mail.gmail.com>

Hm. None of the distinctions brought up so far really hit true with me
(though they all are part of the picture). For example, I think the
distinction between type(x) and x.__class__ is rarely significant -- I
bet that if anyone were to rely on this they'd first have to change a
lot of code that used one or the other without much thinking about the
difference. So we might as well consider these equivalent.

Part of the distinction is probably just in historical usage -- many
idioms hark back to when there *was* a difference.

If I had to invent an artificial difference, I'd say that I use "type"
when talking about the type as a fairly abstract concept, such as the
type of a variable (which may be required to be in a given set, for
example), whereas I'll say "class" when I'm interested in the class as
an object, or its definition (through a class statement). So, if I'm
going to ask for the name, the phrase "the name of the class" rolls
easier off my tongue than "the name of the type". OTOH if I'm going to
just assert set membership, I might slightly prefer "x's type must be
int or str".

It's clear that we ought to at least cross-link the two glossary
entries and point out that they are the same concept (type(x) vs.
x.__class__) notwithstanding. I don't think it's important to try and
eradicate the word type from our conversation or our docs -- so no
sweeping global changes, please. However, if you come across a use of
either one in the docs that seems odd given modern usage, feel free to
clean it up, perhaps after checking with someone else to make sure
your intuition matches that of others.

-- 
--Guido van Rossum (python.org/~guido)

From guido at python.org  Mon Nov 26 16:59:08 2012
From: guido at python.org (Guido van Rossum)
Date: Mon, 26 Nov 2012 07:59:08 -0800
Subject: [Python-Dev] Socket timeout and completion based sockets
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
References: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <CAP7+vJJWDEwAvKB1VVrz-RX6b9kO3TpxnwUGnoq57746MV1WFg@mail.gmail.com>

If you're talking about the standard socket module, I'm not aware that
it uses IOCP on Windows. Are you asking this just in the abstract, or
do you know of a Python implementation that uses IOCP to implement the
standard socket type?

As to the design of the async I/O library (which I am still working
on!), I cannot guarantee anything, and the issue will probably be moot
-- the API won't have the same kind of timeout as the current socket
object (it will have other ways to set deadlines though).

Confusedly yours,

--Guido

On Mon, Nov 26, 2012 at 3:49 AM, Kristj?n Valur J?nsson
<kristjan at ccpgames.com> wrote:
> Regarding the recent discussion on python-ideas about asyncronous IO, I?d
> like to ask a question about python socket?s Timeout feature.
>
> Specifically this:  Is it a documented or a guaranteed feature that a
> send/receive operation that times out with a socket.timeout error is
> re-startable on that socket?
>
>
>
> The reason I ask is that depending on the implementation, a timeout may
> leave a socket in an undefined state.
>
> As it is implemented in the standard cPython implementation, the timeout
> feature is done with an internal select() call.  Thus, if the select() call
> times out, socket send/receive is not even called, so a retry is possible
> without issue.
>
> However, other implementations of python sockets, e.g. ones that rely on IO
> completion, may not have the luxury of using select.  For example, on
> Windows, there is no way to abort an IOCP socket call, so a timeout must be
> implemented by aborting the wait.  Dealing with the resulting race can be an
> interesting challenge.
>
>
>
> K
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)

From shibturn at gmail.com  Mon Nov 26 17:05:04 2012
From: shibturn at gmail.com (Richard Oudkerk)
Date: Mon, 26 Nov 2012 16:05:04 +0000
Subject: [Python-Dev] Socket timeout and completion based sockets
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
References: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <k903vo$2v4$1@ger.gmane.org>

On 26/11/2012 11:49am, Kristj?n Valur J?nsson wrote:
> However, other implementations of python sockets, e.g. ones that rely on
> IO completion, may not have the luxury of using select.  For example, on
> Windows, there is no way to abort an IOCP socket call, so a timeout must
> be implemented by aborting the wait.  Dealing with the resulting race
> can be an interesting challenge.

I am not quite sure what you mean by "aborting the wait".  But you can 
abort an overlapped operation using CancelIo()/CancelIoEx().

I have just done some experimenting.

Using CancelIo()/CancelIoEx() to abort an operation started with 
WSARecv() does not seem to cause a problem -- you just call 
GetOverlappedResult() afterwards to check whether the operation 
completed before it could be aborted.

But aborting an operation started with WSASend() sometimes seems to 
"break" the connection: a subsequent WSARecv()/WSASend() will fail with 
WSAECONNABORTED or WSAECONNRESET depending on which end of the 
connection you are on.

So, as you say, if you abort a send then you cannot expect to 
successfully resend the data later.

-- 
Richard


From glyph at twistedmatrix.com  Mon Nov 26 20:23:55 2012
From: glyph at twistedmatrix.com (Glyph)
Date: Mon, 26 Nov 2012 14:23:55 -0500
Subject: [Python-Dev] Socket timeout and completion based sockets
In-Reply-To: <k903vo$2v4$1@ger.gmane.org>
References: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
	<k903vo$2v4$1@ger.gmane.org>
Message-ID: <50A9C7B9-10FE-4CDE-A348-5C5669B0E569@twistedmatrix.com>

On Nov 26, 2012, at 11:05 AM, Richard Oudkerk <shibturn at gmail.com> wrote:

> Using CancelIo()/CancelIoEx() to abort an operation started with WSARecv() does not seem to cause a problem

(emphasis mine)

Little command-line experiments are not the right way to verify the behavior of high-performance I/O APIs.  You need to do careful reading of the documentation, significant testing under load and experiments on a huge variety of platforms.  Windows runs in _lots_ of horrible, horrible places.

I think that the safest option would really be to better document the somewhat mangled state that a timeout may leave a socket in.  I don't believe the feature was intended for pipelined protocols; if you receive a timeout by using the stdlib timeout functionality you have generally burned the socket.

And, generally, things that care enough about performance or scalability enough to use IOCP operations should never use timeout-sockets anyway; it may do a select() internally (and on Windows, where poll() is never available, it _will_ do a select() internally), which limits the number of file descriptors you might even have in your process before you start encountering spurious errors.  The use case it supports is when you have a little tool that just needs to fetch a URL or something really simple, but wants to be able to get on with things if that doesn't work or takes too long.

-glyph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121126/fd3daa8d/attachment.html>

From kristjan at ccpgames.com  Tue Nov 27 10:35:15 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 27 Nov 2012 09:35:15 +0000
Subject: [Python-Dev] Socket timeout and completion based sockets
In-Reply-To: <k903vo$2v4$1@ger.gmane.org>
References: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
	<k903vo$2v4$1@ger.gmane.org>
Message-ID: <EFE3877620384242A686D52278B7CCD329DEBF5F@RKV-IT-EXCH103.ccp.ad.local>

This is getting off-topic, but:
CancelIoEx() is a new api that I wasn't aware of (my IOCP solution dates back to 2005).  It appears that this can be used, although the documentation is sketchy.
This worries me:
"If the file handle is associated with a completion port, an I/O completion packet is not queued to the port if a synchronous operation is successfully canceled. For asynchronous operations still pending, the cancel operation will queue an I/O completion packet."
This _might_ mean that synchronizing the cancel request with a callback that expects to be called, but then, may not be called, can be difficult.  It is possible that MS didn't think their API completely through (nothing new here.) 

Anyway, that is somewhat beside the point

Even if we can cancel an ongoing operation there needs to be synchronization, so that any data that is received(or sent) is correctly communicated ot the app.  If there isn't a cancel option in the API (not talking about windows here) then this would mean queueing up data for recv(), and no simple solution for send().

So, basically, what I'm saying is, that enabling re-startable socket timeout semantics for sockets implemented with "completion" semantics, rather than "ready" semantics _can_ be difficult, hence my question.

> -----Original Message-----
> From: Python-Dev [mailto:python-dev-
> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Richard
> Oudkerk
> Sent: 26. n?vember 2012 16:05
> To: python-dev at python.org
> Subject: Re: [Python-Dev] Socket timeout and completion based sockets
> 
> On 26/11/2012 11:49am, Kristj?n Valur J?nsson wrote:
> > However, other implementations of python sockets, e.g. ones that rely
> > on IO completion, may not have the luxury of using select.  For
> > example, on Windows, there is no way to abort an IOCP socket call, so
> > a timeout must be implemented by aborting the wait.  Dealing with the
> > resulting race can be an interesting challenge.
> 
> I am not quite sure what you mean by "aborting the wait".  But you can abort
> an overlapped operation using CancelIo()/CancelIoEx().
> 
> I have just done some experimenting.
> 
> Using CancelIo()/CancelIoEx() to abort an operation started with
> WSARecv() does not seem to cause a problem -- you just call
> GetOverlappedResult() afterwards to check whether the operation
> completed before it could be aborted.
> 
> But aborting an operation started with WSASend() sometimes seems to
> "break" the connection: a subsequent WSARecv()/WSASend() will fail with
> WSAECONNABORTED or WSAECONNRESET depending on which end of the
> connection you are on.
> 
> So, as you say, if you abort a send then you cannot expect to successfully
> resend the data later.
> 
> --
> Richard
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-
> dev/kristjan%40ccpgames.com



From kristjan at ccpgames.com  Tue Nov 27 10:23:02 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 27 Nov 2012 09:23:02 +0000
Subject: [Python-Dev] Socket timeout and completion based sockets
In-Reply-To: <CAP7+vJJWDEwAvKB1VVrz-RX6b9kO3TpxnwUGnoq57746MV1WFg@mail.gmail.com>
References: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJJWDEwAvKB1VVrz-RX6b9kO3TpxnwUGnoq57746MV1WFg@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD329DEBF4B@RKV-IT-EXCH103.ccp.ad.local>

Yes, well, as a matter of fact, I do have an IOCP based socket implementation with stackless python.
>From the programmer's perspective, operations appear blocking while IOCP is used to switch tasklets in the background.
But my socket timeout implementation does not guarantee that the socket is left in a valid state for retrying a recv() operation.
I was under the (perhaps mistaken) impression that the current async work _could_ result in a standardized way to create such
alternative socket implementations, ones that might do their magic using greenlets, tasklets, or generators.  But if that were
the case, such loosely defined features of the socket API would need clearer definitions.

K
> -----Original Message-----
> From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf
> Of Guido van Rossum
> Sent: 26. n?vember 2012 15:59
> To: Kristj?n Valur J?nsson
> Cc: Python-Dev (python-dev at python.org)
> Subject: Re: [Python-Dev] Socket timeout and completion based sockets
> 
> If you're talking about the standard socket module, I'm not aware that it uses
> IOCP on Windows. Are you asking this just in the abstract, or do you know of
> a Python implementation that uses IOCP to implement the standard socket
> type?
> 
> As to the design of the async I/O library (which I am still working on!), I
> cannot guarantee anything, and the issue will probably be moot
> -- the API won't have the same kind of timeout as the current socket object
> (it will have other ways to set deadlines though).



From trent at snakebite.org  Tue Nov 27 12:30:11 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 27 Nov 2012 06:30:11 -0500
Subject: [Python-Dev] Tru64 support
In-Reply-To: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
Message-ID: <20121127113011.GF90314@snakebite.org>

On Tue, Sep 04, 2012 at 02:24:10AM -0700, Andrew Svetlov wrote:
> Is it still up to date? Bug has been created in 2004.
> I don't see Tru64 in list of available buildbots.
> 
> Do we need to care about this platform? And how to make sure what
> existing code works fine for that?

    There's a Snakebite Tru64 box now, accessible via t5 from the sb
    menu.  2.7 compiles fine but hangs on some of the subprocess epipe
    tests, I'll set up a slave once I've addressed that.

    One nice thing about Tru64 is that everything is implicitly 64-bit.
    There's no notion of dual 32-bit/64-bit libs/binaries or a 32-bit
    memory architecture.  The 64-bit `python` on Tru64 is in a much
    better state than said binaries on AIX and HP-UX (which still need
    a lot of work).

    As for VMS/OpenVMS, still working with HP to get access to media.

        Trent.

From shibturn at gmail.com  Tue Nov 27 13:33:10 2012
From: shibturn at gmail.com (Richard Oudkerk)
Date: Tue, 27 Nov 2012 12:33:10 +0000
Subject: [Python-Dev] Socket timeout and completion based sockets
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DEBF5F@RKV-IT-EXCH103.ccp.ad.local>
References: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
	<k903vo$2v4$1@ger.gmane.org>
	<EFE3877620384242A686D52278B7CCD329DEBF5F@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <k92buf$r8u$1@ger.gmane.org>

On 27/11/2012 9:35am, Kristj?n Valur J?nsson wrote:
> This worries me:
> "If the file handle is associated with a completion port, an I/O completion
 > packet is not queued to the port if a synchronous operation is 
successfully canceled...

I think you can only abort a synchronous operation if you use 
CancelIoEx() from a different thread.  That won't be an issue if you do 
all your IO stuff in one thread.

-- 
Richard


From guido at python.org  Tue Nov 27 16:54:19 2012
From: guido at python.org (Guido van Rossum)
Date: Tue, 27 Nov 2012 07:54:19 -0800
Subject: [Python-Dev] Socket timeout and completion based sockets
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DEBF4B@RKV-IT-EXCH103.ccp.ad.local>
References: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJJWDEwAvKB1VVrz-RX6b9kO3TpxnwUGnoq57746MV1WFg@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DEBF4B@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <CAP7+vJKpeEDV9ZpvJqN2_joOt4cKj9B1BJ5gYeZusnwcnu_VwQ@mail.gmail.com>

On Tue, Nov 27, 2012 at 1:23 AM, Kristj?n Valur J?nsson
<kristjan at ccpgames.com> wrote:
> Yes, well, as a matter of fact, I do have an IOCP based socket implementation with stackless python.

It would have been nice if you had given more context and stated your
objective upfront instead of asking what appeared to be an obscure
question about a technical detail. I have been distracted by other
stuff temporarily, but I do plan to continue working on tulip more,
and I want to assure you that EVERYTHING IS STILL OPEN. I.e. don't
take the current tulip code base as a draft PEP -- it is just my way
of learning about the issues. Also note that Richard Oudkerk has
developed an alternative, which you can find in the "proactor" branch
of the tulip repository.

> From the programmer's perspective, operations appear blocking while IOCP is used to switch tasklets in the background.
> But my socket timeout implementation does not guarantee that the socket is left in a valid state for retrying a recv() operation.
> I was under the (perhaps mistaken) impression that the current async work _could_ result in a standardized way to create such
> alternative socket implementations, ones that might do their magic using greenlets, tasklets, or generators.  But if that were
> the case, such loosely defined features of the socket API would need clearer definitions.

If you have a specific requirement, please state it explicitly, rather
than just hoping tulip will be your solution. FWIW, it is likely that
tulip will hide the actual socket object completely inside a
higher-level abstraction, so that the actual semantics of sockets
(which are extremely platform-dependent) cannot affect the
specification of the API. There will be platform-specific ways to
reach inside, but they will be of limited use -- it's possible that on
Windows (at least when using IOCP) there won't be a socket object at
all.

Finally, I am not at all interested in greenlets(*). Tulip will
support two API surfaces: a low-level one, suitable for interfacing
with e.g. Twisted or Tornado, that uses callbacks to signal ready-ness
or completion. And a high-level one, useful for e.g. writing protocol
handling libraries and applications, that will somehow use yield
and/or yield-from. The details of each layer are very much unspecified
at this point. NOW WOULD BE A GOOD TIME TO WRITE DOWN YOUR
REQUIREMENTS.

(*) Greenlets are a fine mechanism for some application areas, but
ultimately not fit for the standard library, and they have some
significant downsides.

--Guido

> K
>> -----Original Message-----
>> From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf
>> Of Guido van Rossum
>> Sent: 26. n?vember 2012 15:59
>> To: Kristj?n Valur J?nsson
>> Cc: Python-Dev (python-dev at python.org)
>> Subject: Re: [Python-Dev] Socket timeout and completion based sockets
>>
>> If you're talking about the standard socket module, I'm not aware that it uses
>> IOCP on Windows. Are you asking this just in the abstract, or do you know of
>> a Python implementation that uses IOCP to implement the standard socket
>> type?
>>
>> As to the design of the async I/O library (which I am still working on!), I
>> cannot guarantee anything, and the issue will probably be moot
>> -- the API won't have the same kind of timeout as the current socket object
>> (it will have other ways to set deadlines though).
>
>



-- 
--Guido van Rossum (python.org/~guido)

From brian at python.org  Tue Nov 27 21:41:13 2012
From: brian at python.org (Brian Curtin)
Date: Tue, 27 Nov 2012 14:41:13 -0600
Subject: [Python-Dev] New Contributor Experience in Python and other
 FOSS Communities - A Survey
In-Reply-To: <CAD+XWwoVUSaP1C4qsMbn05ROApkA5fAo2Fr=vXssQy=iAcJSzg@mail.gmail.com>
References: <CAD+XWwoVUSaP1C4qsMbn05ROApkA5fAo2Fr=vXssQy=iAcJSzg@mail.gmail.com>
Message-ID: <CAD+XWwryF-9=9Y2BdeTEmcg4r8+kmbBFkFu0BAL9H3AZ1V0gMA@mail.gmail.com>

On Mon, Nov 19, 2012 at 9:01 AM, Brian Curtin <brian at python.org> wrote:
> Hi all,
>
> Along with a number of other free and open communities, Python is
> being included in a survey of new contributors since January 2010. The
> survey is being done by Kevin Carillo, a PhD student at Victoria
> University of Wellington who is studying various approaches that
> projects use to gain and work with new contributors.
>
> If you have written patches, reviewed issues, or done anything to
> contribute to the development of Python and you started this after
> January 2010, I hope you can spare around 20 minutes to complete this
> survey: https://limesurvey.sim.vuw.ac.nz/index.php?sid=65151&lang=en
>
> There's a longer blog post up at
> http://blog.python.org/2012/11/new-contributor-experience-in-python.html
> if you would like a bit more information.
>
> On behalf of Kevin Carillo, I thank you for your time and
> consideration of this survey.
>
> Brian Curtin

Just a quick reminder - Kevin could really use the help if any of you
recent newcomers can spare the time. I took the survey myself since I
became a committer right after the survey range begins, and I think it
took me 15 minutes.

From trent at snakebite.org  Tue Nov 27 23:49:28 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 27 Nov 2012 17:49:28 -0500
Subject: [Python-Dev] Handling support for newer OS features at run time
Message-ID: <20121127224928.GK91191@snakebite.org>

    The hackiest part of my WSAPoll patch is this:

--- a/PC/pyconfig.h     Sun Nov 18 10:42:42 2012 +0000
+++ b/PC/pyconfig.h     Sun Nov 18 17:27:29 2012 -0500
@@ -158,12 +158,12 @@
 /* set the version macros for the windows headers */
 #ifdef MS_WINX64
 /* 64 bit only runs on XP or greater */
-#define Py_WINVER 0x0501 /* _WIN32_WINNT_WINXP */
-#define Py_NTDDI NTDDI_WINXP
+#define Py_WINVER 0x0600 /* _WIN32_WINNT_WINXP */
+#define Py_NTDDI NTDDI_WIN6
 #else
 /* Python 2.6+ requires Windows 2000 or greater */
-#define Py_WINVER 0x0500 /* _WIN32_WINNT_WIN2K */
-#define Py_NTDDI NTDDI_WIN2KSP4
+#define Py_WINVER 0x0600 /* _WIN32_WINNT_WIN2K */
+#define Py_NTDDI NTDDI_WIN6
 #endif

    Basically, because of the way our build is currently configured, I
    had to bump the minimum supported Windows version from XP to Vista
    just to get access to the WSAPoll headers.

        (Issue: http://bugs.python.org/issue16507
         Patch: http://bugs.python.org/file28038/wsapoll.patch)

    Ideally, a Windows binary should make WSAPoll/select.poll()
    available if running on Vista or above, without impacting
    the ability to run on XP.

    I don't think we've currently got the ability to do this, right?
    Is there a precedent set anywhere else?  I suspect it's not as
    much of an issue on *NIX platforms as you'll typically compile
    from source.  Windows, not so much.

    Thoughts?  A single binary that dynamically loads applicable
    modules seems cleaner but will obviously take more work.  Other
    approach would be to start distributing multiple versions of
    Python based on the underlying Windows version.  Not the nicest
    approach.

        Trent.

From doko at ubuntu.com  Wed Nov 28 00:09:12 2012
From: doko at ubuntu.com (Matthias Klose)
Date: Wed, 28 Nov 2012 00:09:12 +0100
Subject: [Python-Dev] Handling support for newer OS features at run time
In-Reply-To: <20121127224928.GK91191@snakebite.org>
References: <20121127224928.GK91191@snakebite.org>
Message-ID: <50B54818.2090907@ubuntu.com>

Am 27.11.2012 23:49, schrieb Trent Nelson:
>     I don't think we've currently got the ability to do this, right?
>     Is there a precedent set anywhere else?  I suspect it's not as
>     much of an issue on *NIX platforms as you'll typically compile
>     from source.  Windows, not so much.
> 
>     Thoughts?  A single binary that dynamically loads applicable
>     modules seems cleaner but will obviously take more work.  Other
>     approach would be to start distributing multiple versions of
>     Python based on the underlying Windows version.  Not the nicest
>     approach.

Usually I have to build a python package on a build daemon running the kernel of
the latest stable (or latest stable long term support) release.  Thus if a
configure test checks the current kernel, then you may get an unexpected answer
and a missing feature. It is somehow different that I already build different
binaries (per release), but the hard-coding of kernel features during configure
time looks like the same issue. Currently working around it by patching
configure to remove the test and hard-code it for a particular build. Another
solution maybe would to have something like --enable-kernel=<version> (as found
in glibc), and hope that you can deduce the features from the kernel version.


From shibturn at gmail.com  Wed Nov 28 00:14:00 2012
From: shibturn at gmail.com (Richard Oudkerk)
Date: Tue, 27 Nov 2012 23:14:00 +0000
Subject: [Python-Dev] Handling support for newer OS features at run time
In-Reply-To: <20121127224928.GK91191@snakebite.org>
References: <20121127224928.GK91191@snakebite.org>
Message-ID: <k93hg3$2gh$1@ger.gmane.org>

On 27/11/2012 10:49pm, Trent Nelson wrote:
>      Ideally, a Windows binary should make WSAPoll/select.poll()
>      available if running on Vista or above, without impacting
>      the ability to run on XP.

I assume you can do something like

     int WSAAPI (*pWSAPoll)(WSAPOLLFD *, ULONG, INT);
     HINSTANCE hWinsock2 = GetModuleHandle("WS2_32");
     *(FARPROC *)&pWSAPoll = GetProcAddress(hWinsock2, "WSAPoll");
     if (pWSAPoll == NULL)
         ...

to get a function pointer for WSAPoll on versions which support it.

Modules/_winapi.c does something similar to get CancelIoEx.

-- 
Richard


From trent at snakebite.org  Wed Nov 28 00:19:19 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 27 Nov 2012 18:19:19 -0500
Subject: [Python-Dev] Handling support for newer OS features at run time
In-Reply-To: <50B54818.2090907@ubuntu.com>
References: <20121127224928.GK91191@snakebite.org>
	<50B54818.2090907@ubuntu.com>
Message-ID: <20121127231919.GM91191@snakebite.org>

On Tue, Nov 27, 2012 at 03:09:12PM -0800, Matthias Klose wrote:
> Am 27.11.2012 23:49, schrieb Trent Nelson:
> >     I don't think we've currently got the ability to do this, right?
> >     Is there a precedent set anywhere else?  I suspect it's not as
> >     much of an issue on *NIX platforms as you'll typically compile
> >     from source.  Windows, not so much.
> > 
> >     Thoughts?  A single binary that dynamically loads applicable
> >     modules seems cleaner but will obviously take more work.  Other
> >     approach would be to start distributing multiple versions of
> >     Python based on the underlying Windows version.  Not the nicest
> >     approach.
> 
> Usually I have to build a python package on a build daemon running the
> kernel of the latest stable (or latest stable long term support)
> release.  Thus if a configure test checks the current kernel, then you
> may get an unexpected answer and a missing feature. It is somehow
> different that I already build different binaries (per release), but
> the hard-coding of kernel features during configure time looks like
> the same issue. Currently working around it by patching configure to
> remove the test and hard-code it for a particular build. Another
> solution maybe would to have something like --enable-kernel=<version>
> (as found in glibc), and hope that you can deduce the features from
> the kernel version.

    Hmmm.  How often do Linux kernel versions expose new features that
    we can make use of in Python?  i.e. do you run into this problem
    often?  Can you cite some recent examples?

    I'm not sure how much could be shared between Windows and Linux with
    what you've just described.  With Windows, specifically with regards
    to providing dynamic select.poll() support, I was thinking of having
    multiple modules:

        Python33\
            DLLs\
                select.pyd
                select_win6.pyd
                select_win7.pyd
                select_win8.pyd

    And Python would automatically import the appropriate one.  I don't
    think this would be that useful on Linux -- as you say, the decision
    is typically made at ./configure time.

        Trent.

From trent at snakebite.org  Wed Nov 28 00:30:28 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 27 Nov 2012 18:30:28 -0500
Subject: [Python-Dev] Handling support for newer OS features at run time
In-Reply-To: <k93hg3$2gh$1@ger.gmane.org>
References: <20121127224928.GK91191@snakebite.org> <k93hg3$2gh$1@ger.gmane.org>
Message-ID: <20121127233027.GN91191@snakebite.org>

On Tue, Nov 27, 2012 at 03:14:00PM -0800, Richard Oudkerk wrote:
> On 27/11/2012 10:49pm, Trent Nelson wrote:
> >      Ideally, a Windows binary should make WSAPoll/select.poll()
> >      available if running on Vista or above, without impacting
> >      the ability to run on XP.
> 
> I assume you can do something like
> 
>      int WSAAPI (*pWSAPoll)(WSAPOLLFD *, ULONG, INT);
>      HINSTANCE hWinsock2 = GetModuleHandle("WS2_32");
>      *(FARPROC *)&pWSAPoll = GetProcAddress(hWinsock2, "WSAPoll");
>      if (pWSAPoll == NULL)
>          ...
> 
> to get a function pointer for WSAPoll on versions which support it.

    Ah!  Good point, that's probably a much better approach in this
    instance, given I only need one or two more symbols.  Thanks!

        Trent.

From ericsnowcurrently at gmail.com  Wed Nov 28 05:19:28 2012
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Tue, 27 Nov 2012 21:19:28 -0700
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <CADiSq7e0PhtkozwD0-4EP=hRs4V9M7GK8PoaWqz1moRvEeKJ6w@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE2A53@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7ccgu9aMCXUqmSfmRCr3_ZJY5-A8hwemwYTFzUMXHdKzw@mail.gmail.com>
	<50AB8975.3020405@stackless.com>
	<CADiSq7c=E867L7_1PT4dCPVh1UO+kc3xwoXmK3tXbeXBxBApJQ@mail.gmail.com>
	<CAG8k2+6+rRwyTndvf87gGrSGH9jh0h=5L8k+m_nHQyA59DxEJg@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE76CF@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7e0PhtkozwD0-4EP=hRs4V9M7GK8PoaWqz1moRvEeKJ6w@mail.gmail.com>
Message-ID: <CALFfu7BNqM5scr6WZQXfNToFwqup7E_ZLqfK_F_P4BC2dCO25A@mail.gmail.com>

On Thu, Nov 22, 2012 at 5:09 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Thu, Nov 22, 2012 at 9:44 PM, Kristj?n Valur J?nsson
> <kristjan at ccpgames.com> wrote:
>>
>> Where in the tracker?  I tried searching but didn?t find it.
>
>
> This one: http://bugs.python.org/issue13475
>
> This and the issue about being able to configure coverage.py cleanly in
> subprocesses (http://bugs.python.org/issue14803) are the two issues which
> have got me thinking about the interpreter startup and configuration process
> in general lately. Both would add *more* complexity to an already
> complicated and hard to follow initialisation sequence, so I now think we
> should be look at opportunities for rationalisation *first*, before we make
> things even messier.

There's also http://bugs.python.org/issue15716 ("Ability to specify
the PYTHONPATH via a command line flag").

-eric

From ericsnowcurrently at gmail.com  Wed Nov 28 05:46:23 2012
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Tue, 27 Nov 2012 21:46:23 -0700
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <CALFfu7BNqM5scr6WZQXfNToFwqup7E_ZLqfK_F_P4BC2dCO25A@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE2A53@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7ccgu9aMCXUqmSfmRCr3_ZJY5-A8hwemwYTFzUMXHdKzw@mail.gmail.com>
	<50AB8975.3020405@stackless.com>
	<CADiSq7c=E867L7_1PT4dCPVh1UO+kc3xwoXmK3tXbeXBxBApJQ@mail.gmail.com>
	<CAG8k2+6+rRwyTndvf87gGrSGH9jh0h=5L8k+m_nHQyA59DxEJg@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE76CF@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7e0PhtkozwD0-4EP=hRs4V9M7GK8PoaWqz1moRvEeKJ6w@mail.gmail.com>
	<CALFfu7BNqM5scr6WZQXfNToFwqup7E_ZLqfK_F_P4BC2dCO25A@mail.gmail.com>
Message-ID: <CALFfu7Az9u0wRXOtjf0C9X5r+tiO-NF0+1=dER=5wp0Q_ADUnQ@mail.gmail.com>

On Tue, Nov 27, 2012 at 9:19 PM, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> On Thu, Nov 22, 2012 at 5:09 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Thu, Nov 22, 2012 at 9:44 PM, Kristj?n Valur J?nsson
>> <kristjan at ccpgames.com> wrote:
>>>
>>> Where in the tracker?  I tried searching but didn?t find it.
>>
>>
>> This one: http://bugs.python.org/issue13475
>>
>> This and the issue about being able to configure coverage.py cleanly in
>> subprocesses (http://bugs.python.org/issue14803) are the two issues which
>> have got me thinking about the interpreter startup and configuration process
>> in general lately. Both would add *more* complexity to an already
>> complicated and hard to follow initialisation sequence, so I now think we
>> should be look at opportunities for rationalisation *first*, before we make
>> things even messier.
>
> There's also http://bugs.python.org/issue15716 ("Ability to specify
> the PYTHONPATH via a command line flag").
>
> -eric

I knew there was one more: http://bugs.python.org/issue16499 ("CLI
option for isolated mode").

-eric

From greg at krypto.org  Wed Nov 28 06:37:00 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Tue, 27 Nov 2012 21:37:00 -0800
Subject: [Python-Dev] Handling support for newer OS features at run time
In-Reply-To: <20121127231919.GM91191@snakebite.org>
References: <20121127224928.GK91191@snakebite.org>
	<50B54818.2090907@ubuntu.com>
	<20121127231919.GM91191@snakebite.org>
Message-ID: <CAGE7PNL8y28Ud2CxkWzgPLEPi+P1dHNGGwKSraj9Ear1Rqg3Mg@mail.gmail.com>

On Tue, Nov 27, 2012 at 3:19 PM, Trent Nelson <trent at snakebite.org> wrote:

> On Tue, Nov 27, 2012 at 03:09:12PM -0800, Matthias Klose wrote:
> > Am 27.11.2012 23:49, schrieb Trent Nelson:
> > >     I don't think we've currently got the ability to do this, right?
> > >     Is there a precedent set anywhere else?  I suspect it's not as
> > >     much of an issue on *NIX platforms as you'll typically compile
> > >     from source.  Windows, not so much.
> > >
> > >     Thoughts?  A single binary that dynamically loads applicable
> > >     modules seems cleaner but will obviously take more work.  Other
> > >     approach would be to start distributing multiple versions of
> > >     Python based on the underlying Windows version.  Not the nicest
> > >     approach.
> >
> > Usually I have to build a python package on a build daemon running the
> > kernel of the latest stable (or latest stable long term support)
> > release.  Thus if a configure test checks the current kernel, then you
> > may get an unexpected answer and a missing feature. It is somehow
> > different that I already build different binaries (per release), but
> > the hard-coding of kernel features during configure time looks like
> > the same issue. Currently working around it by patching configure to
> > remove the test and hard-code it for a particular build. Another
> > solution maybe would to have something like --enable-kernel=<version>
> > (as found in glibc), and hope that you can deduce the features from
> > the kernel version.
>
>     Hmmm.  How often do Linux kernel versions expose new features that
>     we can make use of in Python?  i.e. do you run into this problem
>     often?  Can you cite some recent examples?
>

Here's an example of using the pipe2() system call. The code is only
compiled in if the C library supports it. If the C library supports it, the
system call can still return an error because the running kernel doesn't
support it (ENOSYS). In that case it falls back to the legacy code:


http://hg.python.org/cpython/file/618ea5612e83/Modules/_posixsubprocess.c#l738

-gps


>
>     I'm not sure how much could be shared between Windows and Linux with
>     what you've just described.  With Windows, specifically with regards
>     to providing dynamic select.poll() support, I was thinking of having
>     multiple modules:
>
>         Python33\
>             DLLs\
>                 select.pyd
>                 select_win6.pyd
>                 select_win7.pyd
>                 select_win8.pyd
>
>     And Python would automatically import the appropriate one.  I don't
>     think this would be that useful on Linux -- as you say, the decision
>     is typically made at ./configure time.
>
>         Trent.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121127/9e642f9f/attachment-0001.html>

From doko at ubuntu.com  Wed Nov 28 07:09:16 2012
From: doko at ubuntu.com (Matthias Klose)
Date: Wed, 28 Nov 2012 07:09:16 +0100
Subject: [Python-Dev] Handling support for newer OS features at run time
In-Reply-To: <CAGE7PNL8y28Ud2CxkWzgPLEPi+P1dHNGGwKSraj9Ear1Rqg3Mg@mail.gmail.com>
References: <20121127224928.GK91191@snakebite.org>
	<50B54818.2090907@ubuntu.com>
	<20121127231919.GM91191@snakebite.org>
	<CAGE7PNL8y28Ud2CxkWzgPLEPi+P1dHNGGwKSraj9Ear1Rqg3Mg@mail.gmail.com>
Message-ID: <50B5AA8C.9070401@ubuntu.com>

Am 28.11.2012 06:37, schrieb Gregory P. Smith:
> On Tue, Nov 27, 2012 at 3:19 PM, Trent Nelson <trent at snakebite.org> wrote:
> 
>> On Tue, Nov 27, 2012 at 03:09:12PM -0800, Matthias Klose wrote:
>>> Am 27.11.2012 23:49, schrieb Trent Nelson:
>>>>     I don't think we've currently got the ability to do this, right?
>>>>     Is there a precedent set anywhere else?  I suspect it's not as
>>>>     much of an issue on *NIX platforms as you'll typically compile
>>>>     from source.  Windows, not so much.
>>>>
>>>>     Thoughts?  A single binary that dynamically loads applicable
>>>>     modules seems cleaner but will obviously take more work.  Other
>>>>     approach would be to start distributing multiple versions of
>>>>     Python based on the underlying Windows version.  Not the nicest
>>>>     approach.
>>>
>>> Usually I have to build a python package on a build daemon running the
>>> kernel of the latest stable (or latest stable long term support)
>>> release.  Thus if a configure test checks the current kernel, then you
>>> may get an unexpected answer and a missing feature. It is somehow
>>> different that I already build different binaries (per release), but
>>> the hard-coding of kernel features during configure time looks like
>>> the same issue. Currently working around it by patching configure to
>>> remove the test and hard-code it for a particular build. Another
>>> solution maybe would to have something like --enable-kernel=<version>
>>> (as found in glibc), and hope that you can deduce the features from
>>> the kernel version.
>>
>>     Hmmm.  How often do Linux kernel versions expose new features that
>>     we can make use of in Python?  i.e. do you run into this problem
>>     often?  Can you cite some recent examples?
>>
> 
> Here's an example of using the pipe2() system call. The code is only
> compiled in if the C library supports it. If the C library supports it, the
> system call can still return an error because the running kernel doesn't
> support it (ENOSYS). In that case it falls back to the legacy code:
> 
> 
> http://hg.python.org/cpython/file/618ea5612e83/Modules/_posixsubprocess.c#l738

another one is the check for working semaphores for the multiprocessing module,
seen at https://launchpad.net/bugs/630511


From ncoghlan at gmail.com  Wed Nov 28 12:20:12 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 28 Nov 2012 21:20:12 +1000
Subject: [Python-Dev] Generally boared by installation (Re: Setting
 project home path the best way)
In-Reply-To: <CALFfu7Az9u0wRXOtjf0C9X5r+tiO-NF0+1=dER=5wp0Q_ADUnQ@mail.gmail.com>
References: <27E194BD-5864-46CC-82F5-F77DC7977D03@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DDC6B1@RKV-IT-EXCH103.ccp.ad.local>
	<2842C4E9-FEF4-46D1-B09D-C755B33A1925@stackless.com>
	<EFE3877620384242A686D52278B7CCD329DE0BF0@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7dfP8kjFXWrqan6uzJi7xJp_KL=153VnzsrTyhdaFq3og@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE2A53@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7ccgu9aMCXUqmSfmRCr3_ZJY5-A8hwemwYTFzUMXHdKzw@mail.gmail.com>
	<50AB8975.3020405@stackless.com>
	<CADiSq7c=E867L7_1PT4dCPVh1UO+kc3xwoXmK3tXbeXBxBApJQ@mail.gmail.com>
	<CAG8k2+6+rRwyTndvf87gGrSGH9jh0h=5L8k+m_nHQyA59DxEJg@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DE76CF@RKV-IT-EXCH103.ccp.ad.local>
	<CADiSq7e0PhtkozwD0-4EP=hRs4V9M7GK8PoaWqz1moRvEeKJ6w@mail.gmail.com>
	<CALFfu7BNqM5scr6WZQXfNToFwqup7E_ZLqfK_F_P4BC2dCO25A@mail.gmail.com>
	<CALFfu7Az9u0wRXOtjf0C9X5r+tiO-NF0+1=dER=5wp0Q_ADUnQ@mail.gmail.com>
Message-ID: <CADiSq7eusbcTEwbJB7Jwxu=fp4eBS_4oCB5j_MnfAZKd9+qpKg@mail.gmail.com>

On Wed, Nov 28, 2012 at 2:46 PM, Eric Snow <ericsnowcurrently at gmail.com>wrote:

> I knew there was one more: http://bugs.python.org/issue16499 ("CLI
> option for isolated mode").
>

Along with another PYIOENCODING related one that the Blender folks reported
(Christian Heimes pointed it out to me earlier today).

Anyway, I created a page on the wiki for the data gathering process:

http://wiki.python.org/moin/CPythonInterpreterInitialization

It's now a matter of going through and sorting out:

1. What gets set during startup?
2. Where does it get set (or modified)?
3. How is that configured?

Once we have a good view of that, *then* we can start looking for ways to
simplify the code, make the whole system more embedding friendly (i.e. by
giving the embedding app total control via a simple and clean API) and
still support the proposals for improvements.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121128/9fc6bafa/attachment.html>

From kristjan at ccpgames.com  Wed Nov 28 13:13:15 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 28 Nov 2012 12:13:15 +0000
Subject: [Python-Dev] Socket timeout and completion based sockets
In-Reply-To: <CAP7+vJKpeEDV9ZpvJqN2_joOt4cKj9B1BJ5gYeZusnwcnu_VwQ@mail.gmail.com>
References: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJJWDEwAvKB1VVrz-RX6b9kO3TpxnwUGnoq57746MV1WFg@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DEBF4B@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJKpeEDV9ZpvJqN2_joOt4cKj9B1BJ5gYeZusnwcnu_VwQ@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD329DED074@RKV-IT-EXCH103.ccp.ad.local>

I'm sorry, I thought it was something that people did more often, to create different implementations of of the "socket" api, for which cPython provided a mere reference implementation.  I know of at least three different alternative implementations, so I thought that the question were clear enough:  Is the timeout mechanism "supposed" to be re-startable for an api that aims to conform to the "socket" module, or is that a mere coincidence falling out from the select/bsd based reference implementation in cPython?  The docs don't say either way.

(For c-level timeout mechanisms implemented for various c implementations of the bsd socket api,  it is not uncommon to see it stated that after a socket operation times out, the socket is in an undefined state and should be discarded, e.g. here:  http://msdn.microsoft.com/en-us/library/windows/desktop/ms740476(v=vs.85).aspx " If a send or receive operation times out on a socket, the socket state is indeterminate, and should not be used; TCP sockets in this state have a potential for data loss, since the operation could be canceled at the same moment the operation was to be completed.") 

Anyway, as for concrete requirements:  The issue I have always seen with various asynchronous libraries is their lack of composability.  Everyone writes their own application loop and event queue.  Merely having a standard spec and reference implementation of an application main loop object, and main event queue object, in the spirit of WSGI, would possibly remedy this.  You could then hopefully assemble various different libraries in the same application, including greenlet(*) based ones.

(*) Greenlets or stackless can be just another way of hiding asynchronous operations from the programmer.  My favourite one, in fact.  The main trick here is unwinding and replaying of calling contexts, the specific implementation by stack-slicking is mere technical detail, since it can be achieved in other ways (see soft-switching in stackless python)

Cheers,

K

> -----Original Message-----
> From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf
> Of Guido van Rossum
> Sent: 27. n?vember 2012 15:54
with stackless python.
> 
> It would have been nice if you had given more context and stated your
> objective upfront instead of asking what appeared to be an obscure question
> about a technical detail

> Finally, I am not at all interested in greenlets
> ...
> very much unspecified at this point. NOW WOULD BE A GOOD TIME TO
> WRITE DOWN YOUR REQUIREMENTS.
> 
> (*) Greenlets are a fine mechanism for some application areas, but ultimately
> not fit for the standard library, and they have some significant downsides.
> 



From solipsis at pitrou.net  Wed Nov 28 17:29:43 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 28 Nov 2012 17:29:43 +0100
Subject: [Python-Dev] Socket timeout and completion based sockets
References: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJJWDEwAvKB1VVrz-RX6b9kO3TpxnwUGnoq57746MV1WFg@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DEBF4B@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJKpeEDV9ZpvJqN2_joOt4cKj9B1BJ5gYeZusnwcnu_VwQ@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DED074@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <20121128172943.521e5de5@pitrou.net>

Le Wed, 28 Nov 2012 12:13:15 +0000,
Kristj?n Valur J?nsson <kristjan at ccpgames.com> a ?crit :
> I'm sorry, I thought it was something that people did more often, to
> create different implementations of of the "socket" api, for which
> cPython provided a mere reference implementation.  I know of at least
> three different alternative implementations, so I thought that the
> question were clear enough:  Is the timeout mechanism "supposed" to
> be re-startable for an api that aims to conform to the "socket"
> module, or is that a mere coincidence falling out from the select/bsd
> based reference implementation in cPython?

I think recv() and send() (and other simple ops) should certainly be
restartable. sendall() is another matter, I think it should be
considered a best effort thing.

> Anyway, as for concrete requirements:  The issue I have always seen
> with various asynchronous libraries is their lack of composability.

Note: if you are using an asynchronous library, you probably shouldn't
be using any form of socket timeout. Instead, you should be using
timer callbacks as provided by the asynchronous library.
(and the sockets themselves, of course, should be put in non-blocking
mode)

Regards

Antoine.



From guido at python.org  Wed Nov 28 18:04:26 2012
From: guido at python.org (Guido van Rossum)
Date: Wed, 28 Nov 2012 09:04:26 -0800
Subject: [Python-Dev] Socket timeout and completion based sockets
In-Reply-To: <EFE3877620384242A686D52278B7CCD329DED074@RKV-IT-EXCH103.ccp.ad.local>
References: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJJWDEwAvKB1VVrz-RX6b9kO3TpxnwUGnoq57746MV1WFg@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DEBF4B@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJKpeEDV9ZpvJqN2_joOt4cKj9B1BJ5gYeZusnwcnu_VwQ@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DED074@RKV-IT-EXCH103.ccp.ad.local>
Message-ID: <CAP7+vJJKJrYbXFF8EjBSYdicVMKZv1J4A_rc4rdw6VkMkEg6Fg@mail.gmail.com>

On Wed, Nov 28, 2012 at 4:13 AM, Kristj?n Valur J?nsson
<kristjan at ccpgames.com> wrote:
> I'm sorry, I thought it was something that people did more often, to create different implementations of of the "socket" api, for which cPython provided a mere reference implementation.  I know of at least three different alternative implementations, so I thought that the question were clear enough:  Is the timeout mechanism "supposed" to be re-startable for an api that aims to conform to the "socket" module, or is that a mere coincidence falling out from the select/bsd based reference implementation in cPython?  The docs don't say either way.

We're going to have to decide here, since nobody has thought about
this enough apparently. I see two possible answers: we can make it
implementation-dependent, or we can require conforming implementations
to implement properly restartable semantics (if they support timeouts
at all). A third option would be to require these semantics *if the
timeout option is supported* but leave it up to the implementation to
support it at all (ditto for nonblocking, i.e. timeout=0).

> (For c-level timeout mechanisms implemented for various c implementations of the bsd socket api,  it is not uncommon to see it stated that after a socket operation times out, the socket is in an undefined state and should be discarded, e.g. here:  http://msdn.microsoft.com/en-us/library/windows/desktop/ms740476(v=vs.85).aspx " If a send or receive operation times out on a socket, the socket state is indeterminate, and should not be used; TCP sockets in this state have a potential for data loss, since the operation could be canceled at the same moment the operation was to be completed.")

Is this relevant to CPython though? Its socket implementation uses
select() to implement timeouts, even on Windows, AFAIK.

> Anyway, as for concrete requirements:  The issue I have always seen with various asynchronous libraries is their lack of composability.  Everyone writes their own application loop and event queue.  Merely having a standard spec and reference implementation of an application main loop object, and main event queue object, in the spirit of WSGI, would possibly remedy this.  You could then hopefully assemble various different libraries in the same application, including greenlet(*) based ones.

Hm. I agree with the first part of this -- and indeed I am planning to
make it so that tulip's event loop can easily be replaced by another
one. I'm less sure about the yield-from-based scheduler, that's the
kind of thing for which it doesn't really make sense to have multiple
implementations. If greenlets can work with the standard event loop
interface, good for them. (Either by providing a conforming
implementation that also supports greenlets, or by just using the
standard implementation.)

> (*) Greenlets or stackless can be just another way of hiding asynchronous operations from the programmer.  My favourite one, in fact.  The main trick here is unwinding and replaying of calling contexts, the specific implementation by stack-slicking is mere technical detail, since it can be achieved in other ways (see soft-switching in stackless python)

Yes. While none of these belong in the stdlib, I certainly want to
support their viability as a 3rd party alternative to yield-from.

Note however that even Christian Tismer has expressed doubts about
hiding async blocks completely -- for the same reasons I'm not keen on
them myself: when it's not obvious whether a particular call can cause
a context switch (e.g. due to something it uses indirectly doing some
kind of I/O that requires a context switch to avoid blocking), you're
back in the world of threads and explicit locks and all the nightmares
that come with that. Using yield or yield-from to mark switch points
means you will always be aware of the possibility that a call switches
(unfortunately there are other costs).

--Guido

> Cheers,
>
> K
>
>> -----Original Message-----
>> From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf
>> Of Guido van Rossum
>> Sent: 27. n?vember 2012 15:54
> with stackless python.
>>
>> It would have been nice if you had given more context and stated your
>> objective upfront instead of asking what appeared to be an obscure question
>> about a technical detail
>
>> Finally, I am not at all interested in greenlets
>> ...
>> very much unspecified at this point. NOW WOULD BE A GOOD TIME TO
>> WRITE DOWN YOUR REQUIREMENTS.
>>
>> (*) Greenlets are a fine mechanism for some application areas, but ultimately
>> not fit for the standard library, and they have some significant downsides.
>>
>
>



-- 
--Guido van Rossum (python.org/~guido)

From glyph at twistedmatrix.com  Wed Nov 28 20:15:07 2012
From: glyph at twistedmatrix.com (Glyph)
Date: Wed, 28 Nov 2012 14:15:07 -0500
Subject: [Python-Dev] Socket timeout and completion based sockets
In-Reply-To: <CAP7+vJJKJrYbXFF8EjBSYdicVMKZv1J4A_rc4rdw6VkMkEg6Fg@mail.gmail.com>
References: <EFE3877620384242A686D52278B7CCD329DEB553@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJJWDEwAvKB1VVrz-RX6b9kO3TpxnwUGnoq57746MV1WFg@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DEBF4B@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJKpeEDV9ZpvJqN2_joOt4cKj9B1BJ5gYeZusnwcnu_VwQ@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD329DED074@RKV-IT-EXCH103.ccp.ad.local>
	<CAP7+vJJKJrYbXFF8EjBSYdicVMKZv1J4A_rc4rdw6VkMkEg6Fg@mail.gmail.com>
Message-ID: <1D9BE0CD-5BF4-480D-8D40-5A409E40760D@twistedmatrix.com>


On Nov 28, 2012, at 12:04 PM, Guido van Rossum <guido at python.org> wrote:

>> Anyway, as for concrete requirements:  The issue I have always seen with various asynchronous libraries is their lack of composability.  Everyone writes their own application loop and event queue.  Merely having a standard spec and reference implementation of an application main loop object, and main event queue object, in the spirit of WSGI, would possibly remedy this.  You could then hopefully assemble various different libraries in the same application, including greenlet(*) based ones.
> 
> Hm. I agree with the first part of this -- and indeed I am planning to
> make it so that tulip's event loop can easily be replaced by another
> one. I'm less sure about the yield-from-based scheduler, that's the
> kind of thing for which it doesn't really make sense to have multiple
> implementations. If greenlets can work with the standard event loop
> interface, good for them. (Either by providing a conforming
> implementation that also supports greenlets, or by just using the
> standard implementation.)

I'm really happy that you are building this in as a core feature of Tulip.  It's really important.

Very early on, Twisted attempted to avoid this lack of composability by explicitly delegating to other application loops; it's one of my favorite features of Twisted.  Granted, no two loops we have attempted to use have themselves been composable, but there's not much we can do about that :).  Still, code written on top of Twisted can always be plugged in to any other loop by simply using the appropriate reactor.  (There's also a plug-in interface for the reactor and a plug-in discovery mechanism so that third parties can easily provide their own reactors if they have an unusual main loop that isn't supported by twisted itself.)

I would also like to bring up <https://github.com/lvh/async-pep> again.  If anyone really wants to dig in and enumerate the use-cases for the _lower-level_ event delivery portions of Tulip, something that would be compatible with Twisted and Tornado and so on, that PEP already has a good skeleton and could use some pull requests.

-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121128/d75584b0/attachment.html>

From eliben at gmail.com  Fri Nov 30 05:55:54 2012
From: eliben at gmail.com (Eli Bendersky)
Date: Thu, 29 Nov 2012 20:55:54 -0800
Subject: [Python-Dev] type vs. class terminology
In-Reply-To: <CAOTb1wdDvta+DFBeGHaOtU4Ftsq2xLZrfrL2i-Sdk22sZoYReQ@mail.gmail.com>
References: <CAOTb1wdDvta+DFBeGHaOtU4Ftsq2xLZrfrL2i-Sdk22sZoYReQ@mail.gmail.com>
Message-ID: <CAF-Rda9V88s+51zqf42_uz0seVeTpM0=K-Nhc+vtNk30J_DaeA@mail.gmail.com>

On Sun, Nov 25, 2012 at 9:01 PM, Chris Jerdonek <chris.jerdonek at gmail.com>wrote:

> I would like to know when we should use "class" in the Python 3
> documentation, and when we should use "type."  Are these terms
> synonymous in Python 3, and do we have a preference for which to use
> and when?
>
> I'm sure this has been discussed before.  But if this terminology
> issue has already been resolved, the resolution doesn't seem to be
> reflected in the docs.  For example, the glossary entries for type and
> class don't reference each other.
>

Good question,

[shameless plug follows, I post this because I truly believe it's very
relevant to the discussion]

I had the same doubts some months ago, which led to writing this article
(relevant to Python 3):
http://eli.thegreenplace.net/2012/03/30/python-objects-types-classes-and-instances-a-glossary/

It examines the class vs. type issue, as well as object vs. instance
And this diagram can also be useful to understand how similar built-in and
user-defined types are in Python 3:
http://eli.thegreenplace.net/2012/04/03/the-fundamental-types-of-python-a-diagram/

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121129/bf5d19a4/attachment.html>

From status at bugs.python.org  Fri Nov 30 18:07:23 2012
From: status at bugs.python.org (Python tracker)
Date: Fri, 30 Nov 2012 18:07:23 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20121130170723.D44591C98E@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-11-23 - 2012-11-30)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3792 ( -7)
  closed 24566 (+48)
  total  28358 (+41)

Open issues with patches: 1653 


Issues opened (27)
==================

#8824: Improve documentation of exec
http://bugs.python.org/issue8824  reopened by mark.dickinson

#9400: multiprocessing.pool.AsyncResult.get() messes up exceptions
http://bugs.python.org/issue9400  reopened by sbt

#15587: IDLE is pixelated on the Macbook Pro with Retina Display
http://bugs.python.org/issue15587  reopened by Tyler.Crompton

#16541: tk_setPalette doesn't accept keyword parameters
http://bugs.python.org/issue16541  opened by HJarausch

#16543: improve TypeError messages for missing arguments (meta issue)
http://bugs.python.org/issue16543  opened by ezio.melotti

#16544: Add external link to ast docs
http://bugs.python.org/issue16544  opened by asvetlov

#16547: IDLE raises an exception in tkinter after fresh file's text ha
http://bugs.python.org/issue16547  opened by ebfe

#16550: pickletools.py treats 32bit lengths as signed, but pickle.py a
http://bugs.python.org/issue16550  opened by serhiy.storchaka

#16551: Cleanup the pure Python pickle implementation
http://bugs.python.org/issue16551  opened by serhiy.storchaka

#16554: The description of the argument of MAKE_FUNCTION and MAKE_CLOS
http://bugs.python.org/issue16554  opened by daniel.urban

#16555: Add es_cu to locale aliases
http://bugs.python.org/issue16555  opened by Leiser.Fern??ndez.Gallo

#16557: PEP 380 isn't reflected in the Functional Programming HOWTO
http://bugs.python.org/issue16557  opened by msmhrt

#16561: Windows installer doesn't use UAC, then crashes
http://bugs.python.org/issue16561  opened by Redoute

#16562: Optimize dict equality test
http://bugs.python.org/issue16562  opened by rhettinger

#16564: email.generator.BytesGenerator fails with bytes payload
http://bugs.python.org/issue16564  opened by Alexander.Kruppa

#16565: Increase Py_AddPendingCall array size
http://bugs.python.org/issue16565  opened by felipecruz

#16566: Structure._anonymous_ should not allow strings
http://bugs.python.org/issue16566  opened by techtonik

#16568: allow constructors to be documented separately from class
http://bugs.python.org/issue16568  opened by chris.jerdonek

#16569: Preventing errors of simultaneous access in zipfile
http://bugs.python.org/issue16569  opened by serhiy.storchaka

#16572: Bad multi-inheritance support in some libs like threading or m
http://bugs.python.org/issue16572  opened by thomas.chiroux

#16574: clarify policy on updates to final peps
http://bugs.python.org/issue16574  opened by chris.jerdonek

#16575: ctypes: unions as arguments
http://bugs.python.org/issue16575  opened by arigo

#16576: ctypes: structure with bitfields as argument
http://bugs.python.org/issue16576  opened by arigo

#16577: Suspect test.test_codeccallbacks.test_mutatingdecodehandler
http://bugs.python.org/issue16577  opened by amaury.forgeotdarc

#16579: .pyw disturb multiprocessing behavior
http://bugs.python.org/issue16579  opened by Alex.stein

#16580: Add examples to int.to_bytres and int.from_bytes
http://bugs.python.org/issue16580  opened by paddy3118

#16581: define "PEP editor" in PEP 1
http://bugs.python.org/issue16581  opened by chris.jerdonek



Most recent 15 issues with no replies (15)
==========================================

#16581: define "PEP editor" in PEP 1
http://bugs.python.org/issue16581

#16580: Add examples to int.to_bytres and int.from_bytes
http://bugs.python.org/issue16580

#16575: ctypes: unions as arguments
http://bugs.python.org/issue16575

#16561: Windows installer doesn't use UAC, then crashes
http://bugs.python.org/issue16561

#16557: PEP 380 isn't reflected in the Functional Programming HOWTO
http://bugs.python.org/issue16557

#16551: Cleanup the pure Python pickle implementation
http://bugs.python.org/issue16551

#16550: pickletools.py treats 32bit lengths as signed, but pickle.py a
http://bugs.python.org/issue16550

#16516: argparse types (and actions) must be hashable
http://bugs.python.org/issue16516

#16509: sqlite3 docs do not explain check_same_thread
http://bugs.python.org/issue16509

#16494: Add a method on importlib.SourceLoader for creating bytecode f
http://bugs.python.org/issue16494

#16492: Add a load_parents argument to importlib.find_loader()
http://bugs.python.org/issue16492

#16486: Add context manager support to aifc module
http://bugs.python.org/issue16486

#16463: test_timeout failure on the RHEL buildbot
http://bugs.python.org/issue16463

#16450: test_missing_localfile masks problems in urlopen
http://bugs.python.org/issue16450

#16429: Emit SyntaxWarning for code that risks UnboundLocalError
http://bugs.python.org/issue16429



Most recent 15 issues waiting for review (15)
=============================================

#16572: Bad multi-inheritance support in some libs like threading or m
http://bugs.python.org/issue16572

#16569: Preventing errors of simultaneous access in zipfile
http://bugs.python.org/issue16569

#16562: Optimize dict equality test
http://bugs.python.org/issue16562

#16554: The description of the argument of MAKE_FUNCTION and MAKE_CLOS
http://bugs.python.org/issue16554

#16551: Cleanup the pure Python pickle implementation
http://bugs.python.org/issue16551

#16550: pickletools.py treats 32bit lengths as signed, but pickle.py a
http://bugs.python.org/issue16550

#16543: improve TypeError messages for missing arguments (meta issue)
http://bugs.python.org/issue16543

#16537: Python???s setup.py raises a ValueError when self.extensions i
http://bugs.python.org/issue16537

#16525: wave file module does not support 32bit float format
http://bugs.python.org/issue16525

#16523: attrgetter and itemgetter signatures in docs need cleanup
http://bugs.python.org/issue16523

#16515: TypeError message incorrect for max() with one keyword arg
http://bugs.python.org/issue16515

#16512: imghdr doesn't support jpegs with an ICC profile
http://bugs.python.org/issue16512

#16511: IDLE configuration file: blank height and width fields trip up
http://bugs.python.org/issue16511

#16510: Using appropriate checks in tests
http://bugs.python.org/issue16510

#16507: Patch selectmodule.c to support WSAPoll on Windows
http://bugs.python.org/issue16507



Top 10 most discussed issues (10)
=================================

#16543: improve TypeError messages for missing arguments (meta issue)
http://bugs.python.org/issue16543  21 msgs

#16518: add "buffer protocol" to glossary
http://bugs.python.org/issue16518  15 msgs

#16574: clarify policy on updates to final peps
http://bugs.python.org/issue16574  15 msgs

#4591: 32-bits unsigned user/group identifier
http://bugs.python.org/issue4591  11 msgs

#4945: json checks True/False by identity, not boolean value
http://bugs.python.org/issue4945  11 msgs

#16547: IDLE raises an exception in tkinter after fresh file's text ha
http://bugs.python.org/issue16547   9 msgs

#7976: warnings should provide a public API for accessing its option 
http://bugs.python.org/issue7976   8 msgs

#16565: Increase Py_AddPendingCall array size
http://bugs.python.org/issue16565   8 msgs

#16566: Structure._anonymous_ should not allow strings
http://bugs.python.org/issue16566   8 msgs

#11175: allow argparse FileType to accept encoding and errors argument
http://bugs.python.org/issue11175   6 msgs



Issues closed (45)
==================

#1827: svnversion_init() doesn't support svn urls in sandbox/trunk
http://bugs.python.org/issue1827  closed by christian.heimes

#1977: Python reinitialization test
http://bugs.python.org/issue1977  closed by christian.heimes

#2039: Pymalloc patch for int and float objects
http://bugs.python.org/issue2039  closed by christian.heimes

#3410: platform.version() don't work as expected in Vista in portugue
http://bugs.python.org/issue3410  closed by ezio.melotti

#4473: POP3 missing support for starttls
http://bugs.python.org/issue4473  closed by pitrou

#9011: ast_for_factor unary minus optimization changes AST
http://bugs.python.org/issue9011  closed by mark.dickinson

#9176: module termios doesn't build on HP-UX
http://bugs.python.org/issue9176  closed by skrah

#10259: Entry text not set if all of 'Font', 'Foreground' and 'Justify
http://bugs.python.org/issue10259  closed by gpolo

#11076: Iterable argparse Namespace
http://bugs.python.org/issue11076  closed by asvetlov

#12848: pickle.py treats 32bit lengths as signed, but _pickle.c as uns
http://bugs.python.org/issue12848  closed by pitrou

#14478: Decimal hashing very slow, could be cached
http://bugs.python.org/issue14478  closed by mark.dickinson

#15990: solidify argument/parameter terminology
http://bugs.python.org/issue15990  closed by chris.jerdonek

#16205: update :class:`str` references to link to the str type section
http://bugs.python.org/issue16205  closed by chris.jerdonek

#16209: add a "class str" entry to the docs
http://bugs.python.org/issue16209  closed by chris.jerdonek

#16323: Wrong C API documentation for locale encoding
http://bugs.python.org/issue16323  closed by asvetlov

#16333: Trailing whitespace in json dump when using indent
http://bugs.python.org/issue16333  closed by ezio.melotti

#16339: Document "exec(stmt, global_dict, local_dict)" form in Python 
http://bugs.python.org/issue16339  closed by mark.dickinson

#16423: urllib data URL
http://bugs.python.org/issue16423  closed by pitrou

#16464: urllib.request: opener not resetting content-length
http://bugs.python.org/issue16464  closed by asvetlov

#16477: tarfile fails to close file handles in case of exception
http://bugs.python.org/issue16477  closed by asvetlov

#16483: Make int(float('inf')) raise ValueError rather than OverflowEr
http://bugs.python.org/issue16483  closed by mark.dickinson

#16519: site.venv() should use abspath(executable)
http://bugs.python.org/issue16519  closed by python-dev

#16521: logging.basicConfig creates empty file when using handlers
http://bugs.python.org/issue16521  closed by python-dev

#16524: File access not always working with Python for Windows 32 bits
http://bugs.python.org/issue16524  closed by terry.reedy

#16530: documentation of os.wait3
http://bugs.python.org/issue16530  closed by ezio.melotti

#16532: AMD64 Windows 7 build failures
http://bugs.python.org/issue16532  closed by pitrou

#16534: -Olimit: unsupported option: warnings and ./configure failures
http://bugs.python.org/issue16534  closed by skrah

#16540: Make itertools count, cycle, and repeat objects subscriptable 
http://bugs.python.org/issue16540  closed by rhettinger

#16542: http//bugs.python/joko.suwito
http://bugs.python.org/issue16542  closed by ezio.melotti

#16545: ast.FunctionDef sets a bad value for kw_defaults when keyword-
http://bugs.python.org/issue16545  closed by brett.cannon

#16546: ast.YieldFrom needlessly has its expr value as optional
http://bugs.python.org/issue16546  closed by mark.dickinson

#16548: os.system won't run any command and there is no error message 
http://bugs.python.org/issue16548  closed by r.david.murray

#16549: regression: -m json.tool module is broken
http://bugs.python.org/issue16549  closed by ezio.melotti

#16552: os.path.basename() docs should link to os.path.split()
http://bugs.python.org/issue16552  closed by chris.jerdonek

#16553: named kwd form of OrderedDict ctor yield random key order
http://bugs.python.org/issue16553  closed by benjamin.peterson

#16556: Update string.Formatter.vformat documentation to say "**kwargs
http://bugs.python.org/issue16556  closed by ezio.melotti

#16558: multiprocessing fails to raise exception with parameters
http://bugs.python.org/issue16558  closed by sbt

#16559: Update JSON??tests
http://bugs.python.org/issue16559  closed by serhiy.storchaka

#16560: Python sighandlers delayed for no reason
http://bugs.python.org/issue16560  closed by zdenek.pavlas

#16563: re.match loops forever on simple regexp
http://bugs.python.org/issue16563  closed by mark.dickinson

#16567: Implementing .= for variable operator
http://bugs.python.org/issue16567  closed by amaury.forgeotdarc

#16570: Absolute imports fail to take full path into account?
http://bugs.python.org/issue16570  closed by brett.cannon

#16571: Iterating over inconsistently-indented code block causes execu
http://bugs.python.org/issue16571  closed by mark.dickinson

#16573: 2to3 should treat enumerate like sorted for zip, map, filter, 
http://bugs.python.org/issue16573  closed by python-dev

#16578: Regular expressions with empty named groups break isname check
http://bugs.python.org/issue16578  closed by Gabriel.Rodr??guez.Alberich

From brett at python.org  Fri Nov 30 20:38:12 2012
From: brett at python.org (Brett Cannon)
Date: Fri, 30 Nov 2012 14:38:12 -0500
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <20121130170723.D44591C98E@psf.upfronthosting.co.za>
References: <20121130170723.D44591C98E@psf.upfronthosting.co.za>
Message-ID: <CAP1=2W6U6z0ZwnWYUEdqQKvEuTmsJd0fiuyPeRJN-xv_SR5z9w@mail.gmail.com>

Do we have a graph of the historical trend of the number of bugs (or at
least the historical details stored somewhere)? I think we have had a net
decrease in open bugs the last couple of weeks and it would be neat to see
an absolute and relative graph of the overall trend since Python 3.3.0 was
released. Also might make a nice motivator to try to close issues faster. =)

Otherwise is the code public for this somewhere? I assume it's making an
XML-RPC call or something every week to get the results, but if I decide to
do a little App Engine app to store historical data and do a graph I would
rather not have to figure all of this out from scratch. =) Although I could
I guess also parse the email if I wanted to ignore all other emails.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121130/716191d4/attachment.html>

From tjreedy at udel.edu  Fri Nov 30 21:15:54 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 30 Nov 2012 15:15:54 -0500
Subject: [Python-Dev] type vs. class terminology
In-Reply-To: <CAF-Rda9V88s+51zqf42_uz0seVeTpM0=K-Nhc+vtNk30J_DaeA@mail.gmail.com>
References: <CAOTb1wdDvta+DFBeGHaOtU4Ftsq2xLZrfrL2i-Sdk22sZoYReQ@mail.gmail.com>
	<CAF-Rda9V88s+51zqf42_uz0seVeTpM0=K-Nhc+vtNk30J_DaeA@mail.gmail.com>
Message-ID: <k9b46e$j6i$1@ger.gmane.org>

On 11/29/2012 11:55 PM, Eli Bendersky wrote:
> On Sun, Nov 25, 2012 at 9:01 PM, Chris Jerdonek
> <chris.jerdonek at gmail.com <mailto:chris.jerdonek at gmail.com>> wrote:
>
>     I would like to know when we should use "class" in the Python 3
>     documentation, and when we should use "type."  Are these terms
>     synonymous in Python 3, and do we have a preference for which to use
>     and when?
>
>     I'm sure this has been discussed before.  But if this terminology
>     issue has already been resolved, the resolution doesn't seem to be
>     reflected in the docs.  For example, the glossary entries for type and
>     class don't reference each other.
>
>
> Good question,
>
> [shameless plug follows, I post this because I truly believe it's very
> relevant to the discussion]
>
> I had the same doubts some months ago, which led to writing this article
> (relevant to Python 3):
> http://eli.thegreenplace.net/2012/03/30/python-objects-types-classes-and-instances-a-glossary/

> It examines the class vs. type issue, as well as object vs. instance

You usage seems to me to be stuck in the now mostly useless Python 1 
distinction between built-in types and user-defined classes. In Python 
3, all instances of type are classes, whether defined with the C or 
Python API. Indeed, the existence of a C API to make classes is an 
implementation detail, not a language feature. The second parameter of 
isinstance or issubclass is a class or set thereof (implemented as a 
(homogeneous!) tuple for historical reasons), without distinction of 
definition mode. When using an imported class, it mostly does not matter 
how the class was defined.

I agree with Guido that it is more useful to use 'class' for the 
concrete run-time object and 'type' (except when referring to the object 
of that name) for abstract (and static) types. (From this viewpoint, 
duck-typing rather than duck-classing *is* the proper term).

Consider the quote from the manual. "An object?s type determines the 
operations that the object supports (e.g., ?does it have a length??)." 
An object potentially supports len(), and one might say should support 
it, if and only if it is a 'finite collection'. That is a 'type' (duck 
type) of object, but not a class in the Python sense. Whether an object 
actually supports len depends on its run-time class. Similarly, an 
object 'should' support sqrt if it is a non-negative scalar number or a 
complex number. Square-root-able is also an abstract type, not a 
concrete class.

I might suggest that types are used to reason about programs while 
classes are used to execute programs.

-- 
Terry Jan Reedy



From storchaka at gmail.com  Fri Nov 30 21:34:37 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Fri, 30 Nov 2012 22:34:37 +0200
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <CAP1=2W6U6z0ZwnWYUEdqQKvEuTmsJd0fiuyPeRJN-xv_SR5z9w@mail.gmail.com>
References: <20121130170723.D44591C98E@psf.upfronthosting.co.za>
	<CAP1=2W6U6z0ZwnWYUEdqQKvEuTmsJd0fiuyPeRJN-xv_SR5z9w@mail.gmail.com>
Message-ID: <k9b58u$rvs$1@ger.gmane.org>

On 30.11.12 21:38, Brett Cannon wrote:
> Also might make a nice motivator to try to close issues faster. =)

May be introduce the title of "Issue closer of the month"? ;)



From rdmurray at bitdance.com  Fri Nov 30 22:07:47 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 30 Nov 2012 16:07:47 -0500
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <CAP1=2W6U6z0ZwnWYUEdqQKvEuTmsJd0fiuyPeRJN-xv_SR5z9w@mail.gmail.com>
References: <20121130170723.D44591C98E@psf.upfronthosting.co.za>
	<CAP1=2W6U6z0ZwnWYUEdqQKvEuTmsJd0fiuyPeRJN-xv_SR5z9w@mail.gmail.com>
Message-ID: <20121130210748.3679B250117@webabinitio.net>

On Fri, 30 Nov 2012 14:38:12 -0500, Brett Cannon <brett at python.org> wrote:
> Do we have a graph of the historical trend of the number of bugs (or at
> least the historical details stored somewhere)? I think we have had a net

Not really.  Ezio made one by hand once, but there is nothing automated.

The historical details are stored only in the mailing list archives, as
far as I know.  In theory I think you could re-calculate them from the
Roundup DB, but for various reasons the numbers would probably come out
slightly different.  Still, getting the data from the DB would be better
than parsing the emails, since for one reason and another there are
missing Friday reports, and reports that were issued on non-Friday
dates.

> decrease in open bugs the last couple of weeks and it would be neat to see
> an absolute and relative graph of the overall trend since Python 3.3.0 was
> released. Also might make a nice motivator to try to close issues faster. =)
> 
> Otherwise is the code public for this somewhere? I assume it's making an

Yes.  It is in the software repository for our roundup instances:

    http://hg.python.org/tracker/python-dev/file/default/scripts/roundup-summary

(Be warned that that isn't the location from which the script is
executed, so it is possible for what is actually running to get out of
sync with what is checked in at that location.)

> XML-RPC call or something every week to get the results, but if I decide to

Nope, it talks directly to the DB.  And as you will see, it is more
than a bit gnarly.

> do a little App Engine app to store historical data and do a graph I would
> rather not have to figure all of this out from scratch. =) Although I could
> I guess also parse the email if I wanted to ignore all other emails.

I'm not sure how one would go about integrating the above with an App
Engine app.  I suspect that not quite enough information is available
through the XML-RPC interface to replicate that script, but maybe you
could manage just the open-close counting part of it.  I haven't
looked at what it would take.

--David

From brett at python.org  Fri Nov 30 22:52:43 2012
From: brett at python.org (Brett Cannon)
Date: Fri, 30 Nov 2012 16:52:43 -0500
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <20121130210748.3679B250117@webabinitio.net>
References: <20121130170723.D44591C98E@psf.upfronthosting.co.za>
	<CAP1=2W6U6z0ZwnWYUEdqQKvEuTmsJd0fiuyPeRJN-xv_SR5z9w@mail.gmail.com>
	<20121130210748.3679B250117@webabinitio.net>
Message-ID: <CAP1=2W53qoCp6fy4haHrOwSEH7SReSJdaWNT72HfKfKG6SC4ag@mail.gmail.com>

On Fri, Nov 30, 2012 at 4:07 PM, R. David Murray <rdmurray at bitdance.com>wrote:

> On Fri, 30 Nov 2012 14:38:12 -0500, Brett Cannon <brett at python.org> wrote:
> > Do we have a graph of the historical trend of the number of bugs (or at
> > least the historical details stored somewhere)? I think we have had a net
>
> Not really.  Ezio made one by hand once, but there is nothing automated.
>
> The historical details are stored only in the mailing list archives, as
> far as I know.  In theory I think you could re-calculate them from the
> Roundup DB, but for various reasons the numbers would probably come out
> slightly different.  Still, getting the data from the DB would be better
> than parsing the emails, since for one reason and another there are
> missing Friday reports, and reports that were issued on non-Friday
> dates.
>
> > decrease in open bugs the last couple of weeks and it would be neat to
> see
> > an absolute and relative graph of the overall trend since Python 3.3.0
> was
> > released. Also might make a nice motivator to try to close issues
> faster. =)
> >
> > Otherwise is the code public for this somewhere? I assume it's making an
>
> Yes.  It is in the software repository for our roundup instances:
>
>
> http://hg.python.org/tracker/python-dev/file/default/scripts/roundup-summary
>
> (Be warned that that isn't the location from which the script is
> executed, so it is possible for what is actually running to get out of
> sync with what is checked in at that location.)
>
> > XML-RPC call or something every week to get the results, but if I decide
> to
>
> Nope, it talks directly to the DB.  And as you will see, it is more
> than a bit gnarly.
>
>
I think I could also download the csv file and parse that to get whatever
data I wanted.


>  > do a little App Engine app to store historical data and do a graph I
> would
> > rather not have to figure all of this out from scratch. =) Although I
> could
> > I guess also parse the email if I wanted to ignore all other emails.
>
> I'm not sure how one would go about integrating the above with an App
> Engine app.  I suspect that not quite enough information is available
> through the XML-RPC interface to replicate that script, but maybe you
> could manage just the open-close counting part of it.  I haven't
> looked at what it would take.
>

It really depends on what statistics I cared about (e.g. there are less
than 4000 bugs while there are less than 25,000 closed bugs). If I just did
high-level statistics it wouldn't be bad, but if I try to track every issue
independently that might be annoying (and actually cost money for me,
although I already personally pay for py3ksupport.appspot.com so I can
probably piggyback off of that app's quota). We will see if this ever goes
anywhere. =)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121130/376f598f/attachment.html>