From ncoghlan at gmail.com Fri Oct 1 00:12:12 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 1 Oct 2010 08:12:12 +1000 Subject: [Python-Dev] We should be using a tool for code reviews In-Reply-To: References: <20100930145218.2022.353483588.divmod.xquotient.55@localhost.localdomain> Message-ID: On Fri, Oct 1, 2010 at 12:56 AM, Guido van Rossum wrote: >> (I am strongly in favor of this, but I don't think many core committers >> are.) > > Having worked in this style for almost 5 years now, I am also strongly > in favor. Jesse expressed it better than I could. I'll be one of those to object (but only slightly). I think one of the privileges/responsibilities that goes with commit access is the ability to make the call between: - "this is a simple change/fix, I'll just check it in with possible post hoc review via python-checkins" - "I want feedback on the idea and/or details before I commit this, I'll post a patch for review to the tracker" - "I may want help in getting this working and/or this may take a while to get right, so I'll create a branch for it" (with the balance between 2 and 3 apparently shifting more in favour of 3 once we have hg to play with) Particularly for user visible API changes, I think getting a sanity check from at least one other dev before committing is a good idea. For smaller stuff, I think python-checkins after the fact reviews are enough to cover it (particularly now that one person asking a question will kick the entire diff over to python-dev for broader review). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From martin at v.loewis.de Fri Oct 1 01:50:42 2010 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 01 Oct 2010 01:50:42 +0200 Subject: [Python-Dev] Celebrating issue #10000 Message-ID: <4CA52252.4060106@v.loewis.de> Amaury just filed issue #10000 yesterday; as counting started with 1000, we are now into 9000 roundup issues. I have become quite fond of roundup over the years, and would like to thank Ka-Ping Yee, Richard Jones, and Erik Forsberg for getting us here. There are many contributions to this infrastructure, both from individuals and software projects, but I'd like to single out two of them which have I also appreciate very much: the folks at Upfront Hosting have helped a lot to keep the system running, and the PostgreSQL database has really validated it's own claim of being the world's most advanced open source database. Kind regards, Martin From solipsis at pitrou.net Fri Oct 1 03:13:35 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 1 Oct 2010 03:13:35 +0200 Subject: [Python-Dev] Docs rebuild Message-ID: <20101001031335.769394be@pitrou.net> Hello, It seems the py3k docs (both dev and 3.1) haven't been rebuilt for a few days. Is there anything that needs to be done to trigger rebuilding? Thank you, Antoine. From ben+python at benfinney.id.au Fri Oct 1 03:40:00 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 01 Oct 2010 11:40:00 +1000 Subject: [Python-Dev] Celebrating issue #10000 References: <4CA52252.4060106@v.loewis.de> Message-ID: <87aamy1yj3.fsf@benfinney.id.au> "Martin v. L?wis" writes: > Amaury just filed issue #10000 yesterday; as counting started with > 1000, we are now into 9000 roundup issues. Congratulations! > I have become quite fond of roundup over the years, and would like to > thank Ka-Ping Yee, Richard Jones, and Erik Forsberg for getting us > here. Yes, thanks to all those involved in moving to a free issue tracker for Python and maintaining it past the initial stages. I'm glad Roundup and PostgreSQL are both up to the task and, by Martin's account, good to use. > There are many contributions to this infrastructure, both from > individuals and software projects, but I'd like to single out two of > them which have I also appreciate very much: the folks at Upfront > Hosting have helped a lot to keep the system running Yay! Donating resources to free software projects *and* allowing administrative control over the software is really valuable. -- \ ?It's up to the masses to distribute [music] however they want | `\ ? The laws don't matter at that point. People sharing music in | _o__) their bedrooms is the new radio.? ?Neil Young, 2008-05-06 | Ben Finney From stephen at xemacs.org Fri Oct 1 06:09:24 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 01 Oct 2010 13:09:24 +0900 Subject: [Python-Dev] Branching without losing your build products [was: We should be using a tool for code reviews] In-Reply-To: <20100930102755.43ab03d3@mission> References: <20100929204125.416e2013@pitrou.net> <1285792363.3194.41.camel@localhost.localdomain> <20100929173010.2acf0b3d@mission> <20100930000420.3b7d4999@pitrou.net> <20100929183347.0d95304e@mission> <87bp7fboak.fsf@uwakimon.sk.tsukuba.ac.jp> <8762xnbko2.fsf@uwakimon.sk.tsukuba.ac.jp> <20100930102755.43ab03d3@mission> Message-ID: <8739sqbll7.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > I should note that I don't particularly like colocated/named branches. I > personally much prefer separate directories for each feature or bug I'm > working on. It helps me keep track of what I'm doing. I have a fast machine > so recompiling all of Python is no big deal. Note that once you have a branch with all of Python built, for any of hg, git, bzr you may prefer to clone the branch/repo with "cp -r" or "rsync" rather than "$VCS clone". The reason for doing it with "$VCS clone" is to get a pristine workspace without any cruft like editor backups or build products. If you *want* a crufty workspace (and in this case some people do), a recursive copy is a better tool. From g.brandl at gmx.net Fri Oct 1 07:22:39 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 01 Oct 2010 07:22:39 +0200 Subject: [Python-Dev] Docs rebuild In-Reply-To: <20101001031335.769394be@pitrou.net> References: <20101001031335.769394be@pitrou.net> Message-ID: Am 01.10.2010 03:13, schrieb Antoine Pitrou: > > Hello, > > It seems the py3k docs (both dev and 3.1) haven't been rebuilt for a few > days. Is there anything that needs to be done to trigger rebuilding? Yes, I noticed it in my cronjob email. It seems latex has a problem with c-api.tex; I'll have a look at it. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Fri Oct 1 07:23:20 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 01 Oct 2010 07:23:20 +0200 Subject: [Python-Dev] Celebrating issue #10000 In-Reply-To: <4CA52252.4060106@v.loewis.de> References: <4CA52252.4060106@v.loewis.de> Message-ID: Am 01.10.2010 01:50, schrieb "Martin v. L?wis": > Amaury just filed issue #10000 yesterday; as counting started > with 1000, we are now into 9000 roundup issues. So, nitpickly, it would be 9001. But of course, we're already at 10003 anyway :) > I have become quite fond of roundup over the years, and would > like to thank Ka-Ping Yee, Richard Jones, and Erik Forsberg > for getting us here. > > There are many contributions to this infrastructure, both > from individuals and software projects, but I'd like to single > out two of them which have I also appreciate very much: > the folks at Upfront Hosting have helped a lot to keep the system > running, and the PostgreSQL database has really validated it's > own claim of being the world's most advanced open source > database. I'd like to add my thanks. Where would we be without the tracker? Probably in holidays... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Fri Oct 1 07:49:23 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 01 Oct 2010 07:49:23 +0200 Subject: [Python-Dev] We should be using a tool for code reviews In-Reply-To: References: Message-ID: Am 30.09.2010 10:22, schrieb Dirkjan Ochtman: > On Wed, Sep 29, 2010 at 20:32, Guido van Rossum wrote: >> I would like to recommend that the Python core developers start using >> a code review tool such as Rietveld or Reviewboard. I don't really >> care which tool we use (I'm sure there are plenty of pros and cons to >> each) but I do think we should get out of the stone age and start >> using a tool for the majority of our code reviews. > > Rambling thoughts about some of the things mentioned in this thread. > > I think hg-review looks interesting, though it may not (yet) have the > level of sophistication of Rietveld. (Public test instance at > http://review.stevelosh.com/.) > > It might be interesting to integrate Rietveld uploads in a Mercurial > extension, particularly if it gets integrated with mq somehow. That would be totally awesome! Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From albrecht.andi at googlemail.com Fri Oct 1 08:30:59 2010 From: albrecht.andi at googlemail.com (Andi Albrecht) Date: Fri, 01 Oct 2010 08:30:59 +0200 Subject: [Python-Dev] We should be using a tool for code reviews In-Reply-To: (Georg Brandl's message of "Fri, 01 Oct 2010 07:49:23 +0200") References: Message-ID: <878w2ijufw.fsf@gmail.com> Georg Brandl writes: > Am 30.09.2010 10:22, schrieb Dirkjan Ochtman: >> On Wed, Sep 29, 2010 at 20:32, Guido van Rossum wrote: >>> I would like to recommend that the Python core developers start using >>> a code review tool such as Rietveld or Reviewboard. I don't really >>> care which tool we use (I'm sure there are plenty of pros and cons to >>> each) but I do think we should get out of the stone age and start >>> using a tool for the majority of our code reviews. >> >> Rambling thoughts about some of the things mentioned in this thread. >> >> I think hg-review looks interesting, though it may not (yet) have the >> level of sophistication of Rietveld. (Public test instance at >> http://review.stevelosh.com/.) >> >> It might be interesting to integrate Rietveld uploads in a Mercurial >> extension, particularly if it gets integrated with mq somehow. > > That would be totally awesome! The Go (the language) project has a Mercurial extension that integrates Rietveld. It seems to provide a few commands for both directions: uploading changelists to a review server and for the reviewer downloading from the server and applying the changelists to a working copy. I never used this extension myself so I can't tell anything about the workflow introduced by these commands. The sources for this extension are here: http://code.google.com/p/go/source/browse/lib/codereview/codereview.py Andi > > Georg From ziade.tarek at gmail.com Fri Oct 1 08:59:09 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 1 Oct 2010 08:59:09 +0200 Subject: [Python-Dev] Celebrating issue #10000 In-Reply-To: <4CA52252.4060106@v.loewis.de> References: <4CA52252.4060106@v.loewis.de> Message-ID: On Fri, Oct 1, 2010 at 1:50 AM, "Martin v. L?wis" wrote: > Amaury just filed issue #10000 yesterday; as counting started > with 1000, we are now into 9000 roundup issues. > Happy birthday, bugs ! :) From barry at python.org Fri Oct 1 17:02:35 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 1 Oct 2010 11:02:35 -0400 Subject: [Python-Dev] Branching without losing your build products [was: We should be using a tool for code reviews] In-Reply-To: <8739sqbll7.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20100929204125.416e2013@pitrou.net> <1285792363.3194.41.camel@localhost.localdomain> <20100929173010.2acf0b3d@mission> <20100930000420.3b7d4999@pitrou.net> <20100929183347.0d95304e@mission> <87bp7fboak.fsf@uwakimon.sk.tsukuba.ac.jp> <8762xnbko2.fsf@uwakimon.sk.tsukuba.ac.jp> <20100930102755.43ab03d3@mission> <8739sqbll7.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101001110235.4a90f9f9@mission> On Oct 01, 2010, at 01:09 PM, Stephen J. Turnbull wrote: >Barry Warsaw writes: > > > I should note that I don't particularly like colocated/named > > branches. I personally much prefer separate directories for each > > feature or bug I'm working on. It helps me keep track of what I'm > > doing. I have a fast machine so recompiling all of Python is no > > big deal. > >Note that once you have a branch with all of Python built, for any of >hg, git, bzr you may prefer to clone the branch/repo with "cp -r" or >"rsync" rather than "$VCS clone". The reason for doing it with "$VCS >clone" is to get a pristine workspace without any cruft like editor >backups or build products. If you *want* a crufty workspace (and in >this case some people do), a recursive copy is a better tool. Yep, absolutely agree. I also use 'cp -a' or 'rsync -avz' when I want that crufty workspace transfered to another (local) machine. In general though, when I'm working on feature or bug branches, I want a pristine working directory. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Fri Oct 1 17:52:47 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 1 Oct 2010 11:52:47 -0400 Subject: [Python-Dev] Celebrating issue #10000 In-Reply-To: <4CA52252.4060106@v.loewis.de> References: <4CA52252.4060106@v.loewis.de> Message-ID: <20101001115247.6c8c8bfa@mission> On Oct 01, 2010, at 01:50 AM, Martin v. L?wis wrote: >Amaury just filed issue #10000 yesterday; as counting started >with 1000, we are now into 9000 roundup issues. > >I have become quite fond of roundup over the years, and would >like to thank Ka-Ping Yee, Richard Jones, and Erik Forsberg >for getting us here. > >There are many contributions to this infrastructure, both >from individuals and software projects, but I'd like to single >out two of them which have I also appreciate very much: >the folks at Upfront Hosting have helped a lot to keep the system >running, and the PostgreSQL database has really validated it's >own claim of being the world's most advanced open source >database. Agreed! Roundup has been a joy to use. Martin is to be profusely thanked too for all of his ongoing work on our python.org resources. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From status at bugs.python.org Fri Oct 1 18:12:01 2010 From: status at bugs.python.org (Python tracker) Date: Fri, 1 Oct 2010 18:12:01 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20101001161201.96A37780BC@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2010-09-24 - 2010-10-01) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues stats: open 2548 (+45) closed 19241 (+50) total 21789 (+67) Open issues with patches: 1072 Issues opened (45) ================== #9899: tkinter test_font fails on OS X with Aqua Tk http://bugs.python.org/issue9899 reopened by pitrou #9941: Unify trace and profile interfaces http://bugs.python.org/issue9941 opened by belopolsky #9942: Allow memory sections to be OS MERGEABLE http://bugs.python.org/issue9942 opened by hunteke #9943: TypeError message became less helpful in Python 2.7 http://bugs.python.org/issue9943 opened by gjb1002 #9948: findCaller is slow and loses case information on Windows http://bugs.python.org/issue9948 opened by aronacher #9949: os.path.realpath on Windows does not follow symbolic links http://bugs.python.org/issue9949 opened by stutzbach #9951: introduce bytes.hex method http://bugs.python.org/issue9951 opened by wiggin15 #9955: multiprocessing.Pipe segmentation fault when recv of unpicklab http://bugs.python.org/issue9955 opened by Zbynek.Winkler #9957: SpooledTemporayFile.truncate should take size parameter http://bugs.python.org/issue9957 opened by rfk #9960: test_structmembers fails on s390x (bigendian 64-bit): int/Py_s http://bugs.python.org/issue9960 opened by dmalcolm #9962: GzipFile doesn't have peek() http://bugs.python.org/issue9962 opened by pitrou #9964: Test failures with -OO http://bugs.python.org/issue9964 opened by michael.foord #9965: Loading malicious pickle may cause excessive memory usage http://bugs.python.org/issue9965 opened by alexandre.vassalotti #9967: encoded_word regular expression in email.header.decode_header( http://bugs.python.org/issue9967 opened by tkikuchi #9968: Let cgi.FieldStorage have named uploaded file http://bugs.python.org/issue9968 opened by phep #9969: tokenize: add support for tokenizing 'str' objects http://bugs.python.org/issue9969 opened by meador.inge #9971: Optimize BufferedReader.readinto http://bugs.python.org/issue9971 opened by stutzbach #9972: PyGILState_XXX missing in Python builds without threads http://bugs.python.org/issue9972 opened by dalcinl #9973: Sometimes buildbot fails to cleanup working copy http://bugs.python.org/issue9973 opened by ocean-city #9974: tokenizer.untokenize not invariant with line continuations http://bugs.python.org/issue9974 opened by Brian.Boss?? #9975: Incorrect use of flowinfo and scope_id in IPv6 sockaddr tuple http://bugs.python.org/issue9975 opened by vnebehaj #9976: Make TestCase._formatMessage public http://bugs.python.org/issue9976 opened by mattheww #9977: TestCase.assertItemsEqual's description of differences http://bugs.python.org/issue9977 opened by mattheww #9978: test_os failures on XP-4 buildbot http://bugs.python.org/issue9978 opened by pitrou #9980: str(float) failure http://bugs.python.org/issue9980 opened by Kiriakos.Vlahos #9981: let make_buildinfo use a temporary directory on windows http://bugs.python.org/issue9981 opened by krisvale #9985: difflib.SequenceMatcher has slightly buggy and undocumented ca http://bugs.python.org/issue9985 opened by marienz #9986: PDF files of python docs have text missing http://bugs.python.org/issue9986 opened by amberj #9988: test_warnings fails with PYTHONFSENCODING=latin-1 on UNIX/BSD http://bugs.python.org/issue9988 opened by haypo #9989: ctypes bitfield problem http://bugs.python.org/issue9989 opened by stutzbach #9990: PyMemoryView_FromObject alters the Py_buffer after calling PyO http://bugs.python.org/issue9990 opened by kermode #9991: xmlrpc client ssl check faulty http://bugs.python.org/issue9991 opened by jniehof #9992: Command line arguments are not correctly decodedif locale and http://bugs.python.org/issue9992 opened by haypo #9993: shutil.move fails on symlink source http://bugs.python.org/issue9993 opened by jniehof #9995: "setup.py register sdist upload" requires pass to be saved http://bugs.python.org/issue9995 opened by techtonik #9996: binascii should convert unicode strings http://bugs.python.org/issue9996 opened by wiggin15 #9997: function named 'top' gets unexpected namespace/scope behaviour http://bugs.python.org/issue9997 opened by iivvoo #9998: find_library should search LD_LIBRARY_PATH on linux http://bugs.python.org/issue9998 opened by jniehof #9999: test_shutil cross-file-system tests are fragile (may not test http://bugs.python.org/issue9999 opened by r.david.murray #10000: mark more tests as CPython specific http://bugs.python.org/issue10000 opened by amaury.forgeotdarc #10001: ~Py_buffer.obj field is undocumented, though not hidden http://bugs.python.org/issue10001 opened by kermode #10002: Installer doesn't install on Windows Server 2008 DataCenter R2 http://bugs.python.org/issue10002 opened by joblack #10003: signal.SIGBREAK regression on windows http://bugs.python.org/issue10003 opened by gz #10006: non-Pythonic fate of __abstractmethods__ http://bugs.python.org/issue10006 opened by Yaroslav.Halchenko #10007: Visual C++ cannot build _ssl and _hashlib if newer OpenSSL is http://bugs.python.org/issue10007 opened by ocean-city Most recent 15 issues with no replies (15) ========================================== #10006: non-Pythonic fate of __abstractmethods__ http://bugs.python.org/issue10006 #9998: find_library should search LD_LIBRARY_PATH on linux http://bugs.python.org/issue9998 #9995: "setup.py register sdist upload" requires pass to be saved http://bugs.python.org/issue9995 #9991: xmlrpc client ssl check faulty http://bugs.python.org/issue9991 #9990: PyMemoryView_FromObject alters the Py_buffer after calling PyO http://bugs.python.org/issue9990 #9977: TestCase.assertItemsEqual's description of differences http://bugs.python.org/issue9977 #9976: Make TestCase._formatMessage public http://bugs.python.org/issue9976 #9975: Incorrect use of flowinfo and scope_id in IPv6 sockaddr tuple http://bugs.python.org/issue9975 #9974: tokenizer.untokenize not invariant with line continuations http://bugs.python.org/issue9974 #9973: Sometimes buildbot fails to cleanup working copy http://bugs.python.org/issue9973 #9972: PyGILState_XXX missing in Python builds without threads http://bugs.python.org/issue9972 #9971: Optimize BufferedReader.readinto http://bugs.python.org/issue9971 #9967: encoded_word regular expression in email.header.decode_header( http://bugs.python.org/issue9967 #9960: test_structmembers fails on s390x (bigendian 64-bit): int/Py_s http://bugs.python.org/issue9960 #9955: multiprocessing.Pipe segmentation fault when recv of unpicklab http://bugs.python.org/issue9955 Most recent 15 issues waiting for review (15) ============================================= #10003: signal.SIGBREAK regression on windows http://bugs.python.org/issue10003 #9998: find_library should search LD_LIBRARY_PATH on linux http://bugs.python.org/issue9998 #9996: binascii should convert unicode strings http://bugs.python.org/issue9996 #9993: shutil.move fails on symlink source http://bugs.python.org/issue9993 #9992: Command line arguments are not correctly decodedif locale and http://bugs.python.org/issue9992 #9991: xmlrpc client ssl check faulty http://bugs.python.org/issue9991 #9985: difflib.SequenceMatcher has slightly buggy and undocumented ca http://bugs.python.org/issue9985 #9981: let make_buildinfo use a temporary directory on windows http://bugs.python.org/issue9981 #9978: test_os failures on XP-4 buildbot http://bugs.python.org/issue9978 #9975: Incorrect use of flowinfo and scope_id in IPv6 sockaddr tuple http://bugs.python.org/issue9975 #9973: Sometimes buildbot fails to cleanup working copy http://bugs.python.org/issue9973 #9967: encoded_word regular expression in email.header.decode_header( http://bugs.python.org/issue9967 #9964: Test failures with -OO http://bugs.python.org/issue9964 #9962: GzipFile doesn't have peek() http://bugs.python.org/issue9962 #9960: test_structmembers fails on s390x (bigendian 64-bit): int/Py_s http://bugs.python.org/issue9960 Top 10 most discussed issues (10) ================================= #9980: str(float) failure http://bugs.python.org/issue9980 35 msgs #9873: Allow bytes in some APIs that use string literals internally http://bugs.python.org/issue9873 12 msgs #9962: GzipFile doesn't have peek() http://bugs.python.org/issue9962 10 msgs #8445: buildbot: test_ttk_guionly failures (test_traversal, test_tab_ http://bugs.python.org/issue8445 9 msgs #8670: c_types.c_wchar should not assume that sizeof(wchar_t) == size http://bugs.python.org/issue8670 9 msgs #1589: New SSL module doesn't seem to verify hostname against commonN http://bugs.python.org/issue1589 8 msgs #3873: Unpickling is really slow http://bugs.python.org/issue3873 7 msgs #9784: _msi.c warnings under 64-bit Windows http://bugs.python.org/issue9784 7 msgs #9942: Allow memory sections to be OS MERGEABLE http://bugs.python.org/issue9942 7 msgs #4111: Add Systemtap/DTrace probes http://bugs.python.org/issue4111 6 msgs Issues closed (50) ================== #1491: BaseHTTPServer incorrectly implements response code 100 http://bugs.python.org/issue1491 closed by orsenthil #3612: Add some missing basic types in ctypes.wintypes http://bugs.python.org/issue3612 closed by ocean-city #3674: test_dbm_ndbm skip is unexpected on win32? http://bugs.python.org/issue3674 closed by ocean-city #4684: sys.exit() exits program when non-daemonic threads are still r http://bugs.python.org/issue4684 closed by twouters #6608: asctime does not check its input http://bugs.python.org/issue6608 closed by belopolsky #7110: Output test failures on stderr in regrtest.py http://bugs.python.org/issue7110 closed by r.david.murray #7346: Redirected stdout fires [Errno 9] http://bugs.python.org/issue7346 closed by amaury.forgeotdarc #7397: __import__ docs should reference importlib.import_module http://bugs.python.org/issue7397 closed by brett.cannon #8274: test_run failing http://bugs.python.org/issue8274 closed by eric.araujo #8521: Allow some winreg functions to accept named arguments http://bugs.python.org/issue8521 closed by brian.curtin #8831: recv and recvfrom on UDP socket do not return/throw exception http://bugs.python.org/issue8831 closed by loewis #9082: warnings.filterwarnings doesn't work with -O http://bugs.python.org/issue9082 closed by brett.cannon #9090: Error code 10035 calling socket.recv() on a socket with a time http://bugs.python.org/issue9090 closed by pitrou #9360: nntplib cleanup http://bugs.python.org/issue9360 closed by pitrou #9562: Slightly misleading wording in documentation of dict.update http://bugs.python.org/issue9562 closed by georg.brandl #9568: test_urllib2_localnet fails on OS X 10.3 http://bugs.python.org/issue9568 closed by ronaldoussoren #9594: typo on Mac/Makefile.in? s/pythonw/python/ http://bugs.python.org/issue9594 closed by ronaldoussoren #9602: PyObject_AsCharBuffer() should only accept read-only objects http://bugs.python.org/issue9602 closed by pitrou #9628: runtests.sh -x doesn't work with more than two args (sed error http://bugs.python.org/issue9628 closed by r.david.murray #9630: Reencode filenames when setting the filesystem encoding http://bugs.python.org/issue9630 closed by haypo #9701: Update ZSH profile on Mac OS installation http://bugs.python.org/issue9701 closed by ronaldoussoren #9841: sysconfig and distutils.sysconfig differ in subtle ways http://bugs.python.org/issue9841 closed by eric.araujo #9869: long_subtype_new segfault in pure-Python code http://bugs.python.org/issue9869 closed by mark.dickinson #9910: Add Py_SetPath API for embedding python http://bugs.python.org/issue9910 closed by krisvale #9934: Python Docs Typo http://bugs.python.org/issue9934 closed by brian.curtin #9940: Strange error reporting with "with" statement http://bugs.python.org/issue9940 closed by benjamin.peterson #9944: Typo in doc for itertools recipe of consume http://bugs.python.org/issue9944 closed by georg.brandl #9945: Improper locking in logging http://bugs.python.org/issue9945 closed by vinay.sajip #9946: lock use in logging http://bugs.python.org/issue9946 closed by georg.brandl #9947: Weird locking in logging config system http://bugs.python.org/issue9947 closed by vinay.sajip #9950: socket.sendall() crash when receiving a signal http://bugs.python.org/issue9950 closed by pitrou #9952: Martin Rinehart wants to stay in touch on LinkedIn http://bugs.python.org/issue9952 closed by loewis #9953: 2 scripts running from crontab simultaneously reference the sa http://bugs.python.org/issue9953 closed by exarkun #9954: include sqlite3.exe in PythonXX/Scripts http://bugs.python.org/issue9954 closed by loewis #9956: memoryview type documentation lacks versionadded http://bugs.python.org/issue9956 closed by benjamin.peterson #9958: (c)elementTree missing children http://bugs.python.org/issue9958 closed by amaury.forgeotdarc #9959: int(math.log(4,2)) == 1 instead of 2 http://bugs.python.org/issue9959 closed by mark.dickinson #9961: Identity Crisis! variable assignment using strftime fails comp http://bugs.python.org/issue9961 closed by exarkun #9963: test_sysconfig when LDFLAGS defined in the user's environment http://bugs.python.org/issue9963 closed by pitrou #9966: platform : a boolean to know easily when a system is posix http://bugs.python.org/issue9966 closed by brian.curtin #9970: PyMemoryView_FromBuffer is not documented http://bugs.python.org/issue9970 closed by pitrou #9979: Create PyUnicode_AsWideCharString() function http://bugs.python.org/issue9979 closed by haypo #9982: Pointer problem in initializing array of arrays http://bugs.python.org/issue9982 closed by pitrou #9983: please add a large NOTE explaining that urllib does not perfor http://bugs.python.org/issue9983 closed by pitrou #9984: please add a large NOTE explaining that urllib2 does not perfo http://bugs.python.org/issue9984 closed by pitrou #9987: usenetrc option broken http://bugs.python.org/issue9987 closed by pitrou #9994: Python becoming orphaned over ssh http://bugs.python.org/issue9994 closed by gregory.p.smith #10004: email/quoprimime.py Exception (with patch) http://bugs.python.org/issue10004 closed by r.david.murray #10005: Postgresql calls undefined method in __str__ http://bugs.python.org/issue10005 closed by r.david.murray #1595365: Urllib2 user-agent header added by an opener is "frozen" http://bugs.python.org/issue1595365 closed by orsenthil From barry at python.org Fri Oct 1 18:31:38 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 1 Oct 2010 12:31:38 -0400 Subject: [Python-Dev] We should be using a tool for code reviews In-Reply-To: References: <20100930145218.2022.353483588.divmod.xquotient.55@localhost.localdomain> Message-ID: <20101001123138.6d1a8131@mission> On Sep 30, 2010, at 01:46 PM, Brett Cannon wrote: >Once we have a good workflow in place we would have to start shifting >our development culture towards requiring a review of code no matter >who the author is (which I support doing). I should note one other thing, in reference to my previous posting about reviews. Launchpad does have a backdoor for getting changes in without formal review. It's called "rubber stamping" and shows up in commit messages, e.g.: $VCS commit -m"[rs=me] Fix trivial misspelling in comment" You can also get a rubber stamp from a reviewer: Alice: can you review my branch that fixes all incorrect uses of "it's"? Bob: If that's your only change, I trust you. rs=me Alice: $VCS commit -m"[rs=bob] The Grammar Nanny strikes again" Usually rubber stamps are reserved for cases where the fix really is trivial, or a change is large but mechanical, or when no reviewer can be found for a time-sensitive fix (very rare). You at least need to record the rubber stamp in the commit message, and be prepared to defend it if it trips up someone's post-commit eyeball filter. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From fdrake at acm.org Fri Oct 1 18:48:36 2010 From: fdrake at acm.org (Fred Drake) Date: Fri, 1 Oct 2010 12:48:36 -0400 Subject: [Python-Dev] We should be using a tool for code reviews In-Reply-To: <20101001123138.6d1a8131@mission> References: <20100930145218.2022.353483588.divmod.xquotient.55@localhost.localdomain> <20101001123138.6d1a8131@mission> Message-ID: On Fri, Oct 1, 2010 at 12:31 PM, Barry Warsaw wrote: > I should note one other thing, in reference to my previous posting about > reviews. ?Launchpad does have a backdoor for getting changes in without > formal review. ?It's called "rubber stamping" and shows up in commit messages, This makes a lot of sense; having to branch & get spelling corrections in docs or comments would be a *huge* pain, and not provide value. ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From janssen at parc.com Fri Oct 1 19:42:25 2010 From: janssen at parc.com (Bill Janssen) Date: Fri, 1 Oct 2010 10:42:25 PDT Subject: [Python-Dev] PPC Leopard build slave going down for an upgrade Message-ID: <23316.1285954945@parc.com> I'm replacing the PPC Leopard build slave with a dual 2GHz G5 machine... Bill From wiggin15 at gmail.com Fri Oct 1 21:17:22 2010 From: wiggin15 at gmail.com (Arnon Yaari) Date: Fri, 1 Oct 2010 21:17:22 +0200 Subject: [Python-Dev] Problems with hex-conversion functions In-Reply-To: References: <8449046c0909051426g2eaa3f94s4c5842c25af17abf@mail.gmail.com> <4AA331A9.70209@gmail.com> Message-ID: Hello again. I submitted two patches to resolve the issues from my first post. Patch 9951 - implement bytes.hex (http://bugs.python.org/issue9951) Patch 9996 - fix input and output of binascii functions ( http://bugs.python.org/issue9996) Fix #1 - patch 9951 implements bytes.hex Fix #2 - this is not fixed for now, no deprecation Fix #3 - this is not fixed for now. I will probably submit another patch if patch 9996 is accepted (create shared conversion functions to be used by both binascii and bytes, maybe) Fix #4 - patch 9996 makes binascii behave correctly in this conversion Fix #5 - same as #4 (strict input and output) As you can see, patch 9996 was rejected and I was referred to this mailing list to continue the discussion. I would like to hear your thoughts about the backward compatibility issue in patch 9996, and getting patch 9951 commited. Thanks. On Fri, Sep 24, 2010 at 12:04 AM, Nick Coghlan wrote: > On Thu, Sep 23, 2010 at 7:31 PM, Ender Wiggin wrote: > > Sorry for the late reply. I would really like to see this fixed. > > > >>> Or we [...] deprecate binascii.(un)hexlify(). > > ... > >>> binascii is the legacy approach here, so if anything was to go, those > >>> functions would be it > > ... > > > > I'm not entirely convinced binascii is the legacy approach. What makes > this > > module "legacy"? > > Because the binascii functions predate the bytes type, and we added > the bytes methods knowing full well that the hexlify/unhexlify > functions already existed. > > > On the contrary, I'm pretty sure modularity is better than sticking all > the > > functionality in the core. > > As was written in this issue: > > http://psf.upfronthosting.co.za/roundup/tracker/issue3532 > > "If you wanted to produce base-85 (say), then you can extend the > > functionality of bytes by providing a > > function that does that, whereas you can't extend the existing bytes > type." > > This example shows that "hex" is actually getting a special treatment by > > having builtin methods associated > > with the bytes type. Why don't we add ".base64" methods? Or even ".zlib"? > > After all, these options were present > > in Python 2.x using the "encode" method of string. In my opinion, having > > modules to deal with these types of > > conversions is better, and this is why I suggested sticking to binascii. > > This *is* a matter of opinion, but python-dev's collective opinion was > already expressed in the decision to include these methods in the > bytes API. > > Base 16 *is* given special treatment by many parts of Python, > precisely because it *is* special: it's the most convenient way to > express binary numbers in a vaguely human readable format. > > No other coding even comes close to that level of importance in > computer science. > > > If no one else is willing to do it (that would be a > > little disappoiting) > > Why would it be disappointing? While it's untidy, nothing's actually > broken and there are ways for programmers to do everything they want > to do. I (and many others here) already have a pretty long list of > "things I'd like to improve/fix but haven't got around to yet", so it > isn't uncommon for things to have to wait awhile before someone looks > at them. > > As Terry said though, there *are* ways to expedite that process (In > this case, providing a patch that adds a .hex method in accordance > with PEP 358, or, as a more ambitious, extensible alternative, > consider updating the hex builtin to support the PEP 3118 API, which > would allow it to automatically provide a hex dump of any object that > exposes a view of a contiguous sequence of data bytes). > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > -------------- next part -------------- An HTML attachment was scrubbed... URL: From slobberchops at gmail.com Fri Oct 1 22:51:20 2010 From: slobberchops at gmail.com (Rafe Kaplan) Date: Fri, 1 Oct 2010 13:51:20 -0700 Subject: [Python-Dev] We should be using a tool for code reviews Message-ID: Sorry I am not replying to the original thread with the same name. I've been working on an HG extension for helping with Rietveld code reviews. The main reason is so that Rietveld can remember state (what issue id is being used) and handle the uploading and downloading to Rietveld. The version I wrote is a little primitive and was written before I fully understood how to use HG. At any rate, I would be interested in participating in creating an hg-codereview extension. -- - Rafe -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Sat Oct 2 01:06:37 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 1 Oct 2010 18:06:37 -0500 Subject: [Python-Dev] [Python-checkins] r85152 - tracker/instances/python-dev/config.ini.template In-Reply-To: <20101001225727.01CC7EE9EC@mail.python.org> References: <20101001225727.01CC7EE9EC@mail.python.org> Message-ID: 2010/10/1 martin.v.loewis : > Author: martin.v.loewis > Date: Sat Oct ?2 00:57:26 2010 > New Revision: 85152 > > Log: > Add django settings to template. > > > Modified: > ? tracker/instances/python-dev/config.ini.template > > Modified: tracker/instances/python-dev/config.ini.template > ============================================================================== > --- tracker/instances/python-dev/config.ini.template ? ?(original) > +++ tracker/instances/python-dev/config.ini.template ? ?Sat Oct ?2 00:57:26 2010 > @@ -381,3 +381,7 @@ > ?# each recipient as a CC address. > ?# Default: single > ?email_sending = multiple > + > +[django] > +# generate on a per-instance basis > +secret_key = 'el at 4s$*(idwm5-87teftxlksckmy8$tyo7(tm!n-5x)zeuheex' I assume the secrecy of this is rather irrelevant? -- Regards, Benjamin From barry at python.org Sat Oct 2 02:06:57 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 1 Oct 2010 20:06:57 -0400 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types Message-ID: <20101001200657.472c2f60@mission> I am continuing my quest to be able to install multiple versions and builds of Python simultaneously, so that they all nicely coexist. E.g. you could have a "normal" build of Python 3.2 and a --with-wide-unicode --with-pydebug build both installed and all packages with extensions would get built and loaded for the proper binary, etc. (Caveat: I'm only intending to do the following work for configure-based builds.) I'm tracking most of this work in this bug: http://bugs.python.org/issue9807 I have a Bazaar branch from py3k which is pretty close to working, and I've put my work so far up for review on Rietveld (see the bug comments for details). I've hit a snag though and I'm not sure what the best way out is. I'm hoping one of you have some better ideas. First, let me set up the scenario. Let's say I build py3k like this: % ./configure --prefix=/tmp/python && make altinstall % make distclean % ./configure --prefix=/tmp/python --with-wide-unicode --with-pydebug \ && make altinstall With my branch, you'll end up with this in /tmp/python: bin/python3.2m - the normal build binary bin/python3.2dmu - the wide+pydebug build binary bin/python3.2m-config bin/python3.2dmu-config ... lib/libpython3.2.so.1.0.m lib/libpython3.2.so.1.0.dmum lib/python3.2m/config/Makefile lib/python3.2dmu/config/Makefile ... include/python3.2m/pyconfig.h include/python3.2dmu/pyconfig.h and of course lots more. This essentially segregates the build-specific bits and pieces according to the $SOABI_QUALIFIER flags (build-flags) calculated in the configure script. This, along with the work for PEP 3149 gives us very nice coexistence without too much duplication. I.e. lib/python3.2/site-packages/foo-0.0-py3.2-plat.egg can contain a shared .py file for all the builds, and .so files with PEP 3149 tags that distinguish between the various build flags. Ah, but now here's the problem I'm stuck on. Let's say I build the foo module, which has an _foo extension for both the 3.2m and 3.2dmu builds. Everything gets installed correctly, and you'll see: lib/python3.2/site-packages/foo-...egg/_foo.cpython-32m.so lib/python3.2/site-packages/foo-...egg/_foo.cpython-32dmu.so If you now run bin/python3.2dmu and try to import _foo, the right thing happens (foreshadowing) assuming that the directory listing for the foo.egg is alphabetical. But what happens if you run bin/python3.2m and import _foo? Yeah, well, even though there's a _foo.cpython-32m.so sitting right there waiting for you to load it, what actually happens is that Python tries to dynload _foo.cpython-32dmu.so and crashes miserably. Why did it do that? The reason is that the import.c logic that uses the struct filedescr tables built from _PyImport_DynLoadFiletab are just not smart enough to handle this case. All it knows about are suffix, and for backwards compatibility, we have dynload_shlib.c matching both .SOABI.so *and* bare .so. So when it's searching the directories for .cpython-32m.so files, it finds the .cpython-32dmu.so first based on the bare .so match. I can think of a couple of ways out, none of which are totally satisfying. Probably the easiest out is to change the PEP 3149 naming so that the files don't end in ".so". E.g. use this instead: foo.cpython-32dmu-so foo.cpython-32m-so or similar. I think that'd be a fairly straightforward change, but it might break some useful assumptions (we'd still fall back to .so of course). Other ideas: - make import.c smarter so that you can match against other than just the suffix. probably a lot of work. - extend struct filedescr to add callbacks into dynload_shlib.c for file matching. this callback would reject .SOABI.so files where the SOABI doesn't match. again a lot of work and more expensive due to tests more complicated than simple strcmps. - scan filesystem for .SOABI.so before .so. you lose locality of reference and probably has lots of other really nasty semantic disturbances. ;) - remove match on bare .so. not good for non-distutils based build processes. Do you have any other ideas? If not, what would your preference from the above be? Right now I think I'd go with changing the PEP 3149 naming scheme to not end in .so (i.e. option #1). Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From solipsis at pitrou.net Sat Oct 2 02:24:18 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 2 Oct 2010 02:24:18 +0200 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types References: <20101001200657.472c2f60@mission> Message-ID: <20101002022418.64ef3ba0@pitrou.net> On Fri, 1 Oct 2010 20:06:57 -0400 Barry Warsaw wrote: > > With my branch, you'll end up with this in /tmp/python: > > bin/python3.2m - the normal build binary > bin/python3.2dmu - the wide+pydebug build binary > bin/python3.2m-config > bin/python3.2dmu-config Do users really want to see such idiosyncratic suffixes? > ... > lib/libpython3.2.so.1.0.m > lib/libpython3.2.so.1.0.dmum Ditto here. This seems to break well-known conventions. If I look at /usr/lib{,64} on my machine, I can't see a single shared libary file that ends neither in ".so" nor ".so.". Before trying to find a solution to your problem, I think it would be nice to get a consensus that this is really a desired feature. Regards Antoine. From barry at python.org Sat Oct 2 02:29:57 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 1 Oct 2010 20:29:57 -0400 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101002022418.64ef3ba0@pitrou.net> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> Message-ID: <20101001202957.6e5bb292@mission> On Oct 02, 2010, at 02:24 AM, Antoine Pitrou wrote: >On Fri, 1 Oct 2010 20:06:57 -0400 >Barry Warsaw wrote: >> >> With my branch, you'll end up with this in /tmp/python: >> >> bin/python3.2m - the normal build binary >> bin/python3.2dmu - the wide+pydebug build binary >> bin/python3.2m-config >> bin/python3.2dmu-config > >Do users really want to see such idiosyncratic suffixes? Do note that "make install" will symlink all that away, so you'll still end up with a single "python3" and "python3-config" pointing to the last one you installed. >> ... >> lib/libpython3.2.so.1.0.m >> lib/libpython3.2.so.1.0.dmum > >Ditto here. This seems to break well-known conventions. >If I look at /usr/lib{,64} on my machine, I can't see a single >shared libary file that ends neither in ".so" nor ".so.". > >Before trying to find a solution to your problem, I think it would be >nice to get a consensus that this is really a desired feature. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From benjamin at python.org Sat Oct 2 02:36:25 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 1 Oct 2010 19:36:25 -0500 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101001200657.472c2f60@mission> References: <20101001200657.472c2f60@mission> Message-ID: 2010/10/1 Barry Warsaw : > I can think of a couple of ways out, none of which are totally satisfying. > Probably the easiest out is to change the PEP 3149 naming so that the files > don't end in ".so". ?E.g. use this instead: > > ? ?foo.cpython-32dmu-so > ? ?foo.cpython-32m-so -1 Doesn't that break not only Python's convention for extensions on shared modules but also any *nix shared object? > > or similar. ?I think that'd be a fairly straightforward change, but it might > break some useful assumptions (we'd still fall back to .so of course). > > Other ideas: > > - make import.c smarter so that you can match against other than just the > ?suffix. ?probably a lot of work. Although it would be more work, I think this is the most "correct" option. -- Regards, Benjamin From ncoghlan at gmail.com Sat Oct 2 08:11:25 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 2 Oct 2010 16:11:25 +1000 Subject: [Python-Dev] We should be using a tool for code reviews In-Reply-To: <20101001123138.6d1a8131@mission> References: <20100930145218.2022.353483588.divmod.xquotient.55@localhost.localdomain> <20101001123138.6d1a8131@mission> Message-ID: On Sat, Oct 2, 2010 at 2:31 AM, Barry Warsaw wrote: > Usually rubber stamps are reserved for cases where the fix really is trivial, > or a change is large but mechanical, or when no reviewer can be found for a > time-sensitive fix (very rare). ?You at least need to record the rubber stamp > in the commit message, and be prepared to defend it if it trips up someone's > post-commit eyeball filter. A system like that, which still trusts committers to make the call that rubber stamping is appropriate, sounds significantly more workable to me than one which required review even for trivial changes. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sat Oct 2 08:36:21 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 2 Oct 2010 16:36:21 +1000 Subject: [Python-Dev] Problems with hex-conversion functions In-Reply-To: References: <8449046c0909051426g2eaa3f94s4c5842c25af17abf@mail.gmail.com> <4AA331A9.70209@gmail.com> Message-ID: On Sat, Oct 2, 2010 at 5:17 AM, Arnon Yaari wrote: > Hello again. I submitted two patches to resolve the issues from my first > post. > > Patch 9951 - implement bytes.hex (http://bugs.python.org/issue9951) > Patch 9996 - fix input and output of binascii functions > (http://bugs.python.org/issue9996) > > Fix #1 - patch 9951 implements bytes.hex > Fix #2 - this is not fixed for now, no deprecation > Fix #3 - this is not fixed for now. I will probably submit another patch if > patch 9996 is accepted (create shared conversion functions to be used by > both binascii and bytes, maybe) > Fix #4 - patch 9996 makes binascii behave correctly in this conversion > Fix #5 - same as #4 (strict input and output) > > As you can see, patch 9996 was rejected and I was referred to this mailing > list to continue the discussion. I actually agree with that rejection. You appear to be thinking of hex coding solely as a data display format, when it is also used fairly often as a data interchange format (usually embedded inside a larger formatting scheme rather than standalone). For data interchange, you want the hex values as ASCII-encoded bytes, for display to the user, you want it as a string. The conversion of the binascii API to Py3k took a data interchange view of the world, bytes.fromhex is more user I/O oriented. > I would like to hear your thoughts about the backward compatibility issue in > patch 9996, and getting patch 9951 commited. Thanks. The 9951 patch looks pretty good on a quick read through. I put some specific feedback on the tracker. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sat Oct 2 08:43:04 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 2 Oct 2010 16:43:04 +1000 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: References: <20101001200657.472c2f60@mission> Message-ID: On Sat, Oct 2, 2010 at 10:36 AM, Benjamin Peterson wrote: > 2010/10/1 Barry Warsaw : >> I can think of a couple of ways out, none of which are totally satisfying. >> Probably the easiest out is to change the PEP 3149 naming so that the files >> don't end in ".so". ?E.g. use this instead: >> >> ? ?foo.cpython-32dmu-so >> ? ?foo.cpython-32m-so > > -1 Doesn't that break not only Python's convention for extensions on > shared modules but also any *nix shared object? > >> >> or similar. ?I think that'd be a fairly straightforward change, but it might >> break some useful assumptions (we'd still fall back to .so of course). >> >> Other ideas: >> >> - make import.c smarter so that you can match against other than just the >> ?suffix. ?probably a lot of work. > > Although it would be more work, I think this is the most "correct" option. I agree with Benjamin here - making import.c handle the situation properly seems like a much better option (and import.c isn't *quite* as ugly as it was before Victor started cleaning it up to handle Unicode paths properly). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sat Oct 2 08:50:28 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 2 Oct 2010 16:50:28 +1000 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101002022418.64ef3ba0@pitrou.net> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> Message-ID: On Sat, Oct 2, 2010 at 10:24 AM, Antoine Pitrou wrote: > On Fri, 1 Oct 2010 20:06:57 -0400 > Barry Warsaw wrote: >> >> With my branch, you'll end up with this in /tmp/python: >> >> ? ? bin/python3.2m ? - the normal build binary >> ? ? bin/python3.2dmu - the wide+pydebug build binary >> ? ? bin/python3.2m-config >> ? ? bin/python3.2dmu-config > > Do users really want to see such idiosyncratic suffixes? Ordinary users won't be building Python from source. Developers won't care so long as we clearly document the sundry suffixes and describe them in the README (or in a PEP, with a pointer from the README). >> ? ? ... >> ? ? lib/libpython3.2.so.1.0.m >> ? ? lib/libpython3.2.so.1.0.dmum > > Ditto here. This seems to break well-known conventions. > If I look at /usr/lib{,64} on my machine, I can't see a single > shared libary file that ends neither in ".so" nor ".so.". Having some characters on the end to flag different kinds of custom build seems like it fits within the .so naming conventions I'm aware of, but I'm sure the *nix packaging folks will pipe up if Barry starts wandering too far afield in this area. > Before trying to find a solution to your problem, I think it would be > nice to get a consensus that this is really a desired feature. Having multiple parallel "altinstall" installations be genuinely non-interfering out of the box certainly seems like a desirable feature to me. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From martin at v.loewis.de Sat Oct 2 09:44:16 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 02 Oct 2010 09:44:16 +0200 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> Message-ID: <4CA6E2D0.9010407@v.loewis.de> >>> With my branch, you'll end up with this in /tmp/python: >>> >>> bin/python3.2m - the normal build binary >>> bin/python3.2dmu - the wide+pydebug build binary >>> bin/python3.2m-config >>> bin/python3.2dmu-config >> >> Do users really want to see such idiosyncratic suffixes? > > Ordinary users won't be building Python from source. Developers won't > care so long as we clearly document the sundry suffixes and describe > them in the README (or in a PEP, with a pointer from the README). I think this is not true. Developers *will* care, and they will cry foul very loudly, asking what nonsense this is. Antoine is proof of that: he is a developer, and he understands the motivation well, but it still goes against his notions of beauty (channeling him here). > Having multiple parallel "altinstall" installations be genuinely > non-interfering out of the box certainly seems like a desirable > feature to me. I think this should not use automatically generated suffixes, though. Perhaps I want an altinstall that is in some kind restrict? Or one where user "peter" has write access into site-packages? I could accept that a suffix is parameter to configure (or some such), and then gets used throughout. By default, Python will not add a suffix. However, I still wonder why people couldn't just install Python in a different prefix if they want separate installations. Regards, Martin From g.brandl at gmx.net Sat Oct 2 12:36:21 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 02 Oct 2010 10:36:21 +0000 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101001200657.472c2f60@mission> References: <20101001200657.472c2f60@mission> Message-ID: Am 02.10.2010 00:06, schrieb Barry Warsaw: > Let's say I build the foo module, which has an _foo extension for both the > 3.2m and 3.2dmu builds. Everything gets installed correctly, and you'll see: > > lib/python3.2/site-packages/foo-...egg/_foo.cpython-32m.so > lib/python3.2/site-packages/foo-...egg/_foo.cpython-32dmu.so > > If you now run bin/python3.2dmu and try to import _foo, the right thing > happens (foreshadowing) assuming that the directory listing for the foo.egg is > alphabetical. > > But what happens if you run bin/python3.2m and import _foo? Yeah, well, even > though there's a _foo.cpython-32m.so sitting right there waiting for you to > load it, what actually happens is that Python tries to dynload > _foo.cpython-32dmu.so and crashes miserably. Why did it do that? > > The reason is that the import.c logic that uses the struct filedescr tables > built from _PyImport_DynLoadFiletab are just not smart enough to handle this > case. All it knows about are suffix, and for backwards compatibility, we have > dynload_shlib.c matching both .SOABI.so *and* bare .so. So when it's > searching the directories for .cpython-32m.so files, it finds the > ..cpython-32dmu.so first based on the bare .so match. I don't understand -- wouldn't "foo.sometag.so" (where sometag is not SOABI) only be found a match for a suffix of ".so" if the module name requested is "foo.sometag"? (And if not, isn't implementing that the easiest solution?) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Sat Oct 2 13:40:51 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 2 Oct 2010 13:40:51 +0200 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <4CA6E2D0.9010407@v.loewis.de> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> Message-ID: <20101002134051.5a7902b1@pitrou.net> On Sat, 02 Oct 2010 09:44:16 +0200 "Martin v. L?wis" wrote: > > I could accept that a suffix is parameter to configure (or some such), > and then gets used throughout. By default, Python will not add a suffix. > However, I still wonder why people couldn't just install Python in a > different prefix if they want separate installations. Indeed. It might make sense to inquire what other packages do here. Apparently they don't do anything special, and they let users install in different prefixes if necessary. Besides, mingling different installations together makes uninstalling much more difficult. Regards Antoine. From janssen at parc.com Sat Oct 2 20:21:50 2010 From: janssen at parc.com (Bill Janssen) Date: Sat, 2 Oct 2010 11:21:50 PDT Subject: [Python-Dev] the "PPC Leopard" buildbot is back online Message-ID: <41722.1286043710@parc.com> I've finished upgrading the PPC Leopard builder to a dual 2 GHz G5 machine. Tests should run a bit faster now that the eMac is out of the loop :-). I'm looking for a faster machine for the PPC Tiger buildbot. Bill From janssen at parc.com Sat Oct 2 20:50:05 2010 From: janssen at parc.com (Bill Janssen) Date: Sat, 2 Oct 2010 11:50:05 PDT Subject: [Python-Dev] sad state of OS X Python testing... Message-ID: <42516.1286045405@parc.com> Folks, I was looking at the buildbots again. Do you realize that we have no OS X Snow Leopard buildbot? No Intel Leopard buildbot? Well over half of Mac users are using Snow Leopard, and we're not testing on that platform. In fact, the only Intel OS X machine we're testing on is a Core Duo, not a Core 2 Duo, and that's running Tiger. Surely someone could volunteer an old Intel Mac to run some tests? (Actually, two someones -- we'd need separate machines for Leopard and Snow Leopard.) I'd be happy to stick it in a server room at PARC and keep an eye on it. (Right now I've got a rack full of old eMacs and a G5 running PPC buildbots.) Perhaps we could get Apple to contribute some "seconds"? I believe donations to the PSF are deductible. Bill From martin at v.loewis.de Sat Oct 2 22:03:57 2010 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 02 Oct 2010 22:03:57 +0200 Subject: [Python-Dev] Rietveld integration into Roundup Message-ID: <4CA7902D.6060504@v.loewis.de> Following up to the recent thread, I have now integrated Rietveld into Roundup. This is a rough draft still, and highly experimental. Please try it out, but expect that it may be necessary to discard all data (including comments) to start over (of course, I'll try to avoid having to do so). The Rietveld integration currently works this way: - the installation lives at http://bugs.python.org/review/ - for each roundup user, a rietveld user is created; log into roundup to get access to rietveld - for each roundup issue, a rietveld issue is created with the same issue id. Please don't create additional Rietveld issues. - regularly (currently every hour), each patch is considered. Patches are skipped if: * they have been added to Rietveld already * they have no clear base version (i.e. they don't originate from svn diff) * they belong to no or a closed issue * they apply to a patch that is not currently supported (only trunk, 2.6, 2.7, 3.1, py3k are currently supported) - for each such patch, a patchset is created - if that is successful, a "review" link is added to roundup Feel free to discuss this here; bug reports and feature requests should go to the meta tracker. Contributions are welcome; I won't be able to work on this much for the next four days. Regards, Martin From martin at v.loewis.de Sat Oct 2 22:08:09 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 02 Oct 2010 22:08:09 +0200 Subject: [Python-Dev] sad state of OS X Python testing... In-Reply-To: <42516.1286045405@parc.com> References: <42516.1286045405@parc.com> Message-ID: <4CA79129.8050900@v.loewis.de> > Surely someone could volunteer an old Intel Mac to run some tests? > (Actually, two someones -- we'd need separate machines for Leopard and > Snow Leopard.) I'd be happy to stick it in a server room at PARC and > keep an eye on it. (Right now I've got a rack full of old eMacs and a > G5 running PPC buildbots.) Perhaps we could get Apple to contribute > some "seconds"? I believe donations to the PSF are deductible. The issue isn't really to get the hardware. The issue is to find somebody to volunteer running a build slave. Regards, Martin From jcea at jcea.es Sat Oct 2 22:14:28 2010 From: jcea at jcea.es (Jesus Cea) Date: Sat, 02 Oct 2010 22:14:28 +0200 Subject: [Python-Dev] Catching exceptions and convert them to warnings in C level code (was: Re: Pronouncement needed in issue9675) In-Reply-To: <20100928180013.0077ba38@pitrou.net> References: <4CA15678.3070004@jcea.es> <20100928180013.0077ba38@pitrou.net> Message-ID: <4CA792A4.3030806@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 28/09/10 18:00, Antoine Pitrou wrote: > By "fails" you mean "crashes the interpreter". > While the deprecation warning can be discussed, bsddb shouldn't crash > when PyCObject_FromVoidPtr() returns NULL, but instead bail out > cleanly (or perhaps even ignore the error and simply display a warning > that the C API won't be available). I plan to deploy two patches for this issue. One, bsddb surviving the warning->error flag. The other, changing the deprecation warning in "CObject" to a py3k warning. While doing this, I think would be useful if bsddb could intercept the exception and provide sensible feedback without raising an exception at import time. I have checked . "PyErr_Fetch" is a bit overkill, it seems. My idea is to convert the CObject exception in bsddb to print a warning like "CObject creation failed. The bsddb.cAPI will be not available. Reason is: XXXX". Is there a "canonical way" of doing this?. Sorry if the answer is trivial... - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBTKeSpJlgi5GaxT1NAQIMkgP9FCFewuhLHzExwi+aMSuevb56aoFOrBSE q/XKN+9JxE59n2BwxgaJtUBrBdUraLe17SswgVgjxzReHKI+eNgC1fTwvDV+KqGa 9AMvlNGBfNCLP/TcSRzIGc9w0MsZaS8MvBKJWw+4hINclcp4a/AgxkCxc9sbm/dM ATijPsG0RPY= =qAIG -----END PGP SIGNATURE----- From alan.mcintyre at gmail.com Sat Oct 2 22:16:50 2010 From: alan.mcintyre at gmail.com (Alan McIntyre) Date: Sat, 2 Oct 2010 13:16:50 -0700 Subject: [Python-Dev] sad state of OS X Python testing... In-Reply-To: <4CA79129.8050900@v.loewis.de> References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> Message-ID: I have an Intel Core 2 Duo Mac that I was going to convert into a home file server in the next couple of weeks, and I'd be glad to set it up as a build slave as well. I don't remember what version of OS X it has on it (it's still packed up in the box), but it certainly won't be the latest one. On Sat, Oct 2, 2010 at 1:08 PM, "Martin v. L?wis" wrote: >> Surely someone could volunteer an old Intel Mac to run some tests? >> (Actually, two someones -- we'd need separate machines for Leopard and >> Snow Leopard.) ?I'd be happy to stick it in a server room at PARC and >> keep an eye on it. ?(Right now I've got a rack full of old eMacs and a >> G5 running PPC buildbots.) ?Perhaps we could get Apple to contribute >> some "seconds"? ?I believe donations to the PSF are deductible. > > The issue isn't really to get the hardware. The issue is to find > somebody to volunteer running a build slave. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/alan.mcintyre%40gmail.com > From alan.mcintyre at gmail.com Sat Oct 2 22:25:44 2010 From: alan.mcintyre at gmail.com (Alan McIntyre) Date: Sat, 2 Oct 2010 13:25:44 -0700 Subject: [Python-Dev] sad state of OS X Python testing... In-Reply-To: References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> Message-ID: On Sat, Oct 2, 2010 at 1:16 PM, Alan McIntyre wrote: > I have an Intel Core 2 Duo Mac that I was going to convert into a home > file server in the next couple of weeks, and I'd be glad to set it up > as a build slave as well. ?I don't remember what version of OS X it > has on it (it's still packed up in the box), but it certainly won't be > the latest one. I just looked and the install disc says it's 10.4.7. From g.rodola at gmail.com Sat Oct 2 22:27:00 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Sat, 2 Oct 2010 22:27:00 +0200 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: <4CA7902D.6060504@v.loewis.de> References: <4CA7902D.6060504@v.loewis.de> Message-ID: Thanks for this. It looks very nice. 2010/10/2 "Martin v. L?wis" : > Following up to the recent thread, I have now integrated Rietveld > into Roundup. This is a rough draft still, and highly experimental. > Please try it out, but expect that it may be necessary to discard > all data (including comments) to start over (of course, I'll try > to avoid having to do so). > > The Rietveld integration currently works this way: > - the installation lives at http://bugs.python.org/review/ > - for each roundup user, a rietveld user is created; > ?log into roundup to get access to rietveld > - for each roundup issue, a rietveld issue is created with the > ?same issue id. Please don't create additional Rietveld issues. > - regularly (currently every hour), each patch is considered. > ?Patches are skipped if: > ?* they have been added to Rietveld already > ?* they have no clear base version (i.e. they don't originate > ? ?from svn diff) > ?* they belong to no or a closed issue > ?* they apply to a patch that is not currently supported > ? ?(only trunk, 2.6, 2.7, 3.1, py3k are currently supported) > - for each such patch, a patchset is created > - if that is successful, a "review" link is added to roundup > > Feel free to discuss this here; bug reports and feature requests > should go to the meta tracker. Contributions are welcome; > I won't be able to work on this much for the next four days. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > From solipsis at pitrou.net Sat Oct 2 22:27:56 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 2 Oct 2010 22:27:56 +0200 Subject: [Python-Dev] Catching exceptions and convert them to warnings in C level code (was: Re: Pronouncement needed in issue9675) References: <4CA15678.3070004@jcea.es> <20100928180013.0077ba38@pitrou.net> <4CA792A4.3030806@jcea.es> Message-ID: <20101002222756.37412266@pitrou.net> On Sat, 02 Oct 2010 22:14:28 +0200 Jesus Cea wrote: > > My idea is to convert the CObject exception in bsddb to print a warning > like "CObject creation failed. The bsddb.cAPI will be not available. > Reason is: XXXX". > > Is there a "canonical way" of doing this?. Sorry if the answer is trivial... Try PyErr_WriteUnraisable(). Regards Antoine. From solipsis at pitrou.net Sat Oct 2 22:32:45 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 2 Oct 2010 22:32:45 +0200 Subject: [Python-Dev] Rietveld integration into Roundup References: <4CA7902D.6060504@v.loewis.de> Message-ID: <20101002223245.05177b74@pitrou.net> Hello, On Sat, 02 Oct 2010 22:03:57 +0200 "Martin v. L?wis" wrote: > Following up to the recent thread, I have now integrated Rietveld > into Roundup. This is a rough draft still, and highly experimental. > Please try it out, but expect that it may be necessary to discard > all data (including comments) to start over (of course, I'll try > to avoid having to do so). Thank you! This looks promising. I have some questions: 1) who is the notification e-mail sent to when a review is published? I just tried on one of my patches and the e-mail came with the following headers: From: pitrou at free.fr Reply-to: pitrou at free.fr, None at psf.upfronthosting.co.za To: pitrou at free.fr Cc: None at psf.upfronthosting.co.za Sujet: Fast path for small int-indexing of lists and tuples (issue9800) Am I right in assuming that nobody except me (even other people nosied in the Roundup issue) received the notification? (is None at psf some kind of /dev/null?) 2) if I look at http://bugs.python.org/issue9962, only the second patch of all three has been enabled for review. Yet they were all created through "svn diff" against a recent py3k checkout. Thanks again Antoine. From martin at v.loewis.de Sat Oct 2 22:37:40 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 02 Oct 2010 22:37:40 +0200 Subject: [Python-Dev] [Pythonmac-SIG] sad state of OS X Python testing... In-Reply-To: <20101002203205.GA24940@illinois.edu> References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> <20101002203205.GA24940@illinois.edu> Message-ID: <4CA79814.3030302@v.loewis.de> > I'm already running a Jython buildslave on an Intel Mac Pro which is > pretty underused - I'd be happy to run a CPython one there too, if > it'd be worthwhile. I think Bill was specifically after Snow Leopard - what system are you using? Regards, Martin From jcea at jcea.es Sat Oct 2 22:42:11 2010 From: jcea at jcea.es (Jesus Cea) Date: Sat, 02 Oct 2010 22:42:11 +0200 Subject: [Python-Dev] We should be using a tool for code reviews In-Reply-To: References: <20100929204125.416e2013@pitrou.net> Message-ID: <4CA79923.6090507@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 30/09/10 22:41, Brett Cannon wrote: > Don't see why not, but those of us who use OpenID would need to start > caring about a password which would be unfortunate. +1. OpenID or OAuth is a must. Moreover, I am a bit worried of needing a google account. Google already knows too much about me, I don't want to navigate with a google cookie in my browser. Guido says that Rietveld can run outside of Google. That is not a bad option. I could host it, I guess, but my server uptime is only around 99.7% (half an hour per week). - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBTKeZI5lgi5GaxT1NAQLcRgP/YldvoSubtVYUC3IxdXtC+XjiKWaC28eJ xIp3CxWwtuzAGrYl/kfiyrxLpk40jL/T6xEdU8r/lXXxKttbBBThLsNf93LrECCB 4uxmIdEY+SkK5Mj2gU3FWTQ4PQfRspAwYtfjGvnaPEbVLTWOccqAK4SsyEpkZ2n9 wQRUNYURv44= =qoXI -----END PGP SIGNATURE----- From martin at v.loewis.de Sat Oct 2 22:57:15 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 02 Oct 2010 22:57:15 +0200 Subject: [Python-Dev] We should be using a tool for code reviews In-Reply-To: <4CA79923.6090507@jcea.es> References: <20100929204125.416e2013@pitrou.net> <4CA79923.6090507@jcea.es> Message-ID: <4CA79CAB.90802@v.loewis.de> Am 02.10.2010 22:42, schrieb Jesus Cea: > On 30/09/10 22:41, Brett Cannon wrote: >> Don't see why not, but those of us who use OpenID would need to start >> caring about a password which would be unfortunate. > > +1. OpenID or OAuth is a must. > > Moreover, I am a bit worried of needing a google account. Google already > knows too much about me, I don't want to navigate with a google cookie > in my browser. > > Guido says that Rietveld can run outside of Google. That is not a bad > option. I could host it, I guess, but my server uptime is only around > 99.7% (half an hour per week). Don't worry. It's hosted on bugs.python.org now. Regards, Martin From solipsis at pitrou.net Sat Oct 2 23:01:51 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 02 Oct 2010 23:01:51 +0200 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: <4CA79C58.90106@v.loewis.de> References: <4CA7902D.6060504@v.loewis.de> <20101002223245.05177b74@pitrou.net> <4CA79C58.90106@v.loewis.de> Message-ID: <1286053311.3242.6.camel@localhost.localdomain> Le samedi 02 octobre 2010 ? 22:55 +0200, "Martin v. L?wis" a ?crit : > > > 2) if I look at http://bugs.python.org/issue9962, only the second patch > > of all three has been enabled for review. Yet they were all created > > through "svn diff" against a recent py3k checkout. > > They had both the same problem: it could figure out the revision number > the patch applied to (e.g. 85039), but not the branch that this revision > is for. That's because the revision is 85039, and in r85039, only > /peps got modified. > > I see... so the revision number is mostly useless when trying to > identify the branch. > > You should be able to fix this by manually filling out the branch on > the file. I'll have to come up with a better way to determine the branch > which a patch was created on. Perhaps going back in history and taking > the first branch where the patch cleanly applies can do the trick. I think a heuristic would be to try py3k HEAD first. That's what most patches are supposed to work against. Regards Antoine. From martin at v.loewis.de Sat Oct 2 22:55:52 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 02 Oct 2010 22:55:52 +0200 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: <20101002223245.05177b74@pitrou.net> References: <4CA7902D.6060504@v.loewis.de> <20101002223245.05177b74@pitrou.net> Message-ID: <4CA79C58.90106@v.loewis.de> > 1) who is the notification e-mail sent to when a review is published? I haven't figured that out yet. I'm not even sure whether email gets sent at all. I'll look into this a week from now, unless someone resolves that earlier. But... > I > just tried on one of my patches and the e-mail came with the following > headers: > > From: pitrou at free.fr > Reply-to: pitrou at free.fr, None at psf.upfronthosting.co.za > To: pitrou at free.fr > Cc: None at psf.upfronthosting.co.za > Sujet: Fast path for small int-indexing of lists and tuples > (issue9800) > > Am I right in assuming that nobody except me (even other people > nosied in the Roundup issue) received the notification? So email did get send ... good. It certainly doesn't consider the nosy list at all at the moment. > (is None at psf some kind of /dev/null?) I have no idea where that came from. > 2) if I look at http://bugs.python.org/issue9962, only the second patch > of all three has been enabled for review. Yet they were all created > through "svn diff" against a recent py3k checkout. They had both the same problem: it could figure out the revision number the patch applied to (e.g. 85039), but not the branch that this revision is for. That's because the revision is 85039, and in r85039, only /peps got modified. I see... so the revision number is mostly useless when trying to identify the branch. You should be able to fix this by manually filling out the branch on the file. I'll have to come up with a better way to determine the branch which a patch was created on. Perhaps going back in history and taking the first branch where the patch cleanly applies can do the trick. Regards, Martin From njriley at illinois.edu Sat Oct 2 22:32:05 2010 From: njriley at illinois.edu (Nicholas Riley) Date: Sat, 2 Oct 2010 15:32:05 -0500 Subject: [Python-Dev] [Pythonmac-SIG] sad state of OS X Python testing... In-Reply-To: <4CA79129.8050900@v.loewis.de> References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> Message-ID: <20101002203205.GA24940@illinois.edu> On Sat, Oct 02, 2010 at 10:08:09PM +0200, "Martin v. L?wis" wrote: > > Surely someone could volunteer an old Intel Mac to run some tests? > > (Actually, two someones -- we'd need separate machines for Leopard and > > Snow Leopard.) I'd be happy to stick it in a server room at PARC and > > keep an eye on it. (Right now I've got a rack full of old eMacs and a > > G5 running PPC buildbots.) Perhaps we could get Apple to contribute > > some "seconds"? I believe donations to the PSF are deductible. > > The issue isn't really to get the hardware. The issue is to find > somebody to volunteer running a build slave. I'm already running a Jython buildslave on an Intel Mac Pro which is pretty underused - I'd be happy to run a CPython one there too, if it'd be worthwhile. -- Nicholas Riley From rdmurray at bitdance.com Sun Oct 3 01:00:27 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Sat, 02 Oct 2010 19:00:27 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes Message-ID: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> A while back on some issue or another I remember telling someone that if there was any sort of clever hack that would allow the current email package (email5) to work with bytes we would have implemented it. Well, I've come up with a clever hack. The idea came out of a conversation with Antoine. I was saying that it was ironic that Unicode could only be used as a 7bit-clean data transmission channel for email, and he remarked that by using surrogate escape you *could* use unicode as a transmission channel for 8bit data. At first I dismissed this observation as irrelevant to email, since email has to transform the 8bit data at some point. But I started thinking. And then I started experimenting. And it turns out that it works. The clever hack (thanks ultimately to Martin) is to accept 8bit data by encoding it using the ASCII codec and the surrogateescape error handler. Then, inside the email module at any point where bytes might be meaningful or might be about to escape, it can check to see if there are any surrogates and act accordingly. The API additions are few, and in fact for most programs (he says bravely, not really knowing) there are really only two changes you need to make when converting a program that handles bytes data to py3k. The first is the encoding of binary input data as mentioned. The second is that when you want to get the bytes back out, you use the new BytesGenerator instead of Generator. BytesGenerator is just like Generator except that it writes bytes to its file argument instead of strings, and it recovers any bytes that were in the original input. So given this sequence: msg = email.msg_from_file(open('myfile', encoding='ascii', errors='surrogateescape')) email.generator.BytesGenerator(open('myfile2', 'wb')).flatten(msg) myfile and myflie2 will theoretically be identical (modulo universal newline and _mangle_from issues). I've additionally added a 'message_from_bytes' convenience function. One nice feature of this patch is that once you've got the model built from surrogateescaped input, if you do a get_payload() on a message body whose ContentTransferEncoding is '8bit' you will get the body decoded to unicode using the charset declared in the Content-Type header (assuming Python supports that charset). You can always get at the bytes version of the body of a message part by using get_payload(decode=True) [*]. You can't really get at the bytes version of message headers, though...for safety if you access a header whose value contains non-ASCII chars (that aren't RFC2047 encoded to be ASCII) the 8bit characters get replaced with '?'s. (But BytesGenerator will emit the original 8bit characters if the headers haven't been modified.) I do not propose that this is a *good* API, since it has the classic problem that if there are coding bugs in the email module strings may "escape" that have surrogates in them and we end up with programs that work most of the time....except when they fail with mysterious errors because of unusual bytes input data. On the other hand you always *know* when you have bytes data in an unknown encoding (because they are surrogate escaped), so it is ever so much better than the Python2 situation. The advantage of this patch is that it means Python3.2 can have an email module that is capable of handling a significant proportion of the applications where the ability to process binary email data is required. I've uploaded the patch to issue 4661 (http://bugs.python.org/issue4661). I uploaded it to rietveld as well just before Martin's announcement. After the announcement I uploaded the svn patch to the tracker, so hopefully there will be an automated review button as well. Here is your chance to exercise the new review tools :) This patch does break two of Barry's patch-for-review rules: it is more than 800 lines of diff (but not a lot more, and less than 800 if you count only code diff and not docs), and it did not have a very extensive design discussion beforehand. I did talk with people on IRC, particularly Barry, before finishing the patch, and I did post a summary to the email-sig mailing list (but got no response). Now it is time to see what the wider community thinks. There is some question of whether this is a bending of the string/bytes separation that doesn't belong as part of the standard library, but after working my way through it I think it is a fairly clean hack[**], and most likely a case where practicality beats purity. Regardless of whether or not this patch or a descendant thereof is accepted I still intend to continue working on email6. There are many other bugs in the current email package that require a rewrite of parts of its infrastructure, and the email-sig is agreed that the email API needs revision quite apart from the bytes/string issues. However, there is something pleasing about the simplicity of this way of handling bytes that I intend to consider carefully while we work further on email6. --David [*] It is counterintuitive that 'decode=True' gives you bytes and 'decode=False' gives you strings, but in this case 'decode' refers to the ContentTransferEncoding...and this confusion is one of the reasons I think the email API needs a big overhaul. [**] There are a couple places where generator pokes into the internals of Message in a way it hasn't before, but this could be fixed by defining a 'bytes access' API on Message, which would probably be a good idea anyway. There is also the possibility of wrapping up the 'ascii+surrogateesape' stuff inside APIs that accept input data, to hide that 'implementation detail' from the email package user. From benjamin at python.org Sun Oct 3 02:15:57 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 2 Oct 2010 19:15:57 -0500 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> Message-ID: 2010/10/2 R. David Murray : > Regardless of whether or not this patch or a descendant thereof is > accepted I still intend to continue working on email6. ?There are many > other bugs in the current email package that require a rewrite of parts > of its infrastructure, and the email-sig is agreed that the email API > needs revision quite apart from the bytes/string issues. ?However, there > is something pleasing about the simplicity of this way of handling bytes > that I intend to consider carefully while we work further on email6. And how would this addition interact with changes in email6? -- Regards, Benjamin From nirsof at gmail.com Sun Oct 3 02:35:01 2010 From: nirsof at gmail.com (Nir Soffer) Date: Sun, 3 Oct 2010 02:35:01 +0200 Subject: [Python-Dev] os.path.normcase rationale? In-Reply-To: <026371F8-8933-4371-A2F5-EDF2F7D18DEE@fuhm.net> References: <4C9531A7.10405@simplistix.co.uk> <4C9C79DA.7000506@simplistix.co.uk> <20100924121737.309071FA5C2@kimball.webabinitio.net> <026371F8-8933-4371-A2F5-EDF2F7D18DEE@fuhm.net> Message-ID: On Sat, Sep 25, 2010 at 1:36 AM, James Y Knight wrote: > > An OSX code sketch is available here (summary: call FSPathMakeRef to get an > FSRef from a path string, then FSRefMakePath to make it back into a path, > which will then have the correct case). And note that it only works if the > file actually exists. > > > http://stackoverflow.com/questions/370186/how-do-i-find-the-correct-case-of-a-filename > > It would indeed be useful to have that be available in Python. > There is a much simpler way: >>> from Carbon import File >>> File.FSRef('/tmp/foo').as_pathname() '/private/tmp/Foo' Note that this is much slower compared to os.path.exists. -------------- next part -------------- An HTML attachment was scrubbed... URL: From krstic at solarsail.hcs.harvard.edu Sun Oct 3 02:25:48 2010 From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?Q?Ivan_Krsti=C4=87?=) Date: Sat, 2 Oct 2010 17:25:48 -0700 Subject: [Python-Dev] sad state of OS X Python testing... In-Reply-To: <42516.1286045405@parc.com> References: <42516.1286045405@parc.com> Message-ID: <7BA24ABE-163C-4A11-B25F-7D833B5B1457@solarsail.hcs.harvard.edu> On Oct 2, 2010, at 11:50 AM, Bill Janssen wrote: > Perhaps we could get Apple to contribute some "seconds"? If you don't get a good solution soon, let me know off-list and I'll see if Apple can help. Cheers, -- Ivan Krsti? | http://radian.org From rdmurray at bitdance.com Sun Oct 3 04:40:35 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Sat, 02 Oct 2010 22:40:35 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> Message-ID: <20101003024035.B2CFA22AB0F@kimball.webabinitio.net> On Sat, 02 Oct 2010 19:15:57 -0500, Benjamin Peterson wrote: > 2010/10/2 R. David Murray : > > Regardless of whether or not this patch or a descendant thereof is > > accepted I still intend to continue working on email6. =C2=A0There are ma= > ny > > other bugs in the current email package that require a rewrite of parts > > of its infrastructure, and the email-sig is agreed that the email API > > needs revision quite apart from the bytes/string issues. =C2=A0However, t= > here > > is something pleasing about the simplicity of this way of handling bytes > > that I intend to consider carefully while we work further on email6. > > And how would this addition interact with changes in email6? It will be no harder to do the backward compatibility support for this than for the rest of the email5 API, if that's what you are asking. Assuming my plan for backward compatibility works at all (which it should). --David From ncoghlan at gmail.com Sun Oct 3 04:43:19 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 3 Oct 2010 12:43:19 +1000 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> Message-ID: On Sun, Oct 3, 2010 at 9:00 AM, R. David Murray wrote: > I do not propose that this is a *good* API, since it has the classic > problem that if there are coding bugs in the email module strings may > "escape" that have surrogates in them and we end up with programs that > work most of the time....except when they fail with mysterious errors > because of unusual bytes input data. ?On the other hand you always > *know* when you have bytes data in an unknown encoding (because they > are surrogate escaped), so it is ever so much better than the Python2 > situation. It's a similar concept to one Antoine and I (and some others) have been considering in the tracker for making urllib.parse able to handle ASCII-compatible bytes-encodings. I've already implemented a version of that patch which has parallel bytes and str versions of all the ASCII constants, and the result is pretty ugly. My next goal is to implement a version that uses the same trick you have here for email and see how the code complexity compares. We do need to tread carefully to make sure the pseudo strings don't escape, but the other approach requires similar care all the way through the internal algorithms to make sure they aren't assuming bytes or str instances anywhere. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From njriley at illinois.edu Sun Oct 3 04:24:39 2010 From: njriley at illinois.edu (Nicholas Riley) Date: Sat, 2 Oct 2010 21:24:39 -0500 Subject: [Python-Dev] [Pythonmac-SIG] sad state of OS X Python testing... In-Reply-To: <4CA79814.3030302@v.loewis.de> References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> <20101002203205.GA24940@illinois.edu> <4CA79814.3030302@v.loewis.de> Message-ID: <20101003022438.GA4247@illinois.edu> On Sat, Oct 02, 2010 at 10:37:40PM +0200, "Martin v. L?wis" wrote: > > I'm already running a Jython buildslave on an Intel Mac Pro which is > > pretty underused - I'd be happy to run a CPython one there too, if > > it'd be worthwhile. > > I think Bill was specifically after Snow Leopard - what system are you > using? It's still on Leopard but I could use another (slower, but still probably fine) machine that has Snow Leopard on it. -- Nicholas Riley From danchr at gmail.com Sun Oct 3 15:18:39 2010 From: danchr at gmail.com (Dan Villiom Podlaski Christiansen) Date: Sun, 3 Oct 2010 15:18:39 +0200 Subject: [Python-Dev] os.path.normcase rationale? In-Reply-To: References: <4C9531A7.10405@simplistix.co.uk> <4C9C79DA.7000506@simplistix.co.uk> <20100924121737.309071FA5C2@kimball.webabinitio.net> <026371F8-8933-4371-A2F5-EDF2F7D18DEE@fuhm.net> Message-ID: <22D2544A-83FA-4C02-8FF9-A3499CFC7A8B@gmail.com> On 3 Oct 2010, at 02:35, Nir Soffer wrote: > On Sat, Sep 25, 2010 at 1:36 AM, James Y Knight wrote: >> >> An OSX code sketch is available here (summary: call FSPathMakeRef >> to get an >> FSRef from a path string, then FSRefMakePath to make it back into a >> path, >> which will then have the correct case). And note that it only works >> if the >> file actually exists. >> >> >> http://stackoverflow.com/questions/370186/how-do-i-find-the-correct-case-of-a-filename >> >> It would indeed be useful to have that be available in Python. >> > > There is a much simpler way: > >>>> from Carbon import File >>>> File.FSRef('/tmp/foo').as_pathname() > '/private/tmp/Foo' > > Note that this is much slower compared to os.path.exists. This won't work in py3k; the Carbon modules were removed in 3.0. A simpler alternative would probably be the F_GETPATH fcntl. An example: Python 3.1.2 (r312:79147, Jul 11 2010, 18:21:56) [GCC 4.2.1 (Apple Inc. build 5664)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from fcntl import fcntl >>> from os.path import basename, exists >>> from os import remove >>> >>> F_GETPATH = 50 >>> >>> if exists('/tmp/?'): ... remove('/tmp/?') ... >>> open('/tmp/?', 'w').close() >>> f = open(b'/tmp/A\xcc\x8a') >>> >>> a = f.name >>> b = fcntl(f, F_GETPATH, b'\0' * 1024).rstrip(b'\0') >>> >>> a, b (b'/tmp/A\xcc\x8a', b'/private/tmp/\xc3\xa5') >>> a.decode('utf-8'), b.decode('utf-8') ('/tmp/?', '/private/tmp/?') -- Dan Villiom Podlaski Christiansen danchr at gmail.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 1943 bytes Desc: not available URL: From foom at fuhm.net Sun Oct 3 15:32:44 2010 From: foom at fuhm.net (James Y Knight) Date: Sun, 3 Oct 2010 09:32:44 -0400 Subject: [Python-Dev] os.path.normcase rationale? In-Reply-To: <22D2544A-83FA-4C02-8FF9-A3499CFC7A8B@gmail.com> References: <4C9531A7.10405@simplistix.co.uk> <4C9C79DA.7000506@simplistix.co.uk> <20100924121737.309071FA5C2@kimball.webabinitio.net> <026371F8-8933-4371-A2F5-EDF2F7D18DEE@fuhm.net> <22D2544A-83FA-4C02-8FF9-A3499CFC7A8B@gmail.com> Message-ID: On Oct 3, 2010, at 9:18 AM, Dan Villiom Podlaski Christiansen wrote: > A simpler alternative would probably be the F_GETPATH fcntl. An example: That requires that you have permission to open the file (and to actually do so which might have other effects), while the File Manager's FSRef method does not. If Python adds a cross-platform function to do this canonicalization, users don't have to worry about how easy it is to invoke in pure-python... James From doug.hellmann at gmail.com Sun Oct 3 17:29:14 2010 From: doug.hellmann at gmail.com (Doug Hellmann) Date: Sun, 3 Oct 2010 11:29:14 -0400 Subject: [Python-Dev] framework build for 2.7 using old sqlite? Message-ID: <9207678A-3EA2-40F4-BEA0-F009DA224353@gmail.com> I'm trying to write a little program that uses the full text search extension module for sqlite with Python 2.7 on Snow Leopard. I installed Python by downloading the DMG file from python.org. According to the Python docs (http://docs.python.org/library/sqlite3.html#sqlite3.Connection.enable_load_extension), 2.7 should include the functions for handling extension modules, but when I try to use them they are not defined (I get an AttributeError when I call the related methods on the Connection object). In Modules/_sqlite/connection.c I see that there is an #ifdef for HAVE_LOAD_EXTENSION, which is in turn only defined if both the version number is high enough and SQLITE_OMIT_LOAD_EXTENSION is not set. I think the problem is that the build of Python in the DMG I download was created with an old version of the SQLite libraries: farnsworth:dhellmann:~:503 $ which python /Library/Frameworks/Python.framework/Versions/2.7/bin/python farnsworth:dhellmann:~:501 $ python Python 2.7 (r27:82508, Jul 3 2010, 21:12:11) [GCC 4.0.1 (Apple Inc. build 5493)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sqlite3 >>> sqlite3.version_info (2, 6, 0) Can anyone confirm that the installer image for OS X 10.5 and later (http://python.org/ftp/python/2.7/python-2.7-macosx10.5.dmg) was created with an old SQLite library? Is there some way to update that DMG, or do I need to build Python from source myself? Thanks, Doug From doug.hellmann at gmail.com Sun Oct 3 18:39:35 2010 From: doug.hellmann at gmail.com (Doug Hellmann) Date: Sun, 3 Oct 2010 12:39:35 -0400 Subject: [Python-Dev] framework build for 2.7 using old sqlite? In-Reply-To: <9207678A-3EA2-40F4-BEA0-F009DA224353@gmail.com> References: <9207678A-3EA2-40F4-BEA0-F009DA224353@gmail.com> Message-ID: <698BF31E-E9D4-4820-A75D-D8BF7D04BF6D@gmail.com> On Oct 3, 2010, at 11:29 AM, Doug Hellmann wrote: > I'm trying to write a little program that uses the full text search extension module for sqlite with Python 2.7 on Snow Leopard. I installed Python by downloading the DMG file from python.org. According to the Python docs (http://docs.python.org/library/sqlite3.html#sqlite3.Connection.enable_load_extension), 2.7 should include the functions for handling extension modules, but when I try to use them they are not defined (I get an AttributeError when I call the related methods on the Connection object). > > In Modules/_sqlite/connection.c I see that there is an #ifdef for HAVE_LOAD_EXTENSION, which is in turn only defined if both the version number is high enough and SQLITE_OMIT_LOAD_EXTENSION is not set. > > I think the problem is that the build of Python in the DMG I download was created with an old version of the SQLite libraries: > > farnsworth:dhellmann:~:503 $ which python > /Library/Frameworks/Python.framework/Versions/2.7/bin/python > > farnsworth:dhellmann:~:501 $ python > Python 2.7 (r27:82508, Jul 3 2010, 21:12:11) > [GCC 4.0.1 (Apple Inc. build 5493)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> import sqlite3 > >>> sqlite3.version_info > (2, 6, 0) Forget that, the version info is for pysqlite, not the underlying libraries. I found that Python's setup.py has SQLITE_OMIT_LOAD_EXTENSION set, which disables this feature (http://svn.python.org/view?view=rev&revision=78688). Doug From martin at v.loewis.de Sun Oct 3 21:32:23 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 03 Oct 2010 21:32:23 +0200 Subject: [Python-Dev] [Python-checkins] r85152 - tracker/instances/python-dev/config.ini.template In-Reply-To: References: <20101001225727.01CC7EE9EC@mail.python.org> Message-ID: <4CA8DA47.7090700@v.loewis.de> >> Modified: tracker/instances/python-dev/config.ini.template >> +[django] >> +# generate on a per-instance basis >> +secret_key = 'el at 4s$*(idwm5-87teftxlksckmy8$tyo7(tm!n-5x)zeuheex' > > I assume the secrecy of this is rather irrelevant? It's only the template. In the instance, you have to replace it with some value, if you care about the secrecy (which I did). Regards, Martin From daniel at stutzbachenterprises.com Mon Oct 4 03:56:41 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sun, 3 Oct 2010 20:56:41 -0500 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: <4CA79C58.90106@v.loewis.de> References: <4CA7902D.6060504@v.loewis.de> <20101002223245.05177b74@pitrou.net> <4CA79C58.90106@v.loewis.de> Message-ID: On Sat, Oct 2, 2010 at 3:55 PM, "Martin v. L?wis" wrote: > I'll have to come up with a better way to determine the branch > which a patch was created on. > That would also be helpful for those of us using DVCS software to talk to the svn server. :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From orsenthil at gmail.com Mon Oct 4 05:39:58 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Mon, 4 Oct 2010 09:09:58 +0530 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: <4CA7902D.6060504@v.loewis.de> References: <4CA7902D.6060504@v.loewis.de> Message-ID: Thanks Martin, this is really good. On Sun, Oct 3, 2010 at 1:33 AM, "Martin v. L?wis" wrote: > ?Patches are skipped if: > ?* they have been added to Rietveld already > ?* they have no clear base version (i.e. they don't originate > ? ?from svn diff) > ?* they belong to no or a closed issue > ?* they apply to a patch that is not currently supported > ? ?(only trunk, 2.6, 2.7, 3.1, py3k are currently supported) One comment: I feel that, only if a roundup issue has patch, the corresponding rietveld issue be created, it is more helpful there and avoids needless duplication. -- Senthil From eviatarbach at gmail.com Mon Oct 4 05:04:55 2010 From: eviatarbach at gmail.com (Eviatar Bach) Date: Sun, 3 Oct 2010 20:04:55 -0700 Subject: [Python-Dev] Inclusive Range In-Reply-To: References: Message-ID: Hello, I have a proposal of making the range() function inclusive; that is, range(3) would generate 0, 1, 2, and 3, as opposed to 0, 1, and 2. Not only is it more intuitive, it also seems to be used often, with coders often writing range(0, example+1) to get the intended result. It would be easy to implement, and though significant, is not any more drastic than changing print to a function in Python 3. Of course, if this were done, slicing behaviour would have to be adjusted accordingly. What are your thoughts? -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Mon Oct 4 06:00:31 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 3 Oct 2010 23:00:31 -0500 Subject: [Python-Dev] Inclusive Range In-Reply-To: References: Message-ID: 2010/10/3 Eviatar Bach : > Hello, > > I have a proposal See the python-ideas mailing list. > of making the range() function inclusive; that is, > range(3) would generate 0, 1, 2, and 3, as opposed to 0, 1, and 2. Not only > is it more intuitive, it also seems to be used often, with coders often > writing range(0, example+1) to get the intended result. It would be easy to > implement, and though significant, is not any more drastic than changing > print to a function in Python 3.. Yes, it is. It's much more subtle. > > What are your thoughts? Completely backwards incompatible. -1 -- Regards, Benjamin From cs at zip.com.au Mon Oct 4 05:58:20 2010 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 4 Oct 2010 14:58:20 +1100 Subject: [Python-Dev] Inclusive Range In-Reply-To: References: Message-ID: <20101004035820.GA28494@cskk.homeip.net> On 03Oct2010 20:04, Eviatar Bach wrote: | I have a proposal of making the range() function inclusive; that is, | range(3) would generate 0, 1, 2, and 3, as opposed to 0, 1, and 2. Not only | is it more intuitive, it also seems to be used often, with coders often | writing range(0, example+1) to get the intended result. It would be easy to | implement, and though significant, is not any more drastic than changing | print to a function in Python 3. Of course, if this were done, slicing | behaviour would have to be adjusted accordingly. 1: take this to python-ieas please instead of python-dev. 2: -1; the current behaviour is, IMHO, highly desirable. Example: x=3; y=7; z=12 s='ahahahahahahahaha' assert s[:x]+s[x:y]+s[y:z]+s[z:] == s No icky +1 hacks in there. 3: I find myself doing the +1 thing very _un_often, myself. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ The hardest years in life are those between ten and seventy. - Helen Hayes (at age 84) From python-dev at masklinn.net Mon Oct 4 09:27:59 2010 From: python-dev at masklinn.net (Xavier Morel) Date: Mon, 4 Oct 2010 09:27:59 +0200 Subject: [Python-Dev] Inclusive Range In-Reply-To: References: Message-ID: On 2010-10-04, at 05:04 , Eviatar Bach wrote: > Hello, > > I have a proposal of making the range() function inclusive; that is, > range(3) would generate 0, 1, 2, and 3, as opposed to 0, 1, and 2. Not only > is it more intuitive, it also seems to be used often, with coders often > writing range(0, example+1) to get the intended result. It would be easy to > implement, and though significant, is not any more drastic than changing > print to a function in Python 3. Of course, if this were done, slicing > behaviour would have to be adjusted accordingly. > > What are your thoughts? Same as the others: 0. This is a discussion for python-ideas, I'm CCing that list 1. This is a major backwards compatibility breakage, and one which is entirely silent (`print` from keyword to function wasn't) 2. It loses not only well-known behavior but interesting properties as well (`range(n)` has exactly `n` elements. With your proposal, it has ``n+1`` breaking ``for i in range(5)`` to iterate 5 times as well as ``for i in range(len(collection))`` for cases where e.g. ``enumerate`` is not good enough or too slow) 3. As well as the relation between range and slices 4. I fail to see how it is more intuitive (let alone more practical, see previous points) 5. If you want an inclusive range, I'd recommend proposing a flag on `range` (e.g. ``inclusive=True``) rather than such a drastic breakage of ``range``'s behavior. That, at least, might have a chance. Changing the existing default behavior of range most definitely doesn't. I'd be ?1 on your proposal, ?0 on adding a flag to ``range`` (I can't recall the half-open ``range`` having bothered me recently, if ever) From scott+python-dev at scottdial.com Mon Oct 4 18:32:26 2010 From: scott+python-dev at scottdial.com (Scott Dial) Date: Mon, 04 Oct 2010 12:32:26 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> Message-ID: <4CAA019A.8030309@scottdial.com> On 10/2/2010 7:00 PM, R. David Murray wrote: > The clever hack (thanks ultimately to Martin) is to accept 8bit data > by encoding it using the ASCII codec and the surrogateescape error > handler. I've seen this idea pop up in a number of threads. I worry that you are all inventing a new kind of dual that is a direct parallel to Python 2.x strings. That is to say, 3.x>>> b = b'\xc2\xa1' 3.x>>> s = b.decode('utf8') 3.x>>> v = b.decode('ascii', 'surrogateescape') , where s and v should be the same "thing" in 3.x but they are not due to an encoding trick. I believe this trick generates more-or-less the same issues as strings did in 2.x: 2.x>>> b = '\xc2\xa1' 2.x>>> s = b.decode('utf8') 2.x>>> v = b Any reasonable 2.x code has to guard on str/unicode and it would seem in 3.x, if this idiom spreads, reasonable code will have to guard on surrogate escapes (which actually seems like a more expensive test). As in, 3.x>>> print(v) Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'utf-8' codec can't encode character '\udcc2' in position 0: surrogates not allowed It seems like this hack is about making the 3.x unicode type more like the 2.x string type, and I thought we decided that was a bad idea. How will developers not have to ask themselves whether a given string is a "real" string or a byte sequence masquerading as a string? Am I missing something here? -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From rdmurray at bitdance.com Mon Oct 4 19:38:35 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 04 Oct 2010 13:38:35 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <4CAA019A.8030309@scottdial.com> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> Message-ID: <20101004173835.9FD1E21B120@kimball.webabinitio.net> On Mon, 04 Oct 2010 12:32:26 -0400, Scott Dial wrote: > On 10/2/2010 7:00 PM, R. David Murray wrote: > > The clever hack (thanks ultimately to Martin) is to accept 8bit data > > by encoding it using the ASCII codec and the surrogateescape error > > handler. > > I've seen this idea pop up in a number of threads. I worry that you are > all inventing a new kind of dual that is a direct parallel to Python 2.x > strings. Yes, that is exactly my worry. > That is to say, > > 3.x>>> b = b'\xc2\xa1' > 3.x>>> s = b.decode('utf8') > 3.x>>> v = b.decode('ascii', 'surrogateescape') > > , where s and v should be the same "thing" in 3.x but they are not due > to an encoding trick. Why "should" they be the same thing in 3.x? One is an ASCII string with some escaped bytes in an unknown encoding, the other is a valid unicode string. The surrogateescape trick is used only when we don't *know* the encoding (a priori) of the bytes in question. > I believe this trick generates more-or-less the same issues as strings > did in 2.x: > > 2.x>>> b = '\xc2\xa1' > 2.x>>> s = b.decode('utf8') > 2.x>>> v = b The difference is that in 2.x people could and would operate on strings as if they knew the encoding, and get in trouble. In 3.x you can't do that. If you've got escaped bytes you *know* that you don't know the encoding, and the program can't get around that except by re-encoding to bytes and properly decoding them. > Any reasonable 2.x code has to guard on str/unicode and it would seem in > 3.x, if this idiom spreads, reasonable code will have to guard on > surrogate escapes (which actually seems like a more expensive test). As in, > > 3.x>>> print(v) > Traceback (most recent call last): > File "", line 1, in > UnicodeEncodeError: 'utf-8' codec can't encode character '\udcc2' in > position 0: surrogates not allowed Right, I mentioned that concern in my post. In this case at least, however, the *goal* is that the surrogates are never seen outside the email internals. In reflection of this, my latest thought is that I should add a 'message_from_binary_file' helper method and a 'feedbytes' method to feedparser, making the surrogates a 100% internal implementation detail[*]. Only if the email package contains a coding error would the surrogates escape and cause problems for user code. > It seems like this hack is about making the 3.x unicode type more like > the 2.x string type, and I thought we decided that was a bad idea. How > will developers not have to ask themselves whether a given string is a > "real" string or a byte sequence masquerading as a string? Am I missing > something here? I think this question is something that needs to be considered any time using surrogates is proposed. I hope that in the email package proposal I've addressed it. What do you think? --David [*] And you are right that there is a performance concern as a result of needing to detect surrogates at various points in the code. From barry at python.org Mon Oct 4 19:41:09 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 4 Oct 2010 13:41:09 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> Message-ID: <20101004134109.13488619@mission> On Oct 02, 2010, at 07:00 PM, R. David Murray wrote: >The advantage of this patch is that it means Python3.2 can have an >email module that is capable of handling a significant proportion of >the applications where the ability to process binary email data is >required. Like others, I'm concerned that we're perpetuating the Python 2 problems with bytes vs. strings. OTOH, I went down a similar road (though much more hacky and less successful) in one of my failed branches, so I sympathize with this nod to practicality that actually works. If the choice is the current brokenness staying in Python 3.2 or this hack being added for now, I'd go with the latter. email6 will make it all better, right? :) In the meantime, I do think it would be good to give our users something that's practical. >I've uploaded the patch to issue 4661 (http://bugs.python.org/issue4661). I >uploaded it to rietveld as well just before Martin's announcement. After the >announcement I uploaded the svn patch to the tracker, so hopefully there will >be an automated review button as well. Here is your chance to exercise the >new review tools :) I see no automatically generated link to the review, but I did add some comments to the Rietveld issue you linked to in one of your comments. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Mon Oct 4 20:28:51 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 4 Oct 2010 14:28:51 -0400 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: References: <20101001200657.472c2f60@mission> Message-ID: <20101004142851.5ef9dc47@mission> On Oct 02, 2010, at 10:36 AM, Georg Brandl wrote: >Am 02.10.2010 00:06, schrieb Barry Warsaw: > >> The reason is that the import.c logic that uses the struct filedescr >> tables built from _PyImport_DynLoadFiletab are just not smart enough >> to handle this case. All it knows about are suffix, and for >> backwards compatibility, we have dynload_shlib.c matching >> both .SOABI.so *and* bare .so. So when it's searching the >> directories for .cpython-32m.so files, it finds >> the ..cpython-32dmu.so first based on the bare .so match. > >I don't understand -- wouldn't "foo.sometag.so" (where sometag is not >SOABI) only be found a match for a suffix of ".so" if the module name >requested is "foo.sometag"? (And if not, isn't implementing that the >easiest solution?) Yep, my analysis was faulty. Python's own import machinery does exactly the right thing. I think the problem is in distribute, which writes a _foo.py file that bootstraps into loading the wrong .so file. E.g. for an extension names _stupid, you end up with this _stupid.py in the egg: def __bootstrap__(): global __bootstrap__, __loader__, __file__ import sys, pkg_resources, imp __file__ = pkg_resources.resource_filename(__name__,'_stupid.cpython-32dmu.so') __loader__ = None; del __bootstrap__, __loader__ imp.load_dynamic(__name__,__file__) __bootstrap__() Python's built-in import finds _stupid.py, but this is hardcoded to the build-flags of the last Python used to install the package. If instead this looked like: def __bootstrap__(): global __bootstrap__, __loader__, __file__ import sys, pkg_resources, imp, sysconfig __file__ = pkg_resources.resource_filename(__name__,'_stupid.{soabi}.so'.format(soabi=sysconfig.get_config_var('SOABI'))) __loader__ = None; del __bootstrap__, __loader__ imp.load_dynamic(__name__,__file__) __bootstrap__() then everything works out just fine. If you install only the 'dmu' version and try to import it with the 'm' Python, you get an ImportError as expected (i.e. not a crash!). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Mon Oct 4 20:34:22 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 4 Oct 2010 14:34:22 -0400 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> Message-ID: <20101004143422.600fb50e@mission> On Oct 02, 2010, at 04:50 PM, Nick Coghlan wrote: >On Sat, Oct 2, 2010 at 10:24 AM, Antoine Pitrou >wrote: >> On Fri, 1 Oct 2010 20:06:57 -0400 >> Barry Warsaw wrote: >>> >>> With my branch, you'll end up with this in /tmp/python: >>> >>> ? ? bin/python3.2m ? - the normal build binary >>> ? ? bin/python3.2dmu - the wide+pydebug build binary >>> ? ? bin/python3.2m-config >>> ? ? bin/python3.2dmu-config >> >> Do users really want to see such idiosyncratic suffixes? > >Ordinary users won't be building Python from source. Developers won't >care so long as we clearly document the sundry suffixes and describe >them in the README (or in a PEP, with a pointer from the README). I agree with this. >>> ? ? lib/libpython3.2.so.1.0.m >>> ? ? lib/libpython3.2.so.1.0.dmum >> >> Ditto here. This seems to break well-known conventions. >> If I look at /usr/lib{,64} on my machine, I can't see a single >> shared libary file that ends neither in ".so" nor ".so.> digits>". > >Having some characters on the end to flag different kinds of custom >build seems like it fits within the .so naming conventions I'm aware >of, but I'm sure the *nix packaging folks will pipe up if Barry starts >wandering too far afield in this area. Because -Wl,-h is used, the right soname will get compiled in, so it will generally just DTRT. The situation where it can break is if you are not using distutils to build things. However, in that case, the symlinks added by 'make install' should still make things just work. However, if you don't use distutils and still want to link against the correct shared library on a multi-build system, you'll have to modify your rules to grab the build flags from the Makefile (the easiest route being via sysconfig). >> Before trying to find a solution to your problem, I think it would be >> nice to get a consensus that this is really a desired feature. > >Having multiple parallel "altinstall" installations be genuinely >non-interfering out of the box certainly seems like a desirable >feature to me. Right. Isn't that kind of the whole point of altinstall? :) Well, multiple coexisting versions of Python, but I think multiple coexisting builds of the same Python version falls into the same category. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Mon Oct 4 20:41:11 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 4 Oct 2010 14:41:11 -0400 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <4CA6E2D0.9010407@v.loewis.de> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> Message-ID: <20101004144111.61561231@mission> On Oct 02, 2010, at 09:44 AM, Martin v. L?wis wrote: >>>> With my branch, you'll end up with this in /tmp/python: >>>> >>>> bin/python3.2m - the normal build binary >>>> bin/python3.2dmu - the wide+pydebug build binary >>>> bin/python3.2m-config >>>> bin/python3.2dmu-config >>> >>> Do users really want to see such idiosyncratic suffixes? >> >> Ordinary users won't be building Python from source. Developers won't >> care so long as we clearly document the sundry suffixes and describe >> them in the README (or in a PEP, with a pointer from the README). > >I think this is not true. Developers *will* care, and they will cry >foul very loudly, asking what nonsense this is. Antoine is proof of >that: he is a developer, and he understands the motivation well, >but it still goes against his notions of beauty (channeling him here). Well, it may be surprising at first, but since it doesn't break any normal usage I don't think most developers will care. But I could be wrong. >> Having multiple parallel "altinstall" installations be genuinely >> non-interfering out of the box certainly seems like a desirable >> feature to me. > >I think this should not use automatically generated suffixes, though. >Perhaps I want an altinstall that is in some kind restrict? >Or one where user "peter" has write access into site-packages? I'm not sure how this relates to the suffix question... >I could accept that a suffix is parameter to configure (or some such), >and then gets used throughout. By default, Python will not add a >suffix. However, I still wonder why people couldn't just install >Python in a different prefix if they want separate installations. For a distro, all those Python binaries have to go in /usr/bin. We already symlink /usr/bin/python to pythonX.Y so I don't see the harm in a few extra symlinks. However, if people *really* don't want to see this by default then I can think of a few options: * Enable the extra build-flag suffixes through a configure option and/or new Makefile target. Could end up duplicating the altinstall rule if the current rule can't be refactored easily. * Expose just the necessary low-level stuff to allow the distro installation scripts to move things around and create the symlinks after the fact. This would mean that other distros (or from-source installers) wouldn't benefit from the isolation feature without some extra work on their part though. It would be nice if this was a feature everybody could just have. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Mon Oct 4 20:42:02 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 4 Oct 2010 14:42:02 -0400 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101002134051.5a7902b1@pitrou.net> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> <20101002134051.5a7902b1@pitrou.net> Message-ID: <20101004144202.40fa1085@mission> On Oct 02, 2010, at 01:40 PM, Antoine Pitrou wrote: >Besides, mingling different installations together makes uninstalling >much more difficult. Not for a distro I think. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Mon Oct 4 20:43:35 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 4 Oct 2010 14:43:35 -0400 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: References: <20101001200657.472c2f60@mission> Message-ID: <20101004144335.50066cbc@mission> On Oct 01, 2010, at 07:36 PM, Benjamin Peterson wrote: >2010/10/1 Barry Warsaw : >> I can think of a couple of ways out, none of which are totally >> satisfying. Probably the easiest out is to change the PEP 3149 >> naming so that the files don't end in ".so". ?E.g. use this instead: >> >> ? ?foo.cpython-32dmu-so >> ? ?foo.cpython-32m-so > >-1 Doesn't that break not only Python's convention for extensions on >shared modules but also any *nix shared object? It shouldn't (i.e. if -Wl,-h is used to get the soname compiled in there), but still I don't like it, and now I know it's not necessary. >> or similar. ?I think that'd be a fairly straightforward change, but >> it might break some useful assumptions (we'd still fall back to .so >> of course). >> >> Other ideas: >> >> - make import.c smarter so that you can match against other than >> just the suffix. ?probably a lot of work. > >Although it would be more work, I think this is the most "correct" >option. Phew! Extra work averted! (see my other reply :). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From solipsis at pitrou.net Mon Oct 4 21:10:09 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 4 Oct 2010 21:10:09 +0200 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> <20101004144111.61561231@mission> Message-ID: <20101004211010.78c5f745@pitrou.net> On Mon, 4 Oct 2010 14:41:11 -0400 Barry Warsaw wrote: > > For a distro, all those Python binaries have to go in /usr/bin. We already > symlink /usr/bin/python to pythonX.Y so I don't see the harm in a few extra > symlinks. Why would a distro want to provide all combinations of Python builds? One important issue for me is guessability. While "d" is reasonably guessable (and "dbg" or "debug" would be even better), "u" and "m" are not. (actually, "u" could lead to misunderstandings such as "is this a unicode-enabled version of Python?"; as for "m", I don't know what it's for) As for the SOABI, you could use a different mangling which would preserve the ".so" suffix -- e.g. "-debug.so" instead of ".so.d". At least then well-known conventions would be preserved. Regards Antoine. From larry at hastings.org Mon Oct 4 21:12:03 2010 From: larry at hastings.org (Larry Hastings) Date: Mon, 04 Oct 2010 12:12:03 -0700 Subject: [Python-Dev] Pronouncement needed in issue9675 In-Reply-To: References: <4CA15678.3070004@jcea.es> Message-ID: <4CAA2703.9000707@hastings.org> Hi--sorry to be a little late to the discussion. I'm the putz who backported capsules (and monkeyed with CObject) for 2.7. On 09/27/2010 07:44 PM, Jesus Cea wrote: > http://bugs.python.org/issue9675 > > Long history sort: Python 2.7 backported Capsule support and > (incorrectly, in my opinion) marked CObject as deprecated. > Not strictly true. CObject is marked with a "Pending Deprecation" warning. But this is still turned into an error by -We. > All C modules in the stdlib were updated to Capsule (with a CObject > compatibility layer), except BSDDB, because this change was done late in > the cycle, the proposed patch was buggy (solvable) and a pronouncement > was done that CObject was not actually deprecated. > I did the stdlib conversion from CObjects to capsules. I was told it'd be improper for me to convert bsddb to capsules because it's externally maintained (by you!). You and I discussed converting bsddb to capsules for 2.7 at the very last minute but it didn't happen. > Since I think that adopting Capsule in BSDDB for 2.7.1 would break the > API compatibility (maybe the CObject proxy would solve this), and since > a previous pronouncement was done abour CObject not-deprecated in 2.7.x, > I would like comments. > By "CObject proxy" I assume you mean CObject support for opening capsules. Specifically: PyCObject_AsVoidPtr()--and therefore functions that call it, such as PyCObject_Import()--understand capsules. So yes, if the bsddb that shipped with Python 2.7.1 used a capsule, external users /should/ work unchanged. I further theorize that external callers would continue to work fine even if they'd been compiled against previous versions of Python (though I admit I've never tried it). On 09/28/2010 06:49 AM, Guido van Rossum wrote: > My guess is that the warning was added to 2.7 before it was clear that > there would never be a 2.8. I understood at the time how unlikely it is that there will ever be a 2.8. I'm just that paranoid/crazy. Even if the Python core dev community never release a 2.7 (or 2.8) external luddites might, and I wanted my foot in the door. I'm fine with changing the warning so it's only active with -3. I still want it left as a PendingDeprecationWarning; PyErr_WarnPy3k() issues a DeprecationWarning, and it'd be inappropriate to upgrade it to that. (Though if there ever is a 2.8 (crosses self) maybe we could drop the -3 requirement then?) I'd be willing to contribute this change. I'd be happy to convert every use of PendingDeprecationWarning, or just the one in cobject.c, either way. Sorry about all the wailing and gnashing of teeth, /larry/ From larry at hastings.org Mon Oct 4 21:34:04 2010 From: larry at hastings.org (Larry Hastings) Date: Mon, 04 Oct 2010 12:34:04 -0700 Subject: [Python-Dev] API for binary operations on Sets In-Reply-To: <62BC914E-8B5A-424C-A068-739D0E71FF55@gmail.com> References: <62BC914E-8B5A-424C-A068-739D0E71FF55@gmail.com> Message-ID: <4CAA2C2C.7080000@hastings.org> On 09/29/2010 08:50 PM, Raymond Hettinger wrote: > 1. Liberalize setobject.c binary operator methods to accept anything > registered to the Set ABC and add a backwards incompatible restriction > to the Set ABC binary operator methods to only accept Set ABC > instances (they currently accept any iterable). > > This approach has a backwards incompatible tightening of the Set ABC, > but that will probably affect very few people. It also has the > disadvantage of not providing a straight-forward way to handle general > iterable arguments (either the implementer needs to write named binary > methods like update, difference, etc for that purpose or the user will > need to cast the the iterable to a set before operating on it). The > positive side of this option is that keeps the current advantages of > the setobject API and its NotImplemented return value. > > 1a. Liberalize setobject.c binary operator methods, restrict SetABC > methods, and add named methods (like difference, update, etc) that > accept any iterable. I prefer 1 to 1a, but either is acceptable. 1 just forces you to call set() on your iterable before operating on it, which I think helps ("explicit is better than implicit"). In 1a, by "add named methods that accept any iterable", you mean add those to the SetABC? If so, then I'm more strongly for 1 and against 1a. I'm not convinced all classes implementing the Set ABC should have to implement all those methods. /larry/ From barry at python.org Mon Oct 4 22:01:17 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 4 Oct 2010 16:01:17 -0400 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101004211010.78c5f745@pitrou.net> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> <20101004144111.61561231@mission> <20101004211010.78c5f745@pitrou.net> Message-ID: <20101004160117.5ebabd8e@mission> On Oct 04, 2010, at 09:10 PM, Antoine Pitrou wrote: >On Mon, 4 Oct 2010 14:41:11 -0400 >Barry Warsaw wrote: >> >> For a distro, all those Python binaries have to go in /usr/bin. We >> already symlink /usr/bin/python to pythonX.Y so I don't see the harm >> in a few extra symlinks. > >Why would a distro want to provide all combinations of Python builds? Maybe not all, but definitely several. At least a normal build and a debug build, but a wide unicode build possibly also. >One important issue for me is guessability. While "d" is >reasonably guessable (and "dbg" or "debug" would be even better), "u" >and "m" are not. >(actually, "u" could lead to misunderstandings such as "is this a >unicode-enabled version of Python?"; as for "m", I don't know what it's >for) I think symlinks will make this discoverable. I like that the binary name's suffix flags matches the flags used in PEP 3149, which also makes it easy to document. I could imagine python3-dbg would be symlinked to python3.2d (or whatever). >As for the SOABI, you could use a different mangling which would >preserve the ".so" suffix -- e.g. "-debug.so" instead of ".so.d". At >least then well-known conventions would be preserved. We already have libpython3.2.so.1.0 which also doesn't end in .so. I suppose we could put the build flags before the .so. part, but I think Matthias had a problem with that (I don't remember the details). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From solipsis at pitrou.net Mon Oct 4 22:18:34 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 4 Oct 2010 22:18:34 +0200 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> <20101004144111.61561231@mission> <20101004211010.78c5f745@pitrou.net> <20101004160117.5ebabd8e@mission> Message-ID: <20101004221834.500d1185@pitrou.net> On Mon, 4 Oct 2010 16:01:17 -0400 Barry Warsaw wrote: > > > >Why would a distro want to provide all combinations of Python builds? > > Maybe not all, but definitely several. At least a normal build and a debug > build, but a wide unicode build possibly also. What is the point of shipping a different unicode representation? Is there any practical use case? I could understand a motivated user trying different build flags for the purpose of experimentation and personal enlightenment, but a Linux distribution? (also, Debian's Python already defaults on wide unicode) > >As for the SOABI, you could use a different mangling which would > >preserve the ".so" suffix -- e.g. "-debug.so" instead of ".so.d". At > >least then well-known conventions would be preserved. > > We already have libpython3.2.so.1.0 which also doesn't end in .so. ".so." is a well-understood Unix convention, while ".so." doesn't seem to be. (this also means that tools such as file managers etc. may not display the file type properly) Regards Antoine. From stephen at xemacs.org Tue Oct 5 07:41:12 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 05 Oct 2010 14:41:12 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101004173835.9FD1E21B120@kimball.webabinitio.net> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> Message-ID: <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> R. David Murray writes: > On Mon, 04 Oct 2010 12:32:26 -0400, Scott Dial wrote: > > On 10/2/2010 7:00 PM, R. David Murray wrote: > > > The clever hack (thanks ultimately to Martin) is to accept 8bit data > > > by encoding it using the ASCII codec and the surrogateescape error > > > handler. > > > > I've seen this idea pop up in a number of threads. I worry that you are > > all inventing a new kind of dual that is a direct parallel to Python 2.x > > strings. > > Yes, that is exactly my worry. I don't worry about this. Strings generated by decoding with surrogate-escape are *different* from other strings: they contain invalid code units (the naked surrogates). These cannot be encoded except with a surrogate-escape flag to .encode(), and sane developers won't do that unless she knows precisely what she's doing. This is not true with Python 2 strings, where all bytes are valid. > > Any reasonable 2.x code has to guard on str/unicode and it would seem in > > 3.x, if this idiom spreads, reasonable code will have to guard on > > surrogate escapes (which actually seems like a more expensive test). > > Right, I mentioned that concern in my post. Again, I don't worry about this. It is *not* an *extra* cost. Those messages are *already broken*, they *will* crash the email module if you fail to guard against them. Decoding them to surrogates actually makes it easier to guard, because you know that even if broken encodings are present, the parser will still work. Broken encodings can no longer crash the parser. That is a Very Good Thing IMHO. > Only if the email package contains a coding error would the > surrogates escape and cause problems for user code. I don't think it is reasonable to internalize surrogates that way; some applications *will* want to look at them and do something useful with them (delete them or replace them with U+FFFD or ...). However, I argue below that the presence of surrogates already means the user code is under fire, and this puts the problem in a canonical form so the user code can prepare for it (if that is desirable). > > It seems like this hack is about making the 3.x unicode type more like > > the 2.x string type, Not at all. It's about letting the parser be a parser, and letting the application handle broken content, or discard it, or whatever. Modularity is improved. This has been a major PITA for Mailman support over the years: every time the spammers and virus writers come up with a new idea, there's a chance it will leak out and the email parser will explode, stopping the show. These kinds of errors are a FAQ on the Mailman lists (although much less so in recent years). > > How will developers not have to ask themselves whether a given > > string is a "real" string or a byte sequence masquerading as a > > string? Am I missing something here? There are two things to say, actually. First, you're in a war zone. *All* email is bytes sequences masquerading as text, and if you're not wearing armor, you're going to get burned. The idea here is to have the email package provide the armor and enough instrumentation so you can do bomb detection yourself (or perhaps just let it blow, if you're hacking up a quick and dirty script). Second, there are developers who will not care whether strings are "real" or "byte sequences in drag", because they're writing MTAs and the like. Those people get really upset, and rightly so, when the parser pukes on broken headers; it is not their app's job at all to deal with that breakage. > I think this question is something that needs to be considered any > time using surrogates is proposed. I don't agree. The presence of naked surrogates is *always* (assuming sane programmers) an indication of invalid input. The question is, should the parser signal invalidity, or should it allow the application to decide? The email module *doesn't have enough information to decide* whether the invalid input is a "real" problem, or how to handle it (cf the example of a MTA app). Note that a completely naive app doesn't care -- it will crash either way because it doesn't handle the exception, whether it's raised by the parser or by a codec when the app tries to do I/O. A robust app *does* care: if the parser raises, then the app must provide an alternative parser good enough to find and fix the invalid bytes. Clearly it's much better to pass invalid (but fully parsed) text back to the app in this case. Note that if the app really wants the parser to raise rather than pass on the input, that should be easy to implement at fairly low cost; you just provide a variable rather than hardcoding the surrogate-escape flag. From chris at simplistix.co.uk Tue Oct 5 10:21:15 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 05 Oct 2010 09:21:15 +0100 Subject: [Python-Dev] os.path.normcase rationale? In-Reply-To: <201009251325.28749.steve@pearwood.info> References: <4C9531A7.10405@simplistix.co.uk> <4C9D298A.3010407@g.nevcal.com> <201009251325.28749.steve@pearwood.info> Message-ID: <4CAADFFB.50205@simplistix.co.uk> On 25/09/2010 04:25, Steven D'Aprano wrote: > 1. Return the case of a filename in some canonical form which depends > on the file system? > 2. Return the case of a filename as it is actually stored on disk? How do 1 and 2 differ? FWIW, the use case that setuptools has (and for which it currently incorrectly uses normpath) is number 2. > 4. Return the case of a filename in some arbitrarily-chosen canonical > form which does not depend on the file system? This is what normpath does, but only if you're on Windows ;-) I still don't really get the use case of normpath in its current form, at all... > Various people have posted links to recipes that solve case #2. Note > though that this necessarily demands that if the file doesn't exist, it > should raise an exception. Fine by me, shame it seems to require iteration to find an answer though :-S > The very concept of canonical form for file names is troublesome. I would have thought "whatever is shown when doing an ls/dir/etc" (and don't be smart and think about mentioning that oyu can get dir to output 8.3 as well as the full path ;-) ) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Oct 5 10:24:21 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 05 Oct 2010 09:24:21 +0100 Subject: [Python-Dev] os.path.normcase rationale? In-Reply-To: References: <4C9531A7.10405@simplistix.co.uk> <4C9D298A.3010407@g.nevcal.com> <201009251325.28749.steve@pearwood.info> Message-ID: <4CAAE0B5.70906@simplistix.co.uk> On 25/09/2010 15:45, Guido van Rossum wrote: > The solution may well be OS specific. Solutions for Windows and OS X > have already been pointed out. If it can't be done for other Unix > versions, I think returning the input unchanged on those platform is a > fine fallback (as it is for non-existent filenames). Spot on, especially as the "default" of case perserving and case sensitive will likely cover anything where an FS-specific solution can't be found - I'd hazard a guess that the reason the FS-sepcific solution can't be found in some cases is because for case preserving and case sensitive situations, there really is no need for such an api ;-) Chris From asmodai at in-nomine.org Tue Oct 5 10:50:57 2010 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Tue, 5 Oct 2010 10:50:57 +0200 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101004160117.5ebabd8e@mission> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> <20101004144111.61561231@mission> <20101004211010.78c5f745@pitrou.net> <20101004160117.5ebabd8e@mission> Message-ID: <20101005085057.GD68173@nexus.in-nomine.org> -On [20101004 22:03], Barry Warsaw (barry at python.org) wrote: >We already have libpython3.2.so.1.0 which also doesn't end in .so. I suppose >we could put the build flags before the .so. part, but I think Matthias had a >problem with that (I don't remember the details). Using major and minor numbers after the .so designation is an old aout (de facto) standard. ELF settled on just major numbers. And typically you will find a symbolic link from libwhatever.so to libwhatever.so.1 Also, from FreeBSD's ldconfig manual page: "Filenames must conform to the lib*.so.[0-9] pattern in order to be added to the hints file." So ending a shared object with the build flags will cause problems on FreeBSD and probably the other BSDs (DragonFly, NetBSD, OpenBSD) and most likely Mac OS X as well. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Time is merely a residue of Reality... From asmodai at in-nomine.org Tue Oct 5 10:45:27 2010 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Tue, 5 Oct 2010 10:45:27 +0200 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101004144202.40fa1085@mission> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> <20101002134051.5a7902b1@pitrou.net> <20101004144202.40fa1085@mission> Message-ID: <20101005084527.GC68173@nexus.in-nomine.org> -On [20101004 20:48], Barry Warsaw (barry at python.org) wrote: >On Oct 02, 2010, at 01:40 PM, Antoine Pitrou wrote: > >>Besides, mingling different installations together makes uninstalling >>much more difficult. > >Not for a distro I think. It does. On BSD the ports and packages are referred to a specific location in the ports tree, e.g. ports/lang/python26, so any Python 2.6 compiled and installed from ports with specific options will *always* point to this location. So if you want to install 2 versions of Python 2.6 with different options, you are out of luck using the standard way. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Looking for the Sun that eclipsed behind black feathered wings... From steve at pearwood.info Tue Oct 5 13:04:39 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 5 Oct 2010 22:04:39 +1100 Subject: [Python-Dev] os.path.normcase rationale? In-Reply-To: <4CAADFFB.50205@simplistix.co.uk> References: <4C9531A7.10405@simplistix.co.uk> <201009251325.28749.steve@pearwood.info> <4CAADFFB.50205@simplistix.co.uk> Message-ID: <201010052204.40773.steve@pearwood.info> On Tue, 5 Oct 2010 07:21:15 pm Chris Withers wrote: > On 25/09/2010 04:25, Steven D'Aprano wrote: > > 1. Return the case of a filename in some canonical form which > > depends on the file system? > > 2. Return the case of a filename as it is actually stored on disk? > > How do 1 and 2 differ? Case #1 imposes a particular canonical form, regardless of what is actually stored on disk. It is similar to normpath, except that we could have different canonical forms depending on what the file system was. normpath merely generalises from the operating system, and never looks at the file system. Some file systems are case-preserving, and don't have a canonical form. We might choose to arbitrarily impose one, as normcase already does. Some are case-folding, in which case it might be sensible to choose the same canonical form as the file system actually uses. However, this may be implementation dependent e.g. under FAT12 or FAT16, the file system will take a file name like pArRoT.tXt and fold it to PARROT.TXT, or possibly parrot.txt, or Parrot.txt. Even if that's not the case for FAT12, it may be the case for other case-folding file systems. And the behaviour of FAT16 will differ according to whether or not it has been built with support for long file names. Case #2 says to actually look at the file and see what the file system considers it's name to be. Consider a NTFS file system. By default it is case-preserving and case-insensitive, although that can be changed. (Just because a file system is NTFS doesn't mean that will be case-insensitive. NTFS can also run in a POSIX mode which is case-sensitive. But I digress.) For simplicity, suppose you're on Windows using NTFS with the standard non-POSIX behaviour. You create a file named pArRoT.tXt. This will be stored on disk using the exact characters that you typed. The file system does no case-folding and merely uses whatever characters are fed to it, which in the case of Windows apps is likely to be whatever characters the user types. In this case, we don't try to impose a particular case on file names, but return whatever actually exists on disk. > FWIW, the use case that setuptools has (and > for which it currently incorrectly uses normpath) is number 2. > > > 4. Return the case of a filename in some arbitrarily-chosen > > canonical form which does not depend on the file system? > > This is what normpath does, but only if you're on Windows ;-) Not quite. macpath.normcase() also lowercases the path. So does the module for OS/2. In any case, Windows is not a file system. It is quite possible to have virtually any combination of case-destroying, case-preserving, -sensitive and -insensitive file systems on the one Windows system. Say, a FAT12 floppy, an NTFS partition, and an ext2 USB stick. Windows doesn't ship with native support for ext2, but that doesn't mean it can't be installed with third party drivers. normpath pays no attention to any of this, and just lowercases the path. At least that's cheap, and consistent, even if it solves the wrong problem :) -- Steven D'Aprano From fuzzyman at voidspace.org.uk Tue Oct 5 13:39:00 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 05 Oct 2010 12:39:00 +0100 Subject: [Python-Dev] Adding Wing 4 project to Message-ID: <4CAB0E54.4010300@voidspace.org.uk> Hello all, There is a Wing IDE [1] project file in Misc/ which I use for working on Python. The project file is 'python-wing.wpr'. Wing 4 is now in beta and I have switched to using it. Wing 4 uses an updated, backwards incompatible, project file format. I would like to add a Wing 4 project file to Misc, called 'python-wing4.wpr' to all the branches I work on (likely py3k, python27-maint and python31-maint). Any objections to this? Once Wing 4 is out of beta there are several options. Major version updates to Wing are infrequent, so the options are listed in my order of preference: 1. rename the old file 'python-wing3.wpr' and rename 'python-wing4.wpr' to 'python-wing.wpr' 2. delete the wing 3 project file altogether and rename 'python-wing4.wpr' to 'python-wing.wpr' 3. stay with versioned project file names and rename 'python-wing.wpr' to 'python-wing3.wpr' (Option 3 could be done immediately of course.) All the best, Michael Foord [1] http://wingware.com/ -- http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From orsenthil at gmail.com Tue Oct 5 13:51:37 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Tue, 5 Oct 2010 17:21:37 +0530 Subject: [Python-Dev] Adding Wing 4 project to In-Reply-To: <4CAB0E54.4010300@voidspace.org.uk> References: <4CAB0E54.4010300@voidspace.org.uk> Message-ID: <20101005115137.GA17796@remy> On Tue, Oct 05, 2010 at 12:39:00PM +0100, Michael Foord wrote: > Wing 4 is now in beta and I have switched to using it. Wing 4 uses > an updated, backwards incompatible, project file format. I would > like to add a Wing 4 project file to Misc, called 'python-wing4.wpr' > to all the branches I work on (likely py3k, python27-maint and > python31-maint). > > Any objections to this? One of the concerns would be size. How much does the new extra file add to the tarball (and to Windows/Mac builds, if they are bundling). -- Senthil From fuzzyman at voidspace.org.uk Tue Oct 5 13:55:18 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 05 Oct 2010 12:55:18 +0100 Subject: [Python-Dev] Adding Wing 4 project to In-Reply-To: <20101005115137.GA17796@remy> References: <4CAB0E54.4010300@voidspace.org.uk> <20101005115137.GA17796@remy> Message-ID: <4CAB1226.2030800@voidspace.org.uk> On 05/10/2010 12:51, Senthil Kumaran wrote: > On Tue, Oct 05, 2010 at 12:39:00PM +0100, Michael Foord wrote: >> Wing 4 is now in beta and I have switched to using it. Wing 4 uses >> an updated, backwards incompatible, project file format. I would >> like to add a Wing 4 project file to Misc, called 'python-wing4.wpr' >> to all the branches I work on (likely py3k, python27-maint and >> python31-maint). >> >> Any objections to this? > One of the concerns would be size. How much does the new extra file > add to the tarball (and to Windows/Mac builds, if they are bundling). >>> len(open('Misc/python-wing.wpr').read()) 555 >>> len(open('Misc/python-wing4.wpr').read()) 888 Michael -- http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From orsenthil at gmail.com Tue Oct 5 13:59:16 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Tue, 5 Oct 2010 17:29:16 +0530 Subject: [Python-Dev] Adding Wing 4 project to In-Reply-To: <4CAB1226.2030800@voidspace.org.uk> References: <4CAB0E54.4010300@voidspace.org.uk> <20101005115137.GA17796@remy> <4CAB1226.2030800@voidspace.org.uk> Message-ID: On Tue, Oct 5, 2010 at 5:25 PM, Michael Foord wrote: >>>> len(open('Misc/python-wing.wpr').read()) > 555 >>>> len(open('Misc/python-wing4.wpr').read()) > 888 So, "size doesn't matter". :) -- Senthil From solipsis at pitrou.net Tue Oct 5 14:00:41 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Oct 2010 14:00:41 +0200 Subject: [Python-Dev] Adding Wing 4 project to References: <4CAB0E54.4010300@voidspace.org.uk> Message-ID: <20101005140041.6f96103c@pitrou.net> On Tue, 05 Oct 2010 12:39:00 +0100 Michael Foord wrote: > > 1. rename the old file 'python-wing3.wpr' and rename 'python-wing4.wpr' > to 'python-wing.wpr' > 2. delete the wing 3 project file altogether and rename > 'python-wing4.wpr' to 'python-wing.wpr' > 3. stay with versioned project file names and rename 'python-wing.wpr' > to 'python-wing3.wpr' > > (Option 3 could be done immediately of course.) Option 3 looks the most reasonable to me. Regards Antoine. From ncoghlan at gmail.com Tue Oct 5 14:05:33 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 5 Oct 2010 22:05:33 +1000 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Tue, Oct 5, 2010 at 3:41 PM, Stephen J. Turnbull wrote: > R. David Murray writes: > ?> Only if the email package contains a coding error would the > ?> surrogates escape and cause problems for user code. > > I don't think it is reasonable to internalize surrogates that way; > some applications *will* want to look at them and do something useful > with them (delete them or replace them with U+FFFD or ...). ?However, > I argue below that the presence of surrogates already means the user > code is under fire, and this puts the problem in a canonical form so > the user code can prepare for it (if that is desirable). Hang on here, this objection doesn't seem to quite mesh with what RDM is proposing (and the similar trick I am considering for urllib.parse). The basic issue is having an algorithm that is designed to operate on character data and depends on multiple ASCII constants stored as str objects. In Python 2.x, those algorithms could innately operate on str objects in any ASCII compatible encoding, as well as on unicode objects (due to the implicit promotion of the ASCII constants to unicode when unicode input was encountered). In Py3k, that trick broke. Now those algorithms only operate on str objects, and bytes input fails, even when it uses an ASCII compatible encoding. For urllib.parse, the external API will be "str in -> str out, bytes in -> bytes out". Whether that is internally implemented by duplicating all the ASCII constants with both bytes and str flavours (as my current patch does), or implicitly (and temporarily) "decoding" the bytes values using ascii+surrogateescape or latin-1 (a pair of alternative approaches I plan to explore soon) should be completely transparent to the user of the API. If a user can easily tell which of these I am doing just through the external behaviour of the documented API, then I'll have made a mistake somewhere. My understanding is that email6 in 3.3 will essentially follow that same model. What I believe RDM is suggesting is an in-between approach for the 3.2 email module: - if you pass in bytes data that isn't 7-bit clean and naively use the str APIs to access the headers, then it will complain loudly if it is about to return escaped data (but will decode the body in accordance with the Content Transfer Encoding) - if you pass in bytes data and know what you are doing, then you can access that raw bytes data and do your own decoding I've probably grossly oversimplified what RDM is suggesting, but it sounds plausible as a useful interim stepping stone to the more comprehensive type separation in email6. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From martin at v.loewis.de Tue Oct 5 11:19:23 2010 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 05 Oct 2010 11:19:23 +0200 Subject: [Python-Dev] Who wants to donate an AMD64 Linux build slave? Message-ID: <4CAAED9B.405@v.loewis.de> We just lost Neal's AMD-64 build slave due to hardware problems. There is another AMD6-64 slave in the pool, however, it runs in a non-standard environment, so it shows test failures at the moment. Would anybody be willing to run a build slave on a machine that you have available? The main requirement is that the machine is always connected to the internet, although not necessarily with a fixed IP address. Regards, Martin From dirkjan at ochtman.nl Tue Oct 5 16:00:15 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Tue, 5 Oct 2010 16:00:15 +0200 Subject: [Python-Dev] Who wants to donate an AMD64 Linux build slave? In-Reply-To: <4CAAED9B.405@v.loewis.de> References: <4CAAED9B.405@v.loewis.de> Message-ID: On Tue, Oct 5, 2010 at 11:19, "Martin v. L?wis" wrote: > Would anybody be willing to run a build slave on a machine that > you have available? The main requirement is that the machine is > always connected to the internet, although not necessarily with > a fixed IP address. I can probably run a build slave on one of my boxes (Gentoo, Athlon 64 x2). Where are the setup docs? Cheers, Dirkjan From van.lindberg at gmail.com Tue Oct 5 16:18:11 2010 From: van.lindberg at gmail.com (VanL) Date: Tue, 05 Oct 2010 09:18:11 -0500 Subject: [Python-Dev] PyCon 2011 Call for Tutorials Message-ID: NB: Posting on behalf of Greg Lindstrom, our tutorial coordinator: PyCon 2011 will be held March 9th through the 17th, 2011 in Atlanta, Georgia. (Home of some of the best southern food you can possibly find on Earth!) The PyCon conference days will be March 11-13, preceded by two tutorial days (March 9-10), and followed by four days of development sprints (March 14-17). The call for tutorial proposals is open until November 1 and we want to encourage you to submit a class idea! Teachers are paid for their efforts and it's a great opportunity for you to teach to a room full of people who have paid to hear what you have to say? Tutorials are 3-hour long classes (with a refreshment break) taught be some of the leading minds in the Python community. Classes range from beginner (Introduction to Python) to advanced (OOP, Data Storage and Optimization) and everything in between. If you don't feel up to teaching a class, you can always encourage your favorite mentor to teach it! Anything Python may be proposed for a class and a variety of topics is always presented but we can't offer what isn't proposed! Get more information at http://us.pycon.org/2011/about/ under the "Tutorial Days" section. Submit your idea on the site under the "Speakers" tab. That's it! You could be teaching a class at PyCon next March. See you in Atlanta! From barry at python.org Tue Oct 5 16:19:46 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 5 Oct 2010 10:19:46 -0400 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101004221834.500d1185@pitrou.net> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> <20101004144111.61561231@mission> <20101004211010.78c5f745@pitrou.net> <20101004160117.5ebabd8e@mission> <20101004221834.500d1185@pitrou.net> Message-ID: <20101005101946.0524bead@mission> On Oct 04, 2010, at 10:18 PM, Antoine Pitrou wrote: >What is the point of shipping a different unicode representation? Is >there any practical use case? I could understand a motivated user >trying different build flags for the purpose of experimentation and >personal enlightenment, but a Linux distribution? >(also, Debian's Python already defaults on wide unicode) True, it may not make sense to ship different unicode builds, but certainly it makes sense to ship debug and non-debug builds. But I don't think it makes sense to special case that when we already handle the build flags according to PEP 3149. Also, I do think there is a valid use case for developers here. I might indeed want to install into /usr/local various different builds of Python for testing purposes (e.g. user reported my extension module crashes when --without-unicode --with-pydebug is given). To me, we're mostly discussing user interface. What binary names to we want to present to the end user? Through the use of symlinks we can present any name we want. I don't personally think having additional build-flagged names hurts the user, but if there's strong disagreement, then I'm not opposed to exposing them optionally through a configure flag and/or new Makefile target. I just don't think it's necessary. >> >As for the SOABI, you could use a different mangling which would >> >preserve the ".so" suffix -- e.g. "-debug.so" instead of ".so.d". At >> >least then well-known conventions would be preserved. >> >> We already have libpython3.2.so.1.0 which also doesn't end in .so. > >".so." is a well-understood Unix convention, while >".so." doesn't seem to be. >(this also means that tools such as file managers etc. may not display >the file type properly) Okay. I'll try to modify the patch to put the build-flags before the .so when --enable-shared is given. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Tue Oct 5 16:21:22 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 5 Oct 2010 10:21:22 -0400 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101005084527.GC68173@nexus.in-nomine.org> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> <20101002134051.5a7902b1@pitrou.net> <20101004144202.40fa1085@mission> <20101005084527.GC68173@nexus.in-nomine.org> Message-ID: <20101005102122.602a46a2@mission> On Oct 05, 2010, at 10:45 AM, Jeroen Ruigrok van der Werven wrote: >-On [20101004 20:48], Barry Warsaw (barry at python.org) wrote: >>On Oct 02, 2010, at 01:40 PM, Antoine Pitrou wrote: >> >>>Besides, mingling different installations together makes uninstalling >>>much more difficult. >> >>Not for a distro I think. > >It does. On BSD the ports and packages are referred to a specific >location in the ports tree, e.g. ports/lang/python26, so any Python >2.6 compiled and installed from ports with specific options will >*always* point to this location. So if you want to install 2 versions >of Python 2.6 with different options, you are out of luck using the >standard way. Do any BSD distros provide multiple different builds of Python to their users? How do they handle having a debug build or non-debug build? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Tue Oct 5 16:22:20 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 5 Oct 2010 10:22:20 -0400 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101005085057.GD68173@nexus.in-nomine.org> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> <20101004144111.61561231@mission> <20101004211010.78c5f745@pitrou.net> <20101004160117.5ebabd8e@mission> <20101005085057.GD68173@nexus.in-nomine.org> Message-ID: <20101005102220.07ea1b5a@mission> On Oct 05, 2010, at 10:50 AM, Jeroen Ruigrok van der Werven wrote: >Also, from FreeBSD's ldconfig manual page: > >"Filenames must conform to the lib*.so.[0-9] pattern in order to be >added to the hints file." > >So ending a shared object with the build flags will cause problems on >FreeBSD and probably the other BSDs (DragonFly, NetBSD, OpenBSD) and >most likely Mac OS X as well. Fair enough. I will change the patch to move the build flags when --enable-shared is used. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From dsdale24 at gmail.com Tue Oct 5 16:56:11 2010 From: dsdale24 at gmail.com (Darren Dale) Date: Tue, 5 Oct 2010 10:56:11 -0400 Subject: [Python-Dev] question/comment about documentation of relative imports Message-ID: I have a couple questions/comments about the use of PEP 328-style relative imports. For example, the faq at http://docs.python.org/py3k/faq/programming.html#what-are-the-best-practices-for-using-import-in-a-module reads: "Never use relative package imports. If you?re writing code that?s in the package.sub.m1 module and want to import package.sub.m2, do not just write from . import m2, even though it?s legal. Write from package.sub import m2 instead. See PEP 328 for details." There is no explanation to support the claim that relative imports should "never" be used. It seems to me that someone read the following in PEP 328:: from .moduleY import spam from .moduleY import spam as ham from . import moduleY from ..subpackage1 import moduleY from ..subpackage2.moduleZ import eggs from ..moduleA import foo from ...package import bar from ...sys import path Note that while that last case is legal, it is certainly discouraged ("insane" was the word Guido used). ... and interpreted it to mean that relative imports are in general discouraged. I interpreted it to mean that relative imports should not be used to import from python's standard library. There are cases where it is necessary to use relative imports, like a package that is included as a subpackage of more than one other project (when it is not acceptable to add an external dependency, for example due to version/compatibility issues). There is some additional context on relative imports in the programming faq for python-2.7 at http://docs.python.org/faq/programming.html#what-are-the-best-practices-for-using-import-in-a-module : "Never use relative package imports. If you?re writing code that?s in the package.sub.m1 module and want to import package.sub.m2, do not just write import m2, even though it?s legal. Write from package.sub import m2 instead. Relative imports can lead to a module being initialized twice, leading to confusing bugs. See PEP 328 for details." Is there some documentation explaining why the module may be initialized twice? I don't see it in PEP 328. Is this also the case for python-3, or does it only apply to the old-style (pre-PEP 328) relative imports in python-2? If relative imports are truly so strongly discouraged, then perhaps warnings should also be included in places like http://docs.python.org/library/functions.html#__import__ , and especially http://docs.python.org/tutorial/modules.html#intra-package-references and http://www.python.org/dev/peps/pep-0328/ (which, if I have misinterpreted, is ambiguously written. Though I doubt this is the case). There is also this warning against relative imports in PEP 8: - Relative imports for intra-package imports are highly discouraged. Always use the absolute package path for all imports. Even now that PEP 328 [7] is fully implemented in Python 2.5, its style of explicit relative imports is actively discouraged; absolute imports are more portable and usually more readable. ... but one could argue, as I just have, that relative imports are more portable, not less. In a sense, the statement "explicit relative imports is actively discouraged" is objectively false. They are passively discouraged. If they were actively discouraged, perhaps performing a relative import would raise a warning, or maybe distutils would raise a warning at install time, or maybe an additional import would be required to enable them. Up until now, I was not aware that use of PEP 328 relative imports might be discouraged. I'm still unclear as to why they might be discouraged. I recently helped convert a popular package to use PEP 328 relative imports. Would the python devs consider this a mistake? Thanks, Darren From rdmurray at bitdance.com Tue Oct 5 17:05:23 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 05 Oct 2010 11:05:23 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101005150523.CA7D6228E86@kimball.webabinitio.net> On Tue, 05 Oct 2010 22:05:33 +1000, Nick Coghlan wrote: > On Tue, Oct 5, 2010 at 3:41 PM, Stephen J. Turnbull wrote: > > R. David Murray writes: > > > Only if the email package contains a coding error would the > > > surrogates escape and cause problems for user code. > > > > I don't think it is reasonable to internalize surrogates that way; > > some applications *will* want to look at them and do something useful > > with them (delete them or replace them with U+FFFD or ...). However, > > I argue below that the presence of surrogates already means the user > > code is under fire, and this puts the problem in a canonical form so > > the user code can prepare for it (if that is desirable). > > Hang on here, this objection doesn't seem to quite mesh with what RDM > is proposing (and the similar trick I am considering for > urllib.parse). [snip Nick's clear explanation of the issue and using surrogates to allow string-based algorithms to work] > My understanding is that email6 in 3.3 will essentially follow that > same model. What I believe RDM is suggesting is an in-between approach > for the 3.2 email module: > > - if you pass in bytes data that isn't 7-bit clean and naively use the > str APIs to access the headers, then it will complain loudly if it is > about to return escaped data (but will decode the body in accordance > with the Content Transfer Encoding) Almost correct. What it will do when it does not have the information needed to decode the bytes correctly (ie: the message is not RFC compliant) is to replace the unknown bytes with '?' characters. This means that you can render a "dirty" email to the terminal, for example, and the invalid bytes will show as '?'s.[*] > - if you pass in bytes data and know what you are doing, then you can > access that raw bytes data and do your own decoding With the current patch this is a true statement for message bodies, but not for message headers. There is no easy way to add access to the bytes version of headers to the email5 API, but since any such data would be non-RFC compliant anyway, that will just have to be good enough for now. > I've probably grossly oversimplified what RDM is suggesting, but it > sounds plausible as a useful interim stepping stone to the more > comprehensive type separation in email6. The more I look at the patch the more I think this can be an internal implementation detail in email5 just like you might do for urllib. So the email5 API will have a way to put bytes in, a way to get decoded data out, and a way to get a bytes out (except for individual header values). The model object will be the same no matter what you put in or take out. The additional methods added to the email5 API to make this possible will be: message_from_bytes (and Parser.parsebytes) message_from_binary_file Feedparser.feedbytes BytesGenerator message_from_bytes and message_from_binary_file are currently part of the proposed email6 API, and I was thinking about some version of Feedparser.feedbytes[**]. BytesGenerator wasn't, but now perhaps it will be (and certainly will be in the backward compatibility interface). -- R. David Murray www.bitdance.com [*] Why '?' and not the unicode invalid character character? Well, the email5 Generate.flatten can be used to generate data for transmission over the wire *if* the source is RFC compliant and 7bit-only, and this would be a normal email5 usage pattern (that is, smtplib.SMTP.sendmail expects ASCII-only strings as input!). So the data generated by Generator.flatten should not include unicode...which raises a problem for CTE 8bit sections that the patch doesn't currently address. [**] Benjamin asked how the patch would affect backward compatibility support in email6, and I said it wouldn't make it harder. However, if feedbytes calls can be mixed with feed calls, which in the simplest implementation they could be, then if email6 does *not* use surrogates internally its feedparser algorithm would need to be considerably more complicated to be backward compatible with this. So when I add Feedparser.parsebytes to my patch, I am at least initially going to disallow mixing calls to feed and feedbytes. Which is another reason to add that method so as to keep the use of the surrogateescape an implementation detail. From martin at v.loewis.de Tue Oct 5 17:06:40 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 05 Oct 2010 17:06:40 +0200 Subject: [Python-Dev] Who wants to donate an AMD64 Linux build slave? In-Reply-To: References: <4CAAED9B.405@v.loewis.de> Message-ID: <4CAB3F00.7050107@v.loewis.de> > I can probably run a build slave on one of my boxes (Gentoo, Athlon 64 > x2). Where are the setup docs? http://wiki.python.org/moin/BuildBot Regards, Martin From dirkjan at ochtman.nl Tue Oct 5 17:14:08 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Tue, 5 Oct 2010 17:14:08 +0200 Subject: [Python-Dev] Who wants to donate an AMD64 Linux build slave? In-Reply-To: <4CAB3F00.7050107@v.loewis.de> References: <4CAAED9B.405@v.loewis.de> <4CAB3F00.7050107@v.loewis.de> Message-ID: On Tue, Oct 5, 2010 at 17:06, "Martin v. L?wis" wrote: >> I can probably run a build slave on one of my boxes (Gentoo, Athlon 64 >> x2). Where are the setup docs? > > http://wiki.python.org/moin/BuildBot Cool, can you get me username/passwd? Cheers, Dirkjan From martin at v.loewis.de Tue Oct 5 17:31:05 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 05 Oct 2010 17:31:05 +0200 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: References: <4CA7902D.6060504@v.loewis.de> Message-ID: <4CAB44B9.2060403@v.loewis.de> > I feel that, only if a roundup issue has patch, the corresponding > rietveld issue be created, it is more helpful there and avoids > needless duplication. I have changed that now. Regards, Martin From hodgestar+pythondev at gmail.com Tue Oct 5 18:13:01 2010 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Tue, 5 Oct 2010 18:13:01 +0200 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: References: Message-ID: On Tue, Oct 5, 2010 at 4:56 PM, Darren Dale wrote: > ? from ...sys import path > > ? Note that while that last case is legal, it is certainly > discouraged ("insane" was the word Guido used). Only if by "legal" you mean "happened to work". It stops "happening to work" in Python 2.6.6. :) Generally I'm +0 on relative imports as a whole. Schiavo Simon From fuzzyman at voidspace.org.uk Tue Oct 5 18:18:18 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 05 Oct 2010 17:18:18 +0100 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: References: Message-ID: <4CAB4FCA.4080502@voidspace.org.uk> On 05/10/2010 17:13, Simon Cross wrote: > On Tue, Oct 5, 2010 at 4:56 PM, Darren Dale wrote: >> from ...sys import path >> >> Note that while that last case is legal, it is certainly >> discouraged ("insane" was the word Guido used). > Only if by "legal" you mean "happened to work". It stops "happening to > work" in Python 2.6.6. :) > > Generally I'm +0 on relative imports as a whole. As the OP pointed out, for code that may be *included* in other projects there is no other choice. This is often useful for packages shared between one or two projects that nonetheless don't warrant separate distribution. All the best, Michael > Schiavo > Simon > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From solipsis at pitrou.net Tue Oct 5 18:43:26 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Oct 2010 18:43:26 +0200 Subject: [Python-Dev] question/comment about documentation of relative imports References: <4CAB4FCA.4080502@voidspace.org.uk> Message-ID: <20101005184326.45a2a013@pitrou.net> On Tue, 05 Oct 2010 17:18:18 +0100 Michael Foord wrote: > > > > Generally I'm +0 on relative imports as a whole. > > As the OP pointed out, for code that may be *included* in other projects > there is no other choice. This is often useful for packages shared > between one or two projects that nonetheless don't warrant separate > distribution. You can put several packages in a single distribution. Regards Antoine. From solipsis at pitrou.net Tue Oct 5 19:07:41 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Oct 2010 19:07:41 +0200 Subject: [Python-Dev] stable builders References: <4CAAED9B.405@v.loewis.de> <4CAB3F00.7050107@v.loewis.de> Message-ID: <20101005190741.3d863fc9@pitrou.net> On Tue, 05 Oct 2010 17:06:40 +0200 "Martin v. L?wis" wrote: > > I can probably run a build slave on one of my boxes (Gentoo, Athlon 64 > > x2). Where are the setup docs? > > http://wiki.python.org/moin/BuildBot By the way, is the distinction between "stable" and "unstable" builders still relevant? If I look at http://www.python.org/dev/buildbot/3.x.stable/, two of our four stable builders are offline (and at least one of them - the ia64 one - has been for a long time). Granted, this makes them "stable" in a sense ;) Regards Antoine. From apt.shansen at gmail.com Tue Oct 5 19:08:59 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Tue, 5 Oct 2010 10:08:59 -0700 Subject: [Python-Dev] [Pythonmac-SIG] sad state of OS X Python testing... In-Reply-To: <4CA79814.3030302@v.loewis.de> References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> <20101002203205.GA24940@illinois.edu> <4CA79814.3030302@v.loewis.de> Message-ID: On Sat, Oct 2, 2010 at 1:37 PM, "Martin v. L?wis" wrote: > > I'm already running a Jython buildslave on an Intel Mac Pro which is > > pretty underused - I'd be happy to run a CPython one there too, if > > it'd be worthwhile. > > I think Bill was specifically after Snow Leopard - what system are you > using? > I have a fairly recent MacPro on Snow Leopard, which I keep consistently up to date and its connected all the time. It has more capacity then I can really find use for. If its still needed, I can set up buildbot to run on it today. Is it all pull/poll oriented, or does the slave need to be connected to by the master? Meaning, do I need to poke a hole in the firewall to allow any external access? The BuildBot page only mentions outgoing access (or I'm misunderstanding it). IIUC, I just need a name/password to tell buildbot to connect to, right? -- Stephen -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmodai at in-nomine.org Tue Oct 5 19:27:38 2010 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Tue, 5 Oct 2010 19:27:38 +0200 Subject: [Python-Dev] issue 9807 - a glitch in coexisting builds of different types In-Reply-To: <20101005102122.602a46a2@mission> References: <20101001200657.472c2f60@mission> <20101002022418.64ef3ba0@pitrou.net> <4CA6E2D0.9010407@v.loewis.de> <20101002134051.5a7902b1@pitrou.net> <20101004144202.40fa1085@mission> <20101005084527.GC68173@nexus.in-nomine.org> <20101005102122.602a46a2@mission> Message-ID: <20101005172738.GG85444@nexus.in-nomine.org> -On [20101005 16:21], Barry Warsaw (barry at python.org) wrote: >Do any BSD distros provide multiple different builds of Python to their users? For all I can tell, no. Assuming you are referring to different builds of the same version. >How do they handle having a debug build or non-debug build? Install in the same place. So you either install a debug build or a stripped build. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B The fragrance always stays in the hand that gives the rose... From dsdale24 at gmail.com Tue Oct 5 19:28:19 2010 From: dsdale24 at gmail.com (Darren Dale) Date: Tue, 5 Oct 2010 13:28:19 -0400 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: <20101005184326.45a2a013@pitrou.net> References: <4CAB4FCA.4080502@voidspace.org.uk> <20101005184326.45a2a013@pitrou.net> Message-ID: On Tue, Oct 5, 2010 at 12:43 PM, Antoine Pitrou wrote: > On Tue, 05 Oct 2010 17:18:18 +0100 > Michael Foord wrote: >> > >> > Generally I'm +0 on relative imports as a whole. >> >> As the OP pointed out, for code that may be *included* in other projects >> there is no other choice. This is often useful for packages shared >> between one or two projects that nonetheless don't warrant separate >> distribution. > > You can put several packages in a single distribution. Thats not the point though. Due to compatibility issues, maybe I don't want to expose the code at the top level. Maybe the foo package is distributed elsewhere as a top-level package, but I need to use an older version due to compatibility problems. I certainly don't want to risk overwriting a pre-existing installation of foo with my required version of foo. This is not a hypothetical, we once had exactly this problem when we distributed an old version of enthought.traits with matplotlib (even though we checked for pre-existing installations, crufty build/ directories containing the out-of-date traits package were overwriting existing installations). Darren From solipsis at pitrou.net Tue Oct 5 19:45:09 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 05 Oct 2010 19:45:09 +0200 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: References: <4CAB4FCA.4080502@voidspace.org.uk> <20101005184326.45a2a013@pitrou.net> Message-ID: <1286300709.3399.16.camel@localhost.localdomain> Le mardi 05 octobre 2010 ? 13:28 -0400, Darren Dale a ?crit : > >> > >> As the OP pointed out, for code that may be *included* in other projects > >> there is no other choice. This is often useful for packages shared > >> between one or two projects that nonetheless don't warrant separate > >> distribution. > > > > You can put several packages in a single distribution. > > Thats not the point though. Due to compatibility issues, maybe I don't > want to expose the code at the top level. Maybe the foo package is > distributed elsewhere as a top-level package, but I need to use an > older version due to compatibility problems. I certainly don't want to > risk overwriting a pre-existing installation of foo with my required > version of foo. This is not a hypothetical, we once had exactly this > problem when we distributed an old version of enthought.traits with > matplotlib That use case requires that the third-party package, not your package, use relative imports. I don't think you can require other projects to follow your coding style recommendations (unless of course you maintain both). And of course you can simply set sys.path to point to your private copies of vendor libraries. Or you can manually import them using the "imp" module. > (even though we checked for pre-existing installations, > crufty build/ directories containing the out-of-date traits package > were overwriting existing installations). I'm not sure I understand the issue. If there's a distutils bug you can report it on http://bugs.python.org. In the end, I think Python offers enough packaging and module locating/loading options that relative imports shouldn't be used just for "distribution purposes". Regards Antoine. From merwok at netwok.org Tue Oct 5 20:03:52 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 05 Oct 2010 20:03:52 +0200 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: References: Message-ID: <4CAB6888.1050408@netwok.org> > If they were actively discouraged, perhaps performing a relative > import would raise a warning, This would be done if this import style was deprecated. It?s different from it being discouraged. > or maybe distutils would raise a warning at install time, Distutils does not inspect source files. > Up until now, I was not aware that use of PEP 328 relative imports > might be discouraged. I'm still unclear as to why they might be > discouraged. I think relative imports are a nice thing. I think they?ve been added because there are people who want them, and the previous implicit relative imports caused so much pain. Neither the FAQ you quote nor PEP 8 really explain what?s bad about explicit relative imports. Regards From alexander.belopolsky at gmail.com Tue Oct 5 20:15:20 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 5 Oct 2010 14:15:20 -0400 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: <4CAB44B9.2060403@v.loewis.de> References: <4CA7902D.6060504@v.loewis.de> <4CAB44B9.2060403@v.loewis.de> Message-ID: I followed the review link from issue5109 to arrive at http://bugs.python.org/review/5109/patch/179/325 . On the review page I clicked on Modules/arraymodule.c > View for side-by-side diff and got Error fetching None/Modules/arraymodule.c?rev=83179: InvalidURLError: Protocol '%s' is not supported. On the other hand, the unified diff link works. Note that the "Tracker Branch" shows /branches/release27-maint even though the patch is for py3k. On Tue, Oct 5, 2010 at 11:31 AM, "Martin v. L?wis" wrote: > >> I feel that, only if a roundup issue has patch, the corresponding >> rietveld issue be created, it is more helpful there and avoids >> needless duplication. > > I have changed that now. > > Regards, > Martin > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/alexander.belopolsky%40gmail.com > From dsdale24 at gmail.com Tue Oct 5 20:17:47 2010 From: dsdale24 at gmail.com (Darren Dale) Date: Tue, 5 Oct 2010 14:17:47 -0400 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: <1286300709.3399.16.camel@localhost.localdomain> References: <4CAB4FCA.4080502@voidspace.org.uk> <20101005184326.45a2a013@pitrou.net> <1286300709.3399.16.camel@localhost.localdomain> Message-ID: On Tue, Oct 5, 2010 at 1:45 PM, Antoine Pitrou wrote: > Le mardi 05 octobre 2010 ? 13:28 -0400, Darren Dale a ?crit : >> >> >> >> As the OP pointed out, for code that may be *included* in other projects >> >> there is no other choice. This is often useful for packages shared >> >> between one or two projects that nonetheless don't warrant separate >> >> distribution. >> > >> > You can put several packages in a single distribution. >> >> Thats not the point though. Due to compatibility issues, maybe I don't >> want to expose the code at the top level. Maybe the foo package is >> distributed elsewhere as a top-level package, but I need to use an >> older version due to compatibility problems. I certainly don't want to >> risk overwriting a pre-existing installation of foo with my required >> version of foo. This is not a hypothetical, we once had exactly this >> problem when we distributed an old version of enthought.traits with >> matplotlib > > That use case requires that the third-party package, not your package, > use relative imports. I don't think you can require other projects to > follow your coding style recommendations (unless of course you maintain > both). I'm not talking about requiring other projects to follow my coding style. > I'm not sure I understand the issue. The issue is implementing a PEP with nice support for relative imports, and then documenting that it should never be used. Darren From fuzzyman at voidspace.org.uk Tue Oct 5 20:22:35 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 05 Oct 2010 19:22:35 +0100 Subject: [Python-Dev] Adding Wing 4 project to In-Reply-To: <20101005140041.6f96103c@pitrou.net> References: <4CAB0E54.4010300@voidspace.org.uk> <20101005140041.6f96103c@pitrou.net> Message-ID: <4CAB6CEB.6050400@voidspace.org.uk> On 05/10/2010 13:00, Antoine Pitrou wrote: > On Tue, 05 Oct 2010 12:39:00 +0100 > Michael Foord wrote: >> 1. rename the old file 'python-wing3.wpr' and rename 'python-wing4.wpr' >> to 'python-wing.wpr' >> 2. delete the wing 3 project file altogether and rename >> 'python-wing4.wpr' to 'python-wing.wpr' >> 3. stay with versioned project file names and rename 'python-wing.wpr' >> to 'python-wing3.wpr' >> >> (Option 3 could be done immediately of course.) > Option 3 looks the most reasonable to me. I'll give it a bit more time, but in the absence of dissenting opinions (a rare thing on python-dev) I'll go ahead and do this. At least it can be done now. All the best, Michael > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From solipsis at pitrou.net Tue Oct 5 20:23:17 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Oct 2010 20:23:17 +0200 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: References: <4CAB4FCA.4080502@voidspace.org.uk> <20101005184326.45a2a013@pitrou.net> <1286300709.3399.16.camel@localhost.localdomain> Message-ID: <20101005202317.149174dd@pitrou.net> On Tue, 5 Oct 2010 14:17:47 -0400 Darren Dale wrote: > >> Thats not the point though. Due to compatibility issues, maybe I don't > >> want to expose the code at the top level. Maybe the foo package is > >> distributed elsewhere as a top-level package, but I need to use an > >> older version due to compatibility problems. I certainly don't want to > >> risk overwriting a pre-existing installation of foo with my required > >> version of foo. This is not a hypothetical, we once had exactly this > >> problem when we distributed an old version of enthought.traits with > >> matplotlib > > > > That use case requires that the third-party package, not your package, > > use relative imports. I don't think you can require other projects to > > follow your coding style recommendations (unless of course you maintain > > both). > > I'm not talking about requiring other projects to follow my coding style. Then can you explain your use case a bit better? It is not obvious at all what kind of "solution" relative imports allow, and why not using them is a heavy burden. > > I'm not sure I understand the issue. > > The issue is implementing a PEP with nice support for relative > imports, and then documenting that it should never be used. While I haven't written the FAQ, I certainly sympathize with the idea that relative imports are much less readable than absolute imports (especially when they climb upwards in the directory hierarchy, i.e. "from ..foo import bar"). Regards Antoine. From guido at python.org Tue Oct 5 20:21:56 2010 From: guido at python.org (Guido van Rossum) Date: Tue, 5 Oct 2010 11:21:56 -0700 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: References: <4CAB4FCA.4080502@voidspace.org.uk> <20101005184326.45a2a013@pitrou.net> <1286300709.3399.16.camel@localhost.localdomain> Message-ID: On Tue, Oct 5, 2010 at 11:17 AM, Darren Dale wrote: > The issue is implementing a PEP with nice support for relative > imports, and then documenting that it should never be used. Isn't this mostly historical? Until the new relative-import syntax was implemented there were various problems with relative imports. The short-term solution was to recommend not using them. The long-term solution was to implement an unambiguous syntax. Now it is time to withdraw the anti-recommendation. Of course, without going overboard -- I still find them an acquired taste; but they have their place. -- --Guido van Rossum (python.org/~guido) From martin at v.loewis.de Tue Oct 5 21:21:27 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 05 Oct 2010 21:21:27 +0200 Subject: [Python-Dev] [Pythonmac-SIG] sad state of OS X Python testing... In-Reply-To: References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> <20101002203205.GA24940@illinois.edu> <4CA79814.3030302@v.loewis.de> Message-ID: <4CAB7AB7.8040305@v.loewis.de> > I have a fairly recent MacPro on Snow Leopard, which I > keep consistently up to date and its connected all the time. It has more > capacity then I can really find use for. > > If its still needed, I can set up buildbot to run on it today. That would be nice. > Is it all > pull/poll oriented, or does the slave need to be connected to by the > master? It uses Perspective Broker, if that's of any help. The slave creates an outgoing TCP connection (which should be kept up all the time), but the master pushes commands over that connection, no polling. > IIUC, I just need a name/password to tell buildbot to connect to, right? Correct. However, I wouldn't want to create that unless you had all the software in place already - else I find myself often waiting for some time, then removing the slave again. Regards, Martin From martin at v.loewis.de Tue Oct 5 21:25:34 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 05 Oct 2010 21:25:34 +0200 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: References: <4CA7902D.6060504@v.loewis.de> <4CAB44B9.2060403@v.loewis.de> Message-ID: <4CAB7BAE.8010500@v.loewis.de> Am 05.10.10 20:15, schrieb Alexander Belopolsky: > I followed the review link from issue5109 to arrive at > http://bugs.python.org/review/5109/patch/179/325 . On the review page > I clicked on Modules/arraymodule.c> View for side-by-side diff and > got > > Error fetching None/Modules/arraymodule.c?rev=83179: InvalidURLError: > Protocol '%s' is not supported. That currently happens for a lot of patches. It can't figure out what the base branch is, and goes to the Rietveld Issue object - on which it is None (because I believe Rietveld is misguided in associating base URLs with issues - they belong to patchsets, as different patches on the same issues might work on different branches). > On the other hand, the unified diff link works. It already has the unified diff - just not the base text that it was generated from (and hence also not the new text). > Note that the "Tracker Branch" shows /branches/release27-maint even > though the patch is for py3k. Yes - it miscomputed it, based on the revision number in the patch (which was a revision in which 2.7 was modified). Regards, Martin From tjreedy at udel.edu Tue Oct 5 21:37:47 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 05 Oct 2010 15:37:47 -0400 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: References: <4CAB4FCA.4080502@voidspace.org.uk> <20101005184326.45a2a013@pitrou.net> <1286300709.3399.16.camel@localhost.localdomain> Message-ID: On 10/5/2010 2:21 PM, Guido van Rossum wrote: > On Tue, Oct 5, 2010 at 11:17 AM, Darren Dale wrote: >> The issue is implementing a PEP with nice support for relative >> imports, and then documenting that it should never be used. > > Isn't this mostly historical? Until the new relative-import syntax was > implemented there were various problems with relative imports. The > short-term solution was to recommend not using them. The long-term > solution was to implement an unambiguous syntax. Now it is time to > withdraw the anti-recommendation. Of course, without going overboard > -- I still find them an acquired taste; but they have their place. Darren, if you have not yet done so, open a tracker that quotes the above and gives your recommended changes at which locations. -- Terry Jan Reedy From dsdale24 at gmail.com Tue Oct 5 22:19:57 2010 From: dsdale24 at gmail.com (Darren Dale) Date: Tue, 5 Oct 2010 16:19:57 -0400 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: References: <4CAB4FCA.4080502@voidspace.org.uk> <20101005184326.45a2a013@pitrou.net> <1286300709.3399.16.camel@localhost.localdomain> Message-ID: On Tue, Oct 5, 2010 at 3:37 PM, Terry Reedy wrote: > On 10/5/2010 2:21 PM, Guido van Rossum wrote: >> >> On Tue, Oct 5, 2010 at 11:17 AM, Darren Dale ?wrote: >>> >>> The issue is implementing a PEP with nice support for relative >>> imports, and then documenting that it should never be used. >> >> Isn't this mostly historical? Until the new relative-import syntax was >> implemented there were various problems with relative imports. The >> short-term solution was to recommend not using them. The long-term >> solution was to implement an unambiguous syntax. Now it is time to >> withdraw the anti-recommendation. Of course, without going overboard >> -- I still find them an acquired taste; but they have their place. > > Darren, if you have not yet done so, open a tracker that quotes the above > and gives your recommended changes at which locations. Done: http://bugs.python.org/issue10031 Darren From martin at v.loewis.de Tue Oct 5 22:55:41 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 05 Oct 2010 22:55:41 +0200 Subject: [Python-Dev] stable builders In-Reply-To: <20101005190741.3d863fc9@pitrou.net> References: <4CAAED9B.405@v.loewis.de> <4CAB3F00.7050107@v.loewis.de> <20101005190741.3d863fc9@pitrou.net> Message-ID: <4CAB90CD.1020803@v.loewis.de> Am 05.10.10 19:07, schrieb Antoine Pitrou: > On Tue, 05 Oct 2010 17:06:40 +0200 > "Martin v. L?wis" wrote: >>> I can probably run a build slave on one of my boxes (Gentoo, Athlon 64 >>> x2). Where are the setup docs? >> >> http://wiki.python.org/moin/BuildBot > > By the way, is the distinction between "stable" and "unstable" builders > still relevant? It's still formally the requirement for making releases. > If I look at http://www.python.org/dev/buildbot/3.x.stable/, two of our > four stable builders are offline (and at least one of them - the ia64 > one - has been for a long time). Granted, this makes them "stable" in a > sense ;) I guess somebody would need to do monitoring on them, and ping operators if the buildbot is down for an extended period of time. Feel free to ping any operator whenever you notice that a slave is down (they do get an automated email, but people can get resistant to automated emails). Also, if you would want to propose that a different set than the current ones should be considered stable, please let me know. I believe "stable" was meant in a different way, though - it would reliably pass all tests, and a test failure should be considered a bug, rather than some random failure on the slave. Regards, Martin From ncoghlan at gmail.com Tue Oct 5 23:09:48 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 6 Oct 2010 07:09:48 +1000 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: References: <4CAB4FCA.4080502@voidspace.org.uk> <20101005184326.45a2a013@pitrou.net> <1286300709.3399.16.camel@localhost.localdomain> Message-ID: On Wed, Oct 6, 2010 at 4:21 AM, Guido van Rossum wrote: > On Tue, Oct 5, 2010 at 11:17 AM, Darren Dale wrote: >> The issue is implementing a PEP with nice support for relative >> imports, and then documenting that it should never be used. > > Isn't this mostly historical? Until the new relative-import syntax was > implemented there were various problems with relative imports. The > short-term solution was to recommend not using them. The long-term > solution was to implement an unambiguous syntax. Now it is time to > withdraw the anti-recommendation. Of course, without going overboard > -- I still find them an acquired taste; but they have their place. Indeed, the objections raised in those FAQ entries mostly applied to the problems implicit relative imports could silently cause. Explicit relative imports will throw exceptions for those cases instead. They read like someone took the old text and just modified it to refer to explicit imports instead of implicit ones without softening the language at all. The remaining scenarios we have that can lead to duplication of a module happen regardless of the import style you use*. Cheers, Nick. *For the curious - those scenarios relate to ending up with the same module present both as "__main__" and under its normal module name. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Tue Oct 5 23:12:31 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 6 Oct 2010 07:12:31 +1000 Subject: [Python-Dev] stable builders In-Reply-To: <4CAB90CD.1020803@v.loewis.de> References: <4CAAED9B.405@v.loewis.de> <4CAB3F00.7050107@v.loewis.de> <20101005190741.3d863fc9@pitrou.net> <4CAB90CD.1020803@v.loewis.de> Message-ID: On Wed, Oct 6, 2010 at 6:55 AM, "Martin v. L?wis" wrote: > I guess somebody would need to do monitoring on them, and ping operators > if the buildbot is down for an extended period of time. Feel free to ping > any operator whenever you notice that a slave is down (they do get > an automated email, but people can get resistant to automated emails). > > Also, if you would want to propose that a different set than the current > ones should be considered stable, please let me know. I believe > "stable" was meant in a different way, though - it would reliably pass all > tests, and a test failure should be considered a bug, rather than > some random failure on the slave. To simplify the task of contacting buildbot operators, would it be worth having a "python-buildbot-owners" mailing list? Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From martin at v.loewis.de Tue Oct 5 23:29:26 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 05 Oct 2010 23:29:26 +0200 Subject: [Python-Dev] stable builders In-Reply-To: References: <4CAAED9B.405@v.loewis.de> <4CAB3F00.7050107@v.loewis.de> <20101005190741.3d863fc9@pitrou.net> <4CAB90CD.1020803@v.loewis.de> Message-ID: <4CAB98B6.7020506@v.loewis.de> > To simplify the task of contacting buildbot operators, would it be > worth having a "python-buildbot-owners" mailing list? Depends on the traffic. It might spam those owners who are never the target of any of these messages, because they are "well-behaving". Do you really find it difficult to determine the email address of any specific operator? Who? If you really need to broadcast a message to all owners, let me know and I can forward it to them (or you can collect the addresses yourself). Regards, Martin From solipsis at pitrou.net Tue Oct 5 23:34:12 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 5 Oct 2010 23:34:12 +0200 Subject: [Python-Dev] stable builders In-Reply-To: <4CAB90CD.1020803@v.loewis.de> References: <4CAAED9B.405@v.loewis.de> <4CAB3F00.7050107@v.loewis.de> <20101005190741.3d863fc9@pitrou.net> <4CAB90CD.1020803@v.loewis.de> Message-ID: <20101005233412.64bc8e4b@pitrou.net> On Tue, 05 Oct 2010 22:55:41 +0200 "Martin v. L?wis" wrote: > > I guess somebody would need to do monitoring on them, and ping operators > if the buildbot is down for an extended period of time. Feel free to > ping any operator whenever you notice that a slave is down (they do get > an automated email, but people can get resistant to automated emails). Well, I suppose manual pinging can become pretty much like automated emails if it becomes frequent. (in any case, I think I've pinged Matthias at least twice on IRC, but perhaps he isn't really present on that medium, although he doesn't appear away) > Also, if you would want to propose that a different set than the current > ones should be considered stable, please let me know. I believe > "stable" was meant in a different way, though - it would reliably pass > all tests, and a test failure should be considered a bug, rather than > some random failure on the slave. I see. Well, apart from the current ones, "alpha Debian" and "i386 Ubuntu" have been quite stable. Antoine. From phd at phd.pp.ru Tue Oct 5 23:41:17 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Wed, 6 Oct 2010 01:41:17 +0400 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: References: <4CAB4FCA.4080502@voidspace.org.uk> <20101005184326.45a2a013@pitrou.net> <1286300709.3399.16.camel@localhost.localdomain> Message-ID: <20101005214117.GA22929@phd.pp.ru> On Wed, Oct 06, 2010 at 07:09:48AM +1000, Nick Coghlan wrote: > The remaining scenarios we have that can lead to duplication of a > module happen regardless of the import style you use*. > > Cheers, > Nick. > > *For the curious - those scenarios relate to ending up with the same > module present both as "__main__" and under its normal module name. I remember falling in the trap of importing the same module twice by manipulating sys.path and changing cwd (current working directory); the module was imported from different paths - from the current directory (entry '.' in the sys.path) and by its full path. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From greg.ewing at canterbury.ac.nz Wed Oct 6 01:54:10 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 06 Oct 2010 12:54:10 +1300 Subject: [Python-Dev] question/comment about documentation of relative imports In-Reply-To: References: <4CAB4FCA.4080502@voidspace.org.uk> <20101005184326.45a2a013@pitrou.net> <1286300709.3399.16.camel@localhost.localdomain> Message-ID: <4CABBAA2.4090008@canterbury.ac.nz> Guido van Rossum wrote: > Now it is time to > withdraw the anti-recommendation. Or at least re-word them all to make it clear that they're talking about the *old* style of relative import in 2.x. -- Greg From alexander.belopolsky at gmail.com Wed Oct 6 02:17:45 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 5 Oct 2010 20:17:45 -0400 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: <4CAB7BAE.8010500@v.loewis.de> References: <4CA7902D.6060504@v.loewis.de> <4CAB44B9.2060403@v.loewis.de> <4CAB7BAE.8010500@v.loewis.de> Message-ID: On Tue, Oct 5, 2010 at 3:25 PM, "Martin v. L?wis" wrote: .. >> >> Error fetching None/Modules/arraymodule.c?rev=83179: InvalidURLError: >> Protocol '%s' is not supported. > > That currently happens for a lot of patches. It can't figure out what > the base branch is, and goes to the Rietveld Issue object - on which > it is None (because I believe Rietveld is misguided in associating > base URLs with issues - they belong to patchsets, as different patches > on the same issues might work on different branches). > I attempted to fix the branch at http://bugs.python.org/file18220 , but it did not trigger rietveld update. I assume you made "Tracker Branch" editable on purpose, but there should be a way rietveld side to know when this field is changed I think. From stephen at xemacs.org Wed Oct 6 05:22:18 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 06 Oct 2010 12:22:18 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87r5g480ph.fsf@uwakimon.sk.tsukuba.ac.jp> Nick Coghlan writes: > - if you pass in bytes data and know what you are doing, then you can > access that raw bytes data and do your own decoding At what level, though? To take an interesting example I used to see frequently: From: taro at tokyo.jp (Taro Yamada in 8-bit Shift JIS) So I guess you are suggesting that the email module can RFC 822 parse that, and 1. Refuse to return the unwrapped (ie, single line) form of the whole field, except as bytes. 2. Refuse to return the content of the From field, except as bytes. 3. Return the email address parsed from the From field. 4. Refuse to return the comment, except as bytes. That's fine. But suppose I have a private or newly defined header that is structured? Now I have two choices: 1. Write a version of my private parser for both str (the normal case) and bytes (if accessing the value as str raises) 2. Always get the bytes and convert them to str (probably using the same .decode('ascii','surrogate-escape') call that email uses but won't let me have the value of!), then use a common str parser. Note that this is more problematic than it looks, since the appropriate base codec may require information from higher-level structures (eg, qp codec tags or a Content-Type header's charset field). Why should I reproduce email's logic here? I don't care if the default or concise API raises on surrogates in the str value. But I'm pretty sure that I will want to use str values containing surrogates in these contexts (for the same reasons that email module does, for example), rather than work with bytes sometimes and strs sometimes. Please provide a way to return strs-with-surrogates if I ask for them. From martin at v.loewis.de Wed Oct 6 07:23:22 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 06 Oct 2010 07:23:22 +0200 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: References: <4CA7902D.6060504@v.loewis.de> <4CAB44B9.2060403@v.loewis.de> <4CAB7BAE.8010500@v.loewis.de> Message-ID: <4CAC07CA.1030005@v.loewis.de> > I attempted to fix the branch at http://bugs.python.org/file18220 , > but it did not trigger rietveld update. I assume you made "Tracker > Branch" editable on purpose, but there should be a way rietveld side > to know when this field is changed I think. Please start submitting issues with the Rietveld integration into the meta tracker. Regards, Martin From kalman.gergely at duodecad.hu Wed Oct 6 09:34:05 2010 From: kalman.gergely at duodecad.hu (=?ISO-8859-1?Q?K=E1lm=E1n_Gergely?=) Date: Wed, 06 Oct 2010 09:34:05 +0200 Subject: [Python-Dev] socket.fromfd() documentation problem Message-ID: <4CAC266D.7040105@duodecad.hu> Hello I was having a very nasty fd leak recently where I've leaked more than 200k FDs, allocating more than 1Gbytes of ram in kernel space. It was my fault alright, but I thought I'd mention it here so maybe you'll put a little NOTE section in the documentation mentioning that you have to os.close() the original FD to avoid leakage. Also I'm not completely clear why python does it this way and why not close the original socket - it seems (at least to me) that this would be the right(er) way. Nevertheless what are your thoughts on this? Should I file a bug report for it? (The best part was that the allocated FD memory did not show up as slab, so this was a real pain in the butt to hunt down) Kalman Gergely From victor.stinner at haypocalc.com Wed Oct 6 14:12:48 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Wed, 6 Oct 2010 14:12:48 +0200 Subject: [Python-Dev] socket.fromfd() documentation problem In-Reply-To: <4CAC266D.7040105@duodecad.hu> References: <4CAC266D.7040105@duodecad.hu> Message-ID: <201010061412.48549.victor.stinner@haypocalc.com> Le mercredi 06 octobre 2010 09:34:05, K?lm?n Gergely a ?crit : > Nevertheless what are your thoughts on this? Should I file a bug report > for it? It will be fixed faster if you open an issue and attach a patch ;-) -- Victor Stinner http://www.haypocalc.com/ From stephen at xemacs.org Wed Oct 6 15:55:00 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 06 Oct 2010 22:55:00 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101005150523.CA7D6228E86@kimball.webabinitio.net> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> Message-ID: <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> R. David Murray writes: > version of headers to the email5 API, but since any such data would > be non-RFC compliant anyway, [access to non-conforming headers by > reparsing the bytes] will just have to be good enough for now. But that's potentially unpleasant for, say, Mailman. AFAICS, what you're saying is that Mailman will have to implement a full header parser and repair module, or shunt (and wait for administrator intervention on) any mail that happens to contain even one byte of non-RFC-conforming content in a header it cares about. (Note that we're not talking about moderator-level admins here; we're talking about the Big Cheese with access to the command line on the list host.) That's substantially worse than the current system, where (in theory, and in actual practice where it distributes its own version of email) it can trap the Unicode exception on a per-header basis. I also worry about the implications for backwards compatibility. Eventually email-N needs to handle non-conforming mail in a sensible way, or anybody who gets spam (ie, everybody) and wants a reliable email system will need to implement their own. If you punt completely on handling non-conforming mail now, when is it going to be done? And when it is done, will the backward-compatible interface be able to access the robust implementation, or will people who want robust APIs have to use rather different ones? The way you're going right now, I have to worry about the answer to the second question, at least. > [*] Why '?' and not the unicode invalid character character? Well, the > email5 Generate.flatten can be used to generate data for transmission over > the wire *if* the source is RFC compliant and 7bit-only, and this would > be a normal email5 usage pattern (that is, smtplib.SMTP.sendmail expects > ASCII-only strings as input!). So the data generated by Generator.flatten > should not include unicode... I don't understand this at all. Of course the byte stream generated by Generator.flatten won't contain Unicode (in the headers, anyway); it will contain only ASCII (that happens to conform to QP or Base64 encoding of Unicode in some appropriate UTF in many cases). Why is U+FFFD REPLACEMENT CHARACTER any different from any other non-ASCII character in this respect? (Surely you are not saying that Generator.flatten can't DTRT with non-ASCII content *at all*?) The only thing I can think of is that you might not want to introduce non-ASCII characters into a string that looks like it might simply be corrupted in transmission (eg, it contains only one non-ASCII byte). That's reasonable; there are a lot of people who don't have to deal with anything but ASCII and occasionally Latin-1, and they don't like having Unicode crammed down their throats. > which raises a problem for CTE 8bit sections > that the patch doesn't currently address. AFAIK, there's no requirement, implied or otherwise, that a conforming implementation *produce* CTE 8bit. So just don't do that; that will keep smtplib happy, no? From benjamin at python.org Wed Oct 6 17:02:22 2010 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 6 Oct 2010 10:02:22 -0500 Subject: [Python-Dev] [Python-checkins] r85288 - in python/branches/py3k/Lib: concurrent/futures/_base.py test/test_concurrent_futures.py In-Reply-To: <20101006130545.9D068EE981@mail.python.org> References: <20101006130545.9D068EE981@mail.python.org> Message-ID: 2010/10/6 brian.quinlan : > Author: brian.quinlan > Date: Wed Oct ?6 15:05:45 2010 > New Revision: 85288 > > Log: > Fixes 9903: test_concurrent_futures writes on stderr > > Modified: > ? python/branches/py3k/Lib/concurrent/futures/_base.py > ? python/branches/py3k/Lib/test/test_concurrent_futures.py > > Modified: python/branches/py3k/Lib/concurrent/futures/_base.py > ============================================================================== > --- python/branches/py3k/Lib/concurrent/futures/_base.py ? ? ? ?(original) > +++ python/branches/py3k/Lib/concurrent/futures/_base.py ? ? ? ?Wed Oct ?6 15:05:45 2010 > @@ -40,9 +40,8 @@ > > ?# Logger for internal use by the futures package. > ?LOGGER = logging.getLogger("concurrent.futures") > -_handler = logging.StreamHandler() > -LOGGER.addHandler(_handler) > -del _handler > +STDERR_HANDLER = logging.StreamHandler() > +LOGGER.addHandler(STDERR_HANDLER) > > ?class Error(Exception): > ? ? """Base class for all future-related exceptions.""" > > Modified: python/branches/py3k/Lib/test/test_concurrent_futures.py > ============================================================================== > --- python/branches/py3k/Lib/test/test_concurrent_futures.py ? ?(original) > +++ python/branches/py3k/Lib/test/test_concurrent_futures.py ? ?Wed Oct ?6 15:05:45 2010 > @@ -9,6 +9,8 @@ > ?# without thread support. > ?test.support.import_module('threading') > > +import io > +import logging > ?import multiprocessing > ?import sys > ?import threading > @@ -21,7 +23,8 @@ > > ?from concurrent import futures > ?from concurrent.futures._base import ( > - ? ?PENDING, RUNNING, CANCELLED, CANCELLED_AND_NOTIFIED, FINISHED, Future, wait) > + ? ?PENDING, RUNNING, CANCELLED, CANCELLED_AND_NOTIFIED, FINISHED, Future, > + ? ?LOGGER, STDERR_HANDLER, wait) > ?import concurrent.futures.process > > ?def create_future(state=PENDING, exception=None, result=None): > @@ -617,24 +620,33 @@ > ? ? ? ? self.assertTrue(was_cancelled) > > ? ? def test_done_callback_raises(self): > - ? ? ? ?raising_was_called = False > - ? ? ? ?fn_was_called = False > - > - ? ? ? ?def raising_fn(callback_future): > - ? ? ? ? ? ?nonlocal raising_was_called > - ? ? ? ? ? ?raising_was_called = True > - ? ? ? ? ? ?raise Exception('doh!') > - > - ? ? ? ?def fn(callback_future): > - ? ? ? ? ? ?nonlocal fn_was_called > - ? ? ? ? ? ?fn_was_called = True > - > - ? ? ? ?f = Future() > - ? ? ? ?f.add_done_callback(raising_fn) > - ? ? ? ?f.add_done_callback(fn) > - ? ? ? ?f.set_result(5) > - ? ? ? ?self.assertTrue(raising_was_called) > - ? ? ? ?self.assertTrue(fn_was_called) > + ? ? ? ?LOGGER.removeHandler(STDERR_HANDLER) > + ? ? ? ?logging_stream = io.StringIO() > + ? ? ? ?handler = logging.StreamHandler(logging_stream) > + ? ? ? ?LOGGER.addHandler(handler) > + ? ? ? ?try: > + ? ? ? ? ? ?raising_was_called = False > + ? ? ? ? ? ?fn_was_called = False > + > + ? ? ? ? ? ?def raising_fn(callback_future): > + ? ? ? ? ? ? ? ?nonlocal raising_was_called > + ? ? ? ? ? ? ? ?raising_was_called = True > + ? ? ? ? ? ? ? ?raise Exception('doh!') > + > + ? ? ? ? ? ?def fn(callback_future): > + ? ? ? ? ? ? ? ?nonlocal fn_was_called > + ? ? ? ? ? ? ? ?fn_was_called = True > + > + ? ? ? ? ? ?f = Future() > + ? ? ? ? ? ?f.add_done_callback(raising_fn) > + ? ? ? ? ? ?f.add_done_callback(fn) > + ? ? ? ? ? ?f.set_result(5) > + ? ? ? ? ? ?self.assertTrue(raising_was_called) > + ? ? ? ? ? ?self.assertTrue(fn_was_called) > + ? ? ? ? ? ?self.assertIn('Exception: doh!', logging_stream.getvalue()) > + ? ? ? ?finally: > + ? ? ? ? ? ?LOGGER.removeHandler(handler) > + ? ? ? ? ? ?LOGGER.addHandler(STDERR_HANDLER) You could use TestCase.addCleanup() here. -- Regards, Benjamin From rdmurray at bitdance.com Wed Oct 6 18:18:03 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 06 Oct 2010 12:18:03 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <87r5g480ph.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <87r5g480ph.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101006161803.E968621B04C@kimball.webabinitio.net> On Wed, 06 Oct 2010 12:22:18 +0900, "Stephen J. Turnbull" wrote: > Nick Coghlan writes: > > > - if you pass in bytes data and know what you are doing, then you can > > access that raw bytes data and do your own decoding > > At what level, though? > > To take an interesting example I used to see frequently: > > From: taro at tokyo.jp > (Taro Yamada in 8-bit Shift JIS) > > So I guess you are suggesting that the email module can RFC 822 parse > that, and > > 1. Refuse to return the unwrapped (ie, single line) form of the whole > field, except as bytes. > 2. Refuse to return the content of the From field, except as bytes. > 3. Return the email address parsed from the From field. > 4. Refuse to return the comment, except as bytes. 5. Return the content, with non-ASCII bytes replaced with ? characters. In other words, my proposed patch only makes email5 1/8 to 1/4 broken, instead of half broken as it is now. But not un-broken enough for Mailman, it sounds like. > That's fine. But suppose I have a private or newly defined header > that is structured? Now I have two choices: > > 1. Write a version of my private parser for both str (the normal > case) and bytes (if accessing the value as str raises) > > 2. Always get the bytes and convert them to str (probably using the > same .decode('ascii','surrogate-escape') call that email uses but > won't let me have the value of!), then use a common str parser. Yes, this is exactly the dilemma faced by the entire email package. The current email6 code attempts to do a variation on (1) by having a common parser that handles both strings and bytes using a dual subclass approach. This patch is trying out (2). If you have a private header parser, you would ideally like to be able to use the same mechanism as the email package to solve the problem. For email6 you'd be able to register your header parser and get handed the input like the built in parser and be able to use the tools provided by the built in parser to do your work. In email5 there is no way that I know of for you to register a private parser, so you need access to the raw input for the header in one form or another. If we go this route (as opposed to only handling headers with 8bit data by sanitizing them), then we need to think about the email5 header parsers as well (decode_header and parseaddr). They are of course going to have the same problems as the rest of the email package with parsing bytes, and you are suggesting that access to those header 8bit bytes is needed. One option would be to add a keyword to the get and get_all methods that instructs it to return the string with the surrogate-escaped bytes, which can then be passed onward to decode_header, parseaddr, or a custom decoder. Then I need to look at what needs to be added to those methods to handle the escaped bytes, and from what you say they too need a keyword telling them to preserve the escaped bytes on output (a "yes I know what I'm doing" flag...'preserve_escaped_bytes=True'?). > Note that this is more problematic than it looks, since the > appropriate base codec may require information from higher-level > structures (eg, qp codec tags or a Content-Type header's charset > field). You'll have to give me an example of where this is a problem but is not already a problem in email4. > Why should I reproduce email's logic here? I don't care if the > default or concise API raises on surrogates in the str value. But I'm > pretty sure that I will want to use str values containing surrogates > in these contexts (for the same reasons that email module does, for > example), rather than work with bytes sometimes and strs sometimes. > > Please provide a way to return strs-with-surrogates if I ask for them. Does my proposal make sense? But note, it raises exactly the backward compatibility concerns you mention in your next email (that I will reply to next). It is an open question whether it is worth opening that door in order to be able to do extended handling on non-RFC conforming email (as opposed to just sanitizing it and soldering on). -- R. David Murray www.bitdance.com From rdmurray at bitdance.com Wed Oct 6 19:09:25 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 06 Oct 2010 13:09:25 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101006170925.4EAA022A999@kimball.webabinitio.net> On Wed, 06 Oct 2010 22:55:00 +0900, "Stephen J. Turnbull" wrote: > R. David Murray writes: > > > version of headers to the email5 API, but since any such data would > > be non-RFC compliant anyway, [access to non-conforming headers by > > reparsing the bytes] will just have to be good enough for now. > > But that's potentially unpleasant for, say, Mailman. AFAICS, what > you're saying is that Mailman will have to implement a full header > parser and repair module, or shunt (and wait for administrator > intervention on) any mail that happens to contain even one byte of > non-RFC-conforming content in a header it cares about. (Note that No, it just means that such bytes would not be preserved for presentation in the web UI. They'd show up as '?'s (Or U+FFFDs, perhaps, if I change DeocdedGenerator to use U+FFFD instead of ?s for the unknown bytes). As long as BytesGenerator is used on the output side to send the messages, the bytes will be preserved and presented to the moderator in their email. So the only parsing issue is if Mailman cares about *the non-ASCII bytes* in the headers it cares about. If it has to modify headers that contain non-ASCII bytes (for example, addresses and Subject) and cares about preserving the non-ASCII bytes, then there is indeed an issue; see previous email for a possible way around that. > we're not talking about moderator-level admins here; we're talking > about the Big Cheese with access to the command line on the list > host.) That's substantially worse than the current system, where (in > theory, and in actual practice where it distributes its own version of > email) it can trap the Unicode exception on a per-header basis. I thought mailman no longer distributed its own version of email? And the email API currently promises not to raise during parsing, which is a contract my patch does not change. > I also worry about the implications for backwards compatibility. > Eventually email-N needs to handle non-conforming mail in a sensible > way, or anybody who gets spam (ie, everybody) and wants a reliable > email system will need to implement their own. If you punt completely > on handling non-conforming mail now, when is it going to be done? And We're (in the current patch) not punting on handling non-conforming email, we're punting on handling non-conforming bytes *if the headers that contain them need to be modified*. The headers can still be modified, you just (currently) lose the non-ASCII bytes in the process. > when it is done, will the backward-compatible interface be able to > access the robust implementation, or will people who want robust APIs > have to use rather different ones? The way you're going right now, I > have to worry about the answer to the second question, at least. Well, this is still theory given the current state of the email6 code, but I *think* that working email5 code, even after this patch, will continue to work using email6's backward compatibility interface. And robustness is not the issue, only extended-beyond-the-RFCs handling of non-conforming bytes would be an issue. *But*, as I implied in my previous email, if we allow the surrogates out so that custom header parsers can use them, then making *that* code continue to work may require an extra layer in the compatibility interface to produce the surrogateescaped strings. Still, at the moment I can't see any theoretical reason why that would not be possible, so it may be worth the risk. > > [*] Why '?' and not the unicode invalid character character? Well, the > > email5 Generate.flatten can be used to generate data for transmission over > > the wire *if* the source is RFC compliant and 7bit-only, and this would > > be a normal email5 usage pattern (that is, smtplib.SMTP.sendmail expects > > ASCII-only strings as input!). So the data generated by Generator.flatten > > should not include unicode... > > I don't understand this at all. Of course the byte stream generated > by Generator.flatten won't contain Unicode (in the headers, anyway); > it will contain only ASCII (that happens to conform to QP or Base64 > encoding of Unicode in some appropriate UTF in many cases). Why is > U+FFFD REPLACEMENT CHARACTER any different from any other non-ASCII > character in this respect? > > (Surely you are not saying that Generator.flatten can't DTRT with > non-ASCII content *at all*?) Yes, that is *exactly* what I am saying: >>> m = email.message_from_string("""\ ... From: p??stal ... ... """) >>> str(m) Traceback (most recent call last): .... UnicodeEncodeError: 'ascii' codec can't encode character '\xf6' in position 1: ordinal not in range(128) Remember, email5 is a direct translation of email4, and email4 only handled ASCII and oh-by-the-way-if-there-are-bytes-along-for-the- -ride-fine-we'll-pass-then-along. So if you want to put non-ASCII data into a message you have to encode it properly to ASCII in exactly the same way that you did in email4: >>> m = email.message.Message() >>> m['From'] = email.header.Header("p??stal", charset='utf-8') >>> str(m) 'From: =?utf-8?q?p=C3=B6stal?=\n\n' > The only thing I can think of is that you might not want to introduce > non-ASCII characters into a string that looks like it might simply be > corrupted in transmission (eg, it contains only one non-ASCII byte). > That's reasonable; there are a lot of people who don't have to deal > with anything but ASCII and occasionally Latin-1, and they don't like > having Unicode crammed down their throats. > > > which raises a problem for CTE 8bit sections > > that the patch doesn't currently address. > > AFAIK, there's no requirement, implied or otherwise, that a conforming > implementation *produce* CTE 8bit. So just don't do that; that will > keep smtplib happy, no? Yes, exactly. I need to fix the patch to recode using, say, quoted-printable in that case. DecodedGenerator could still produce the unicode, though, which is what I believe we want. (Although that raises the question of whether DecodedGenerator should also decode the RFC2047 encoded headers....but that raises a backward compatibility issue). --David From stephen at xemacs.org Wed Oct 6 20:31:34 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 07 Oct 2010 03:31:34 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101006161803.E968621B04C@kimball.webabinitio.net> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <87r5g480ph.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006161803.E968621B04C@kimball.webabinitio.net> Message-ID: <87aamr896h.fsf@uwakimon.sk.tsukuba.ac.jp> R. David Murray writes: > 5. Return the content, with non-ASCII bytes replaced with ? > characters. That hadn't occurred to me (and it makes me sick to contemplate it). That said, this is probably good enough for Mailman-like apps to limp along for "most" users. It's certainly good enough for the "might kick your wife and elope with your dog" alpha ports of Mailman to Python 3 (well, as certain as I can be; of course in the end Barry decides). Assuming reasonable backward compatibility of the API, of course! > In other words, my proposed patch only makes email5 1/8 to 1/4 > broken, instead of half broken as it is now. But not un-broken > enough for Mailman, it sounds like. IMO, not in the long run. But realistically, in the applications I know of, most desired traffic is conformant, and since there aren't any Python 3 email apps yet, this isn't even a regression. :-/ I do think that it's important that the parsed object be able to tell you what fields are there (except if the field name itself is invalid) and return field bodies parsed as far as possible. > If we go this route (as opposed to only handling headers with 8bit data by > sanitizing them), then we need to think about the email5 header parsers > as well (decode_header and parseaddr). They are of course going to have > the same problems as the rest of the email package with parsing bytes, > and you are suggesting that access to those header 8bit bytes is needed. Yes, that would be preferable to replacing them with ASCII junk. But I don't see any problem with parsing them; they're syntactically insignificant by definition. The problem is purely on output: do I get verbatim escaped bytes, a sanitized str, or an exception? > One option would be to add a keyword to the get and get_all methods > that instructs it to return the string with the surrogate-escaped > bytes, which can then be passed onward to decode_header, parseaddr, > or a custom decoder. Then I need to look at what needs to be added > to those methods to handle the escaped bytes, and from what you say > they too need a keyword telling them to preserve the escaped bytes > on output (a "yes I know what I'm doing" flag... > 'preserve_escaped_bytes=True'?). The need is not absolute, but I would have a strong preference for being able to get at those bytes. > Does my proposal make sense? But note, it raises exactly the backward > compatibility concerns you mention in your next email (that I will reply > to next). It is an open question whether it is worth opening that door > in order to be able to do extended handling on non-RFC conforming email > (as opposed to just sanitizing it and soldering on). Well, maybe not. However, it is not obvious to me that you won't run into these issues again in Email6. Applications that think of email as textual objects are going to want to make their own choices about handling of non-conforming email, and it's likely to be massively inconvenient to say "OK, but you have to use bytes interfaces exclusively, because the str interfaces don't handle that." From stephen at xemacs.org Wed Oct 6 21:40:03 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 07 Oct 2010 04:40:03 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101006170925.4EAA022A999@kimball.webabinitio.net> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> Message-ID: <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> R. David Murray writes: > So the only parsing issue is if Mailman cares about *the non-ASCII > bytes* in the headers it cares about. If it has to modify headers that > contain non-ASCII bytes (for example, addresses and Subject) and cares > about preserving the non-ASCII bytes, then there is indeed an issue; > see previous email for a possible way around that. OK. > I thought mailman no longer distributed its own version of email? I believe so; the point is that it could do so again. > And the email API currently promises not to raise during parsing, > which is a contract my patch does not change. Which is a contract that has historically been broken frequently. Unhandled UnicodeErrors have been one of the most common causes of queue stoppage in Mailman (exceeded only by configuration errors AFAICS). I haven't seen any reports for a while, but with the email package being reengineered from the ground up, the possibility of regression can't be ignored. Granted, there should be no regression problem in the current model for Email5, AIUI. > We're (in the current patch) not punting on handling non-conforming > email, we're punting on handling non-conforming bytes *if the headers > that contain them need to be modified*. The headers can still be > modified, you just (currently) lose the non-ASCII bytes in the process. Modified *or examined*. I can't think of any important applications offhand that *need* to examine the non-ASCII bytes (in particular, Mailman doesn't need to do that). Verbatim copying of the bytes themselves is almost always the desired usage. > And robustness is not the issue, only extended-beyond-the-RFCs handling > of non-conforming bytes would be an issue. And with that, I'm certain that Jon Postel is really dead. :-( > > (Surely you are not saying that Generator.flatten can't DTRT with > > non-ASCII content *at all*?) > > Yes, that is *exactly* what I am saying: > > >>> m = email.message_from_string("""\ > ... From: p?stal > ... > ... """) > >>> str(m) > Traceback (most recent call last): > .... > UnicodeEncodeError: 'ascii' codec can't encode character '\xf6' in position 1: ordinal not in range(128) But that's not interesting; you did that with Python 3. We want to know what people porting from Python 2 will expect. So, in 2.5.5 or 2.6.6 on Mac, with email v4.0.2, it *doesn't* raise, it returns wideload:~ 4:14$ python Python 2.5.5 (r255:77872, Jul 13 2010, 03:03:57) [GCC 4.0.1 (Apple Inc. build 5490)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import email >>> m=email.message_from_string('From: p?stal\n\n') >>> str(m) 'From nobody Thu Oct 7 04:18:25 2010\nFrom: p\xc3\xb6stal\n\n' >>> m['From'] 'p\xc3\xb6stal' >>> That's hardly helpful! Surely we can and should do better than that now, especially since UTF-8 (with a proper CTE) is now almost universally acceptable to MUAs. When would it be a problem for that to return 'From nobody Thu Oct 7 04:18:25 2010\nFrom: =?UTF-8?Q?p=C3=B6stal?=\n\n' > Remember, email5 is a direct translation of email4, and email4 only > handled ASCII and oh-by-the-way-if-there-are-bytes-along-for-the- > -ride-fine-we'll-pass-then-along. So if you want to put non-ASCII > data into a message you have to encode it properly to ASCII in > exactly the same way that you did in email4: But if you do it right, then it will still work in a version that just encodes non-ASCII characters in UTF-8 with the appropriate CTE. Since you'll never be passing it non-ASCII characters, it's already ASCII and UTF-8, and no CTE will be needed. > Yes, exactly. I need to fix the patch to recode using, say, > quoted-printable in that case. It really should check for proportions of non-ASCII. QP would be horrible for Japanese or Chinese. > DecodedGenerator could still produce the unicode, though, which is > what I believe we want. (Although that raises the question of > whether DecodedGenerator should also decode the RFC2047 encoded > headers....but that raises a backward compatibility issue). Can't really help you there. While I would want the RFC 2047 headers decoded if I were writing new code (which is generally the case for me), I haven't really wrapped my head around the issues of porting old code using Python2 str to Python3 str here. My intuition says "no problem" (there won't be any MIME-words so the app won't try to decode them), but I'm not real sure of that. ;-) From abergeron at gmail.com Wed Oct 6 22:26:57 2010 From: abergeron at gmail.com (Arnaud Bergeron) Date: Wed, 6 Oct 2010 16:26:57 -0400 Subject: [Python-Dev] hashlib bug when built on OS X 10.6 for 10.5 Message-ID: If you build python (at least 2.6.5, but probably other versions as well) in a universal setup using the following command (or similar) while the machine currently has 10.6 installed: ./configure --with-universal-archs= --enable-universalsdk=/Developer/SDKs/MacOSX10.5.sdk/ then the hashlib module will be unimportable with the following error: ImportError: No module named _md5 This is because the openssl detection code in setup.py picks up the system libssl (in /) which is 0.9.8 and selects not to build the additional _sha256, and _sh512 modules. Then, when the builds proceeds the SDK libssl is used (in /Developer/SDKs/MacOSX10.5.sdk/) which is version 0.9.7 and the openssl_sha256 and openssl_sha512 raise ValueError in the _hashlib that is built. Then the fancy code in hashlib.py detects the ValueError, tries to import _sha256 and gets an ImportError which is caught by the code set to catch an ImportError of _hashlib and this codes tries to import _md5 which also fails and gives the error above. In summary, the code in setup.py finds the wrong library and this creates a situation with which hashlib.py is not ready to handle. If the analysis is not clear on certain points, feel free to ask questions. Arnaud From rdmurray at bitdance.com Wed Oct 6 23:39:14 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 06 Oct 2010 17:39:14 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <87aamr896h.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <87r5g480ph.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006161803.E968621B04C@kimball.webabinitio.net> <87aamr896h.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101006213914.E6F4A21B0A8@kimball.webabinitio.net> On Thu, 07 Oct 2010 03:31:34 +0900, "Stephen J. Turnbull" wrote: > R. David Murray writes: > > > 5. Return the content, with non-ASCII bytes replaced with ? > > characters. > > That hadn't occurred to me (and it makes me sick to contemplate it). > > That said, this is probably good enough for Mailman-like apps to limp > along for "most" users. It's certainly good enough for the "might > kick your wife and elope with your dog" alpha ports of Mailman to > Python 3 (well, as certain as I can be; of course in the end Barry > decides). Assuming reasonable backward compatibility of the API, of > course! Yeah, "good enough" is pretty much the goal here. > > In other words, my proposed patch only makes email5 1/8 to 1/4 > > broken, instead of half broken as it is now. But not un-broken > > enough for Mailman, it sounds like. > > IMO, not in the long run. But realistically, in the applications I > know of, most desired traffic is conformant, and since there aren't > any Python 3 email apps yet, this isn't even a regression. :-/ > > I do think that it's important that the parsed object be able to tell > you what fields are there (except if the field name itself is invalid) > and return field bodies parsed as far as possible. Well, email doesn't currently parse the bodies any further by itself. You have to call parsing routines to get further parsing. So maybe what I should do is work on finalizing the patch without addressing the 'give me the escaped bytes issue', and then prepare a follow on patch that adds that keyword and adjusts the header parsing helpers accordingly. > > If we go this route (as opposed to only handling headers with 8bit data by > > sanitizing them), then we need to think about the email5 header parsers > > as well (decode_header and parseaddr). They are of course going to have > > the same problems as the rest of the email package with parsing bytes, > > and you are suggesting that access to those header 8bit bytes is needed. > > Yes, that would be preferable to replacing them with ASCII junk. > > But I don't see any problem with parsing them; they're syntactically > insignificant by definition. The problem is purely on output: do I > get verbatim escaped bytes, a sanitized str, or an exception? Right, the needed changes should be sanitizing by default, and providing the keyword to get the escaped bytes. Mostly it'll be writing tests :) > > Does my proposal make sense? But note, it raises exactly the backward > > compatibility concerns you mention in your next email (that I will reply > > to next). It is an open question whether it is worth opening that door > > in order to be able to do extended handling on non-RFC conforming email > > (as opposed to just sanitizing it and soldering on). > > Well, maybe not. However, it is not obvious to me that you won't run > into these issues again in Email6. Applications that think of email > as textual objects are going to want to make their own choices about > handling of non-conforming email, and it's likely to be massively > inconvenient to say "OK, but you have to use bytes interfaces > exclusively, because the str interfaces don't handle that." The strategy in email6 so far is for the application program to be able to access *any piece* of the parsed data as either text or bytes, and for the header parsers to record defects when there are non-ASCII bytes where there aren't supposed to be. So the application can check for defects and retrieve, say, the comment field that has the non-ASCII *as bytes* and decode it. Or, if it doesn't care about parsing them, it just modifies the fields it wants to modify that *are* valid, and the invalid non-ASCII comment gets carried along and emitted when the message is serialized as bytes. This is more or less what we are talking about enabling in email5 with the 'escape_bytes=True' keyword, it's just a less structured and more error prone approach to it than what we have planned for email6. -- R. David Murray www.bitdance.com From trentm at gmail.com Thu Oct 7 00:53:59 2010 From: trentm at gmail.com (Trent Mick) Date: Wed, 6 Oct 2010 15:53:59 -0700 Subject: [Python-Dev] opinions on issue2142 ('\ No newline at end of file' to difflib.unified_diff)? Message-ID: Soliciting opinions on issue 2142 (http://bugs.python.org/issue2142). There are patched available so this isn't vapour. :) The issue is this (I'll discuss only unified_diff(), but the same applies to context_diff()): >>> from difflib import * >>> gen = unified_diff("one\ntwo\nthree".splitlines(1), ... "one\ntwo\ntrois".splitlines(1)) >>> print ''.join(gen) --- +++ @@ -1,3 +1,3 @@ one two -three+trois Where as with `diff`, `hg` and `git`: diff -r 667b0870428d a --- a/a Wed Oct 06 15:39:50 2010 -0700 +++ b/a Wed Oct 06 15:40:31 2010 -0700 @@ -1,3 +1,3 @@ one two -three \ No newline at end of file +trois \ No newline at end of file While originally marked as a *bug*, the issue was changed to be a *feature* request, because arguably `difflib.unified_diff()` is fine, and the problem is in the naive use of the following to create a patch that will work with `patch`: ''.join(gen) Possiblities: 1. Change `difflib.unified_diff` to emit: ['--- \n', '+++ \n', '@@ -1,3 +1,3 @@\n', ' one\n', ' two\n', '-three\n', '\ No newline at end of file', '+trois\n', '\ No newline at end of file'] instead of: ['--- \n', '+++ \n', '@@ -1,3 +1,3 @@\n', ' one\n', ' two\n', '-three', '+trois'] for this case. 2. Add a `add_end_of_file_newline_markers_to_make_patch_happy` keyword arg (probably with a different name:) to `difflib.unified_diff` to do this additional handling. The reason is to not surprise existing code that would be surprised with those "\No newline at end of file" entries. 3. Not touch `difflib.unified_diff` and instead update http://docs.python.org/library/difflib.html#difflib-interface documentation to discuss the issue and show how users of unified_diff should handle this case themselves. Thoughts? Orthogonal: *After* a decision is made for the Python 3.3 tree we can discuss if including this in either of Python 2.7 or 3.2 would be wanted. -- Trent Mick From rdmurray at bitdance.com Thu Oct 7 02:46:08 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 06 Oct 2010 20:46:08 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101007004608.ED5EA22A3E5@kimball.webabinitio.net> Stephen J. Turnbull xemacs.org> writes: > R. David Murray writes: > > We're (in the current patch) not punting on handling non-conforming > > email, we're punting on handling non-conforming bytes *if the headers > > that contain them need to be modified*. The headers can still be > > modified, you just (currently) lose the non-ASCII bytes in the process. > > Modified *or examined*. I can't think of any important applications > offhand that *need* to examine the non-ASCII bytes (in particular, > Mailman doesn't need to do that). Verbatim copying of the bytes > themselves is almost always the desired usage. Mmm. Yes, or examined. If we allow escaped bytes to be returned, perhaps we also should provide a helper that "unescapes" the bytes and returns the byte string (yes, this is just a call to encode, but by wrapping it we continue to hide the surrogateescape implementation detail.) > > And robustness is not the issue, only extended-beyond-the-RFCs handling > > of non-conforming bytes would be an issue. > > And with that, I'm certain that Jon Postel is really dead. A goal for email6 is to be *at least* as Postel compliant as email4. The goal for my patch is to make email5.1 more Postel compliant than email5.0 is :) > > > (Surely you are not saying that Generator.flatten can't DTRT with > > > non-ASCII content *at all*?) > > > > Yes, that is *exactly* what I am saying: > > > > >>> m = email.message_from_string("""\ > > ... From: p??stal > > ... > > ... """) > > >>> str(m) > > Traceback (most recent call last): > > .... > > UnicodeEncodeError: 'ascii' codec can't encode character '\xf6' in position 1: ordinal not in range(128) > > But that's not interesting; you did that with Python 3. We want to Of course I did it with Python3. It's the Python3 email codebase I'm working with (and have to work *around*). > know what people porting from Python 2 will expect. So, in 2.5.5 or > 2.6.6 on Mac, with email v4.0.2, it *doesn't* raise, it returns > > wideload:~ 4:14$ python > Python 2.5.5 (r255:77872, Jul 13 2010, 03:03:57) > [GCC 4.0.1 (Apple Inc. build 5490)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> import email > >>> m=email.message_from_string('From: p??stal\n\n') > >>> str(m) > 'From nobody Thu Oct 7 04:18:25 2010\nFrom: p\xc3\xb6stal\n\n' > >>> m['From'] > 'p\xc3\xb6stal' > >>> > > That's hardly helpful! Surely we can and should do better than that > now, especially since UTF-8 (with a proper CTE) is now almost > universally acceptable to MUAs. When would it be a problem for that > to return > > 'From nobody Thu Oct 7 04:18:25 2010\nFrom: =?UTF-8?Q?p=C3=B6stal?=\n\n' What's wrong with that is that when we parse the bytes of the message we don't know that b'\xc3\xb6' == '=?UTF-8?Q?=C3=B6?='. It isn't even all that likely to be true, since I would guess that latin1 is still more common than utf-8 (but you might know better). > > Remember, email5 is a direct translation of email4, and email4 only > > handled ASCII and oh-by-the-way-if-there-are-bytes-along-for-the- > > -ride-fine-we'll-pass-then-along. So if you want to put non-ASCII > > data into a message you have to encode it properly to ASCII in > > exactly the same way that you did in email4: > > But if you do it right, then it will still work in a version that just > encodes non-ASCII characters in UTF-8 with the appropriate CTE. Since > you'll never be passing it non-ASCII characters, it's already ASCII > and UTF-8, and no CTE will be needed. So you are suggesting that I should use U+FFFD encoded as UTF-8 rather than '?' as the substitution character? But earlier you said that people would probably rather not be forced to deal with Unicode just because there are invalid bytes in the message. So that's probably not what you meant. Presumably you are suggesting that email5 be smart enough to turn my example into properly UTF-8/CTE encoded text. But *that* problem is what email6 is trying to address. It just doesn't look practical to address it directly in the email5 code base, because the email4 codebase that email5 inherits does not provide the correct distinction between bytes and text. email5 is parsing the input stream *as if* it were ASCII-only CTE text. I'm trying to extend it to also handle non-ASCII bytes gracefully. Extending it to actually handle unicode input is a whole different kettle of sushi[*]. > > Yes, exactly. I need to fix the patch to recode using, say, > > quoted-printable in that case. > > It really should check for proportions of non-ASCII. QP would be > horrible for Japanese or Chinese. Noted. > > DecodedGenerator could still produce the unicode, though, which is > > what I believe we want. (Although that raises the question of > > whether DecodedGenerator should also decode the RFC2047 encoded > > headers....but that raises a backward compatibility issue). > > Can't really help you there. While I would want the RFC 2047 headers > decoded if I were writing new code (which is generally the case for > me), I haven't really wrapped my head around the issues of porting old > code using Python2 str to Python3 str here. My intuition says "no > problem" (there won't be any MIME-words so the app won't try to decode > them), but I'm not real sure of that. Thinking about this further, I think it is unlikely that an application using DecodedGenerator would be further processing the headers generated by it, so I think this is probably a safe enough change, given that there are few if any Python3 email handling applications at this point. If anyone knows of a Python2 application that does post-process DecodedGenerator headers, please let me know. --David [*] And I've had an argument with someone who thinks email should *not* be extended to handle unicode messages with non-ASCII content, on the grounds that they aren't really email. From ronaldoussoren at mac.com Thu Oct 7 10:30:28 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 07 Oct 2010 10:30:28 +0200 Subject: [Python-Dev] hashlib bug when built on OS X 10.6 for 10.5 In-Reply-To: References: Message-ID: <8F642BB2-0B82-4FFF-B996-119D01D9C2CF@mac.com> On 6 Oct, 2010, at 22:26, Arnaud Bergeron wrote: > If you build python (at least 2.6.5, but probably other versions as > well) in a universal setup using the following command (or similar) > while the machine currently has 10.6 installed: > > ./configure --with-universal-archs= > --enable-universalsdk=/Developer/SDKs/MacOSX10.5.sdk/ > > then the hashlib module will be unimportable with the following error: > > ImportError: No module named _md5 > > This is because the openssl detection code in setup.py picks up the > system libssl (in /) which is 0.9.8 and selects not to build the > additional _sha256, and _sh512 modules. Then, when the builds > proceeds the SDK libssl is used (in /Developer/SDKs/MacOSX10.5.sdk/) > which is version 0.9.7 and the openssl_sha256 and openssl_sha512 raise > ValueError in the _hashlib that is built. This definitely looks like a bug I fixed before, see . The patch should not be present in 2.7 and any of the active branches. I can build hashlib just fine with the HEAD of the release26-maint branch, could you please test with 2.6.6? BTW. In general it is better to file bugs at bugs.python.org instead of mailing them here, using the bugtracker ensures that your report doesn't get lost. Ronald From cyril at montrealpython.org Thu Oct 7 03:26:56 2010 From: cyril at montrealpython.org (Cyril Robert) Date: Wed, 6 Oct 2010 21:26:56 -0400 Subject: [Python-Dev] ConFoo 2011 Call for speakers Message-ID: Greetings Python developers, We, Montr?al-Python, are the coordinators of the Python track at ConFoo 2011 and we are very proud to announce our call for speakers. PHP-Qu?bec, Montr?al-Python, Montreal.rb, W3Qc, and OWASP Montr?al are organizing the first edition of the ConFoo conference, which will be held in Montr?al on March 9th through 11th at the Hilton Bonaventure Hotel. With over 500 expected attendees, ConFoo is one of the largest Web development conference in North America. ConFoo is about the Web, but it's also about showcasing the strengths of different technologies. Do you think that Python beats all the other languages out there for Web development? Do you think that Python knocks Perl 6's socks off? Come and tell us why! Sessions are one hour long and you can present in French or in English, which ever your prefer. Submissions are due for November 26; for more details, visit the ConFoo website: http://confoo.ca/en/call-for-papers By the way, we are perfectly aware that there is a slight clash with PyCon. You should not worry about that since all Python talks will happen before Friday, giving you plenty of time for the commute towards Atlanta. Share the word! From tk at giga.or.at Thu Oct 7 14:17:52 2010 From: tk at giga.or.at (Thomas Klausner) Date: Thu, 7 Oct 2010 14:17:52 +0200 Subject: [Python-Dev] python-2.6.6 coredump running newspipe Message-ID: <20101007121752.GB15070@danbala.tuwien.ac.at> Hi! I'm running newspipe-1.1.9, an RSS reader (http://newspipe.sourceforge.net/), on NetBSD-5.99.11/amd64 using Python-2.6.6. Sometimes, it core dumps with particular feeds in the configuration (I guess depending on the feed, because when I comment out the offending feed in the opml file, it runs through to completion). The backtrace looks like this: Core was generated by `python'. Program terminated with signal 10, Bus error. #0 0x00007f7ffdc35a21 in PyOS_snprintf (str=0x7f7ff5dfe3d8 "@", size=120, format=0x1
) at Python/mysnprintf.c:43 43 { (gdb) bt #0 0x00007f7ffdc35a21 in PyOS_snprintf (str=0x7f7ff5dfe3d8 "@", size=120, format=0x1
) at Python/mysnprintf.c:43 #1 0x00007f7ffdc471a6 in PyOS_ascii_formatd (buffer=0x7f7ff5dfe3d8 "@", buf_size=120, format=0x7f7ff5dfe388 "%.2f", d=0.15256118774414062) at Python/pystrtod.c:455 #2 0x00007f7ffdbaa7fa in formatfloat (buf=0x7f7ff5dfe3d8 "@", buflen=120, flags=16, prec=2, type=102, v=0x7f7ffcc6d510) at Objects/stringobject.c:4378 #3 0x00007f7ffdbabd32 in PyString_Format (format=0x7f7ffc8144e0, args=0x7f7ffcc6d510) at Objects/stringobject.c:4943 #4 0x00007f7ffdbaa3b0 in string_mod (v=0x7f7ffc8144e0, w=0x7f7ffcc6d510) at Objects/stringobject.c:4116 #5 0x00007f7ffdb459db in binary_op1 (v=0x7f7ffc8144e0, w=0x7f7ffcc6d510, op_slot=32) at Objects/abstract.c:917 #6 0x00007f7ffdb45c81 in binary_op (v=0x7f7ffc8144e0, w=0x7f7ffcc6d510, op_slot=32, op_name=0x7f7ffdc6c089 "%") at Objects/abstract.c:969 #7 0x00007f7ffdb467ad in PyNumber_Remainder (v=0x7f7ffc8144e0, w=0x7f7ffcc6d510) at Objects/abstract.c:1221 #8 0x00007f7ffdc08a03 in PyEval_EvalFrameEx (f=0x7f7fefa1dab0, throwflag=0) at Python/ceval.c:1180 #9 0x00007f7ffdc1175f in fast_function (func=0x7f7ff8a9bed8, pp_stack=0x7f7ff5dfeae8, n=1, na=1, nk=0) at Python/ceval.c:3836 #10 0x00007f7ffdc11565 in call_function (pp_stack=0x7f7ff5dfeae8, oparg=1) at Python/ceval.c:3771 #11 0x00007f7ffdc0d81f in PyEval_EvalFrameEx (f=0x7f7fee920420, throwflag=0) at Python/ceval.c:2412 #12 0x00007f7ffdc0f715 in PyEval_EvalCodeEx (co=0x7f7ffcc247b0, globals=0x7f7ffd1c5880, locals=0x0, args=0x7f7ff5b0aac8, argcount=8, kws=0x7f7ff5b0ab08, kwcount=0, defs=0x7f7ff8d3c4e8, defcount=5, closure=0x0) at Python/ceval.c:3000 #13 0x00007f7ffdc1184a in fast_function (func=0x7f7ff8a9cc80, pp_stack=0x7f7ff5dfeff8, n=8, na=8, nk=0) at Python/ceval.c:3846 #14 0x00007f7ffdc11565 in call_function (pp_stack=0x7f7ff5dfeff8, oparg=7) at Python/ceval.c:3771 #15 0x00007f7ffdc0d81f in PyEval_EvalFrameEx (f=0x7f7ff5b0a820, throwflag=0) at Python/ceval.c:2412 #16 0x00007f7ffdc1175f in fast_function (func=0x7f7ff8a9e140, pp_stack=0x7f7ff5dff358, n=1, na=1, nk=0) at Python/ceval.c:3836 #17 0x00007f7ffdc11565 in call_function (pp_stack=0x7f7ff5dff358, oparg=0) at Python/ceval.c:3771 #18 0x00007f7ffdc0d81f in PyEval_EvalFrameEx (f=0x7f7ff5b0a420, throwflag=0) at Python/ceval.c:2412 #19 0x00007f7ffdc1175f in fast_function (func=0x7f7ffca1db90, pp_stack=0x7f7ff5dff6b8, n=1, na=1, nk=0) at Python/ceval.c:3836 #20 0x00007f7ffdc11565 in call_function (pp_stack=0x7f7ff5dff6b8, oparg=0) at Python/ceval.c:3771 #21 0x00007f7ffdc0d81f in PyEval_EvalFrameEx (f=0x7f7ff5b03190, throwflag=0) at Python/ceval.c:2412 #22 0x00007f7ffdc0f715 in PyEval_EvalCodeEx (co=0x7f7ffca0d4e0, globals=0x7f7ffca473a0, locals=0x0, args=0x7f7ff04d3e68, argcount=1, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3000 #23 0x00007f7ffdb7a612 in function_call (func=0x7f7ffca1daa0, arg=0x7f7ff04d3e50, kw=0x0) at Objects/funcobject.c:524 #24 0x00007f7ffdb495e8 in PyObject_Call (func=0x7f7ffca1daa0, arg=0x7f7ff04d3e50, kw=0x0) at Objects/abstract.c:2492 #25 0x00007f7ffdb5eca0 in instancemethod_call (func=0x7f7ffca1daa0, arg=0x7f7ff04d3e50, kw=0x0) at Objects/classobject.c:2579 #26 0x00007f7ffdb495e8 in PyObject_Call (func=0x7f7ff8ac2a00, arg=0x7f7ffd112050, kw=0x0) at Objects/abstract.c:2492 #27 0x00007f7ffdc10cd3 in PyEval_CallObjectWithKeywords (func=0x7f7ff8ac2a00, arg=0x7f7ffd112050, kw=0x0) at Python/ceval.c:3619 #28 0x00007f7ffdc4e69f in t_bootstrap (boot_raw=0x7f7ffd1b4590) at ./Modules/threadmodule.c:428 #29 0x00007f7ffd90ba32 in pthread_setcancelstate () from /usr/lib/libpthread.so.1 #30 0x00007f7ffd26e9b0 in ___lwp_park50 () from /usr/lib/libc.so.12 #31 0x0000000000000000 in ?? () (gdb) fr 1 #1 0x00007f7ffdc471a6 in PyOS_ascii_formatd (buffer=0x7f7ff5dfe3d8 "@", buf_size=120, format=0x7f7ff5dfe388 "%.2f", d=0.15256118774414062) at Python/pystrtod.c:455 455 PyOS_snprintf(buffer, buf_size, format, d); (gdb) l 450 format = tmp_format; 451 } 452 453 454 /* Have PyOS_snprintf do the hard work */ 455 PyOS_snprintf(buffer, buf_size, format, d); 456 457 /* Do various fixups on the return string */ 458 459 /* Get the current locale, and find the decimal point string. (gdb) p format $1 = 0x7f7ff5dfe388 "%.2f" (gdb) fr 0 #0 0x00007f7ffdc35a21 in PyOS_snprintf (str=0x7f7ff5dfe3d8 "@", size=120, format=0x1
) at Python/mysnprintf.c:43 43 { (gdb) l 38 CAUTION: Unlike C99, str != NULL and size > 0 are required. 39 */ 40 41 int 42 PyOS_snprintf(char *str, size_t size, const char *format, ...) 43 { 44 int rc; 45 va_list va; 46 47 va_start(va, format); (gdb) It seems that the format argument is corrupted while calling PyOS_snprintf. Any ideas what could cause this or how to fix this? Thanks, Thomas From rdmurray at bitdance.com Thu Oct 7 16:00:03 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 07 Oct 2010 10:00:03 -0400 Subject: [Python-Dev] opinions on issue2142 ('\ No newline at end of file' to difflib.unified_diff)? In-Reply-To: References: Message-ID: <20101007140003.A24A71D6C1B@kimball.webabinitio.net> On Wed, 06 Oct 2010 15:53:59 -0700, Trent Mick wrote: > Soliciting opinions on issue 2142 (http://bugs.python.org/issue2142). > There are patched available so this isn't vapour. :) [...] > Possiblities: > > 1. Change `difflib.unified_diff` to emit: > > ['--- \n', '+++ \n', '@@ -1,3 +1,3 @@\n', ' one\n', ' two\n', > '-three\n', '\ No newline at end of file', '+trois\n', '\ No newline > at end of file'] > > instead of: > > ['--- \n', '+++ \n', '@@ -1,3 +1,3 @@\n', ' one\n', ' two\n', > '-three', '+trois'] > > for this case. > > > 2. Add a `add_end_of_file_newline_markers_to_make_patch_happy` keyword > arg (probably with a different name:) to `difflib.unified_diff` to do > this additional handling. The reason is to not surprise existing code > that would be surprised with those "\No newline at end of file" > entries. > > 3. Not touch `difflib.unified_diff` and instead update > http://docs.python.org/library/difflib.html#difflib-interface > documentation to discuss the issue and show how users of unified_diff > should handle this case themselves. > > Thoughts? I don't think (1) is a good option both for backward compatibility reasons and (as mentioned in the ticket IIRC) because not all programs using difflib use it to generate diffs for direct output. (2) might be worth it given that there is a "standard" to follow so it might be worth coding that standard into the stdlib. > Orthogonal: *After* a decision is made for the Python 3.3 tree we can > discuss if including this in either of Python 2.7 or 3.2 would be > wanted. (3) is the only option for 2.7/3.1. We're still pre-beta on 3.2, so (2) is still an option there. 3.3 doesn't enter the discussion until after 3.2 beta 1. -- R. David Murray www.bitdance.com From dirkjan at ochtman.nl Thu Oct 7 16:20:44 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 7 Oct 2010 16:20:44 +0200 Subject: [Python-Dev] opinions on issue2142 ('\ No newline at end of file' to difflib.unified_diff)? In-Reply-To: References: Message-ID: On Thu, Oct 7, 2010 at 00:53, Trent Mick wrote: > 1. Change `difflib.unified_diff` to emit: > > ? ?['--- ?\n', '+++ ?\n', '@@ -1,3 +1,3 @@\n', ' one\n', ' two\n', > '-three\n', '\ No newline at end of file', '+trois\n', '\ No newline > at end of file'] > > instead of: > > ? ?['--- ?\n', '+++ ?\n', '@@ -1,3 +1,3 @@\n', ' one\n', ' two\n', > '-three', '+trois'] > > for this case. > > > 2. Add a `add_end_of_file_newline_markers_to_make_patch_happy` keyword > arg (probably with a different name:) to `difflib.unified_diff` to do > this additional handling. The reason is to not surprise existing code > that would be surprised with those "\No newline at end of file" > entries. > > 3. Not touch `difflib.unified_diff` and instead update > http://docs.python.org/library/difflib.html#difflib-interface > documentation to discuss the issue and show how users of unified_diff > should handle this case themselves. Mark (in the issue) argues that there is no specification for diffs, and that this is thus a feature, not a bug. On the other hand, in Mercurial we've maintained the idea that diffs are specified by whatever (GNU) patch(1) accepts. So I would support treating this as a bug, not just a feature. As such, I think 3.2 should emit the extra line by default and add a keyword argument to make it easy to revert to the old behavior (and add docs to 2.7, 3.1 and 3.2 about the issue!). Cheers, Dirkjan From rdmurray at bitdance.com Thu Oct 7 17:15:18 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 07 Oct 2010 11:15:18 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <877hhu8rvf.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007004608.ED5EA22A3E5@kimball.webabinitio.net> <877hhu8rvf.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101007151518.AEFC11BF4F3@kimball.webabinitio.net> On Thu, 07 Oct 2010 15:00:04 +0900, "Stephen J. Turnbull" wrote: > R. David Murray writes: > > > > But that's not interesting; you did that with Python 3. We want to > > Of course I did it with Python3. It's the Python3 email codebase > > I'm working with (and have to work *around*). > > Sure. My point is that it has nothing to do with the expections of > people trying to upgrade their apps to Python 3, and meeting those > expectations is an important requirement of the specification of > email5, right? Well, not necessarily, no. Python3 broke backward compatibility. *Some* changes are going to have to be made in user code to make it work with email5. Where we can minimize those changes we should, but it isn't a requirement, no. With my patch, the minimization will be message_from_string --> message_from_bytes, message_from_file --> message_from_binary_file, and in some cases Generator --> BytesGenerator, for those programs that need to deal with wire format data that is not 7bit clean. Programs that only *generate* emails should need few if any changes, but that is already true (that's the half of email that is working :). > Actually, in context we were not talking about a random character that > came in from outside, we were talking about U+FFFD that *we* > generated, and *know* that it's the only non-ASCII character in the > string because we replaced all the others with it. Ah, so that *was* what you were suggesting. > Of course the best we can do with 'From: =?UNKNOWN?Q?p=C3=B6stal' or > 'From: p\xc3\xb6stal' on input is to save the encoded or raw bytes > representation and spit it back out on output. Yes. And I haven't actually dealt with what to do with non-ascii characters or RFC2047 unknown-8bit characters when decoding headers in email6. In issue 6302 we are talking about adding a decode_header_to_string method for email5 where the same issue arises, and so we'll need to make a decision soon. Presumably we'll use U+FFFD to replace them (along with registering defects in email6). > The MIME-charset = UNKNOWN dodge might be a better way of handling > this. The str is all ASCII, so won't raise exceptions unless the app > itself objects to MIME encoded-words for some reason. OTOH, the > presence of encoded words will be a red flag to any human viewer, and > after processing with .flatten(), the receiver is likely to DTRT (from > the receiving human's point of view, per that human's configuration). That is a very interesting idea. It is the *right* thing to do, since it would mean that a message parsed as bytes could be generated via Generator and passed to, say, smtplib without losing any information. However, It's not exactly trivial to implement, since issues of runs of characters and line re-wrapping need need to be dealt with. Perhaps Header can be made to handle bytes in order to do this; I'll have to look in to it. > > So you are suggesting that I should use U+FFFD encoded as UTF-8 > > rather than '?' as the substitution character? But earlier you said > > that people would probably rather not be forced to deal with Unicode > > just because there are invalid bytes in the message. So that's > > probably not what you meant. > > "Suggest" !=3D "recommend". Talking to a wider base of users and > developers, you might or might not find that to be a good idea. I > don't think the 800 million or so Chinese coming online in the next > decade will much care whether you use U+FFFD or '?'. The Japanese > would prefer U+2639 WHITE FROWNING FACE or U+270C VICTORY HAND, no > doubt ("crassly cute" is much beloved here). Americans will likely > prefer '?', as they probably have correspondents with legacy systems > that won't like UTF-8 or perhaps don't have a font to display U+FFFD. For the moment I think I'll stick with '?', with the idea of "fixing that bug" by using the unknown charset trick at a later stage. > > Presumably you are suggesting that email5 be smart enough to turn my > > example into properly UTF-8/CTE encoded text. > > No, in general that's undecidable without asking the originator, > although humans can often make a good guess. But not always: Japanese > are fond of "four-character compound words", and I once found an > 8-byte sequence (four 2-byte characters) that is idiomatic in both > Shift JIS and EUC-JP. Even a dictionary lookup can't determine the > intended encoding for that sequence. I was talking about unicode input, though, where you do know (modulo the language differences that unicode hasn't yet sorted out). > I'm only saying that any Unicode email-N generates itself can be > properly encoded. Agreed. > > But *that* problem is what email6 is trying to address. It just > > doesn't look practical to address it directly in the email5 code > > base, because the email4 codebase that email5 inherits does not > > provide the correct distinction between bytes and text. email5 is > > parsing the input stream *as if* it were ASCII-only CTE text. > > I don't see how this is different from email6. Just because email6 is > trying to DTRT doesn't mean the spammers will, and even Emacs MUA > developers occasionally screw this up in new products. So email-N has > to handle input streams that are *supposed* to be entirely ASCII except > for message bodies that are properly marked as 8bit or binary CTE, but > occasionally will not conform. Right, but I was talking about my python3 example, where I was using the email5 parser to (unsuccessfully) parse unicode. *That's* the thing email5 can't really handle, but email6 will be able to. > > Extending it to actually handle unicode input is a whole different > > kettle of sushi[*]. > > But this is not your problem in email5 AFAICS. Right, but I thought you were suggesting it was. My mistake. > > [*] And I've had an argument with someone who thinks email should > > *not* be extended to handle unicode messages with non-ASCII > > content, on the grounds that they aren't really email. > > That's total nonsense. Don't argue with people like that, educate > them, and if that fails, ignore them. There's good reason for not > extending email5, ie, email4 didn't do it. But that has nothing to do > with what email "really is". [ snip good supporting text ] > In practice, email undoubtably has clients that want to manipulate > bytes directly. I can't blame them, but the RFCs have nothing to say > about that, really. RFC 822 and its family (including MIME) are about > representing human media as octet streams compatible with protocols > like RFC 821, and in Python the human medium for representing text is > str. The result of bytes manipulations should be "as if" the original > stream was decoded, manipulated, and reencoded. So direct bytes > manipulation is an optimization. The RFCs don't provide for it at > all, AFAICS. > > The same thing is true of URIs, except that RFC 3896 makes it fully > explicit that URIs are conceptually text, not octets. Again, there > are many important use cases for bytes manipulation of URIs, but this > is an optimization. Thank you very much for this piece of perspective. I hadn't thought about it that clearly before, but what you say makes perfect sense to me, and is in fact the implicit perspective I've been working from when working on the email6 stuff. -- R. David Murray www.bitdance.com From victor.stinner at haypocalc.com Thu Oct 7 18:19:41 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 7 Oct 2010 18:19:41 +0200 Subject: [Python-Dev] Inconsistencies if locale and filesystem encodings are different Message-ID: <201010071819.41443.victor.stinner@haypocalc.com> Hi, A PYTHONFSENCODING environment variable was added to Python 3.2: issue #8622. This variable introduces an inconstency because the filesystem and the locale encodings can now be different. There are (at least) four issues related to this problem. We have 2 choices to fix these issues: (a) use the same encoding to encode and decode values (it can be different for each issue) (b) remove PYTHONFSENCODING variable and raise an error if locale and filesystem encodings are different (ensure that both encodings are the same) Even if choice (a) is not easy to implement, it is feasible and I already wrote some patches. I don't understand how Python interact with other programs who ignore the PYTHONFSENCODING environment variable. It's like Python uses its own "locale". Choice (b) looks easy to implement, but... there is the problem of Mac OS X. Mac OS X uses utf-8 encoding for the filesystem (and not the locale encoding), whereas it looks like the locale encoding is used for the command line arguments. See issue #4388 for more information. There is also maybe an useful usecase of the PYTHONFSENCODING, but I don't remember which one :-) Issues ------ sys.argv: - decoded from the locale encoding - subprocess encodes process arguments to the filesystem encoding => issue #9992 sys.path: - decoded from the locale encoding - import encodes paths to the filesystem encoding => issue #10014 The script name, read on the command line (eg. python script.py), is decoded using the locale encoding, whereas it is used to fill sys.path[0] (without any encoding conversion) and import encodes paths to the filesystem encoding. => issue #10039 PYTHONWARNINGS environment variable: - decoded from the locale encoding - subprocess encodes environment variables to the filesystem encoding => issue #9988 -- Victor Stinner http://www.haypocalc.com/ From lutz at rmi.net Thu Oct 7 18:03:18 2010 From: lutz at rmi.net (lutz at rmi.net) Date: Thu, 07 Oct 2010 16:03:18 -0000 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes Message-ID: Stephen J. Turnbull wrote (giving me an opening to jump in here): > R. David Murray writes: > > In other words, my proposed patch only makes email5 1/8 to 1/4 > > broken, instead of half broken as it is now. But not un-broken > > enough for Mailman, it sounds like. > > IMO, not in the long run. But realistically, in the applications I > know of, most desired traffic is conformant, and since there aren't > any Python 3 email apps yet, this isn't even a regression. :-/ Well, yes there are, and yes it is. As I pointed out in a thread on this list back in June, there are multiple large Python 3 email "apps" in the new Programming Python, a book which is about to be released, and which will be read by at least tens of thousands of people, many of whom will be evaluating the stability of Python 3. These apps include both a simple webmail site, as well as a more sophisticated 5k-line tkinter email client -- one which I've been using for all my personal and business email over the last 6 months, and which works well with the email package as it is in 3.1 (albeit with a bit of workaround code). This includes support for Unicode, MIME, headers, attachments, and the lot. I'm forwarding a link to the code of these clients to David by private email in case they might be useful as a test case (O'Reilly has already posted them ahead of the book, but they may be a bit too heavy for use in formal testing). The email package is obviously less than ideal today, and there are many other clients for it besides my own, of course. But making it backward incompatible at this point is likely to be seen as a big negative to newcomers evaluating 3.X viability. And as I tried to make clear in June, this list should carefully weigh the PR cost of pulling the rug out from under those brave souls who have already taken the time to accommodate the 3.X world you've mandated. To put that more strongly, the Python user base is much larger than this list's readership. If I'm using 3.1 email, so are many others. People will accept the 3.X world you make up to a point, but it's impossible to code to a moving target, much less base a product on it. At some point, they'll simply stop trying to keep up; in fact, some already have. Fixes are a Good Thing, of course, and this particular change's scope remains to be seen; but to channel most of the users I meet out there in the real world today: Enough with the 3.X changes already, eh? --Mark Lutz (http://learning-python.com, http://rmi.net/~lutz) From mal at egenix.com Thu Oct 7 18:35:09 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 07 Oct 2010 18:35:09 +0200 Subject: [Python-Dev] Inconsistencies if locale and filesystem encodings are different In-Reply-To: <201010071819.41443.victor.stinner@haypocalc.com> References: <201010071819.41443.victor.stinner@haypocalc.com> Message-ID: <4CADF6BD.2050805@egenix.com> Victor Stinner wrote: > Hi, > > A PYTHONFSENCODING environment variable was added to Python 3.2: issue #8622. > This variable introduces an inconstency because the filesystem and the locale > encodings can now be different. > > There are (at least) four issues related to this problem. We have 2 choices to > fix these issues: > > (a) use the same encoding to encode and decode values (it can be different > for each issue) > > (b) remove PYTHONFSENCODING variable and raise an error if locale and > filesystem encodings are different (ensure that both encodings are the same) > > Even if choice (a) is not easy to implement, it is feasible and I already > wrote some patches. > > I don't understand how Python interact with other programs who ignore the > PYTHONFSENCODING environment variable. It's like Python uses its own "locale". > > Choice (b) looks easy to implement, but... there is the problem of Mac OS X. > Mac OS X uses utf-8 encoding for the filesystem (and not the locale encoding), > whereas it looks like the locale encoding is used for the command line > arguments. See issue #4388 for more information. > > There is also maybe an useful usecase of the PYTHONFSENCODING, but I don't > remember which one :-) You have to differentiate between the meaning of a file system encoding and the locale: A file system encoding defines how the applications interact with the file system. A locale defines how the user expects to interact with the application. It is well possible that the two are different. Mac OS X is just one example. Another common example is having a Unix account using the C locale (=ASCII) while working on a UTF-8 file system. BTW: We added that because just like I/O encoding, you need to be able to override the setting determined by Python via locale introspection, which may be wrong. The env var is only meant as a way to solve encoding problems in special situations where the local cannot be used to determine the file system or input/output encoding. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 07 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From phd at phd.pp.ru Thu Oct 7 18:44:19 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Thu, 7 Oct 2010 20:44:19 +0400 Subject: [Python-Dev] Inconsistencies if locale and filesystem encodings are different In-Reply-To: <4CADF6BD.2050805@egenix.com> References: <201010071819.41443.victor.stinner@haypocalc.com> <4CADF6BD.2050805@egenix.com> Message-ID: <20101007164419.GA26143@phd.pp.ru> On Thu, Oct 07, 2010 at 06:35:09PM +0200, M.-A. Lemburg wrote: > It is well possible that the two are different. Mac OS X is > just one example. Another common example is having a Unix > account using the C locale (=ASCII) while working on a UTF-8 > file system. My filesystems are always koi8-r, but sometimes I work with programs in utf-8 locale. Just an example... Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From rdmurray at bitdance.com Thu Oct 7 19:46:02 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 07 Oct 2010 13:46:02 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: References: Message-ID: <20101007174602.309AA21B0BD@kimball.webabinitio.net> On Thu, 07 Oct 2010 16:03:18 -0000, lutz at rmi.net wrote: > I'm forwarding a link to the code of these clients to David by > private email in case they might be useful as a test case (O'Reilly > has already posted them ahead of the book, but they may be a bit too > heavy for use in formal testing). Thanks very much. I will take a look, and expect they will be helpful. > The email package is obviously less than ideal today, and there are > many other clients for it besides my own, of course. But making it > backward incompatible at this point is likely to be seen as a big > negative to newcomers evaluating 3.X viability. And as I tried to > make clear in June, this list should carefully weigh the PR cost of > pulling the rug out from under those brave souls who have already > taken the time to accommodate the 3.X world you've mandated. Well, as I have said before the plan is to provide backward compatibility in email6, so that you only need to change your code if you want to take advantage of improved or new functionality. If this turns out not to be possible for some reason, then we aren't going to suddenly stop supporting email5. That's not the Python Way :) (Example: we added ArgParse post-3.0, and lots of people wanted to deprecate OptParse, but we aren't planning on removing OptParse.) Do you see any issues with the patch I'm proposing? My goal is to make things work that didn't work before, but nothing that worked before should stop working, if I do my job right. The one *potentially* backward-incompatible change that I'm consciously considering (that is, any other backward incompatibilities will be bugs) is having DecodedGenerator fully decode headers and emit full unicode, rather than the ASCII-only unicode that Generator emits. Can you think of any problem that that would cause? A quick grep indicates your own code does not use that generator (possibly because currently it does not do that decoding). I could, of course, only enable header decoding if a flag is passed requesting it, and as I write this I realize that that is indeed what I should do. Even though I haven't been able to think of a case where DecodedGenerator producing non-ASCII unicode would be an issue, that doesn't mean there isn't one :) > To put that more strongly, the Python user base is much larger than > this list's readership. If I'm using 3.1 email, so are many others. > People will accept the 3.X world you make up to a point, but it's > impossible to code to a moving target, much less base a product on > it. At some point, they'll simply stop trying to keep up; in fact, > some already have. > > Fixes are a Good Thing, of course, and this particular change's scope > remains to be seen; but to channel most of the users I meet out there > in the real world today: Enough with the 3.X changes already, eh? Now that Python3 is out, the backward compatibility policy for it is the same as it always was for Python2. Only the transition from 2 to 3 broke backward compatibility in a significant way. From here on, we are as conservative as we always have been at making backward incompatible changes (that is, we don't do it intentionally without a good reason and a deprecation cycle, and if we do it unintentionally it is a regression and treated as such). -- R. David Murray www.bitdance.com From brett at python.org Thu Oct 7 20:15:27 2010 From: brett at python.org (Brett Cannon) Date: Thu, 7 Oct 2010 11:15:27 -0700 Subject: [Python-Dev] python-2.6.6 coredump running newspipe In-Reply-To: <20101007121752.GB15070@danbala.tuwien.ac.at> References: <20101007121752.GB15070@danbala.tuwien.ac.at> Message-ID: It's best to report issues at bugs.python.org. On Thu, Oct 7, 2010 at 05:17, Thomas Klausner wrote: > Hi! > > I'm running newspipe-1.1.9, an RSS reader > (http://newspipe.sourceforge.net/), on NetBSD-5.99.11/amd64 using > Python-2.6.6. > > Sometimes, it core dumps with particular feeds in the configuration (I > guess depending on the feed, because when I comment out the offending > feed in the opml file, it runs through to completion). > > The backtrace looks like this: > Core was generated by `python'. > Program terminated with signal 10, Bus error. > #0 ?0x00007f7ffdc35a21 in PyOS_snprintf (str=0x7f7ff5dfe3d8 "@", size=120, format=0x1
) at Python/mysnprintf.c:43 > 43 ? ? ?{ > (gdb) bt > #0 ?0x00007f7ffdc35a21 in PyOS_snprintf (str=0x7f7ff5dfe3d8 "@", size=120, format=0x1
) at Python/mysnprintf.c:43 > #1 ?0x00007f7ffdc471a6 in PyOS_ascii_formatd (buffer=0x7f7ff5dfe3d8 "@", buf_size=120, format=0x7f7ff5dfe388 "%.2f", d=0.15256118774414062) at Python/pystrtod.c:455 > #2 ?0x00007f7ffdbaa7fa in formatfloat (buf=0x7f7ff5dfe3d8 "@", buflen=120, flags=16, prec=2, type=102, v=0x7f7ffcc6d510) at Objects/stringobject.c:4378 > #3 ?0x00007f7ffdbabd32 in PyString_Format (format=0x7f7ffc8144e0, args=0x7f7ffcc6d510) at Objects/stringobject.c:4943 > #4 ?0x00007f7ffdbaa3b0 in string_mod (v=0x7f7ffc8144e0, w=0x7f7ffcc6d510) at Objects/stringobject.c:4116 > #5 ?0x00007f7ffdb459db in binary_op1 (v=0x7f7ffc8144e0, w=0x7f7ffcc6d510, op_slot=32) at Objects/abstract.c:917 > #6 ?0x00007f7ffdb45c81 in binary_op (v=0x7f7ffc8144e0, w=0x7f7ffcc6d510, op_slot=32, op_name=0x7f7ffdc6c089 "%") at Objects/abstract.c:969 > #7 ?0x00007f7ffdb467ad in PyNumber_Remainder (v=0x7f7ffc8144e0, w=0x7f7ffcc6d510) at Objects/abstract.c:1221 > #8 ?0x00007f7ffdc08a03 in PyEval_EvalFrameEx (f=0x7f7fefa1dab0, throwflag=0) at Python/ceval.c:1180 > #9 ?0x00007f7ffdc1175f in fast_function (func=0x7f7ff8a9bed8, pp_stack=0x7f7ff5dfeae8, n=1, na=1, nk=0) at Python/ceval.c:3836 > #10 0x00007f7ffdc11565 in call_function (pp_stack=0x7f7ff5dfeae8, oparg=1) at Python/ceval.c:3771 > #11 0x00007f7ffdc0d81f in PyEval_EvalFrameEx (f=0x7f7fee920420, throwflag=0) at Python/ceval.c:2412 > #12 0x00007f7ffdc0f715 in PyEval_EvalCodeEx (co=0x7f7ffcc247b0, globals=0x7f7ffd1c5880, locals=0x0, args=0x7f7ff5b0aac8, argcount=8, kws=0x7f7ff5b0ab08, kwcount=0, defs=0x7f7ff8d3c4e8, > ? ?defcount=5, closure=0x0) at Python/ceval.c:3000 > #13 0x00007f7ffdc1184a in fast_function (func=0x7f7ff8a9cc80, pp_stack=0x7f7ff5dfeff8, n=8, na=8, nk=0) at Python/ceval.c:3846 > #14 0x00007f7ffdc11565 in call_function (pp_stack=0x7f7ff5dfeff8, oparg=7) at Python/ceval.c:3771 > #15 0x00007f7ffdc0d81f in PyEval_EvalFrameEx (f=0x7f7ff5b0a820, throwflag=0) at Python/ceval.c:2412 > #16 0x00007f7ffdc1175f in fast_function (func=0x7f7ff8a9e140, pp_stack=0x7f7ff5dff358, n=1, na=1, nk=0) at Python/ceval.c:3836 > #17 0x00007f7ffdc11565 in call_function (pp_stack=0x7f7ff5dff358, oparg=0) at Python/ceval.c:3771 > #18 0x00007f7ffdc0d81f in PyEval_EvalFrameEx (f=0x7f7ff5b0a420, throwflag=0) at Python/ceval.c:2412 > #19 0x00007f7ffdc1175f in fast_function (func=0x7f7ffca1db90, pp_stack=0x7f7ff5dff6b8, n=1, na=1, nk=0) at Python/ceval.c:3836 > #20 0x00007f7ffdc11565 in call_function (pp_stack=0x7f7ff5dff6b8, oparg=0) at Python/ceval.c:3771 > #21 0x00007f7ffdc0d81f in PyEval_EvalFrameEx (f=0x7f7ff5b03190, throwflag=0) at Python/ceval.c:2412 > #22 0x00007f7ffdc0f715 in PyEval_EvalCodeEx (co=0x7f7ffca0d4e0, globals=0x7f7ffca473a0, locals=0x0, args=0x7f7ff04d3e68, argcount=1, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) > ? ?at Python/ceval.c:3000 > #23 0x00007f7ffdb7a612 in function_call (func=0x7f7ffca1daa0, arg=0x7f7ff04d3e50, kw=0x0) at Objects/funcobject.c:524 > #24 0x00007f7ffdb495e8 in PyObject_Call (func=0x7f7ffca1daa0, arg=0x7f7ff04d3e50, kw=0x0) at Objects/abstract.c:2492 > #25 0x00007f7ffdb5eca0 in instancemethod_call (func=0x7f7ffca1daa0, arg=0x7f7ff04d3e50, kw=0x0) at Objects/classobject.c:2579 > #26 0x00007f7ffdb495e8 in PyObject_Call (func=0x7f7ff8ac2a00, arg=0x7f7ffd112050, kw=0x0) at Objects/abstract.c:2492 > #27 0x00007f7ffdc10cd3 in PyEval_CallObjectWithKeywords (func=0x7f7ff8ac2a00, arg=0x7f7ffd112050, kw=0x0) at Python/ceval.c:3619 > #28 0x00007f7ffdc4e69f in t_bootstrap (boot_raw=0x7f7ffd1b4590) at ./Modules/threadmodule.c:428 > #29 0x00007f7ffd90ba32 in pthread_setcancelstate () from /usr/lib/libpthread.so.1 > #30 0x00007f7ffd26e9b0 in ___lwp_park50 () from /usr/lib/libc.so.12 > #31 0x0000000000000000 in ?? () > (gdb) fr 1 > #1 ?0x00007f7ffdc471a6 in PyOS_ascii_formatd (buffer=0x7f7ff5dfe3d8 "@", buf_size=120, format=0x7f7ff5dfe388 "%.2f", d=0.15256118774414062) at Python/pystrtod.c:455 > 455 ? ? ? ? PyOS_snprintf(buffer, buf_size, format, d); > (gdb) l > 450 ? ? ? ? ? ? format = tmp_format; > 451 ? ? ? ? } > 452 > 453 > 454 ? ? ? ? /* Have PyOS_snprintf do the hard work */ > 455 ? ? ? ? PyOS_snprintf(buffer, buf_size, format, d); > 456 > 457 ? ? ? ? /* Do various fixups on the return string */ > 458 > 459 ? ? ? ? /* Get the current locale, and find the decimal point string. > (gdb) p format > $1 = 0x7f7ff5dfe388 "%.2f" > (gdb) fr 0 > #0 ?0x00007f7ffdc35a21 in PyOS_snprintf (str=0x7f7ff5dfe3d8 "@", size=120, format=0x1
) at Python/mysnprintf.c:43 > 43 ? ? ?{ > (gdb) l > 38 ? ? ? ? CAUTION: ?Unlike C99, str != NULL and size > 0 are required. > 39 ? ? ?*/ > 40 > 41 ? ? ?int > 42 ? ? ?PyOS_snprintf(char *str, size_t size, const ?char ?*format, ...) > 43 ? ? ?{ > 44 ? ? ? ? ?int rc; > 45 ? ? ? ? ?va_list va; > 46 > 47 ? ? ? ? ?va_start(va, format); > (gdb) > > It seems that the format argument is corrupted while calling PyOS_snprintf. > > Any ideas what could cause this or how to fix this? > > Thanks, > ?Thomas > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From barry at python.org Thu Oct 7 20:37:48 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 7 Oct 2010 14:37:48 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101007143748.7c548261@mission> On Oct 07, 2010, at 04:40 AM, Stephen J. Turnbull wrote: > > And the email API currently promises not to raise during parsing, > > which is a contract my patch does not change. > >Which is a contract that has historically been broken frequently. >Unhandled UnicodeErrors have been one of the most common causes of >queue stoppage in Mailman (exceeded only by configuration errors >AFAICS). I haven't seen any reports for a while, but with the email >package being reengineered from the ground up, the possibility of >regression can't be ignored. I'm fairly certain that most of the modern causes of this are post-parse modifications of the message. IOW, in Mailman's architecture, we try to parse the raw data into a Message object tree very early in the pipeline, and then a pickled version of that gets passed between the queue runners. If the initial parse fails, there's almost literally nothing Mailman can do with the original data other than delete it. Where we've gotten into trouble before has been things like adding the Subject prefixes and such. That seems like application logic that the email package can't really get involved with, and indeed Mailman has built up a raft of defense for failures of this kind. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From victor.stinner at haypocalc.com Thu Oct 7 21:12:13 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 7 Oct 2010 21:12:13 +0200 Subject: [Python-Dev] =?iso-8859-1?q?Inconsistencies_if_locale_and_filesys?= =?iso-8859-1?q?tem=09encodings_are_different?= In-Reply-To: <20101007164419.GA26143@phd.pp.ru> References: <201010071819.41443.victor.stinner@haypocalc.com> <4CADF6BD.2050805@egenix.com> <20101007164419.GA26143@phd.pp.ru> Message-ID: <201010072112.13622.victor.stinner@haypocalc.com> Le jeudi 07 octobre 2010 18:44:19, Oleg Broytman a ?crit : > On Thu, Oct 07, 2010 at 06:35:09PM +0200, M.-A. Lemburg wrote: > > It is well possible that the two are different. Mac OS X is > > just one example. Another common example is having a Unix > > account using the C locale (=ASCII) while working on a UTF-8 > > file system. > > My filesystems are always koi8-r, but sometimes I work with programs in > utf-8 locale. Just an example... Are programs able to display correctly non-ascii filenames if your locale encoding is different than your filesystem encoding? -- Victor Stinner http://www.haypocalc.com/ From victor.stinner at haypocalc.com Thu Oct 7 21:14:36 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 7 Oct 2010 21:14:36 +0200 Subject: [Python-Dev] =?iso-8859-1?q?Inconsistencies_if_locale_and_filesys?= =?iso-8859-1?q?tem_encodings_are=09different?= In-Reply-To: <4CADF6BD.2050805@egenix.com> References: <201010071819.41443.victor.stinner@haypocalc.com> <4CADF6BD.2050805@egenix.com> Message-ID: <201010072114.36733.victor.stinner@haypocalc.com> Le jeudi 07 octobre 2010 18:35:09, M.-A. Lemburg a ?crit : > Victor Stinner wrote: > > Hi, > > > > A PYTHONFSENCODING environment variable was added to Python 3.2: issue > > #8622. This variable introduces an inconstency because the filesystem > > and the locale encodings can now be different. > > > > There are (at least) four issues related to this problem. We have 2 > > choices to > > > > fix these issues: > > (a) use the same encoding to encode and decode values (it can be > > different > > > > for each issue) > > > > (b) remove PYTHONFSENCODING variable and raise an error if locale and > > > > filesystem encodings are different (ensure that both encodings are the > > same) > > > > Even if choice (a) is not easy to implement, it is feasible and I already > > wrote some patches. > > > > I don't understand how Python interact with other programs who ignore the > > PYTHONFSENCODING environment variable. It's like Python uses its own > > "locale". > > > > Choice (b) looks easy to implement, but... there is the problem of Mac OS > > X. Mac OS X uses utf-8 encoding for the filesystem (and not the locale > > encoding), whereas it looks like the locale encoding is used for the > > command line arguments. See issue #4388 for more information. > > > > There is also maybe an useful usecase of the PYTHONFSENCODING, but I > > don't remember which one :-) > > You have to differentiate between the meaning of a file system > encoding and the locale: > > A file system encoding defines how the applications interact > with the file system. > > A locale defines how the user expects to interact with the > application. What is the encoding of the command line arguments? Locale or filesystem encoding? Is it different if an argument is a filename or a path? -- Victor Stinner http://www.haypocalc.com/ From phd at phd.pp.ru Thu Oct 7 21:42:47 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Thu, 7 Oct 2010 23:42:47 +0400 Subject: [Python-Dev] Inconsistencies if locale and filesystem?encodings are different In-Reply-To: <201010072112.13622.victor.stinner@haypocalc.com> References: <201010071819.41443.victor.stinner@haypocalc.com> <4CADF6BD.2050805@egenix.com> <20101007164419.GA26143@phd.pp.ru> <201010072112.13622.victor.stinner@haypocalc.com> Message-ID: <20101007194247.GA31826@phd.pp.ru> On Thu, Oct 07, 2010 at 09:12:13PM +0200, Victor Stinner wrote: > Le jeudi 07 octobre 2010 18:44:19, Oleg Broytman a ?crit : > > My filesystems are always koi8-r, but sometimes I work with programs in > > utf-8 locale. Just an example... > > Are programs able to display correctly non-ascii filenames if your locale > encoding is different than your filesystem encoding? Most of them don't because - you are right - most programs assume fs encoding to be the same as stdio locale. But some programs are more clever; for example, one can define G_FILENAME_ENCODING env var to guide GTK2/GLib programs; it can be a fixed encoding or a special value "@locale". On the other side there are programs that ignore locale completely and read/write filenames using their own fixed encoding; for example, Transmission bittorrent client read/write files in the encoding defined in the .torrent metafile. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From martin at v.loewis.de Thu Oct 7 23:04:54 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 07 Oct 2010 23:04:54 +0200 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: References: <4CA7902D.6060504@v.loewis.de> <20101002223245.05177b74@pitrou.net> <4CA79C58.90106@v.loewis.de> Message-ID: <4CAE35F6.5050508@v.loewis.de> Am 04.10.2010 03:56, schrieb Daniel Stutzbach: > On Sat, Oct 2, 2010 at 3:55 PM, "Martin v. L?wis" > wrote: > > I'll have to come up with a better way to determine the branch > which a patch was created on. > > > That would also be helpful for those of us using DVCS software to talk > to the svn server. :-) Not sure in what way that would be helpful: I know *have* a better way to determine the branch a patch was created on, but I completely fail to see how it would help the DVCS software users. I take the svn revision number from the patch, and then search back in history until I get a revision where all chunks patch cleanly without any changes to the line numbers; this has already helped a lot. I also have a database listing all file names, so I can deal with patches that were created for a subdirectory. However, if I get something like diff -r e981b6cc56b0 Include/longintrepr.h --- a/Include/longintrepr.h Thu Oct 07 03:12:19 2010 +0200 +++ b/Include/longintrepr.h Thu Oct 07 13:53:41 2010 +0200 I have no clue where I should look for the base revision that the patch was created against. I could guess that the base revision was probably created on Oct 07 3:12:19, which would make that r85299. Not sure how reliable this is, though - will the DVCS software always use the same time stamps in the diff for the base version as are recorded in the original repository? Also, there are no directories called "a" and "b" in the repository. Will DVCS software always use these pseudo directories? Regards, Martin From solipsis at pitrou.net Thu Oct 7 23:10:06 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 7 Oct 2010 23:10:06 +0200 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: <4CAE35F6.5050508@v.loewis.de> References: <4CA7902D.6060504@v.loewis.de> <20101002223245.05177b74@pitrou.net> <4CA79C58.90106@v.loewis.de> <4CAE35F6.5050508@v.loewis.de> Message-ID: <20101007231006.3ffc035c@pitrou.net> On Thu, 07 Oct 2010 23:04:54 +0200 "Martin v. L?wis" wrote: > > However, if I get something like > > diff -r e981b6cc56b0 Include/longintrepr.h > --- a/Include/longintrepr.h Thu Oct 07 03:12:19 2010 +0200 > +++ b/Include/longintrepr.h Thu Oct 07 13:53:41 2010 +0200 > > I have no clue where I should look for the base revision > that the patch was created against. As I said, most patches are supposed to be produced against py3k HEAD, so you could try just that as the primary heuristic. I think you may trying too hard to find smart ways of inferring the correct svn rev and branch, while developing against latest py3k is, most of the time, the required standard. Outdated patches are not really helpful anyway. Regards Antoine. From martin at v.loewis.de Thu Oct 7 23:17:03 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 07 Oct 2010 23:17:03 +0200 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: <20101007231006.3ffc035c@pitrou.net> References: <4CA7902D.6060504@v.loewis.de> <20101002223245.05177b74@pitrou.net> <4CA79C58.90106@v.loewis.de> <4CAE35F6.5050508@v.loewis.de> <20101007231006.3ffc035c@pitrou.net> Message-ID: <4CAE38CF.5070305@v.loewis.de> > As I said, most patches are supposed to be produced against py3k HEAD, > so you could try just that as the primary heuristic. I think this is impractical. There are tons of patches (the majority) which are in the tracker and *not* against py3k head. So this heuristics will only cover a small minority. > I think you may trying too hard to find smart ways of inferring the > correct svn rev and branch, while developing against latest py3k is, > most of the time, the required standard. Outdated patches are not > really helpful anyway Hmm. So how many versions should I go back in py3k until giving up? Regards, Martin From solipsis at pitrou.net Thu Oct 7 23:20:23 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 07 Oct 2010 23:20:23 +0200 Subject: [Python-Dev] Rietveld integration into Roundup In-Reply-To: <4CAE38CF.5070305@v.loewis.de> References: <4CA7902D.6060504@v.loewis.de> <20101002223245.05177b74@pitrou.net> <4CA79C58.90106@v.loewis.de> <4CAE35F6.5050508@v.loewis.de> <20101007231006.3ffc035c@pitrou.net> <4CAE38CF.5070305@v.loewis.de> Message-ID: <1286486423.3143.6.camel@localhost.localdomain> Le jeudi 07 octobre 2010 ? 23:17 +0200, "Martin v. L?wis" a ?crit : > > As I said, most patches are supposed to be produced against py3k HEAD, > > so you could try just that as the primary heuristic. > > I think this is impractical. There are tons of patches (the majority) > which are in the tracker and *not* against py3k head. So this heuristics > will only cover a small minority. I don't understand what the problem is. As long as the patch *applies* cleanly on py3k HEAD, it's ok. If it doesn't, the patch will have to be re-generated anyway. > > I think you may trying too hard to find smart ways of inferring the > > correct svn rev and branch, while developing against latest py3k is, > > most of the time, the required standard. Outdated patches are not > > really helpful anyway > > Hmm. So how many versions should I go back in py3k until giving up? Zero. Regards Antoine. From debatem1 at gmail.com Fri Oct 8 00:00:26 2010 From: debatem1 at gmail.com (geremy condra) Date: Thu, 7 Oct 2010 15:00:26 -0700 Subject: [Python-Dev] python.org going down? Message-ID: Seems like python.org has gone down and come back up a couple of times in the last few minutes, is this intentional? Geremy Condra From martin at v.loewis.de Fri Oct 8 00:52:07 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 08 Oct 2010 00:52:07 +0200 Subject: [Python-Dev] python.org going down? In-Reply-To: References: Message-ID: <4CAE4F17.4020206@v.loewis.de> Am 08.10.2010 00:00, schrieb geremy condra: > Seems like python.org has gone down and come back up a couple of times > in the last few minutes, is this intentional? Nothing on python.org suggests that this has actually happened. Could it be that the issue is on your end? Regards, Martin From tk at giga.or.at Fri Oct 8 00:53:09 2010 From: tk at giga.or.at (Thomas Klausner) Date: Fri, 8 Oct 2010 00:53:09 +0200 Subject: [Python-Dev] python-2.6.6 coredump running newspipe In-Reply-To: References: <20101007121752.GB15070@danbala.tuwien.ac.at> Message-ID: <20101007225309.GE15070@danbala.tuwien.ac.at> On Thu, Oct 07, 2010 at 11:15:27AM -0700, Brett Cannon wrote: > It's best to report issues at bugs.python.org. Are different people reading it there? Following your suggestion, I've created http://bugs.python.org/issue10047 Please let me know what kind of further details you need. I still have the python sources, executable with debugging symbols and core dump lying around, so right now I can give feedback most quickly. Thanks, Thomas From debatem1 at gmail.com Fri Oct 8 01:06:22 2010 From: debatem1 at gmail.com (geremy condra) Date: Thu, 7 Oct 2010 16:06:22 -0700 Subject: [Python-Dev] python.org going down? In-Reply-To: <4CAE4F17.4020206@v.loewis.de> References: <4CAE4F17.4020206@v.loewis.de> Message-ID: On Thu, Oct 7, 2010 at 3:52 PM, "Martin v. L?wis" wrote: > Am 08.10.2010 00:00, schrieb geremy condra: >> Seems like python.org has gone down and come back up a couple of times >> in the last few minutes, is this intentional? > > Nothing on python.org suggests that this has actually happened. Could > it be that the issue is on your end? It's possible, but I was first notified that it was happening by someone literally 3000 miles away. It seems unlikely that we would both have had the same problem at the same time under those conditions, doesn't it? Geremy Condra From brett at python.org Fri Oct 8 01:12:43 2010 From: brett at python.org (Brett Cannon) Date: Thu, 7 Oct 2010 16:12:43 -0700 Subject: [Python-Dev] python-2.6.6 coredump running newspipe In-Reply-To: <20101007225309.GE15070@danbala.tuwien.ac.at> References: <20101007121752.GB15070@danbala.tuwien.ac.at> <20101007225309.GE15070@danbala.tuwien.ac.at> Message-ID: On Thu, Oct 7, 2010 at 15:53, Thomas Klausner wrote: > On Thu, Oct 07, 2010 at 11:15:27AM -0700, Brett Cannon wrote: >> It's best to report issues at bugs.python.org. > > Are different people reading it there? Yes, but we also just don't accept bug reports here as they just get lost in mailing list traffic. > > Following your suggestion, I've created > http://bugs.python.org/issue10047 Thanks. -Brett > > Please let me know what kind of further details you need. > I still have the python sources, executable with debugging symbols and > core dump lying around, so right now I can give feedback most quickly. > > Thanks, > ?Thomas > From martin at v.loewis.de Fri Oct 8 01:17:46 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 08 Oct 2010 01:17:46 +0200 Subject: [Python-Dev] python.org going down? In-Reply-To: References: <4CAE4F17.4020206@v.loewis.de> Message-ID: <4CAE551A.90105@v.loewis.de> >> >> Nothing on python.org suggests that this has actually happened. Could >> it be that the issue is on your end? > > It's possible, but I was first notified that it was happening by > someone literally 3000 miles away. It seems unlikely that we would > both have had the same problem at the same time under those > conditions, doesn't it? True. However, I really cannot see anything on the machines that indicates some outage. I'm still unsure what "it" is that was happening, so it's also difficult to analyse this further. Regards, Martin From tseaver at palladion.com Fri Oct 8 01:33:19 2010 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 07 Oct 2010 19:33:19 -0400 Subject: [Python-Dev] python.org going down? In-Reply-To: <4CAE551A.90105@v.loewis.de> References: <4CAE4F17.4020206@v.loewis.de> <4CAE551A.90105@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/07/2010 07:17 PM, "Martin v. L?wis" wrote: >>> >>> Nothing on python.org suggests that this has actually happened. Could >>> it be that the issue is on your end? >> >> It's possible, but I was first notified that it was happening by >> someone literally 3000 miles away. It seems unlikely that we would >> both have had the same problem at the same time under those >> conditions, doesn't it? > > True. However, I really cannot see anything on the machines that > indicates some outage. I'm still unsure what "it" is that was happening, > so it's also difficult to analyse this further. FWIW, PyPI was inaccessible for some longish period of time this morning. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyuWL8ACgkQ+gerLs4ltQ67gACeML5yTGYeeUPNzkwAQjGY+DFn kOkAnisWvAi0AAYOwPqvwqIcG0h7emPj =kdeJ -----END PGP SIGNATURE----- From martin at v.loewis.de Fri Oct 8 01:42:06 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 08 Oct 2010 01:42:06 +0200 Subject: [Python-Dev] python.org going down? In-Reply-To: References: <4CAE4F17.4020206@v.loewis.de> <4CAE551A.90105@v.loewis.de> Message-ID: <4CAE5ACE.8030206@v.loewis.de> > FWIW, PyPI was inaccessible for some longish period of time this morning. That I can confirm (assuming you are talking about the UTC night). However, it stopped around 5:00 UTC, so it's clearly unrelated to anything that happened reportedly 2 hours ago. Regards, Martin From debatem1 at gmail.com Fri Oct 8 02:16:24 2010 From: debatem1 at gmail.com (geremy condra) Date: Thu, 7 Oct 2010 17:16:24 -0700 Subject: [Python-Dev] python.org going down? In-Reply-To: <4CAE551A.90105@v.loewis.de> References: <4CAE4F17.4020206@v.loewis.de> <4CAE551A.90105@v.loewis.de> Message-ID: On Thu, Oct 7, 2010 at 4:17 PM, "Martin v. L?wis" wrote: >>> >>> Nothing on python.org suggests that this has actually happened. Could >>> it be that the issue is on your end? >> >> It's possible, but I was first notified that it was happening by >> someone literally 3000 miles away. It seems unlikely that we would >> both have had the same problem at the same time under those >> conditions, doesn't it? > > True. However, I really cannot see anything on the machines that > indicates some outage. I'm still unsure what "it" is that was happening, > so it's also difficult to analyse this further. chalk it up to a mystery of the internet, I guess. It still seems strange to me that two people would get the same behavior so far away from each other and not have it be on that end though. Geremy Condra From tseaver at palladion.com Fri Oct 8 02:45:16 2010 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 07 Oct 2010 20:45:16 -0400 Subject: [Python-Dev] python.org going down? In-Reply-To: <4CAE5ACE.8030206@v.loewis.de> References: <4CAE4F17.4020206@v.loewis.de> <4CAE551A.90105@v.loewis.de> <4CAE5ACE.8030206@v.loewis.de> Message-ID: <4CAE699C.2080403@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/07/2010 07:42 PM, "Martin v. L?wis" wrote: >> FWIW, PyPI was inaccessible for some longish period of time this morning. > > That I can confirm (assuming you are talking about the UTC night). > However, it stopped around 5:00 UTC, so it's clearly unrelated to > anything that happened reportedly 2 hours ago. If you have evidence in the logs for one and not the other, then they are likely unrelated. I was speculating about possible issues with, say, the network fabric at the ISP, or something, which could cause apparent outages from the outside viewer's perspective, without necessarily leaving any trace on the machines hosting the sites, except a drop in traffic in the logs. Tres - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyuaZwACgkQ+gerLs4ltQ5GGwCeIVOv1up3AW0wbBtrrnUE70H1 tLgAniCWWmsXG3PRXzxXEVfunMLNSiku =+NY4 -----END PGP SIGNATURE----- From stephen at xemacs.org Fri Oct 8 05:37:38 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 08 Oct 2010 12:37:38 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101007151518.AEFC11BF4F3@kimball.webabinitio.net> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007004608.ED5EA22A3E5@kimball.webabinitio.net> <877hhu8rvf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007151518.AEFC11BF4F3@kimball.webabinitio.net> Message-ID: <8739sh8id9.fsf@uwakimon.sk.tsukuba.ac.jp> R. David Murray writes: > > The MIME-charset = UNKNOWN dodge might be a better way of handling > > this. > > That is a very interesting idea. It is the *right* thing to do, since it > would mean that a message parsed as bytes could be generated via Generator > and passed to, say, smtplib without losing any information. However, > It's not exactly trivial to implement, since issues of runs of characters > and line re-wrapping need need to be dealt with. Perhaps Header can be > made to handle bytes in order to do this; I'll have to look in to > it. Ouch. RFC 822 line wrapping is a bytes->bytes transformation, and the client shouldn't see it at all unless it inspects the wire format. MIME-encoding is a text->bytes transformation, again an internal matter. The constraints on the wire format means that the MIME- encoder needs to careful about encoded-word length. ISTM that all you need to know, assuming that this is a method on a Header, and it's normally invoked just before conversion to bytes, is the codec and the CTE, and both can be optional (default to 'utf-8' and a value depending on the proportion of encodable characters). You take the header, encode according to the codec, then start MIME-encoding according to the CTE. The maximum size of encoded words is chosen to fit on a line within 78 bytes. The number of bytes encoded in each word depends only on the size of metadata associated with the word. (Sure you could make it prettier for those reading it with an "MUA" like less, but I don't think that's really worth anybody's time.) *If* you have an 8-bit value of unknown encoding on input, this will appear in the Header's value as a surrogate. Hm, OK, I see the problem ... as usual, it's that the only efficient thing to do is encode using surrogate-escape which loses the information that these are invalid bytes. Would it really be that bad to add an O(length) component where you examine the string for surrogates (and too-long words, for that matter), and chop off those pieces for MIME encoding? > > > Presumably you are suggesting that email5 be smart enough to turn my > > > example into properly UTF-8/CTE encoded text. > > > > No, in general that's undecidable without asking the originator, > > although humans can often make a good guess. > > I was talking about unicode input, though, where you do know (modulo > the language differences that unicode hasn't yet sorted out). I don't understand why this is difficult. As far as what Unicode has and hasn't sorted out, that's not your job AFAICS. If clients want a specific codec or other language-based style, they'd better specify it themselves. Else, you just stuff the Unicode into a UTF-8-encoded bytes, and go from there. This is *why* Unicode was designed, so that software could do something standard and sane with text which needs to be readable but not exquisitely crafted literary works. No? If you want beauty, then use a markup language. > Right, but I was talking about my python3 example, where I was using > the email5 parser to (unsuccessfully) parse unicode. *That's* the thing > email5 can't really handle, but email6 will be able to. For email5 it would be an extension, yes, but I don't see why it would be hard to handle Unicode input, assuming it's *really* Unicode, unless you want to cater to "legacy" systems that might not understand Unicode (or at least would prefer an alternative encoding). Since it's an extension, I don't think that's your problem, and the people who would really like this extension (eg, the Japanese) are used to dealing with mojibake issues. (Of course, as an extension, you don't need to do it at all. This is just speculation.) The problem would be with careless clients of email5 that find a way to hand it bogus Unicode (eg, by inappropriately using the latin-1 codec to get a binary represention of their bytes in Unicode), but I'm not sure how big a problem that would be. > Thank you very much for this piece of perspective. I hadn't thought > about it that clearly before, but what you say makes perfect sense to me, > and is in fact the implicit perspective I've been working from when > working on the email6 stuff. You're welcome, of course, and it makes me feel much better about email6. (Not that I had any real worries, but here we are about halfway up a 100m cliff, and the trail just widened from 20cm to 2m. :-) From stephen at xemacs.org Fri Oct 8 07:33:22 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 08 Oct 2010 14:33:22 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: References: Message-ID: <871v818d0d.fsf@uwakimon.sk.tsukuba.ac.jp> lutz at rmi.net writes: > To put that more strongly, the Python user base is much larger than > this list's readership. Agreed. Nevertheless, this is the channel (not "channel") that the developers listen on, and substantial effort is made to let Python users know that. I think they do know it, too. > If I'm using 3.1 email, so are many others. That's not obvious. 3.1 email is unusable for several applications. In fact, for human factors reasons (humans are very likely to communicate with other humans who use the same encodings, and to accept occasional glitches they must deal with manually), MUAs are likely to port relatively easily as "good enough" software. But I doubt very much that folks writing MTAs or spam filters that must run unattended, often in long-lived, very active processes, are producing production versions using Python 3 email yet. > People will accept the 3.X world you make up to a point, but it's > impossible to code to a moving target, much less base a product on > it. "Impossible is nothing." It's a decision that each individual developer makes for herself. I haven't heard Mailman devs complain about the impossibility of dealing with the proposed changes, for example. Quite the reverse, in fact. > At some point, they'll simply stop trying to keep up; in fact, > some already have. Predictable and predicted. Where's the balance? I don't know, but "channeling" the users is not a lot of help. There are three worthy goals here: 1. Taking advantage of improvements in to-be-released Pythons. 2. Not changing one's own working code. 3. Not participating in python-dev/email-sig. Take any two; one can't have all three. More specifically, it's interesting that most of the users you talk to care enough to actually say they don't want more incompatible changes. But what are we supposed to take from that? Some fixes have to be incompatible; do the users want the fix or the compatibility? You waffle (as a good representative often must): > Fixes are a Good Thing, of course, and this particular change's scope > remains to be seen; but to channel most of the users I meet out there > in the real world today: Enough with the 3.X changes already, eh? But that's also a decision each developer *can* make for himself. Python does not withdraw products, or even withdraw support, just because the core developers release something they consider better. If having 1 *and* 2 is so important to particular users, but they come into conflict because of proposed changes in Python, then they're going to have to give up 3, come here, and articulate their needs. As you are doing -- but to have real influence, you're going to have to do the review of David's patch that he requests. I really don't see how the process can work any other way. From ziade.tarek at gmail.com Fri Oct 8 09:05:56 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 09:05:56 +0200 Subject: [Python-Dev] Distutils2 scripts Message-ID: Hello, In the Distutils2 project, we'll have quite a few scripts that can be called via -m $ python -m distutils2.depgraph : shows a dependency graph $ python -m distutils2.install : installs a project $ python -m distutils2.run command : runs a distutils2 command etc.. The feedback I received for this is pretty clear: people want a single script that can be called directly. e.g. $ distutils2 depgraph $ distutils2 install $ distutils2 run command Fair enough, I can add that script in the project and get it installed when people install the project. I just wanted to make sure that once distutils2 is back in the stdlib, it's OK for us to add that script in the distribution. I remember that people did not want "yet another script" in there, so I just wanted to double-check before I take this path Thanks ! Tarek -- Tarek Ziad? | http://ziade.org From asmodai at in-nomine.org Fri Oct 8 09:21:08 2010 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 8 Oct 2010 09:21:08 +0200 Subject: [Python-Dev] python.org going down? In-Reply-To: <4CAE551A.90105@v.loewis.de> References: <4CAE4F17.4020206@v.loewis.de> <4CAE551A.90105@v.loewis.de> Message-ID: <20101008072107.GI85444@nexus.in-nomine.org> -On [20101008 01:22], "Martin v. L?wis" (martin at v.loewis.de) wrote: >True. However, I really cannot see anything on the machines that >indicates some outage. I'm still unsure what "it" is that was happening, >so it's also difficult to analyse this further. It's hosted by XS4All, they had some network problems yesterday at least. I am not sure if it would explain this behaviour, might be one of the uplinks or peers had some issues. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Sleep tonight, sweet summer light, scattered yesterdays, the past is far away... From asmodai at in-nomine.org Fri Oct 8 09:22:37 2010 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 8 Oct 2010 09:22:37 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: <20101008072237.GJ85444@nexus.in-nomine.org> -On [20101008 09:07], Tarek Ziad? (ziade.tarek at gmail.com) wrote: >The feedback I received for this is pretty clear: people want a single >script that can be called directly. e.g. > >$ distutils2 depgraph >$ distutils2 install >$ distutils2 run command +1 from me. I sincerely dislike the Perl-esque -m stuff. >I just wanted to make sure that once distutils2 is back in the stdlib, >it's OK for us to add that script in the distribution. No problem from my side, I do, however, not speak for the developers at large of course. ;) -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Care-charmer Sleep, son of the sable Night, Brother to Death, in silent darkness born... From asmodai at in-nomine.org Fri Oct 8 09:25:57 2010 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 8 Oct 2010 09:25:57 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: <20101008072557.GA75788@nexus.in-nomine.org> -On [20101008 09:07], Tarek Ziad? (ziade.tarek at gmail.com) wrote: >I just wanted to make sure that once distutils2 is back in the stdlib, >it's OK for us to add that script in the distribution. Ah, one thing that came to mind: is this script supposed to be installed in /usr{/local}/bin? If so, how would it look like for multiple Python distributions that are installed? -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B To the dull mind nature is leaden. To the illumined mind the whole world burns and sparkles with light... From ziade.tarek at gmail.com Fri Oct 8 09:50:08 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 09:50:08 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008072557.GA75788@nexus.in-nomine.org> References: <20101008072557.GA75788@nexus.in-nomine.org> Message-ID: On Fri, Oct 8, 2010 at 9:25 AM, Jeroen Ruigrok van der Werven wrote: > -On [20101008 09:07], Tarek Ziad? (ziade.tarek at gmail.com) wrote: >>I just wanted to make sure that once distutils2 is back in the stdlib, >>it's OK for us to add that script in the distribution. > > Ah, one thing that came to mind: > > is this script supposed to be installed in /usr{/local}/bin? If so, how > would it look like for multiple Python distributions that are installed? This script will be installed besides the Python executable, so it should follow the same naming rules depending on the OSes. e.g. a -MINOR.MINOR suffix is appended *or* each python version has its own bin/ location etc. -- Tarek Ziad? | http://ziade.org From ziade.tarek at gmail.com Fri Oct 8 09:51:01 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 09:51:01 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008072557.GA75788@nexus.in-nomine.org> Message-ID: On Fri, Oct 8, 2010 at 9:50 AM, Tarek Ziad? wrote: ... > e.g. a -MINOR.MINOR MAJOR.MINOR From chris at simplistix.co.uk Fri Oct 8 10:41:50 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 08 Oct 2010 09:41:50 +0100 Subject: [Python-Dev] os.path.normcase rationale? In-Reply-To: <201010052204.40773.steve@pearwood.info> References: <4C9531A7.10405@simplistix.co.uk> <201009251325.28749.steve@pearwood.info> <4CAADFFB.50205@simplistix.co.uk> <201010052204.40773.steve@pearwood.info> Message-ID: <4CAED94E.7090207@simplistix.co.uk> On 05/10/2010 12:04, Steven D'Aprano wrote: > On Tue, 5 Oct 2010 07:21:15 pm Chris Withers wrote: >> On 25/09/2010 04:25, Steven D'Aprano wrote: >>> 1. Return the case of a filename in some canonical form which >>> depends on the file system? >>> 2. Return the case of a filename as it is actually stored on disk? >> >> How do 1 and 2 differ? > > Case #1 imposes a particular canonical form, regardless of what is > actually stored on disk. It is similar to normpath, except that we > could have different canonical forms depending on what the file system > was. normpath merely generalises from the operating system, and never > looks at the file system. Ah, okay, yeah, that's actually an anti-goal for me ;-) > Case #2 says to actually look at the file and see what the file system > considers it's name to be. Consider a NTFS file system. By default it > is case-preserving and case-insensitive, although that can be changed. > (Just because a file system is NTFS doesn't mean that will be > case-insensitive. NTFS can also run in a POSIX mode which is > case-sensitive. But I digress.) Yeah, this is definitely where I think the missing use case lies... >> FWIW, the use case that setuptools has (and >> for which it currently incorrectly uses normpath) is number 2. >> >>> 4. Return the case of a filename in some arbitrarily-chosen >>> canonical form which does not depend on the file system? >> >> This is what normpath does, but only if you're on Windows ;-) > > Not quite. macpath.normcase() also lowercases the path. So does the > module for OS/2. Interesting, since I develop on MacOS, Linux and Windows and only experienced the problem caused by setuptools normcase'ing distribution names on Windows. The MacOS case also isn't in the docs. > In any case, Windows is not a file system. It is quite possible to have > virtually any combination of case-destroying, case-preserving, > -sensitive and -insensitive file systems on the one Windows system. Say, > a FAT12 floppy, an NTFS partition, and an ext2 USB stick. Windows > doesn't ship with native support for ext2, but that doesn't mean it > can't be installed with third party drivers. yes, exactly! > normpath pays no attention to any of this, and just lowercases the path. > At least that's cheap, and consistent, even if it solves the wrong > problem :) ...and creates a few more along the way ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Oct 8 10:43:48 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 08 Oct 2010 09:43:48 +0100 Subject: [Python-Dev] Adding Wing 4 project to In-Reply-To: <20101005140041.6f96103c@pitrou.net> References: <4CAB0E54.4010300@voidspace.org.uk> <20101005140041.6f96103c@pitrou.net> Message-ID: <4CAED9C4.5060202@simplistix.co.uk> On 05/10/2010 13:00, Antoine Pitrou wrote: >> 3. stay with versioned project file names and rename 'python-wing.wpr' >> to 'python-wing3.wpr' >> >> (Option 3 could be done immediately of course.) > > Option 3 looks the most reasonable to me. I don't use Wing, but option 3 does seem most sensible. Explicit-rather-than-implicit-ly yours, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Fri Oct 8 10:50:20 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 08 Oct 2010 09:50:20 +0100 Subject: [Python-Dev] Another relative imports question In-Reply-To: References: Message-ID: <4CAEDB4C.8040303@simplistix.co.uk> Hi All, The new explicit relative import syntax is great. I wanted to relatively import a module. import .mymoduleinmypackage ...and got a SyntaxError in Python 2.6. I guess I need to do: from . import mymoduleinmypackage ...but it does feel weirdly asymetric that: from .mymoduleinmypackage import something ...while: import .mymoduleinmypackage mymoduleinmypackage.something ...does not. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From fuzzyman at voidspace.org.uk Fri Oct 8 11:34:25 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 08 Oct 2010 10:34:25 +0100 Subject: [Python-Dev] os.path.normcase rationale? In-Reply-To: <4CAED94E.7090207@simplistix.co.uk> References: <4C9531A7.10405@simplistix.co.uk> <201009251325.28749.steve@pearwood.info> <4CAADFFB.50205@simplistix.co.uk> <201010052204.40773.steve@pearwood.info> <4CAED94E.7090207@simplistix.co.uk> Message-ID: <4CAEE5A1.2040507@voidspace.org.uk> On 08/10/2010 09:41, Chris Withers wrote: > On 05/10/2010 12:04, Steven D'Aprano wrote: >> On Tue, 5 Oct 2010 07:21:15 pm Chris Withers wrote: >>> On 25/09/2010 04:25, Steven D'Aprano wrote: >>> [snip...] >>> FWIW, the use case that setuptools has (and >>> for which it currently incorrectly uses normpath) is number 2. >>> >>>> 4. Return the case of a filename in some arbitrarily-chosen >>>> canonical form which does not depend on the file system? >>> >>> This is what normpath does, but only if you're on Windows ;-) >> >> Not quite. macpath.normcase() also lowercases the path. So does the >> module for OS/2. > > Interesting, since I develop on MacOS, Linux and Windows and only > experienced the problem caused by setuptools normcase'ing distribution > names on Windows. The MacOS case also isn't in the docs. > Unless you're using Mac OS 9 you will be using posixpath and not macpath though. :-) Michael -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From solipsis at pitrou.net Fri Oct 8 11:42:20 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 8 Oct 2010 11:42:20 +0200 Subject: [Python-Dev] [Pythonmac-SIG] sad state of OS X Python testing... References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> <20101002203205.GA24940@illinois.edu> <4CA79814.3030302@v.loewis.de> Message-ID: <20101008114220.20369e20@pitrou.net> On Tue, 5 Oct 2010 10:08:59 -0700 Stephen Hansen wrote: > On Sat, Oct 2, 2010 at 1:37 PM, "Martin v. L?wis" wrote: > > > > I'm already running a Jython buildslave on an Intel Mac Pro which is > > > pretty underused - I'd be happy to run a CPython one there too, if > > > it'd be worthwhile. > > > > I think Bill was specifically after Snow Leopard - what system are you > > using? > > > > I have a fairly recent MacPro on Snow Leopard, which I keep consistently up > to date and its connected all the time. It has more capacity then I can > really find use for. Now that the buildbot is up, it is recommended that you try to investigate the failures (and the test_ttk_guionly crash), and that you create bugs reports on http://bugs.python.org for them. Thank you! Antoine. From g.brandl at gmx.net Fri Oct 8 12:24:07 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 08 Oct 2010 12:24:07 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: Am 08.10.2010 09:05, schrieb Tarek Ziad?: > Hello, > > In the Distutils2 project, we'll have quite a few scripts that can be > called via -m > > $ python -m distutils2.depgraph : shows a dependency graph > $ python -m distutils2.install : installs a project > $ python -m distutils2.run command : runs a distutils2 command > > etc.. What happened to "python setup.py action"? Or is this a step towards not requiring setup.py at all? > The feedback I received for this is pretty clear: people want a single > script that can be called directly. e.g. > > $ distutils2 depgraph > $ distutils2 install > $ distutils2 run command > > Fair enough, I can add that script in the project and get it installed > when people install the project. > > I just wanted to make sure that once distutils2 is back in the stdlib, > it's OK for us to add that script in the distribution. > I remember that people did not want "yet another script" in there, so > I just wanted to double-check before I take this path This is certainly preferable to memorizing the modules you can call with -m, because it's easier to provide help functions. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Fri Oct 8 12:31:49 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 08 Oct 2010 12:31:49 +0200 Subject: [Python-Dev] Another relative imports question In-Reply-To: <4CAEDB4C.8040303@simplistix.co.uk> References: <4CAEDB4C.8040303@simplistix.co.uk> Message-ID: Am 08.10.2010 10:50, schrieb Chris Withers: > Hi All, > > The new explicit relative import syntax is great. > I wanted to relatively import a module. > > import .mymoduleinmypackage > > ....and got a SyntaxError in Python 2.6. > > I guess I need to do: > > from . import mymoduleinmypackage > > ....but it does feel weirdly asymetric that: > > from .mymoduleinmypackage import something > > ....while: > > import .mymoduleinmypackage > mymoduleinmypackage.something > > ....does not. The explanation is that everything that comes after "import" is thereafter usable as an identifier (or expression, in the case of dotted names) in code. ".mymodule" is not a valid expression, so the question would be how to refer to it. (This argument is weakened in the present of an "as" renaming, but I doubt you'd like to write import .mymodule as mymodule .) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ncoghlan at gmail.com Fri Oct 8 13:24:00 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 8 Oct 2010 21:24:00 +1000 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008072237.GJ85444@nexus.in-nomine.org> References: <20101008072237.GJ85444@nexus.in-nomine.org> Message-ID: On Fri, Oct 8, 2010 at 5:22 PM, Jeroen Ruigrok van der Werven wrote: > +1 from me. I sincerely dislike the Perl-esque -m stuff. -m is generally for developer utilities, or "incidental" utilities provided by modules that are mainly intended for use as library modules. It's also very handy for running stuff out of an uninstalled Python (such as an svn checkout). For actual end-user facing scripts, a proper script on the system path is a better option. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From dirkjan at ochtman.nl Fri Oct 8 13:24:09 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 8 Oct 2010 13:24:09 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: On Fri, Oct 8, 2010 at 09:05, Tarek Ziad? wrote: > The feedback I received for this is pretty clear: people want a single > script that can be called directly. e.g. > > $ distutils2 depgraph > $ distutils2 install > $ distutils2 run command > > Fair enough, I can add that script in the project and get it installed > when people install the project. It doesn't seem very nice to have a version in the script. Can we just call it distutils? Or py-dist? paint-it-some-other-color-ly yours, Dirkjan From jon+python-dev at unequivocal.co.uk Fri Oct 8 13:31:37 2010 From: jon+python-dev at unequivocal.co.uk (Jon Ribbens) Date: Fri, 8 Oct 2010 12:31:37 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: <20101008113137.GA20222@snowy.squish.net> On Fri, Oct 08, 2010 at 01:24:09PM +0200, Dirkjan Ochtman wrote: > On Fri, Oct 8, 2010 at 09:05, Tarek Ziad? wrote: > > The feedback I received for this is pretty clear: people want a single > > script that can be called directly. e.g. > > > > $ distutils2 depgraph > > $ distutils2 install > > $ distutils2 run command > > > > Fair enough, I can add that script in the project and get it installed > > when people install the project. > > It doesn't seem very nice to have a version in the script. Can we just > call it distutils? Or py-dist? pypackage? From fdrake at acm.org Fri Oct 8 14:02:19 2010 From: fdrake at acm.org (Fred Drake) Date: Fri, 8 Oct 2010 08:02:19 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: On Fri, Oct 8, 2010 at 7:24 AM, Dirkjan Ochtman wrote: > It doesn't seem very nice to have a version in the script. Can we just > call it distutils? Or py-dist? If we go this route, then - "make altinstall" should include the version number in the name of *any* scripts it installs. - "make install" would then add links without the version numbers. This mirrors the name of the Python executable, so is likely as "right" as it's going to get (*if* we install separate scripts). Georg: > What happened to "python setup.py action"? Or is this a step towards > not requiring setup.py at all? I'm in favor of add a top-level setup module that can be invoked using "python -m setup ...". There will be three cases: - d2 projects without a setup.py, where this will invoke the module from the standard library, - d2 projects with a setup.py, where the local setup.py will be invoked, and - d1 projects, which always have a setup.py, which will be invoked. This would provide the most consistent story for package users, where the only command they'll need to remember (for Python versions with the setup module) will be python -m setup ... ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From ronaldoussoren at mac.com Fri Oct 8 14:14:00 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 08 Oct 2010 12:14:00 +0000 (GMT) Subject: [Python-Dev] os.path.normcase rationale? In-Reply-To: <4CAEE5A1.2040507@voidspace.org.uk> Message-ID: <3b13a3d2-4886-4ca4-97bd-e00d132d984e@me.com> On 08 Oct, 2010,at 11:38 AM, Michael Foord wrote: On 08/10/2010 09:41, Chris Withers wrote: >>>> 4. Return the case of a filename in some arbitrarily-chosen >>>> canonical form which does not depend on the file system? ? AFAIK this is what the function is supposed to do: return a platform-dependent canonical form of the filename. And that is hopelessly naive on modern systems, on both linux and OSX some file systems are case insensitive and others are not. The default for Linux is case sensitive, but some filesystems are not (VFAT, CIFS), and the default on OSX is case insensitive, but some filesystems are case sensitive (NFS, case sensitive HFS+) Ronald -------------- next part -------------- An HTML attachment was scrubbed... URL: From dirkjan at ochtman.nl Fri Oct 8 14:15:39 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 8 Oct 2010 14:15:39 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: 2010/10/8 Fred Drake : > Georg: >> What happened to "python setup.py action"? ?Or is this a step towards >> not requiring setup.py at all? > > I'm in favor of add a top-level setup module that can be invoked using > "python -m setup ...". ?There will be three cases: > > - d2 projects without a setup.py, where this will invoke the module from > ?the standard library, > > - d2 projects with a setup.py, where the local setup.py will be invoked, > ?and > > - d1 projects, which always have a setup.py, which will be invoked. > > This would provide the most consistent story for package users, where the > only command they'll need to remember (for Python versions with the setup > module) will be > > ? ?python -m setup ... That seems like a fairly elegant solution. Cheers, Dirkjan From g.brandl at gmx.net Fri Oct 8 14:20:50 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 08 Oct 2010 14:20:50 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: Am 08.10.2010 14:02, schrieb Fred Drake: > Georg: >> What happened to "python setup.py action"? Or is this a step towards >> not requiring setup.py at all? > > I'm in favor of add a top-level setup module that can be invoked using > "python -m setup ...". There will be three cases: > > - d2 projects without a setup.py, where this will invoke the module from > the standard library, > > - d2 projects with a setup.py, where the local setup.py will be invoked, > and > > - d1 projects, which always have a setup.py, which will be invoked. > > This would provide the most consistent story for package users, where the > only command they'll need to remember (for Python versions with the setup > module) will be > > python -m setup ... That's actually a very nice trick, *probably* together for a "shortcut" for "python -m setup". Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ziade.tarek at gmail.com Fri Oct 8 15:14:43 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 15:14:43 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: On Fri, Oct 8, 2010 at 12:24 PM, Georg Brandl wrote: > Am 08.10.2010 09:05, schrieb Tarek Ziad?: >> Hello, >> >> In the Distutils2 project, we'll have quite a few scripts that can be >> called via -m >> >> $ python -m distutils2.depgraph : shows a dependency graph >> $ python -m distutils2.install : installs a project >> $ python -m distutils2.run command : runs a distutils2 command >> >> etc.. > > What happened to "python setup.py action"? ?Or is this a step towards > not requiring setup.py at all? Yes, setup.py is gone and everything is driven by scripts, that read setup.cfg. There are some configurable hooks to call code when wanted bu the idea is that those are restricted to the build part of the process, so a release made at PyPI provides static metadata. Tarek -- Tarek Ziad? | http://ziade.org From ziade.tarek at gmail.com Fri Oct 8 15:22:11 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 15:22:11 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: 2010/10/8 Fred Drake : > On Fri, Oct 8, 2010 at 7:24 AM, Dirkjan Ochtman wrote: >> It doesn't seem very nice to have a version in the script. Can we just >> call it distutils? Or py-dist? > > If we go this route, then > > - "make altinstall" should include the version number in the name of *any* > ?scripts it installs. > > - "make install" would then add links without the version numbers. > > This mirrors the name of the Python executable, so is likely as "right" as > it's going to get (*if* we install separate scripts). Yes that what I was thinking about -- I am not too worried about this, since every Linux deals with the 'more than one python installed' case. > > Georg: >> What happened to "python setup.py action"? ?Or is this a step towards >> not requiring setup.py at all? > > I'm in favor of add a top-level setup module that can be invoked using > "python -m setup ...". ?There will be three cases: Nice idea ! I wouldn't call it setup though, since it does many other things. I can't think of a good name yet, but I'd like such a script to express the idea that it can be used to: - query pypi - browse what's installed - install/remove projects - create releases and upload them pkg_manager ? > > - d2 projects without a setup.py, where this will invoke the module from > ?the standard library, Yes ! > > - d2 projects with a setup.py, where the local setup.py will be invoked, > ?and Mmm.. setup.py is gone in D2, and setup.py will be the marker of d1. Some project might want to provide both setups for backward compatibility: - a setup.py (d1) - a setup,cfg (d2 and optionally some d1 options) It's easy enough in that case to detect if the .cfg has what d2 requires (like the [metadata] section) > > - d1 projects, which always have a setup.py, which will be invoked. Yes > > This would provide the most consistent story for package users, where the > only command they'll need to remember (for Python versions with the setup > module) will be > > ? ?python -m setup ... Nice ! And I'd like to extend this idea with what we said at the last Summit. What if a project like Pip is able to replace d1 and or d2 when it comes to install a package. As long as we agree on the same command line interface, we could provide a way for pip to be called by this global script. Tarek -- Tarek Ziad? | http://ziade.org From dirkjan at ochtman.nl Fri Oct 8 15:28:25 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 8 Oct 2010 15:28:25 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: On Fri, Oct 8, 2010 at 15:22, Tarek Ziad? wrote: > Mmm.. setup.py is gone in D2, and setup.py will be the marker of d1. So, sorry for backing up to this, but isn't it true that many projects do custom stuff in their setup.py that they wouldn't be able to do in setup.cfg? Is the goal really to make that impossible/unnecessary? In Mercurial, for example, we have a fairly large setup.py that helps us deal with many kinds of issues: incomplete Python installations on different platforms, adding packages depending on the platform, do custom stuff to compile gettext files, etc... Cheers, Dirkjan From fuzzyman at voidspace.org.uk Fri Oct 8 15:30:42 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 08 Oct 2010 14:30:42 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: <4CAF1D02.4030604@voidspace.org.uk> On 08/10/2010 14:28, Dirkjan Ochtman wrote: > On Fri, Oct 8, 2010 at 15:22, Tarek Ziad? wrote: >> Mmm.. setup.py is gone in D2, and setup.py will be the marker of d1. > So, sorry for backing up to this, but isn't it true that many projects > do custom stuff in their setup.py that they wouldn't be able to do in > setup.cfg? Is the goal really to make that impossible/unnecessary? The goal is to make it unnecessary. My understanding is that it will still be possible to use a setup.py, just unnecessary for the vast majority of cases. All the best, Michael > In > Mercurial, for example, we have a fairly large setup.py that helps us > deal with many kinds of issues: incomplete Python installations on > different platforms, adding packages depending on the platform, do > custom stuff to compile gettext files, etc... > > Cheers, > > Dirkjan > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From ziade.tarek at gmail.com Fri Oct 8 15:37:03 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 15:37:03 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: On Fri, Oct 8, 2010 at 3:28 PM, Dirkjan Ochtman wrote: > On Fri, Oct 8, 2010 at 15:22, Tarek Ziad? wrote: >> Mmm.. setup.py is gone in D2, and setup.py will be the marker of d1. > > So, sorry for backing up to this, but isn't it true that many projects > do custom stuff in their setup.py that they wouldn't be able to do in > setup.cfg? They will still be able to do it by using hooks. > Is the goal really to make that impossible/unnecessary? In > Mercurial, for example, we have a fairly large setup.py that helps us > deal with many kinds of issues: incomplete Python installations on > different platforms, adding packages depending on the platform, do > custom stuff to compile gettext files, etc... No the goal is to stop having setup.py as the standard for getting metadata from a project. There will always be ways to call some code in the various steps. setup.py is currently called for everything : building, installing, providing metadata etc. We want to break that and make it possible for people to express their metadata into a static file, and offer ways to express them differently depending on the platforms. See PEP 345. So you should have what you need. Now, we are still in early stages in providing these features, and I'd be happy to work on the Mercurial setup.py conversion, if it's a complex use case. So we can see how things go. Tarek > > Cheers, > > Dirkjan > -- Tarek Ziad? | http://ziade.org From ziade.tarek at gmail.com Fri Oct 8 15:38:47 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 15:38:47 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CAF1D02.4030604@voidspace.org.uk> References: <4CAF1D02.4030604@voidspace.org.uk> Message-ID: 2010/10/8 Michael Foord : > ?On 08/10/2010 14:28, Dirkjan Ochtman wrote: >> >> On Fri, Oct 8, 2010 at 15:22, Tarek Ziad? ?wrote: >>> >>> Mmm.. setup.py is gone in D2, and setup.py will be the marker of d1. >> >> So, sorry for backing up to this, but isn't it true that many projects >> do custom stuff in their setup.py that they wouldn't be able to do in >> setup.cfg? Is the goal really to make that impossible/unnecessary? > > The goal is to make it unnecessary. My understanding is that it will still > be possible to use a setup.py, just unnecessary for the vast majority of > cases. Yes. And to make it possible to keep a d1 setup.py, which works with all installers out there, we decided that d2 would ignore that file, and provide other ways to hook code if necessary. IOW one project may provide a d1 setup.py and a d2 configuration in the same release. > > All the best, > > Michael > >> ?In >> Mercurial, for example, we have a fairly large setup.py that helps us >> deal with many kinds of issues: incomplete Python installations on >> different platforms, adding packages depending on the platform, do >> custom stuff to compile gettext files, etc... >> >> Cheers, >> >> Dirkjan >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > > -- > > http://www.voidspace.org.uk/ > > READ CAREFULLY. By accepting and reading this email you agree, > on behalf of your employer, to release me from all obligations > and waivers arising from any and all NON-NEGOTIATED agreements, > licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, > confidentiality, non-disclosure, non-compete and acceptable use > policies (?BOGUS AGREEMENTS?) that I have entered into with your > employer, its partners, licensors, agents and assigns, in > perpetuity, without prejudice to my ongoing rights and privileges. > You further represent that you have the authority to release me > from any BOGUS AGREEMENTS on behalf of your employer. > > -- Tarek Ziad? | http://ziade.org From barry at python.org Fri Oct 8 15:57:38 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 8 Oct 2010 09:57:38 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <8739sh8id9.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007004608.ED5EA22A3E5@kimball.webabinitio.net> <877hhu8rvf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007151518.AEFC11BF4F3@kimball.webabinitio.net> <8739sh8id9.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101008095738.6e40eb7f@mission> On Oct 08, 2010, at 12:37 PM, Stephen J. Turnbull wrote: >Ouch. RFC 822 line wrapping is a bytes->bytes transformation, and the >client shouldn't see it at all unless it inspects the wire format. Header wrapping sucks even more because it's supposed to take the semantic context into account, which means that a generic Header wrapping algorithm cannot work for everything. E.g. Received: headers are supposed to wrap after the semicolon. The current email package does a pretty poor job of emulating this requirement, though it often gets it right enough. David has plans for addressing this problem. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Fri Oct 8 16:26:36 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 8 Oct 2010 10:26:36 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: <20101008102636.3ffede1e@mission> On Oct 08, 2010, at 03:22 PM, Tarek Ziad? wrote: >Yes that what I was thinking about -- I am not too worried about this, >since every Linux deals with the 'more than one python installed' >case. Kind of. but anyway... >> I'm in favor of add a top-level setup module that can be invoked using >> "python -m setup ...". ?There will be three cases: > >Nice idea ! I wouldn't call it setup though, since it does many other >things. I can't think of a good name yet, but I'd like such a script >to express the idea that it can be used to: I like 'python -m setup' too. It's a small step from the familiar thing (python setup.py) to the new and shiny thing, without being confusing. And you won't have to worry about things like version numbers because the Python executable will already have that baked in. >- query pypi >- browse what's installed >- install/remove projects >- create releases and upload them > >pkg_manager ? No underscores, please. :) Actually, a decent wrapper script could just be called 'setup'. My command-not-found on Ubuntu doesn't find a collision, or even close similarities. I still like 'egg' as a command too. There are no collisions that I can see. I know this has been thrown around for years, and it's always been rejected because I think setuptools wanted to claim it, but since it still doesn't exist afaict, distutils2 could easily use it. In any case, these could be a simple shell script wrapping 'python -m setup'. It could even take a --use-python-version option to select the pythonX.Y it used, without having to encode the Python version number in the script name. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From apt.shansen at gmail.com Fri Oct 8 16:26:58 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Fri, 8 Oct 2010 07:26:58 -0700 Subject: [Python-Dev] [Pythonmac-SIG] sad state of OS X Python testing... In-Reply-To: <20101008114220.20369e20@pitrou.net> References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> <20101002203205.GA24940@illinois.edu> <4CA79814.3030302@v.loewis.de> <20101008114220.20369e20@pitrou.net> Message-ID: On Fri, Oct 8, 2010 at 2:42 AM, Antoine Pitrou wrote: > On Tue, 5 Oct 2010 10:08:59 -0700 > Stephen Hansen wrote: > > On Sat, Oct 2, 2010 at 1:37 PM, "Martin v. L?wis" >wrote: > > > > > > I'm already running a Jython buildslave on an Intel Mac Pro which is > > > > pretty underused - I'd be happy to run a CPython one there too, if > > > > it'd be worthwhile. > > > > > > I think Bill was specifically after Snow Leopard - what system are you > > > using? > > > > > > > I have a fairly recent MacPro on Snow Leopard, which I keep consistently > up > > to date and its connected all the time. It has more capacity then I can > > really find use for. > > Now that the buildbot is up, it is recommended that you try to > investigate the failures (and the test_ttk_guionly crash), and that you > create bugs reports on http://bugs.python.org for them. > I shall, I just got busy yesterday. :) The failure is happening just because it can't possibly succeed, I set up the account for the buildbot in such a way that it has no access to a GUI context. I'm going to rectify that today so I can properly test TK. The hard crash I'll report as soon as I have a few minutes to isolate more where its happening and thus to who all should get a report. --Stephen -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Fri Oct 8 16:54:30 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 08 Oct 2010 16:54:30 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008102636.3ffede1e@mission> References: <20101008102636.3ffede1e@mission> Message-ID: Am 08.10.2010 16:26, schrieb Barry Warsaw: >>- query pypi >>- browse what's installed >>- install/remove projects >>- create releases and upload them >> >>pkg_manager ? > > No underscores, please. :) > > Actually, a decent wrapper script could just be called 'setup'. My > command-not-found on Ubuntu doesn't find a collision, or even close > similarities. No generic name, *please*. easy_install was bad enough, no need to repeat that mistake. "egg" would be better, but weren't we phasing out the egg format? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Fri Oct 8 17:00:16 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 08 Oct 2010 17:00:16 +0200 Subject: [Python-Dev] [Pythonmac-SIG] sad state of OS X Python testing... In-Reply-To: References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> <20101002203205.GA24940@illinois.edu> <4CA79814.3030302@v.loewis.de> <20101008114220.20369e20@pitrou.net> Message-ID: <1286550016.17666.8.camel@localhost.localdomain> Hi, > The failure is happening just because it can't possibly succeed, I set > up the account for the buildbot in such a way that it has no access to > a GUI context. I'm going to rectify that today so I can properly test > TK. Well, a nice thing would be for tests to be properly skipped in this situation, rather than fail or crash :) Do you think you can try to write a patch for this? Thanks for helping! From solipsis at pitrou.net Fri Oct 8 17:03:28 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 8 Oct 2010 17:03:28 +0200 Subject: [Python-Dev] Distutils2 scripts References: <20101008102636.3ffede1e@mission> Message-ID: <20101008170328.459fdd3f@pitrou.net> On Fri, 8 Oct 2010 10:26:36 -0400 Barry Warsaw wrote: > >- query pypi > >- browse what's installed > >- install/remove projects > >- create releases and upload them > > > >pkg_manager ? > > No underscores, please. :) > > Actually, a decent wrapper script could just be called 'setup'. My > command-not-found on Ubuntu doesn't find a collision, or even close > similarities. pysetup perhaps? I think "setup" is such a generic name that being a Python-specific tool sounds wrong. I also had that reaction towards easy_install at the beginning. > I still like 'egg' as a command too. There are no collisions that I can see. > I know this has been thrown around for years, and it's always been rejected > because I think setuptools wanted to claim it, but since it still doesn't > exist afaict, distutils2 could easily use it. 'egg' is already the name of a file format, so re-using that name for something more general would be confusing. What would you think if 'tar' allowed you to resize JPEGs? :) From a.badger at gmail.com Fri Oct 8 17:04:35 2010 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 8 Oct 2010 11:04:35 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008102636.3ffede1e@mission> References: <20101008102636.3ffede1e@mission> Message-ID: <20101008150435.GC10153@unaka.lan> On Fri, Oct 08, 2010 at 10:26:36AM -0400, Barry Warsaw wrote: > On Oct 08, 2010, at 03:22 PM, Tarek Ziad? wrote: > > >Yes that what I was thinking about -- I am not too worried about this, > >since every Linux deals with the 'more than one python installed' > >case. > > Kind of. but anyway... > > >> I'm in favor of add a top-level setup module that can be invoked using > >> "python -m setup ...". ?There will be three cases: > > > >Nice idea ! I wouldn't call it setup though, since it does many other > >things. I can't think of a good name yet, but I'd like such a script > >to express the idea that it can be used to: > > I like 'python -m setup' too. It's a small step from the familiar thing > (python setup.py) to the new and shiny thing, without being confusing. And > you won't have to worry about things like version numbers because the Python > executable will already have that baked in. > > >- query pypi > >- browse what's installed > >- install/remove projects > >- create releases and upload them > > > >pkg_manager ? > > No underscores, please. :) > > Actually, a decent wrapper script could just be called 'setup'. My > command-not-found on Ubuntu doesn't find a collision, or even close > similarities. > Simple English names like this are almost never a good idea for commands. A quick google for "/usr/bin/setup" finds that Fedora-derived distros have a /usr/bin/setup as a wrapper for all the text-mode configuration tools. And there's a derivative of opensolaris that has a /usr/bin/setup for configuring the system the first time. > I still like 'egg' as a command too. There are no collisions that I can see. > I know this has been thrown around for years, and it's always been rejected > because I think setuptools wanted to claim it, but since it still doesn't > exist afaict, distutils2 could easily use it. > There's a 2D graphics library that provides a /usr/bin/egg command: http://www.ir.isas.jaxa.jp/~cyamauch/eggx_procall/ Latest Stable Version 0.93r3 (released 2010/4/14) In the larger universe of programs, it might make for more intuitive remembering of the command to use a prefix (either py or python) though. python-setup is a lot like python setup.py pysetup is shorter pyegg is even shorter :-) -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From stephen at xemacs.org Fri Oct 8 16:55:37 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 08 Oct 2010 23:55:37 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101008095738.6e40eb7f@mission> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007004608.ED5EA22A3E5@kimball.webabinitio.net> <877hhu8rvf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007151518.AEFC11BF4F3@kimball.webabinitio.net> <8739sh8id9.fsf@uwakimon.sk.tsukuba.ac.jp> <20101008095738.6e40eb7f@mission> Message-ID: <87tykw7mza.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > Header wrapping sucks even more because it's supposed to take the > semantic context into account, which means that a generic Header > wrapping algorithm cannot work for everything. E.g. Received: > headers are supposed to wrap after the semicolon. Received headers are an easy special case: An Internet mail program MUST NOT change or delete a Received: line that was previously added to the message header section. (RFC 5321, sec. 4.4) So you save them as bytes and Barry's your FLUFL, as they say. If email wants to *produce* them (as a service to say smtplib), then it wants to comply with the detailed recommendations in RFC 5321, sec. 4.4 anyway; I don't think there's a good reason treat Received headers as text since they're conceptually part of the wire protocol. (Except for the information of curious users, but then getting it exactly right is best done by just passing the whole thing, folds and all, to .decode('ascii'), I should think.) I should think you *want* addresses and suchlike structured headers (Content-Type with several RFC 2231 parameters, anyone?) to line up nicely, too. So generic folding algorithms are really only applicable to unstructured text fields like Subject and Summary anyway. You can call that "sucky" if you like, I prefer to call it "tasteful." From barry at python.org Fri Oct 8 17:07:53 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 8 Oct 2010 11:07:53 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008150435.GC10153@unaka.lan> References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> Message-ID: <20101008110753.6cdaf13b@mission> On Oct 08, 2010, at 11:04 AM, Toshio Kuratomi wrote: >python-setup is a lot like python setup.py >pysetup is shorter >pyegg is even shorter :-) wfm! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From apt.shansen at gmail.com Fri Oct 8 17:11:13 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Fri, 8 Oct 2010 08:11:13 -0700 Subject: [Python-Dev] [Pythonmac-SIG] sad state of OS X Python testing... In-Reply-To: <1286550016.17666.8.camel@localhost.localdomain> References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> <20101002203205.GA24940@illinois.edu> <4CA79814.3030302@v.loewis.de> <20101008114220.20369e20@pitrou.net> <1286550016.17666.8.camel@localhost.localdomain> Message-ID: On Fri, Oct 8, 2010 at 8:00 AM, Antoine Pitrou wrote: > > Hi, > > > The failure is happening just because it can't possibly succeed, I set > > up the account for the buildbot in such a way that it has no access to > > a GUI context. I'm going to rectify that today so I can properly test > > TK. > > Well, a nice thing would be for tests to be properly skipped in this > situation, rather than fail or crash :) Do you think you can try to > write a patch for this? > Absolutely, that's on my TODO list. First, figuring out the buildslave control process; then isolating where that crash is happening and reporting it to whomever (which, I think in a cursory look, is actually the TK folks); then figuring out how to convert no-GUI-context-possible situations into a skip. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Fri Oct 8 17:12:44 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 8 Oct 2010 17:12:44 +0200 Subject: [Python-Dev] Distutils2 scripts References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> Message-ID: <20101008171244.75fc2afa@pitrou.net> On Fri, 8 Oct 2010 11:04:35 -0400 Toshio Kuratomi wrote: > > In the larger universe of programs, it might make for more intuitive > remembering of the command to use a prefix (either py or python) though. > > python-setup is a lot like python setup.py > pysetup is shorter > pyegg is even shorter :-) Wouldn't "quiche" be a better alternative for "pyegg"? From fuzzyman at voidspace.org.uk Fri Oct 8 17:21:19 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 08 Oct 2010 16:21:19 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008110753.6cdaf13b@mission> References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008110753.6cdaf13b@mission> Message-ID: <4CAF36EF.10306@voidspace.org.uk> On 08/10/2010 16:07, Barry Warsaw wrote: > On Oct 08, 2010, at 11:04 AM, Toshio Kuratomi wrote: > >> python-setup is a lot like python setup.py >> pysetup is shorter >> pyegg is even shorter :-) > wfm! To avoid any potential confusion, wfm is a common tla for the following phrases: Whole Foods Market Western Federation of Miners Window Fitters Mate Workforce Management Although wfm is possibly being used here as "works for me" given the context... ;-) Michael > -Barry > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon+python-dev at unequivocal.co.uk Fri Oct 8 17:31:21 2010 From: jon+python-dev at unequivocal.co.uk (Jon Ribbens) Date: Fri, 8 Oct 2010 16:31:21 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008150435.GC10153@unaka.lan> References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> Message-ID: <20101008153121.GE20222@snowy.squish.net> On Fri, Oct 08, 2010 at 11:04:35AM -0400, Toshio Kuratomi wrote: > In the larger universe of programs, it might make for more intuitive > remembering of the command to use a prefix (either py or python) though. > > python-setup is a lot like python setup.py > pysetup is shorter > pyegg is even shorter :-) I'd just like to say "pypackage" again. pypackage install thingy pypackage remove thingy pypackage update thingy etc From lutz at rmi.net Fri Oct 8 17:44:45 2010 From: lutz at rmi.net (lutz at rmi.net) Date: Fri, 08 Oct 2010 15:44:45 -0000 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes Message-ID: Thanks for both your reply and work, David. I'm going to have to test my email clients under the 3.2 patch when it gels. It's good to hear that email5 API support remains a goal. I don't mean to single out this change unfairly, of course. My real concern is not as much with the specific technical aspects of this proposal, as with the generally low priority that backward compatibility sometimes receives on this list. The bytecode file model change in 3.2 comes to mind as another example; sound as it may be, I'm not sure this list has any idea how many users, systems, or docs may be impacted by this. Though not always true, the work here does sometimes appear to be conducted in a vacuum. Ultimately, development in the open source world is driven by the very few with time to show up, rather than by the very many who depend on it. This can unfortunately lead to the perception of thrashing by end users. Some even come to see the net effect as not that much different from closed models. I have no solution to offer, except to underscore again that changes made here affect very many people who are too busy using Python to participate here. Especially given the still tentative state of 3.X, stability matters. --Mark Lutz (http://learning-python.com, http://rmi.net/~lutz) > -----Original Message----- > From: "R. David Murray" > To: lutz at rmi.net > Subject: Re: [Python-Dev] Patch making the current email package (mostly) support bytes > Date: Thu, 07 Oct 2010 13:46:02 -0400 > > On Thu, 07 Oct 2010 16:03:18 -0000, lutz at rmi.net wrote: > > I'm forwarding a link to the code of these clients to David by > > private email in case they might be useful as a test case (O'Reilly > > has already posted them ahead of the book, but they may be a bit too > > heavy for use in formal testing). > > Thanks very much. I will take a look, and expect they will > be helpful. > > > The email package is obviously less than ideal today, and there are > > many other clients for it besides my own, of course. But making it > > backward incompatible at this point is likely to be seen as a big > > negative to newcomers evaluating 3.X viability. And as I tried to > > make clear in June, this list should carefully weigh the PR cost of > > pulling the rug out from under those brave souls who have already > > taken the time to accommodate the 3.X world you've mandated. > > Well, as I have said before the plan is to provide backward compatibility > in email6, so that you only need to change your code if you want to > take advantage of improved or new functionality. If this turns out not > to be possible for some reason, then we aren't going to suddenly stop > supporting email5. That's not the Python Way :) (Example: we added > ArgParse post-3.0, and lots of people wanted to deprecate OptParse, > but we aren't planning on removing OptParse.) > > Do you see any issues with the patch I'm proposing? My goal is to make > things work that didn't work before, but nothing that worked before > should stop working, if I do my job right. > > The one *potentially* backward-incompatible change that I'm consciously > considering (that is, any other backward incompatibilities will be bugs) > is having DecodedGenerator fully decode headers and emit full unicode, > rather than the ASCII-only unicode that Generator emits. Can you think > of any problem that that would cause? A quick grep indicates your own > code does not use that generator (possibly because currently it does not > do that decoding). I could, of course, only enable header decoding if > a flag is passed requesting it, and as I write this I realize that that > is indeed what I should do. Even though I haven't been able to think of a > case where DecodedGenerator producing non-ASCII unicode would be an issue, > that doesn't mean there isn't one :) > > > To put that more strongly, the Python user base is much larger than > > this list's readership. If I'm using 3.1 email, so are many others. > > People will accept the 3.X world you make up to a point, but it's > > impossible to code to a moving target, much less base a product on > > it. At some point, they'll simply stop trying to keep up; in fact, > > some already have. > > > > Fixes are a Good Thing, of course, and this particular change's scope > > remains to be seen; but to channel most of the users I meet out there > > in the real world today: Enough with the 3.X changes already, eh? > > Now that Python3 is out, the backward compatibility policy for it is > the same as it always was for Python2. Only the transition from 2 > to 3 broke backward compatibility in a significant way. From here > on, we are as conservative as we always have been at making backward > incompatible changes (that is, we don't do it intentionally without > a good reason and a deprecation cycle, and if we do it unintentionally > it is a regression and treated as such). > > -- > R. David Murray www.bitdance.com > From lutz at rmi.net Fri Oct 8 17:51:45 2010 From: lutz at rmi.net (lutz at rmi.net) Date: Fri, 08 Oct 2010 15:51:45 -0000 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes Message-ID: <4fybbku8mj1ne7ts08102010115107@SMTP> stephen at xemacs.org wrote in the full message below: > If having 1 *and* 2 is so important to particular users, but they come > into conflict because of proposed changes in Python, then they're > going to have to give up 3, come here, and articulate their needs. But I _did_ come here and articulate my needs, and received this antagonistic response for my efforts. If you really value user input, you may want to explore the nature of your reaction to it. Trust me: criticism goes with the territory any time your actions impact a large group of people. This seems inherent here. Frankly, your view of the roles of developers and users seems so upside down to me that I doubt anything I could say here would matter. You're more than welcome to ignore an interjection of reality and adopt a closed group mindset, of course, but you do so at the peril of the system you're working on. For my part, one week from now I'll be standing up again in front of a group of 20 Python beginners, and basically apologizing for both the present and ongoing 3.X changes they must conform to in the near future. Python may not be Perl 6 yet, but its image is already tarnished in the real world where people make technology choices, due to its rapid pace of change. It's a genuine problem. In the end, I suppose I'm just one of those lazy end users you mentioned who are too busy to spend 24/7 hanging out on this list in order to head off changes that will break their code. (Yes, sarcasm intended.) --Mark Lutz (http://learning-python.com, http://rmi.net/~lutz) > -----Original Message----- > From: "Stephen J. Turnbull" > To: lutz at rmi.net > Subject: Re: [Python-Dev] Patch making the current email package > (mostly) support bytes > Date: Fri, 08 Oct 2010 14:33:22 +0900 > > lutz at rmi.net writes: > > > To put that more strongly, the Python user base is much larger than > > this list's readership. > > Agreed. Nevertheless, this is the channel (not "channel") that the > developers listen on, and substantial effort is made to let Python > users know that. I think they do know it, too. > > > If I'm using 3.1 email, so are many others. > > That's not obvious. 3.1 email is unusable for several applications. > In fact, for human factors reasons (humans are very likely to > communicate with other humans who use the same encodings, and to > accept occasional glitches they must deal with manually), MUAs are > likely to port relatively easily as "good enough" software. But I > doubt very much that folks writing MTAs or spam filters that must run > unattended, often in long-lived, very active processes, are producing > production versions using Python 3 email yet. > > > People will accept the 3.X world you make up to a point, but it's > > impossible to code to a moving target, much less base a product on > > it. > > "Impossible is nothing." It's a decision that each individual > developer makes for herself. I haven't heard Mailman devs complain > about the impossibility of dealing with the proposed changes, for > example. Quite the reverse, in fact. > > > At some point, they'll simply stop trying to keep up; in fact, > > some already have. > > Predictable and predicted. Where's the balance? I don't know, but > "channeling" the users is not a lot of help. There are three worthy > goals here: > > 1. Taking advantage of improvements in to-be-released Pythons. > 2. Not changing one's own working code. > 3. Not participating in python-dev/email-sig. > > Take any two; one can't have all three. > > More specifically, it's interesting that most of the users you talk to > care enough to actually say they don't want more incompatible changes. > But what are we supposed to take from that? Some fixes have to be > incompatible; do the users want the fix or the compatibility? You > waffle (as a good representative often must): > > > Fixes are a Good Thing, of course, and this particular change's scope > > remains to be seen; but to channel most of the users I meet out there > > in the real world today: Enough with the 3.X changes already, eh? > > But that's also a decision each developer *can* make for himself. > Python does not withdraw products, or even withdraw support, just > because the core developers release something they consider better. > > If having 1 *and* 2 is so important to particular users, but they come > into conflict because of proposed changes in Python, then they're > going to have to give up 3, come here, and articulate their needs. As > you are doing -- but to have real influence, you're going to have to > do the review of David's patch that he requests. > > I really don't see how the process can work any other way. > From a.badger at gmail.com Fri Oct 8 17:53:42 2010 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 8 Oct 2010 11:53:42 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008171244.75fc2afa@pitrou.net> References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008171244.75fc2afa@pitrou.net> Message-ID: <20101008155342.GD10153@unaka.lan> On Fri, Oct 08, 2010 at 05:12:44PM +0200, Antoine Pitrou wrote: > On Fri, 8 Oct 2010 11:04:35 -0400 > Toshio Kuratomi wrote: > > > > In the larger universe of programs, it might make for more intuitive > > remembering of the command to use a prefix (either py or python) though. > > > > python-setup is a lot like python setup.py > > pysetup is shorter > > pyegg is even shorter :-) > > Wouldn't "quiche" be a better alternative for "pyegg"? > I won't bikeshed as long as we stay away from conflicting names. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From status at bugs.python.org Fri Oct 8 18:13:38 2010 From: status at bugs.python.org (Python tracker) Date: Fri, 8 Oct 2010 18:13:38 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20101008161338.EC23A780D9@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2010-10-01 - 2010-10-08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues stats: open 2557 (+37) closed 19276 (+30) total 21833 (+44) Open issues with patches: 1071 Issues opened (37) ================== #10008: Two links point to same place http://bugs.python.org/issue10008 opened by ocean-city #10009: Automated MSI installation does not work http://bugs.python.org/issue10009 opened by amaury.forgeotdarc #10011: `except` doesn't use `isinstance` http://bugs.python.org/issue10011 opened by cool-RR #10013: fix `./libpython2.6.so: undefined reference to `_PyParser_Gram http://bugs.python.org/issue10013 opened by PaulePanter #10014: sys.path[0] is incorrect if PYTHONFSENCODING is used http://bugs.python.org/issue10014 opened by haypo #10015: Creating a multiproccess.pool.ThreadPool from a child thread b http://bugs.python.org/issue10015 opened by Michael.Olson #10016: shutil.copyfile -- allow sparse copying http://bugs.python.org/issue10016 opened by karaken12 #10017: pprint.pprint raises TypeError on dictionaries with user-defin http://bugs.python.org/issue10017 opened by arno #10018: IDLE not loading in xp pro due to tcl issue http://bugs.python.org/issue10018 opened by Grant.Andrew #10019: json.dumps with indent = 0 not adding newlines http://bugs.python.org/issue10019 opened by dpjanes #10020: docs for sqlite3 describe functions not available without reco http://bugs.python.org/issue10020 opened by doughellmann #10021: Format parser is too permissive http://bugs.python.org/issue10021 opened by belopolsky #10022: Emit more information in decoded SSL certificates http://bugs.python.org/issue10022 opened by pitrou #10023: test_lib2to3 leaks under 3.1 http://bugs.python.org/issue10023 opened by pitrou #10024: Outdated advice in C-API tutorial? http://bugs.python.org/issue10024 opened by pitrou #10025: random.seed not initialized as advertised http://bugs.python.org/issue10025 opened by goddard #10027: os.lstat/os.stat don't set st_nlink on Windows http://bugs.python.org/issue10027 opened by brian.curtin #10028: test_concurrent_futures fails on Windows Server 2003 http://bugs.python.org/issue10028 opened by brian.curtin #10029: "Equivalent to" code for zip is wrong in Python 3 http://bugs.python.org/issue10029 opened by max #10030: Patch for zip decryption speedup http://bugs.python.org/issue10030 opened by shashank #10031: Withdraw anti-recommendation of relative imports from document http://bugs.python.org/issue10031 opened by dsdale24 #10036: compiler warnings for various modules on Ubuntu x86 http://bugs.python.org/issue10036 opened by jfinkels #10037: multiprocessing.pool processes started by worker handler stops http://bugs.python.org/issue10037 opened by asksol #10038: Returntype of json.loads() on strings http://bugs.python.org/issue10038 opened by llnik #10039: python ??.py fails with UnicodeEncodeError if PYTHONFSENCODING http://bugs.python.org/issue10039 opened by haypo #10040: GZipFile failure on large files http://bugs.python.org/issue10040 opened by Robert.Rohde #10041: socket.makefile(mode = 'r').readline() silently removes carria http://bugs.python.org/issue10041 opened by kaizhu #10042: total_ordering stack overflow http://bugs.python.org/issue10042 opened by francescor #10043: UnboundLocalError with local variable set by setattr, caused b http://bugs.python.org/issue10043 opened by ssc #10044: small int optimization http://bugs.python.org/issue10044 opened by pitrou #10045: poor cStringIO.StringO seek performance http://bugs.python.org/issue10045 opened by boogenhagn #10046: Correction to atexit documentation http://bugs.python.org/issue10046 opened by Jason.Baker #10047: python-2.6.6 coredump running newspipe http://bugs.python.org/issue10047 opened by wiz #10048: urllib.request documentation confusing http://bugs.python.org/issue10048 opened by pitrou #10049: Add a "no-op" (null) context manager to contextlib http://bugs.python.org/issue10049 opened by hniksic #10050: urllib.request still has old 2.x urllib primitives http://bugs.python.org/issue10050 opened by pitrou #10051: distutils fail to install unicode-encoded files with POSIX loc http://bugs.python.org/issue10051 opened by mgorny Most recent 15 issues with no replies (15) ========================================== #10051: distutils fail to install unicode-encoded files with POSIX loc http://bugs.python.org/issue10051 #10050: urllib.request still has old 2.x urllib primitives http://bugs.python.org/issue10050 #10048: urllib.request documentation confusing http://bugs.python.org/issue10048 #10039: python ??.py fails with UnicodeEncodeError if PYTHONFSENCODING http://bugs.python.org/issue10039 #10037: multiprocessing.pool processes started by worker handler stops http://bugs.python.org/issue10037 #10036: compiler warnings for various modules on Ubuntu x86 http://bugs.python.org/issue10036 #10025: random.seed not initialized as advertised http://bugs.python.org/issue10025 #10023: test_lib2to3 leaks under 3.1 http://bugs.python.org/issue10023 #10022: Emit more information in decoded SSL certificates http://bugs.python.org/issue10022 #10018: IDLE not loading in xp pro due to tcl issue http://bugs.python.org/issue10018 #10017: pprint.pprint raises TypeError on dictionaries with user-defin http://bugs.python.org/issue10017 #10009: Automated MSI installation does not work http://bugs.python.org/issue10009 #10008: Two links point to same place http://bugs.python.org/issue10008 #9998: find_library should search LD_LIBRARY_PATH on linux http://bugs.python.org/issue9998 #9995: "setup.py register sdist upload" requires pass to be saved http://bugs.python.org/issue9995 Most recent 15 issues waiting for review (15) ============================================= #10049: Add a "no-op" (null) context manager to contextlib http://bugs.python.org/issue10049 #10045: poor cStringIO.StringO seek performance http://bugs.python.org/issue10045 #10044: small int optimization http://bugs.python.org/issue10044 #10041: socket.makefile(mode = 'r').readline() silently removes carria http://bugs.python.org/issue10041 #10039: python ??.py fails with UnicodeEncodeError if PYTHONFSENCODING http://bugs.python.org/issue10039 #10037: multiprocessing.pool processes started by worker handler stops http://bugs.python.org/issue10037 #10030: Patch for zip decryption speedup http://bugs.python.org/issue10030 #10028: test_concurrent_futures fails on Windows Server 2003 http://bugs.python.org/issue10028 #10027: os.lstat/os.stat don't set st_nlink on Windows http://bugs.python.org/issue10027 #10019: json.dumps with indent = 0 not adding newlines http://bugs.python.org/issue10019 #10016: shutil.copyfile -- allow sparse copying http://bugs.python.org/issue10016 #10014: sys.path[0] is incorrect if PYTHONFSENCODING is used http://bugs.python.org/issue10014 #9998: find_library should search LD_LIBRARY_PATH on linux http://bugs.python.org/issue9998 #9993: shutil.move fails on symlink source http://bugs.python.org/issue9993 #9992: Command line arguments are not correctly decodedif locale and http://bugs.python.org/issue9992 Top 10 most discussed issues (10) ================================= #10044: small int optimization http://bugs.python.org/issue10044 22 msgs #4661: email.parser: impossible to read messages encoded in a differe http://bugs.python.org/issue4661 14 msgs #10049: Add a "no-op" (null) context manager to contextlib http://bugs.python.org/issue10049 12 msgs #9800: Fast path for small int-indexing of lists and tuples http://bugs.python.org/issue9800 9 msgs #10021: Format parser is too permissive http://bugs.python.org/issue10021 9 msgs #6302: email.header.decode_header data types are inconsistent and inc http://bugs.python.org/issue6302 8 msgs #9873: urllib.parse: Allow bytes in some APIs that use string literal http://bugs.python.org/issue9873 7 msgs #10029: "Equivalent to" code for zip is wrong in Python 3 http://bugs.python.org/issue10029 7 msgs #5088: optparse: inconsistent default value for append actions http://bugs.python.org/issue5088 6 msgs #9017: doctest option flag to enable/disable some chunk of doctests? http://bugs.python.org/issue9017 6 msgs Issues closed (30) ================== #2982: more tests for pyexpat http://bugs.python.org/issue2982 closed by amaury.forgeotdarc #4487: Add utf8 alias for email charsets http://bugs.python.org/issue4487 closed by r.david.murray #7789: Issue using datetime with format() http://bugs.python.org/issue7789 closed by belopolsky #8017: c_char_p.value does not return a bytes object in Windows. http://bugs.python.org/issue8017 closed by ocean-city #8190: Add a PyPI XML-RPC client module http://bugs.python.org/issue8190 closed by eric.araujo #8584: test_multiprocessing skips some tests http://bugs.python.org/issue8584 closed by brian.curtin #8670: c_types.c_wchar should not assume that sizeof(wchar_t) == size http://bugs.python.org/issue8670 closed by haypo #8980: distutils.tests.test_register.RegisterTestCase.test_strict fai http://bugs.python.org/issue8980 closed by tarek #9060: Python/dup2.c doesn't compile on (at least) newlib http://bugs.python.org/issue9060 closed by amaury.forgeotdarc #9065: tarfile: default root:root ownership is incorrect. http://bugs.python.org/issue9065 closed by lars.gustaebel #9272: CGIHTTPServer poisons os.environ http://bugs.python.org/issue9272 closed by orsenthil #9533: metaclass can't derive from ABC http://bugs.python.org/issue9533 closed by benjamin.peterson #9759: GzipFile object should raise ValueError on .read() after .clos http://bugs.python.org/issue9759 closed by pitrou #9903: test_concurrent_futures writes on stderr http://bugs.python.org/issue9903 closed by bquinlan #9936: trace misreports "missing" lines http://bugs.python.org/issue9936 closed by belopolsky #9978: Better wait for slow machine in test_os (_kill_with_event) http://bugs.python.org/issue9978 closed by ocean-city #9989: ctypes bitfield problem http://bugs.python.org/issue9989 closed by ocean-city #9996: binascii should convert unicode strings http://bugs.python.org/issue9996 closed by r.david.murray #10003: signal.SIGBREAK regression on windows http://bugs.python.org/issue10003 closed by brian.curtin #10006: non-Pythonic fate of __abstractmethods__ http://bugs.python.org/issue10006 closed by benjamin.peterson #10010: ".. index::" position http://bugs.python.org/issue10010 closed by orsenthil #10012: httplib shadows builtin, assumes strings http://bugs.python.org/issue10012 closed by orsenthil #10026: xml.dom.pulldom strange behavior http://bugs.python.org/issue10026 closed by georg.brandl #10032: os.setuid and os.setgid have unexpected influence on serial mo http://bugs.python.org/issue10032 closed by ned.deily #10033: can't run unittest because of issubclass http://bugs.python.org/issue10033 closed by ned.deily #10034: Readline doc issue http://bugs.python.org/issue10034 closed by georg.brandl #10035: sgmllib fail to parse html containing http://bugs.python.org/issue10035 closed by georg.brandl #1050268: rfc822.parseaddr is broken, breaks sendmail call in smtplib http://bugs.python.org/issue1050268 closed by r.david.murray #845560: imaplib: traceback from _checkquote with empty string http://bugs.python.org/issue845560 closed by r.david.murray #1210680: Split email headers near a space http://bugs.python.org/issue1210680 closed by r.david.murray From merwok at netwok.org Fri Oct 8 17:45:30 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 08 Oct 2010 17:45:30 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008153121.GE20222@snowy.squish.net> References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008153121.GE20222@snowy.squish.net> Message-ID: <4CAF3C9A.3020402@netwok.org> Le 08/10/2010 17:31, Jon Ribbens a ?crit : > I'd just like to say "pypackage" again. In the Python world, a package is a directory with an __init__.py file. Distutils and distutils2 try to avoid confusion and call the other things ?distributions?. Of course, everyone outside of the Python world thinks a distribution is an operating system that has a free kernel and a package manager. Maybe we should trade ?distributions? for ?bundles?. Regards From stephen at xemacs.org Fri Oct 8 18:06:29 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 09 Oct 2010 01:06:29 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101007143748.7c548261@mission> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007143748.7c548261@mission> Message-ID: <87sk0g7jp6.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > On Oct 07, 2010, at 04:40 AM, Stephen J. Turnbull wrote: > I'm fairly certain that most of the modern causes of [Unicode > errors in Mailman] are post-parse modifications of the message. > IOW, in Mailman's architecture, we try to parse the raw data into a > Message object tree very early in the pipeline, and then a pickled > version of that gets passed between the queue runners. > > Where we've gotten into trouble before has been things like adding > the Subject prefixes and such. Not to mention those wonderful unremovable addresses containing TAB etc. But I'm pretty sure I've seen reports at least in 2.1.9, and probably more recently than that, where there was 8-bit content in a header of the incoming message and Mailman blew up on that. This is stuff that should have been shunted explicitly, but instead managed to get out of the parser and then blow up. I don't think the errors I'm thinking about were due to Mailman manipulations, but rather insufficient paranoia in handling incoming hazmat. > That seems like application logic that the email package can't > really get involved with, and indeed Mailman has built up a raft of > defense for failures of this kind. But adding Subject prefixes and the like shouldn't be a problem as long is the internal representation of each message object (bytes vs str) is fixed and the representation is opaque, so that the module can do appropriate conversions when necessary. The problem that you face in Python 2 is that that separation is not properly made, and the same values in the message object can often serve as text and as wire format, and it's hard to tell which is which. The Unicode handling is tacked on as an afterthought. That mess is entirely unnecessary in Python 3. Text and wire format can be easily distinguished with three different representations of email: Unicode for the conceptual RFC 822 layer (of course this is an extension, because RFC 822 itself is strictly limited to the ASCII subset), bytes for wire format, and Message objects for modern structured mail (including MIME, etc). *If* email6 is reengineered with that kind of structure, then you should be able to dispense with almost all of the raft of defense, because the email module will give you well-behaved Message objects, whose text components (including the header) are well-behaved character strings that mix seamlessly with other character strings. Maybe even in email5 .... From ziade.tarek at gmail.com Fri Oct 8 18:22:22 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 18:22:22 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> Message-ID: On Fri, Oct 8, 2010 at 4:54 PM, Georg Brandl wrote: > Am 08.10.2010 16:26, schrieb Barry Warsaw: > >>>- query pypi >>>- browse what's installed >>>- install/remove projects >>>- create releases and upload them >>> >>>pkg_manager ? >> >> No underscores, please. :) >> >> Actually, a decent wrapper script could just be called 'setup'. ?My >> command-not-found on Ubuntu doesn't find a collision, or even close >> similarities. > > No generic name, *please*. ?easy_install was bad enough, no need to repeat > that mistake. ?"egg" would be better, but weren't we phasing out the egg > format? -1 on anything containing the word 'egg'. It'll add confusion with egg-the-format > > Georg > > > -- > Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. > Four shall be the number of spaces thou shalt indent, and the number of thy > indenting shall be four. Eight shalt thou not indent, nor either indent thou > two, excepting that thou then proceed to four. Tabs are right out. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com > -- Tarek Ziad? | http://ziade.org From ziade.tarek at gmail.com Fri Oct 8 18:25:59 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 18:25:59 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008155342.GD10153@unaka.lan> References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008171244.75fc2afa@pitrou.net> <20101008155342.GD10153@unaka.lan> Message-ID: On Fri, Oct 8, 2010 at 5:53 PM, Toshio Kuratomi wrote: > On Fri, Oct 08, 2010 at 05:12:44PM +0200, Antoine Pitrou wrote: ... >> > pysetup is shorter Let's use pysetup ! ... > I won't bikeshed as long as we stay away from conflicting names. +1. So. Let's add pysetup in distutils2, that will be installed as a classical script. Once we move distutils2 back in the stdlib, it will be provided in Python's bin dir, so people will have the same "pysetup" name everywhere, Tarek -- Tarek Ziad? | http://ziade.org From alexis at notmyidea.org Fri Oct 8 18:43:33 2010 From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=) Date: Fri, 08 Oct 2010 17:43:33 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008171244.75fc2afa@pitrou.net> <20101008155342.GD10153@unaka.lan> Message-ID: <4CAF4A35.905@notmyidea.org> Le 10/08/2010 05:25 PM, Tarek Ziad? a ?crit : > On Fri, Oct 8, 2010 at 5:53 PM, Toshio Kuratomi wrote: >> On Fri, Oct 08, 2010 at 05:12:44PM +0200, Antoine Pitrou wrote: > ... >>>> pysetup is shorter > > Let's use pysetup ! +1 on pysetup. Reusing the well known "setup" and adding py as a prefix will avoid any conflict. > > ... >> I won't bikeshed as long as we stay away from conflicting names. > > +1. > > So. Let's add pysetup in distutils2, that will be installed as a > classical script. Once we move distutils2 back in the stdlib, it will > be provided in Python's bin dir, so people will have the same > "pysetup" name everywhere, > > > Tarek > From alexis at notmyidea.org Fri Oct 8 18:47:17 2010 From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=) Date: Fri, 08 Oct 2010 17:47:17 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008153121.GE20222@snowy.squish.net> References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008153121.GE20222@snowy.squish.net> Message-ID: <4CAF4B15.9030204@notmyidea.org> Le 10/08/2010 04:31 PM, Jon Ribbens a ?crit : > On Fri, Oct 08, 2010 at 11:04:35AM -0400, Toshio Kuratomi wrote: >> In the larger universe of programs, it might make for more intuitive >> remembering of the command to use a prefix (either py or python) though. >> >> python-setup is a lot like python setup.py >> pysetup is shorter >> pyegg is even shorter :-) > > I'd just like to say "pypackage" again. > > pypackage install thingy > pypackage remove thingy > pypackage update thingy Btw, we are not talking about "packages", as packages are already used in the python concepts (See http://guide.python-distribute.org/glossary.html) So, it would be better to have something like "pydist", but seems to be less obvious than "pysetup". Alexis > > etc > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/alexis%40notmyidea.org From rdmurray at bitdance.com Fri Oct 8 18:57:53 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 08 Oct 2010 12:57:53 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <4fybbku8mj1ne7ts08102010115107@SMTP> References: <4fybbku8mj1ne7ts08102010115107@SMTP> Message-ID: <20101008165753.3D9811FB3D0@kimball.webabinitio.net> On Fri, 08 Oct 2010 15:51:45 -0000, lutz at rmi.net wrote: > For my part, one week from now I'll be standing up again in front > of a group of 20 Python beginners, and basically apologizing for > both the present and ongoing 3.X changes they must conform to in > the near future. Python may not be Perl 6 yet, but its image is > already tarnished in the real world where people make technology > choices, due to its rapid pace of change. It's a genuine problem. > > In the end, I suppose I'm just one of those lazy end users you > mentioned who are too busy to spend 24/7 hanging out on this > list in order to head off changes that will break their code. > (Yes, sarcasm intended.) What would be helpful would be to know what changes it is that we have made between 3.1 and 3.2 that are raising backward compatibility concerns. What are we doing that is perceived as "ongoing 3.X changes"? Generalities will not help, only by looking at specifics can we re-evaluate our actions. In a private message you mentioned "the bytecode file model change", by which I presume you mean PEP 3147. Our view is that this is a backward compatible change: any Python program that was working should continue to work. Barry's original idea was that the new behavior would only be turned on by a flag, but Guido (and others) wanted it to be the default because in his view it is a superior arrangement for normal use. Perhaps we did not fully consider the effect on third party tools (and, as you point out, documentation) that expects .pyc files along side the .py files. Yet this change is no where near the level of change that makes typical Python programs fail. We feel like it is a worthwhile trade-off (and Debian and Ubuntu at least may well backport it to earlier Python versions). But apparently you disagree. So, engage us in dialog about it, please. And *please* mention any other specific changes you think are disruptive between 3.1 and 3.2. We need to know about them, preferably *before* we release 3.2 beta (currently targeted for the end of this month). Because I assure you that it is not our policy to be changing things any more rapidly than we did between python 2.x versions[*]. If you feel like you are apologizing to your groups of beginners, it would be wonderful if you could act as their advocate here. Obviously the issues directly affect you, so hopefully it is worth your time to engage us on this topic. And thank you for the messages you have sent. I know they have made me even more careful than I was already trying to be. -- R. David Murray www.bitdance.com [*] There may be a few exceptions to this where the 3.x library code fails to work in real-world applications, so that a more radical change is made but is, in reality, a bug fix. But even there we try to be conservative. From fdrake at acm.org Fri Oct 8 19:01:40 2010 From: fdrake at acm.org (Fred Drake) Date: Fri, 8 Oct 2010 13:01:40 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: On Fri, Oct 8, 2010 at 9:22 AM, Tarek Ziad? wrote: > pkg_manager ? 1. Underscores are evil. Don't do that. 2. Mixed shortened + written-out names are just nasty. > Mmm.. setup.py is gone in D2, and setup.py will be the marker of d1. Did we finally decide it could be done without setup.py entirely, in all cases? I guess I've been busy elsewhere lately. > Some project might want to provide both setups for backward > compatibility: > > - a setup.py (d1) > - a setup,cfg (d2 and optionally some d1 options) If a project requites setup.py for any reason, it can include the compatibility it needs there, even if there is sometimes a need for d2 to use a setup.py: try: import distutils2 except ImportError: import distutils.core distutils.core.setup(...) else: distutils2.core.setup() Anyway, the pysetup name offered in tis thread works for me as well. ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From me+python at ixokai.io Fri Oct 8 19:02:59 2010 From: me+python at ixokai.io (Stephen Hansen) Date: Fri, 8 Oct 2010 10:02:59 -0700 Subject: [Python-Dev] Build failure in test_cmd_line on OSX-x86 Message-ID: I'm sure this has to be my configuration somehow, but I'm getting a build failure that I don't quite know how to debug, because I can't reproduce it when I run the test manually. Any advice would be appreciated-- I'm a buildslave newbie :-) I'm referring to http://www.python.org/dev/buildbot/builders/x86%20Snow%20Leopard%203.x/builds/13/steps/test/logs/stdio Now, when I saw this the first thing I assumed I should do is to try to reproduce it, so I did: Top-2:~ pythonbuildbot$ cp -R buildarea/3.x.hansen-osx-x86/ ~/test Top-2:~ pythonbuildbot$ cd ~/test/build/ Top-2:build pythonbuildbot$ ./configure --with-pydebug --with-computed-gotos [snip] Top-2:build pythonbuildbot$ make all [snip] I made sure to go check out the stdio for configure/compile so the compiling I'd do would be with the same options as the buildslave did. Then: Top-2:build pythonbuildbot$ ./python.exe -m test.regrtest -uall test_cmd_line.py [1/1] test_cmd_line 1 test OK. [84022 refs] Doh. So, next I try running the whole test suite: Top-2:build pythonbuildbot$ ./python.exe -m test.regrtest -uall [snip] [ 44/348] test_cmd_line And it passes too: but I notice its run way earlier in the test process then it did on the buildbot, and I remember reading awhile ago about a test failure that happened only when a certain test ran in a certain order. So, I go garb the randseed from the buildbot run and use it: Top-2:build pythonbuildbot$ ./python.exe -m test.regrtest -uall -r --randseed=9634655 == CPython 3.2a2+ (py3k:85321, Oct 8 2010, 08:54:05) [GCC 4.2.1 (Apple Inc. build 5664)] == Darwin-10.4.0-i386-64bit little-endian == /Users/pythonbuildbot/test/build/build/test_python_86644 Using random seed 9634655 [snip] And long story short, it gets to 201 and runs test_cmd_line in the same order as the buildbot did, and it succeeds too, and I curse the gods of the netherworld, and am stumped with how to proceed. Two separate buildbot runs and this same failure happened, yet for me, no. Or I'm doing something differently then the buildbot is, and I can't see what. --Stephen -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Fri Oct 8 19:07:26 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 19:07:26 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: On Fri, Oct 8, 2010 at 7:01 PM, Fred Drake wrote: > On Fri, Oct 8, 2010 at 9:22 AM, Tarek Ziad? wrote: >> pkg_manager ? > > 1. Underscores are evil. ?Don't do that. > > 2. Mixed shortened + written-out names are just nasty. > >> Mmm.. setup.py is gone in D2, and setup.py will be the marker of d1. > > Did we finally decide it could be done without setup.py entirely, in > all cases? ?I guess I've been busy elsewhere lately. You mean like, having a distutils1 project without setup.py ? > >> Some project might want to provide both setups for backward >> compatibility: >> >> - a setup.py (d1) >> - a setup,cfg (d2 and optionally some d1 options) > > If a project requites setup.py for any reason, it can include the > compatibility it needs there, even if there is sometimes a need for d2 > to use a setup.py: > > > ? ?try: > ? ? ? ?import distutils2 > ? ?except ImportError: > ? ? ? ?import distutils.core > ? ? ? ?distutils.core.setup(...) > ? ?else: > ? ? ? ?distutils2.core.setup() At the last sprint, we ended up thinking that it would be better to have a setup.py that work only with distutils, and let people using setup.cfg for d2 (and if needed, hooks) This is way simpler for people because we can tell them: leave your setup.py as it is so you are compatible with installers like pip and easy_install for the time being, and add stuff in your .cfg. That way, they don't have to throw distutils2 code in the mix in setup.py, and be confused with all the subtle differences. And they'll follow that way the "no-setup.py" philosophy Tarek From rdmurray at bitdance.com Fri Oct 8 19:09:12 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 08 Oct 2010 13:09:12 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <87sk0g7jp6.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007143748.7c548261@mission> <87sk0g7jp6.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101008170912.28958200D48@kimball.webabinitio.net> On Sat, 09 Oct 2010 01:06:29 +0900, "Stephen J. Turnbull" wrote: > That mess is entirely unnecessary in Python 3. Text and wire format > can be easily distinguished with three different representations of > email: Unicode for the conceptual RFC 822 layer (of course this is an > extension, because RFC 822 itself is strictly limited to the ASCII > subset), bytes for wire format, and Message objects for modern > structured mail (including MIME, etc). > > *If* email6 is reengineered with that kind of structure, then you > should be able to dispense with almost all of the raft of defense, > because the email module will give you well-behaved Message objects, > whose text components (including the header) are well-behaved > character strings that mix seamlessly with other character strings. That engineering is pretty much what we are looking at, although in practice I think you have to hang wire-format and text-format bits off of appropriate places in the model in order to keep everything properly coordinated. > Maybe even in email5 .... I suspect that's pushing it. Patches happily accepted, though :) -- R. David Murray www.bitdance.com From solipsis at pitrou.net Fri Oct 8 19:28:52 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 8 Oct 2010 19:28:52 +0200 Subject: [Python-Dev] Build failure in test_cmd_line on OSX-x86 References: Message-ID: <20101008192852.6d57e6ef@pitrou.net> On Fri, 8 Oct 2010 10:02:59 -0700 Stephen Hansen wrote: > > And long story short, it gets to 201 and runs test_cmd_line in the same > order as the buildbot did, and it succeeds too, and I curse the gods of the > netherworld, and am stumped with how to proceed. Two separate buildbot runs > and this same failure happened, yet for me, no. Or I'm doing something > differently then the buildbot is, and I can't see what. The buildbot user probably has different locale settings. I can simulate the failure with: $ PYTHONFSENCODING=latin1 ./python -m test.regrtest test_cmd_line [1/1] test_cmd_line test test_cmd_line failed -- Traceback (most recent call last): File "/home/antoine/py3k/__svn__/Lib/test/test_cmd_line.py", line 127, in test_run_code 0) AssertionError: 1 != 0 You should therefore see what the locale settings of the buildbot are (the LANG and LC_* environment variables). Of course, the test is also buggy so you should open an issue on the tracker. (and the fact that the test doesn't print the actual error message of the spawned interpreter is unhelpful) Regards Antoine. From rdmurray at bitdance.com Fri Oct 8 19:35:26 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 08 Oct 2010 13:35:26 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: References: Message-ID: <20101008173526.6F5801F9ABC@kimball.webabinitio.net> On Fri, 08 Oct 2010 15:44:45 -0000, lutz at rmi.net wrote: > Thanks for both your reply and work, David. I'm going to have > to test my email clients under the 3.2 patch when it gels. It's > good to hear that email5 API support remains a goal. I just landed the patch (though without the MIME encoding of unknown header bytes or the 'yes-I-really-want-the-escaped-bytes' flags that Stephen and I have been discussing. So it will be present in alpha3. I would greatly appreciate your testing it and making sure it doesn't break any of your code. > I don't mean to single out this change unfairly, of course. My > real concern is not as much with the specific technical aspects > of this proposal, as with the generally low priority that backward > compatibility sometimes receives on this list. The bytecode file I don't perceive that lack of priority myself. Certainly I don't see a lack of priority on backward compatibility in the bug tracker, quite the reverse[*]. As I said in my public email, specific examples would be most helpful. > model change in 3.2 comes to mind as another example; sound as it > may be, I'm not sure this list has any idea how many users, systems, > or docs may be impacted by this. Though not always true, the work > here does sometimes appear to be conducted in a vacuum. Well, we can only react to the input we find out about. Developers *do* read blogs and such about what's going on in the wider community and bring that info back to python-dev, but as is inherent with projects structured as volunteer efforts, what we get is only what someone decides to put in time on. Specific suggestions on how to improve the feedback loop are always welcome; volunteer efforts to improve our fundamental procedures are just as or perhaps more valuable than volunteer code writing (though they probably involve even more politicing effort :). > Ultimately, development in the open source world is driven by the > very few with time to show up, rather than by the very many who > depend on it. This can unfortunately lead to the perception > of thrashing by end users. Some even come to see the net effect > as not that much different from closed models. I have no solution Well, the Python community takes it as a principle to avoid thrashing. So if you see examples where we are failing in that goal, call us on it (with specifics). > to offer, except to underscore again that changes made here affect > very many people who are too busy using Python to participate here. > Especially given the still tentative state of 3.X, stability matters. We do try to remain aware of that. When we fail, someone needs to let us know. -- R. David Murray www.bitdance.com [*] I'm currently aware of one exception to this, the nttplib module. It was pretty much unusable as it stood (I tried, as did Antoine; it had no unit tests so massive breakage is not that surprising), so we broke backward compatibility with 3.1 in order to fix that. From rdmurray at bitdance.com Fri Oct 8 19:39:55 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 08 Oct 2010 13:39:55 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <87tykw7mza.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007004608.ED5EA22A3E5@kimball.webabinitio.net> <877hhu8rvf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007151518.AEFC11BF4F3@kimball.webabinitio.net> <8739sh8id9.fsf@uwakimon.sk.tsukuba.ac.jp> <20101008095738.6e40eb7f@mission> <87tykw7mza.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101008173955.A545F1FC971@kimball.webabinitio.net> On Fri, 08 Oct 2010 23:55:37 +0900, "Stephen J. Turnbull" wrote: > I should think you *want* addresses and suchlike structured headers > (Content-Type with several RFC 2231 parameters, anyone?) to line up > nicely, too. So generic folding algorithms are really only applicable > to unstructured text fields like Subject and Summary anyway. > > You can call that "sucky" if you like, I prefer to call it "tasteful." No, what's sucky is that email4/5 doesn't support that. It only folds headers as unstructured blobs, with a nod in the direction of structure by breaking lines at "obvious" places like ';'s. (Which line breaking algorithm is the subject of at least one bug report....) I'd like to fix that in email6 by adding full support for structured headers. -- R. David Murray www.bitdance.com From rdmurray at bitdance.com Fri Oct 8 19:43:28 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 08 Oct 2010 13:43:28 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <8739sh8id9.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007004608.ED5EA22A3E5@kimball.webabinitio.net> <877hhu8rvf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007151518.AEFC11BF4F3@kimball.webabinitio.net> <8739sh8id9.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101008174328.CBBA2200D48@kimball.webabinitio.net> On Fri, 08 Oct 2010 12:37:38 +0900, "Stephen J. Turnbull" wrote: > *If* you have an 8-bit value of unknown encoding on input, this will > appear in the Header's value as a surrogate. Hm, OK, I see the > problem ... as usual, it's that the only efficient thing to do is > encode using surrogate-escape which loses the information that these > are invalid bytes. Would it really be that bad to add an O(length) > component where you examine the string for surrogates (and too-long > words, for that matter), and chop off those pieces for MIME encoding? Nope, and that's more or less what I think I'm going to do. But I haven't started writing the code yet. > > > > Presumably you are suggesting that email5 be smart enough to turn my > > > > example into properly UTF-8/CTE encoded text. > > > > > > No, in general that's undecidable without asking the originator, > > > although humans can often make a good guess. > > > > I was talking about unicode input, though, where you do know (modulo > > the language differences that unicode hasn't yet sorted out). > > I don't understand why this is difficult. As far as what Unicode has It isn't difficult in principle. It's just difficult in email5. -- R. David Murray www.bitdance.com From eric at trueblade.com Fri Oct 8 16:49:15 2010 From: eric at trueblade.com (Eric Smith) Date: Fri, 08 Oct 2010 10:49:15 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008102636.3ffede1e@mission> References: <20101008102636.3ffede1e@mission> Message-ID: <4CAF2F6B.3030903@trueblade.com> On 10/8/10 10:26 AM, Barry Warsaw wrote: > No underscores, please. :) Indeed! > In any case, these could be a simple shell script wrapping 'python -m setup'. > It could even take a --use-python-version option to select the pythonX.Y it > used, without having to encode the Python version number in the script name. On Windows it can't be a shell script or batch file, but needs to be an executable. setuptools already deals with this. -- Eric. From stephen at xemacs.org Fri Oct 8 19:48:23 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 09 Oct 2010 02:48:23 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101008170912.28958200D48@kimball.webabinitio.net> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007143748.7c548261@mission> <87sk0g7jp6.fsf@uwakimon.sk.tsukuba.ac.jp> <20101008170912.28958200D48@kimball.webabinitio.net> Message-ID: <87pqvk7ezc.fsf@uwakimon.sk.tsukuba.ac.jp> R. David Murray writes: > On Sat, 09 Oct 2010 01:06:29 +0900, "Stephen J. Turnbull" wrote: > > That mess is entirely unnecessary in Python 3. Text and wire format > > can be easily distinguished with three different representations of > > email: Unicode for the conceptual RFC 822 layer (of course this is an > > extension, because RFC 822 itself is strictly limited to the ASCII > > subset), bytes for wire format, and Message objects for modern > > structured mail (including MIME, etc). > That engineering is pretty much what we are looking at, although in > practice I think you have to hang wire-format and text-format bits off > of appropriate places in the model in order to keep everything properly > coordinated. Right. That's where I was going with my comment to Barry about the Received headers. Even if email isn't going to serve clients working with wire format, it needs to deal with those headers. But where I think the headers defined by RFC 822 should be stored as str in email6, I am leaning toward storing Received headers verbatim as bytes (including any RFC 822 folding whitespace) because of the RFC 5321 requirement that they be preserved exactly. From me+python at ixokai.io Fri Oct 8 20:09:41 2010 From: me+python at ixokai.io (Stephen Hansen) Date: Fri, 8 Oct 2010 11:09:41 -0700 Subject: [Python-Dev] Build failure in test_cmd_line on OSX-x86 In-Reply-To: <20101008192852.6d57e6ef@pitrou.net> References: <20101008192852.6d57e6ef@pitrou.net> Message-ID: On Fri, Oct 8, 2010 at 10:28 AM, Antoine Pitrou wrote: > On Fri, 8 Oct 2010 10:02:59 -0700 > Stephen Hansen > wrote: > > > > And long story short, it gets to 201 and runs test_cmd_line in the same > > order as the buildbot did, and it succeeds too, and I curse the gods of > the > > netherworld, and am stumped with how to proceed. Two separate buildbot > runs > > and this same failure happened, yet for me, no. Or I'm doing something > > differently then the buildbot is, and I can't see what. > > The buildbot user probably has different locale settings. I can > simulate the failure with: > I'd find that very surprising: the buildslave is running as the same user I am running the test under, and the LANG is en_US.UTF-8 -- the default. Granted, the slave's running under launchd, and so is launching twisted with the tac directly -- but I can't see any part of that process which would cause the default LANG to change. Interestingly enough, I can't reproduce the failure with: Top-2:build pythonbuildbot$ PYTHONFSENCODING=latin1 ./python.exe -m test.regrtest -uall test_cmd_line.py [1/1] test_cmd_line 1 test OK. [84024 refs] (and just to test--) Top-2:build pythonbuildbot$ PYTHONFSENCODING="utf-8" ./python.exe -m test.regrtest -uall test_cmd_line.py [1/1] test_cmd_line 1 test OK. [84024 refs] But I don't think that environment variable does anything on the Mac; I'm pretty sure the fs encoding is set as utf-8 and mandated as such in the OS. > You should therefore see what the locale settings of the buildbot are > (the LANG and LC_* environment variables). Of course, the test is also > buggy so you should open an issue on the tracker. > I'm just not sure what to say about it or in what way its being buggy yet, so can't open an issue :) > (and the fact that the test doesn't print the actual error message of > the spawned interpreter is unhelpful) > Agreed. --S -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Fri Oct 8 20:20:32 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 8 Oct 2010 14:20:32 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: References: Message-ID: <20101008142032.222bf9dc@mission> On Oct 08, 2010, at 03:44 PM, lutz at rmi.net wrote: >Ultimately, development in the open source world is driven by the >very few with time to show up, rather than by the very many who >depend on it. This can unfortunately lead to the perception >of thrashing by end users. Some even come to see the net effect >as not that much different from closed models. I have no solution >to offer, except to underscore again that changes made here affect >very many people who are too busy using Python to participate here. >Especially given the still tentative state of 3.X, stability matters. I'm reminded of a survey Guido conducted at some long past Python conference. He asked (paraphrasing): raise your hand if you think Python is changing too fast. Lots of hands went up. Then he asked, raise your hand if you have a feature you want to get in the next version. Lots of hands went up. I'm sympathetic to the view that changes in Python can be disruptive to end users. The Python community itself takes this seriously too, as evidenced by the language moratorium[*]. But OTOH, Python cannot stagnate and even fixing things means changing things. The reality too is that Python releases come out approximately every 18 months, and a year and a half can either seem like an excruciatingly long time, or blink of the eye depending on which side of the fence you stand on. Yes, stability matters, but Python 3 is still a new snakeling and I suspect that as the pace of porting picks up, more changes will be necessary. Adding new modules named like distutils2 or unittest2 is less than satisfying but useful for keeping older APIs around. I'm sad to hear that some people think that our development model differs little from closed source development. To me, nothing could be further from the truth. But the adage does go "(s)he who does the work, decides", and this is the forum for those who are doing the work. I think everyone here welcomes advocates for under-represented Python communities, and their concerns should be taken in consideration when changes are discussed. But ultimately, Python must evolve to stay relevant or it will die. This is where competing design trade-offs must be discussed. If not here, by us, then where and by whom? -Barry [*] Mostly instituted to allow alternative implementations to catch up, it does necessarily slow the pace of changes visible to end users. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Fri Oct 8 20:22:28 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 8 Oct 2010 14:22:28 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <87pqvk7ezc.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007143748.7c548261@mission> <87sk0g7jp6.fsf@uwakimon.sk.tsukuba.ac.jp> <20101008170912.28958200D48@kimball.webabinitio.net> <87pqvk7ezc.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101008142228.12424fd2@mission> On Oct 09, 2010, at 02:48 AM, Stephen J. Turnbull wrote: >Right. That's where I was going with my comment to Barry about the >Received headers. Even if email isn't going to serve clients working >with wire format, it needs to deal with those headers. But where I >think the headers defined by RFC 822 should be stored as str in >email6, I am leaning toward storing Received headers verbatim as bytes >(including any RFC 822 folding whitespace) because of the RFC 5321 >requirement that they be preserved exactly. I think then you'd have to have two different APIs. It'll just be too confusing and error prone to return either bytes or strs from header value accessors depending on the header name. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ziade.tarek at gmail.com Fri Oct 8 20:41:03 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 20:41:03 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CAF2F6B.3030903@trueblade.com> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> Message-ID: On Fri, Oct 8, 2010 at 4:49 PM, Eric Smith wrote: ... > > On Windows it can't be a shell script or batch file, but needs to be an > executable. setuptools already deals with this. Why ? The script-wrapping feature Setuptools has is on my radar for d2, but I am not sure why it's an advantage in that case. Tarek -- Tarek Ziad? | http://ziade.org From eric at trueblade.com Fri Oct 8 20:51:01 2010 From: eric at trueblade.com (Eric Smith) Date: Fri, 08 Oct 2010 14:51:01 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> Message-ID: <4CAF6815.6050606@trueblade.com> On 10/8/10 2:41 PM, Tarek Ziad? wrote: > On Fri, Oct 8, 2010 at 4:49 PM, Eric Smith wrote: > ... >> >> On Windows it can't be a shell script or batch file, but needs to be an >> executable. setuptools already deals with this. > > Why ? The script-wrapping feature Setuptools has is on my radar for > d2, but I am not sure why it's an advantage in that case. It can't be a batch file because then it can't be called from other batch files (without using the "call" syntax). I guess there are other options like using WSH (or newer technology), but then you have to pick your target OS carefully. I think the setuptools approach is the preferred solution, although of course it's a hassle. -- Eric. From me+python at ixokai.io Fri Oct 8 21:27:37 2010 From: me+python at ixokai.io (Stephen Hansen) Date: Fri, 8 Oct 2010 12:27:37 -0700 Subject: [Python-Dev] Build failure in test_cmd_line on OSX-x86 In-Reply-To: References: <20101008192852.6d57e6ef@pitrou.net> Message-ID: On Fri, Oct 8, 2010 at 11:09 AM, Stephen Hansen > wrote: > On Fri, Oct 8, 2010 at 10:28 AM, Antoine Pitrou wrote: > >> On Fri, 8 Oct 2010 10:02:59 -0700 >> Stephen Hansen > wrote: >> > >> > And long story short, it gets to 201 and runs test_cmd_line in the same >> > order as the buildbot did, and it succeeds too, and I curse the gods of >> the >> > netherworld, and am stumped with how to proceed. Two separate buildbot >> runs >> > and this same failure happened, yet for me, no. Or I'm doing something >> > differently then the buildbot is, and I can't see what. >> >> The buildbot user probably has different locale settings. I can >> simulate the failure with: >> > > I'd find that very surprising: the buildslave is running as the same user I > am running the test under, and the LANG is en_US.UTF-8 -- the default. > Granted, the slave's running under launchd, and so is launching twisted with > the tac directly -- but I can't see any part of that process which would > cause the default LANG to change. > I edited the launchd config to force LANG = "en_US.UTF-8" and the test suddenly passes, which is good. I have no idea why the LANG would end up different when the app is launched from launchd -- even though it was running as the same user as I was doing the testing against -- but, apparently, it was. But, issue4388 and issue9992 seem to already be in, and I commented on them. Thanks for the help in figuring this out. :) --Stephen/Ixokai -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Fri Oct 8 21:31:07 2010 From: brett at python.org (Brett Cannon) Date: Fri, 8 Oct 2010 12:31:07 -0700 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008171244.75fc2afa@pitrou.net> <20101008155342.GD10153@unaka.lan> Message-ID: On Fri, Oct 8, 2010 at 09:25, Tarek Ziad? wrote: > On Fri, Oct 8, 2010 at 5:53 PM, Toshio Kuratomi wrote: >> On Fri, Oct 08, 2010 at 05:12:44PM +0200, Antoine Pitrou wrote: > ... >>> > pysetup is shorter > > Let's use pysetup ! > > ... >> I won't bikeshed as long as we stay away from conflicting names. > > +1. > > So. Let's add pysetup in distutils2, that will be installed as a > classical script. Once we move distutils2 back in the stdlib, it will > be provided in Python's bin dir, so people will have the same > "pysetup" name everywhere, I am not about to bikeshed on the name, but I would like to publicly shed a single tear for no one even suggesting a Monty Python name closer than "quiche". I think going with PyPI over Cheeseshop helped put an end to that naming scheme, and that's a shame. Anyway, I can always alias pysetup to cheeseshop or ohmightytim on my machine and reminisce. From rdmurray at bitdance.com Fri Oct 8 21:40:00 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 08 Oct 2010 15:40:00 -0400 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <87pqvk7ezc.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007143748.7c548261@mission> <87sk0g7jp6.fsf@uwakimon.sk.tsukuba.ac.jp> <20101008170912.28958200D48@kimball.webabinitio.net> <87pqvk7ezc.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20101008194000.0E7A622B1B5@kimball.webabinitio.net> On Sat, 09 Oct 2010 02:48:23 +0900, "Stephen J. Turnbull" wrote: > R. David Murray writes: > > On Sat, 09 Oct 2010 01:06:29 +0900, "Stephen J. Turnbull" wrote: > > > That mess is entirely unnecessary in Python 3. Text and wire format > > > can be easily distinguished with three different representations of > > > email: Unicode for the conceptual RFC 822 layer (of course this is an > > > extension, because RFC 822 itself is strictly limited to the ASCII > > > subset), bytes for wire format, and Message objects for modern > > > structured mail (including MIME, etc). > > > That engineering is pretty much what we are looking at, although in > > practice I think you have to hang wire-format and text-format bits off > > of appropriate places in the model in order to keep everything properly > > coordinated. > > Right. That's where I was going with my comment to Barry about the > Received headers. Even if email isn't going to serve clients working > with wire format, it needs to deal with those headers. But where I > think the headers defined by RFC 822 should be stored as str in > email6, I am leaning toward storing Received headers verbatim as bytes > (including any RFC 822 folding whitespace) because of the RFC 5321 > requirement that they be preserved exactly. Well, the plan for email6 is to *allow* clients to work with wire format, though it will probably be a bit more awkward than working with the text interface. And my current strategy is in general to preserve the input bytes and, as long as the header in question hasn't been modified, emit those bytes when serialization back to bytes is done. My current plan is that conversion to text is only done at the point where text is requested, at which point the conversion is cached for later use. And if the header is modified, the source bytes version is discarded. Conversely if the source of the header was text input (msg['Subject'] = 'Hi'), then the conversion to bytes is only done when serialization to bytes is requested. None of this is implemented yet. -- R. David Murray www.bitdance.com From stephen at xemacs.org Fri Oct 8 21:44:20 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 09 Oct 2010 04:44:20 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <20101008142228.12424fd2@mission> References: <20101002230027.9A3DB22ABC5@kimball.webabinitio.net> <4CAA019A.8030309@scottdial.com> <20101004173835.9FD1E21B120@kimball.webabinitio.net> <87y6ad8adj.fsf@uwakimon.sk.tsukuba.ac.jp> <20101005150523.CA7D6228E86@kimball.webabinitio.net> <87bp778lzf.fsf@uwakimon.sk.tsukuba.ac.jp> <20101006170925.4EAA022A999@kimball.webabinitio.net> <878w2b860c.fsf@uwakimon.sk.tsukuba.ac.jp> <20101007143748.7c548261@mission> <87sk0g7jp6.fsf@uwakimon.sk.tsukuba.ac.jp> <20101008170912.28958200D48@kimball.webabinitio.net> <87pqvk7ezc.fsf@uwakimon.sk.tsukuba.ac.jp> <20101008142228.12424fd2@mission> Message-ID: <87mxqo79m3.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > On Oct 09, 2010, at 02:48 AM, Stephen J. Turnbull wrote: > > >Right. That's where I was going with my comment to Barry about the > >Received headers. Even if email isn't going to serve clients working > >with wire format, it needs to deal with those headers. But where I > >think the headers defined by RFC 822 should be stored as str in > >email6, I am leaning toward storing Received headers verbatim as bytes > >(including any RFC 822 folding whitespace) because of the RFC 5321 > >requirement that they be preserved exactly. > > I think then you'd have to have two different APIs. It'll just be too > confusing and error prone to return either bytes or strs from header value > accessors depending on the header name. Ah, but you wouldn't *return* those bytes with the value accessor, you'd return str, because that's what clients expect. And you wouldn't store bytes with a mutator -- or store a str for that matter: you get an error because the value is immutable. You would initialize it with a special initializer accepting the RFC 5321 components (and those arguments would be str and DateTimes and stuff like that). IOW, the implementation would share basically nothing with ordinary Headers, but the API would seem familiar, up to specific semantics like immutability. (And you also wouldn't be able to delete it with the usual interface; there would be a special API meaning "I really do want to delete this Received header, and I do understand that deletion is prohibited where RFC 5321 rules.") The choice of storage medium is purely an optimization (and to remind you that this header is actually part of the wire protocol). It's probably more trouble than it's worth. From rdmurray at bitdance.com Fri Oct 8 23:00:44 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 08 Oct 2010 17:00:44 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008171244.75fc2afa@pitrou.net> <20101008155342.GD10153@unaka.lan> Message-ID: <20101008210044.3691822B2F9@kimball.webabinitio.net> On Fri, 08 Oct 2010 12:31:07 -0700, Brett Cannon wrote: > I am not about to bikeshed on the name, but I would like to publicly > shed a single tear for no one even suggesting a Monty Python name > closer than "quiche". I think going with PyPI over Cheeseshop helped > put an end to that naming scheme, and that's a shame. > > Anyway, I can always alias pysetup to cheeseshop or ohmightytim on my > machine and reminisce. Or thebookofarmaments. -- R. David Murray www.bitdance.com From gisle at activestate.com Fri Oct 8 23:24:08 2010 From: gisle at activestate.com (Gisle Aas) Date: Fri, 8 Oct 2010 23:24:08 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101008072237.GJ85444@nexus.in-nomine.org> References: <20101008072237.GJ85444@nexus.in-nomine.org> Message-ID: On Oct 8, 2010, at 9:22 , Jeroen Ruigrok van der Werven wrote: > +1 from me. I sincerely dislike the Perl-esque -m stuff. As a Perl/Python guy I have to object to calling the -m stuff Perl-esque. This is a very Pythonish thing. In the Perl world we never treat modules as scripts; they are separate concepts written separately and installed in separate locations. There is no feature of perl similar to the Pythonish -m stuff. Regards, Gisle From nad at acm.org Fri Oct 8 23:40:42 2010 From: nad at acm.org (Ned Deily) Date: Fri, 08 Oct 2010 14:40:42 -0700 Subject: [Python-Dev] Build failure in test_cmd_line on OSX-x86 References: <20101008192852.6d57e6ef@pitrou.net> Message-ID: In article , Stephen Hansen wrote: > I edited the launchd config to force LANG = "en_US.UTF-8" and the test > suddenly passes, which is good. I have no idea why the LANG would end up > different when the app is launched from launchd -- even though it was > running as the same user as I was doing the testing against -- but, > apparently, it was. OS X does not automatically set LANG in all "launch" environments. Terminal.app in 10.5 and 10.6 has a preference to "set locale environment variables on startup" so, if you are running under it, you'll see LANG set. But if you execute via launchd or ssh (for instance), it will not automatically be set unless you do something like explicitly set and export it in a shell startup file. -- Ned Deily, nad at acm.org From ziade.tarek at gmail.com Fri Oct 8 23:44:35 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 8 Oct 2010 23:44:35 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008171244.75fc2afa@pitrou.net> <20101008155342.GD10153@unaka.lan> Message-ID: On Fri, Oct 8, 2010 at 9:31 PM, Brett Cannon wrote: > On Fri, Oct 8, 2010 at 09:25, Tarek Ziad? wrote: >> On Fri, Oct 8, 2010 at 5:53 PM, Toshio Kuratomi wrote: >>> On Fri, Oct 08, 2010 at 05:12:44PM +0200, Antoine Pitrou wrote: >> ... >>>> > pysetup is shorter >> >> Let's use pysetup ! >> >> ... >>> I won't bikeshed as long as we stay away from conflicting names. >> >> +1. >> >> So. Let's add pysetup in distutils2, that will be installed as a >> classical script. Once we move distutils2 back in the stdlib, it will >> be provided in Python's bin dir, so people will have the same >> "pysetup" name everywhere, > > I am not about to bikeshed on the name, but I would like to publicly > shed a single tear for no one even suggesting a Monty Python name > closer than "quiche". I think going with PyPI over Cheeseshop helped > put an end to that naming scheme, and that's a shame. > > Anyway, I can always alias pysetup to cheeseshop or ohmightytim on my > machine and reminisce. Hehe. What's the story behind changing the name from Cheeseshop to PyPI btw ? I found the first one much nicer > -- Tarek Ziad? | http://ziade.org From martin at v.loewis.de Sat Oct 9 00:12:14 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 09 Oct 2010 00:12:14 +0200 Subject: [Python-Dev] python.org going down? In-Reply-To: <20101008072107.GI85444@nexus.in-nomine.org> References: <4CAE4F17.4020206@v.loewis.de> <4CAE551A.90105@v.loewis.de> <20101008072107.GI85444@nexus.in-nomine.org> Message-ID: <4CAF973E.2060301@v.loewis.de> Am 08.10.2010 09:21, schrieb Jeroen Ruigrok van der Werven: > -On [20101008 01:22], "Martin v. L?wis" (martin at v.loewis.de) wrote: >> True. However, I really cannot see anything on the machines that >> indicates some outage. I'm still unsure what "it" is that was happening, >> so it's also difficult to analyse this further. > > It's hosted by XS4All, they had some network problems yesterday at least. I > am not sure if it would explain this behaviour, might be one of the uplinks > or peers had some issues. Ah. That would explain it, indeed. Regards, Martin From brett at python.org Sat Oct 9 00:40:08 2010 From: brett at python.org (Brett Cannon) Date: Fri, 8 Oct 2010 15:40:08 -0700 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008171244.75fc2afa@pitrou.net> <20101008155342.GD10153@unaka.lan> Message-ID: On Fri, Oct 8, 2010 at 14:44, Tarek Ziad? wrote: > On Fri, Oct 8, 2010 at 9:31 PM, Brett Cannon wrote: >> On Fri, Oct 8, 2010 at 09:25, Tarek Ziad? wrote: >>> On Fri, Oct 8, 2010 at 5:53 PM, Toshio Kuratomi wrote: >>>> On Fri, Oct 08, 2010 at 05:12:44PM +0200, Antoine Pitrou wrote: >>> ... >>>>> > pysetup is shorter >>> >>> Let's use pysetup ! >>> >>> ... >>>> I won't bikeshed as long as we stay away from conflicting names. >>> >>> +1. >>> >>> So. Let's add pysetup in distutils2, that will be installed as a >>> classical script. Once we move distutils2 back in the stdlib, it will >>> be provided in Python's bin dir, so people will have the same >>> "pysetup" name everywhere, >> >> I am not about to bikeshed on the name, but I would like to publicly >> shed a single tear for no one even suggesting a Monty Python name >> closer than "quiche". I think going with PyPI over Cheeseshop helped >> put an end to that naming scheme, and that's a shame. >> >> Anyway, I can always alias pysetup to cheeseshop or ohmightytim on my >> machine and reminisce. > > Hehe. What's the story behind changing the name from Cheeseshop to PyPI btw ? > I found the first one much nicer Richard Jones is the authority on the story, but from what I can remember from the discussion it was decided that managers would have had issues with using a service called the Cheeseshop. So basically the idea of professional-sounding name won out. I still use cheeseshop.python.org to access the package index. From foom at fuhm.net Sat Oct 9 01:07:13 2010 From: foom at fuhm.net (James Y Knight) Date: Fri, 8 Oct 2010 19:07:13 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008072237.GJ85444@nexus.in-nomine.org> Message-ID: <57250427-A81E-471A-810B-5C1444EE1B1C@fuhm.net> On Oct 8, 2010, at 5:24 PM, Gisle Aas wrote: > On Oct 8, 2010, at 9:22 , Jeroen Ruigrok van der Werven wrote: > >> +1 from me. I sincerely dislike the Perl-esque -m stuff. > > As a Perl/Python guy I have to object to calling the -m stuff Perl-esque. This is a very Pythonish thing. In the Perl world we never treat modules as scripts; they are separate concepts written separately and installed in separate locations. There is no feature of perl similar to the Pythonish -m stuff. Yes there is. -m and -M. E.g., the widely advertised perl -MCPAN -e install. It's not identical to python's -m, to be sure, but it's *similar*. James From greg.ewing at canterbury.ac.nz Sat Oct 9 01:35:38 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 09 Oct 2010 12:35:38 +1300 Subject: [Python-Dev] Another relative imports question In-Reply-To: References: <4CAEDB4C.8040303@simplistix.co.uk> Message-ID: <4CAFAACA.3040200@canterbury.ac.nz> Georg Brandl wrote: > The explanation is that everything that comes after "import" is thereafter > usable as an identifier (or expression, in the case of dotted names) in > code. ".mymodule" is not a valid expression, so the question would be how > to refer to it. I think a reasonable answer is that you should be able to refer to it simply as 'mymodule'. This may require treating it as a bit of a special case, but it would make intuitive sense. -- Greg From gisle at activestate.com Sat Oct 9 09:02:24 2010 From: gisle at activestate.com (Gisle Aas) Date: Sat, 9 Oct 2010 09:02:24 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <57250427-A81E-471A-810B-5C1444EE1B1C@fuhm.net> References: <20101008072237.GJ85444@nexus.in-nomine.org> <57250427-A81E-471A-810B-5C1444EE1B1C@fuhm.net> Message-ID: On Oct 9, 2010, at 1:07 , James Y Knight wrote: > On Oct 8, 2010, at 5:24 PM, Gisle Aas wrote: > >> On Oct 8, 2010, at 9:22 , Jeroen Ruigrok van der Werven wrote: >> >>> +1 from me. I sincerely dislike the Perl-esque -m stuff. >> >> As a Perl/Python guy I have to object to calling the -m stuff Perl-esque. This is a very Pythonish thing. In the Perl world we never treat modules as scripts; they are separate concepts written separately and installed in separate locations. There is no feature of perl similar to the Pythonish -m stuff. > > > Yes there is. -m and -M. > > E.g., the widely advertised perl -MCPAN -e install. It's not identical to python's -m, to be sure, but it's *similar*. It might look similar but it's not. If it was similar 'python -mfoo' would be a shortcut for 'python -c "import foo"' and 'python -Mfoo' would be a shortcut for 'python -c "from foo import *". It would also have to be possible to repeat the -c option. Then we could have written the Perl-esque: python -Mdistutils2.depgraph -c "main()" There is no way to do something similar to 'python -mfoo ...' from perl. The closest thing I could think of would be 'perl $(perldoc -l foo) ...', assuming a bash-like shell. Regards, Gisle From solipsis at pitrou.net Sat Oct 9 14:28:27 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 9 Oct 2010 14:28:27 +0200 Subject: [Python-Dev] [Pythonmac-SIG] sad state of OS X Python testing... In-Reply-To: References: <42516.1286045405@parc.com> <4CA79129.8050900@v.loewis.de> <20101002203205.GA24940@illinois.edu> <4CA79814.3030302@v.loewis.de> <20101008114220.20369e20@pitrou.net> <1286550016.17666.8.camel@localhost.localdomain> Message-ID: <20101009142827.1cae2bdb@pitrou.net> Congratulations Stephen, you are now the owner of our first green OS X buildbot! cheers Antoine. On Fri, 8 Oct 2010 08:11:13 -0700 Stephen Hansen wrote: [snip] From martin at v.loewis.de Sat Oct 9 19:39:33 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 09 Oct 2010 19:39:33 +0200 Subject: [Python-Dev] Another relative imports question In-Reply-To: <4CAFAACA.3040200@canterbury.ac.nz> References: <4CAEDB4C.8040303@simplistix.co.uk> <4CAFAACA.3040200@canterbury.ac.nz> Message-ID: <4CB0A8D5.4060907@v.loewis.de> Am 09.10.2010 01:35, schrieb Greg Ewing: > Georg Brandl wrote: >> The explanation is that everything that comes after "import" is >> thereafter >> usable as an identifier (or expression, in the case of dotted names) in >> code. ".mymodule" is not a valid expression, so the question would be >> how >> to refer to it. > > I think a reasonable answer is that you should be able > to refer to it simply as 'mymodule'. I don't think that's reasonable: import xml.dom doesn't give you dom, but xml. So import .dom shouldn't give you dom, but . (which is nonsensical, of course). Regards, Martin From rrr at ronadam.com Sat Oct 9 23:06:55 2010 From: rrr at ronadam.com (Ron Adam) Date: Sat, 09 Oct 2010 16:06:55 -0500 Subject: [Python-Dev] Another relative imports question In-Reply-To: <4CB0A8D5.4060907@v.loewis.de> References: <4CAEDB4C.8040303@simplistix.co.uk> <4CAFAACA.3040200@canterbury.ac.nz> <4CB0A8D5.4060907@v.loewis.de> Message-ID: On 10/09/2010 12:39 PM, "Martin v. L?wis" wrote: > Am 09.10.2010 01:35, schrieb Greg Ewing: >> Georg Brandl wrote: >>> The explanation is that everything that comes after "import" is >>> thereafter >>> usable as an identifier (or expression, in the case of dotted names) in >>> code. ".mymodule" is not a valid expression, so the question would be >>> how >>> to refer to it. >> >> I think a reasonable answer is that you should be able >> to refer to it simply as 'mymodule'. > > I don't think that's reasonable: > > import xml.dom > > doesn't give you dom, but xml. > > So > > import .dom > > shouldn't give you dom, but . (which is nonsensical, of course). I don't think it would be "import .dom", but... from . import dom It would be another module in xml doing the importing, so xml will have already been imported. Ron From ncoghlan at gmail.com Sun Oct 10 06:21:38 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 10 Oct 2010 14:21:38 +1000 Subject: [Python-Dev] Build failure in test_cmd_line on OSX-x86 In-Reply-To: <20101008192852.6d57e6ef@pitrou.net> References: <20101008192852.6d57e6ef@pitrou.net> Message-ID: On Sat, Oct 9, 2010 at 3:28 AM, Antoine Pitrou wrote: > You should therefore see what the locale settings of the buildbot are > (the LANG and LC_* environment variables). Of course, the test is also > buggy so you should open an issue on the tracker. > > (and the fact that the test doesn't print the actual error message of > the spawned interpreter is unhelpful) If someone wants to throw an issue my way, I can take a look at dumping stdout/stderr from the various test_cmd_line tests (I may not get to it until post-beta1 though). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From solipsis at pitrou.net Sun Oct 10 09:54:36 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 10 Oct 2010 09:54:36 +0200 Subject: [Python-Dev] Build failure in test_cmd_line on OSX-x86 In-Reply-To: References: <20101008192852.6d57e6ef@pitrou.net> Message-ID: <1286697276.3155.0.camel@localhost.localdomain> > If someone wants to throw an issue my way, I can take a look at > dumping stdout/stderr from the various test_cmd_line tests (I may not > get to it until post-beta1 though). It's done in r85324. Regards Antoine. From ncoghlan at gmail.com Sun Oct 10 12:25:59 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 10 Oct 2010 20:25:59 +1000 Subject: [Python-Dev] Build failure in test_cmd_line on OSX-x86 In-Reply-To: <1286697276.3155.0.camel@localhost.localdomain> References: <20101008192852.6d57e6ef@pitrou.net> <1286697276.3155.0.camel@localhost.localdomain> Message-ID: On Sun, Oct 10, 2010 at 5:54 PM, Antoine Pitrou wrote: > >> If someone wants to throw an issue my way, I can take a look at >> dumping stdout/stderr from the various test_cmd_line tests (I may not >> get to it until post-beta1 though). > > It's done in r85324. Even better :) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From techtonik at gmail.com Sun Oct 10 17:01:15 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Sun, 10 Oct 2010 18:01:15 +0300 Subject: [Python-Dev] Allow registered users to link bugs Message-ID: Can anybody remind me why we don't allow registered tracker users to link dependent bugs between each other? -- anatoly t. From techtonik at gmail.com Sun Oct 10 17:27:44 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Sun, 10 Oct 2010 18:27:44 +0300 Subject: [Python-Dev] Relative imports in Py3k In-Reply-To: References: Message-ID: On Sat, Sep 25, 2010 at 4:52 PM, Georg Brandl wrote: >> >> I wonder if situation with relative imports in packages is improved in >> Python 3k or we are still doomed to a chain of hacks? >> >> My user story: ... >> PEP 328 http://www.python.org/dev/peps/pep-0328/ ?proposes: >> >> from ... import config >> from ..utils.qthelpers import translate, add_actions, create_action >> >> But this doesn't work, and I couldn't find any short user level >> explanation why it is >> not possible to make this work at least in Py3k without additional magic. > > Uh, "this doesn't work" is a report that every developer loves. ?I suppose > Python does raise an exception with a message? It is not a bug report. It is a "user story" - that means the thing that I, as a user of Python, assumed to work. But instead have got: ValueError: Attempted relative import in non-package There are __init__.py files in every directory of spyderlib, but I don't want to debug or seek a workarounds against this in Python 2. I just want to know if relative imports are fixed in Python 3. -- anatoly t. From techtonik at gmail.com Sun Oct 10 17:54:20 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Sun, 10 Oct 2010 18:54:20 +0300 Subject: [Python-Dev] Relative imports in Py3k In-Reply-To: References: Message-ID: On Sun, Sep 26, 2010 at 2:32 PM, Nick Coghlan wrote: >> >> I wonder if situation with relative imports in packages is improved in >> Python 3k or >> we are still doomed to a chain of hacks? This is The question. Is Python 3k more friendly to users or require them to learn the "zen of import" (which is not available in concise format, unfortunately, unlike "this" thing)? >> relatives. All imports are like: >> >> from spyderlib.config import get_icon >> from spyderlib.utils.qthelpers import translate, add_actions, create_action > > This is almost certainly failing because the directory containing the > spyderlib package isn't on sys.path anywhere (instead, whichever > directory contains the script you executed directly will be in there, > which will be somewhere inside the package instead of outside it). Put > the appropriate directory in PYTHONPATH and these tests should start > working. This is a hack. I use relative imports, because I don't want to care about PYTHONPATH issues. I work with two clones of spyderlib (reference point and feature branch). You propose to switch PYTHONPATH every time I want to execute debug code in the __main__ section from either of them. Of course, I can write scripts to automate the thing, but it is a _basic_ debug operation, and as a one of the hundreds Python users I don't want to go through this misery just to be able to run "python submodule.py". Hope this clarifies The question. >> PEP 328 http://www.python.org/dev/peps/pep-0328/ ?proposes: >> >> from ... import config >> from ..utils.qthelpers import translate, add_actions, create_action > > This fails for two reasons: > 1. "__main__" is missing the module namespace information PEP 328 > needs to do its thing > 2. Even if 1 is resolved, PEP 328 will resolve to the same absolute > imports you're already using and likely fail for the same reason (i.e. > spyderlib not found on sys.path) This is a hack. There is no explanation why this hack is required, or it is not a user-friendly explanation. >> But this doesn't work, and I couldn't find any short user level >> explanation why it is >> not possible to make this work at least in Py3k without additional magic. > > If you use the -m switch to run your module from the directory that > contains the spyderlib package directory, it will work. The use of -m > provides the module namespace information that PEP 328 needs, while > running from the directory that contains the spyderlib package ensures > it is visible through sys.path. > > The one caveat is that the specified module is run as "__main__" and > hence does not exist in sys.modules under its normal name. If it gets > imported by another module, then you will end up with two copies of > the module (one as "__main__" and one under the normal name). This may > or may not cause problems, depending on the nature of the module being > executed. > > While PEP 366 describes the boilerplate you can put at the top of your > module to allow a directly executed module to try to find its > siblings, it still requires that PYTHONPATH be set appropriately. And > if you set PYTHONPATH appropriately, then direct execution with > absolute imports will work. > > (The explanation of the failures applies for all Python versions that > I am aware of, but the -m based fix only became available in 2.6) > (The impact of various command line options and the PYTHONPATH > environment variable on sys.path are described at > http://docs.python.org/using/cmdline.html) > (The basic import search algorithm is described in the tutorial: > http://docs.python.org/tutorial/modules.html#the-module-search-path) > (And the full gory specification details are in PEP 302, with a few > subsequent tweaks courtesy of PEP 328 and PEP 366). This what I call "zen of import". I don't see why such awkward way should be necessary in Python 3k, which breaks backwards compatibility. Why it can't "just work" for my user story? -- anatoly t. From ptmcg at austin.rr.com Sun Oct 10 20:57:21 2010 From: ptmcg at austin.rr.com (Paul McGuire) Date: Sun, 10 Oct 2010 13:57:21 -0500 Subject: [Python-Dev] [Python-ideas] minmax() function returning (minimum, maximum) tuple of a sequence Message-ID: <1ADBA76FE1F94E2CAA5211B04E950B48@PTMCG> Just as an exercise, I wanted to try my hand at adding a function to the compiled Python C code. An interesting optimization that I read about (where? don't recall) finds the minimum and maximum elements of a sequence in a single pass, with a 25% reduction in number of comparison operations: - the sequence elements are read in pairs - each pair is compared to find smaller/greater - the smaller is compared to current min - the greater is compared to current max So each pair is applied to the running min/max values using 3 comparisons, vs. 4 that would be required if both were compared to both min and max. This feels somewhat similar to how divmod returns both quotient and remainder of a single division operation. This would be potentially interesting for those cases where min and max are invoked on the same sequence one after the other, and especially so if the sequence elements were objects with expensive comparison operations. However, it *would* add "minmax" to the namespace bloat wherever this function might land (builtin? itertools? collections?). And I'm sure we wouldn't want to subsume the current min and max to just be minmax(s)[0] or minmax(s)[1], since that would *increase* the number of comparisons by 50% for either function when used alone. I did my prototyping using Python 2.5.5 source, since I only have Visual Studio 2005 - here is a diff to the that version's bltinmodule.c file: http://pastebin.com/4xv6SLBM Through the magic of Google, I've found these minmax implementations in the wild: http://www2-pcmdi.llnl.gov/cdat/source/api-reference/genutil.minmax.html http://mth.uct.ac.za/~lab/chap1/ans/ans2.pdf I also found a few other minmax references, but these methods were different implementations (minimum of sequence of maxima, or a generic min_or_max function taking a comparison flag or function in order to perform min or max). Unfortunately, I did not come up with a good way to search for cases where min and max were called one after the other. Any comments? Interest? Should I write up a PEP? Go back to my pyparsing hole? -- Paul McGuire From martin at v.loewis.de Sun Oct 10 21:48:23 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 10 Oct 2010 21:48:23 +0200 Subject: [Python-Dev] [Python-ideas] minmax() function returning (minimum, maximum) tuple of a sequence In-Reply-To: <1ADBA76FE1F94E2CAA5211B04E950B48@PTMCG> References: <1ADBA76FE1F94E2CAA5211B04E950B48@PTMCG> Message-ID: <4CB21887.9080704@v.loewis.de> > Any comments? For 3.2, this is out of scope, because of the moratorium. For later versions, I'd rather not see a builtin name polluting the global namespace for this. Regards, Martin From solipsis at pitrou.net Sun Oct 10 22:09:48 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 10 Oct 2010 22:09:48 +0200 Subject: [Python-Dev] [Python-ideas] minmax() function returning (minimum, maximum) tuple of a sequence References: <1ADBA76FE1F94E2CAA5211B04E950B48@PTMCG> Message-ID: <20101010220948.06fbb3ec@pitrou.net> On Sun, 10 Oct 2010 13:57:21 -0500 "Paul McGuire" wrote: > > Any comments? Interest? Should I write up a PEP? Go back to my pyparsing > hole? Generally, these things are discussed on python-ideas first. I don't think a PEP required for a single function, but you'll have to convince people that it's useful ;) Personnally, I'm not convinced that a maximum 25% improvement on a rather uncommon use case (min() and max() on a sequence of objects which take a long time to compare) is a compelling argument for a builtin. On the other hand, it would be a rather simple and intuitive builtin, so why not? Regards Antoine. From steve at pearwood.info Mon Oct 11 01:17:54 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 11 Oct 2010 10:17:54 +1100 Subject: [Python-Dev] [Python-ideas] minmax() function returning (minimum, maximum) tuple of a sequence In-Reply-To: <1ADBA76FE1F94E2CAA5211B04E950B48@PTMCG> References: <1ADBA76FE1F94E2CAA5211B04E950B48@PTMCG> Message-ID: <201010111017.56101.steve@pearwood.info> On Mon, 11 Oct 2010 05:57:21 am Paul McGuire wrote: > Just as an exercise, I wanted to try my hand at adding a function to > the compiled Python C code. An interesting optimization that I read > about (where? don't recall) finds the minimum and maximum elements of > a sequence in a single pass, with a 25% reduction in number of > comparison operations: > - the sequence elements are read in pairs > - each pair is compared to find smaller/greater > - the smaller is compared to current min > - the greater is compared to current max > > So each pair is applied to the running min/max values using 3 > comparisons, vs. 4 that would be required if both were compared to > both min and max. > > This feels somewhat similar to how divmod returns both quotient and > remainder of a single division operation. > > This would be potentially interesting for those cases where min and > max are invoked on the same sequence one after the other, and > especially so if the sequence elements were objects with expensive > comparison operations. Perhaps more importantly, it is ideal for the use-case where you have an iterator. You can't call min() and then max(), as min() consumes the iterator leaving nothing for max(). It may be undesirable to convert the iterator to a list first -- it may be that the number of items in the data stream is too large to fit into memory all at once, but even if it is small, it means you're now walking the stream three times when one would do. To my mind, minmax() is as obvious and as useful a built-in as divmod(), but if there is resistance to making such a function a built-in, perhaps it could go into itertools. (I would prefer it to keep the same signature as min() and max(), namely that it will take either a single iterable argument or multiple arguments.) I've experimented with minmax() myself. Not surprisingly, the performance of a pure Python version doesn't even come close to the built-ins. I'm +1 on the idea. Presumably follow-ups should go to python-ideas. -- Steven D'Aprano From rdmurray at bitdance.com Mon Oct 11 01:22:53 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 10 Oct 2010 19:22:53 -0400 Subject: [Python-Dev] Relative imports in Py3k In-Reply-To: References: Message-ID: <20101010232253.5641322B249@kimball.webabinitio.net> > On Sun, 10 Oct 2010 18:54:20 +0300, anatoly techtonik wrote: > > (The explanation of the failures applies for all Python versions that > > I am aware of, but the -m based fix only became available in 2.6) > > (The impact of various command line options and the PYTHONPATH > > environment variable on sys.path are described at > > http://docs.python.org/using/cmdline.html) > > (The basic import search algorithm is described in the tutorial: > > http://docs.python.org/tutorial/modules.html#the-module-search-path) > > (And the full gory specification details are in PEP 302, with a few > > subsequent tweaks courtesy of PEP 328 and PEP 366). > > This what I call "zen of import". > > I don't see why such awkward way should be necessary in Python 3k, > which breaks backwards compatibility. Why it can't "just work" for my > user story? Because you weren't around advocating and implementing a change when Python 3 was developed? It's too late now to arbitrarily break backward compatibility, so you'll have to advocate and develop any change inside the parameters of Python's normal backward compatibility policy. In other words, you should move this discussion to python-ideas. -- R. David Murray www.bitdance.com From benjamin at python.org Mon Oct 11 02:15:08 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 10 Oct 2010 19:15:08 -0500 Subject: [Python-Dev] [Python-ideas] minmax() function returning (minimum, maximum) tuple of a sequence In-Reply-To: <20101010220948.06fbb3ec@pitrou.net> References: <1ADBA76FE1F94E2CAA5211B04E950B48@PTMCG> <20101010220948.06fbb3ec@pitrou.net> Message-ID: 2010/10/10 Antoine Pitrou : > Personnally, I'm not convinced that a maximum 25% improvement on a > rather uncommon use case (min() and max() on a sequence of objects > which take a long time to compare) is a compelling argument for a > builtin. ?On the other hand, it would be a rather simple and intuitive > builtin, so why not? Because new builtins should have a better reason that there's no good reason not to. :) -- Regards, Benjamin From zac256 at gmail.com Mon Oct 11 02:55:51 2010 From: zac256 at gmail.com (Zac Burns) Date: Sun, 10 Oct 2010 17:55:51 -0700 Subject: [Python-Dev] [Python-ideas] minmax() function returning (minimum, maximum) tuple of a sequence In-Reply-To: <201010111017.56101.steve@pearwood.info> References: <1ADBA76FE1F94E2CAA5211B04E950B48@PTMCG> <201010111017.56101.steve@pearwood.info> Message-ID: This could be generalized and placed into itertools if we create a function (say, apply for lack of a better name at the moment) that takes in an iterable and creates new iterables that yield each from the original (avoiding the need for a list) holding only one in memory. Then you could pass the whatever function you wanted to run the iterables over an get the result back in a tuple. Eg: itertools.apply(iterable, min, max) ~= (min(iterable), max(iterable)) This class that creates 'associated iterables' from an original iterable where each new iterable has to be iterated over at the same time might also be useful in other contexts and could be added to itertools as well. Unfortunately this solution seems incompatable with the implementations with for loops in min and max (EG: How do you switch functions at the right time?) So it might take some tweaking. -- Zachary Burns (407)590-4814 Aim - Zac256FL On Sun, Oct 10, 2010 at 4:17 PM, Steven D'Aprano wrote: > On Mon, 11 Oct 2010 05:57:21 am Paul McGuire wrote: > > Just as an exercise, I wanted to try my hand at adding a function to > > the compiled Python C code. An interesting optimization that I read > > about (where? don't recall) finds the minimum and maximum elements of > > a sequence in a single pass, with a 25% reduction in number of > > comparison operations: > > - the sequence elements are read in pairs > > - each pair is compared to find smaller/greater > > - the smaller is compared to current min > > - the greater is compared to current max > > > > So each pair is applied to the running min/max values using 3 > > comparisons, vs. 4 that would be required if both were compared to > > both min and max. > > > > This feels somewhat similar to how divmod returns both quotient and > > remainder of a single division operation. > > > > This would be potentially interesting for those cases where min and > > max are invoked on the same sequence one after the other, and > > especially so if the sequence elements were objects with expensive > > comparison operations. > > Perhaps more importantly, it is ideal for the use-case where you have an > iterator. You can't call min() and then max(), as min() consumes the > iterator leaving nothing for max(). It may be undesirable to convert > the iterator to a list first -- it may be that the number of items in > the data stream is too large to fit into memory all at once, but even > if it is small, it means you're now walking the stream three times when > one would do. > > To my mind, minmax() is as obvious and as useful a built-in as divmod(), > but if there is resistance to making such a function a built-in, > perhaps it could go into itertools. (I would prefer it to keep the same > signature as min() and max(), namely that it will take either a single > iterable argument or multiple arguments.) > > I've experimented with minmax() myself. Not surprisingly, the > performance of a pure Python version doesn't even come close to the > built-ins. > > I'm +1 on the idea. > > Presumably follow-ups should go to python-ideas. > > > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmjohnson.mailinglist at gmail.com Mon Oct 11 07:56:47 2010 From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson) Date: Sun, 10 Oct 2010 19:56:47 -1000 Subject: [Python-Dev] [Python-ideas] minmax() function returning (minimum, maximum) tuple of a sequence In-Reply-To: References: <1ADBA76FE1F94E2CAA5211B04E950B48@PTMCG> <201010111017.56101.steve@pearwood.info> Message-ID: On Sun, Oct 10, 2010 at 2:55 PM, Zac Burns wrote: > This could be generalized and placed into itertools if we create a function > (say, apply for lack of a better name at the moment) that takes in an > iterable and creates new iterables that yield each from the original > (avoiding the need for a list) holding only one in memory. Then you could > pass the whatever function you wanted to run the iterables over an get the > result back in a tuple. Time machine partially beat you to this one. Look at the docs on itertools.tee tee(it, n=2) --> (it1, it2 , ... itn) splits one iterator into n Can be used like so: >>> it = iter(range(100)) >>> it1, it2 = itertools.tee(it) >>> max(it1) 99 >>> min(it2) 0 This doesn't quite have the memory characteristics you describe, but it's about as good as you can expect in a single threaded environment. From python-dev at masklinn.net Mon Oct 11 08:06:13 2010 From: python-dev at masklinn.net (Xavier Morel) Date: Mon, 11 Oct 2010 08:06:13 +0200 Subject: [Python-Dev] [Python-ideas] minmax() function returning (minimum, maximum) tuple of a sequence In-Reply-To: References: <1ADBA76FE1F94E2CAA5211B04E950B48@PTMCG> <201010111017.56101.steve@pearwood.info> Message-ID: On 2010-10-11, at 07:56 , Carl M. Johnson wrote: > On Sun, Oct 10, 2010 at 2:55 PM, Zac Burns wrote: >> This could be generalized and placed into itertools if we create a function >> (say, apply for lack of a better name at the moment) that takes in an >> iterable and creates new iterables that yield each from the original >> (avoiding the need for a list) holding only one in memory. Then you could >> pass the whatever function you wanted to run the iterables over an get the >> result back in a tuple. > > Time machine partially beat you to this one. Look at the docs on itertools.tee > > tee(it, n=2) --> (it1, it2 , ... itn) splits one iterator into n > > Can be used like so: > >>>> it = iter(range(100)) >>>> it1, it2 = itertools.tee(it) >>>> max(it1) > 99 >>>> min(it2) > 0 > > This doesn't quite have the memory characteristics you describe That's an understatement. As the documentation indicates, if you're going to fully consume the iterator the first time around instead of iterating both in near-lockstep (combining islice and map for instance) you're better off serializing to a list and then using that list twice, memory-wise. > but it's about as good as you can expect in a single threaded environment. You could also have coroutines, but they cooperate explicitly so you'd have to "fix" min and max (and any other function you can get your hands on, really) in order to allow them to act as coroutines, I think. From ncoghlan at gmail.com Mon Oct 11 14:27:47 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 11 Oct 2010 22:27:47 +1000 Subject: [Python-Dev] Relative imports in Py3k In-Reply-To: References: Message-ID: On Mon, Oct 11, 2010 at 1:54 AM, anatoly techtonik wrote: > On Sun, Sep 26, 2010 at 2:32 PM, Nick Coghlan wrote: >> This is almost certainly failing because the directory containing the >> spyderlib package isn't on sys.path anywhere (instead, whichever >> directory contains the script you executed directly will be in there, >> which will be somewhere inside the package instead of outside it). Put >> the appropriate directory in PYTHONPATH and these tests should start >> working. > > This is a hack. I use relative imports, because I don't want to care > about PYTHONPATH issues. I work with two clones of spyderlib > (reference point and feature branch). You propose to switch PYTHONPATH > every time I want to execute debug code in the __main__ section from > either of them. Anatoly, unconstructive responses like this are why people often react negatively to your attempts to be "helpful". I specifically mentioned 2 things you could do: - modify PYTHONPATH - use -m to execute your modules and just switch your current working directory depending on which version of spyderlib you want to execute Responding to that by telling me that modifying PYTHONPATH when working with in-development code is clumsy isn't telling me anything I don't already know. There's a *reason* why you can now use -m to have relative imports "just work" so long as your current working directory includes the top level directory for your package. There's a *reason* we added the ability to execute directories and zipfiles and have them automatically add themselves to the head of sys.path. And it's the same reason: we've been working on making it easier to work with uninstalled (i.e. not on PYTHONPATH) code for years. Throwing your hands up and saying "bah, it doesn't work, it should just read my mind and know what I meant!" really isn't helpful. Based on your latest response, giving your spyderlib package a top-level __main__.py and running the directory directly may even be an option for you. But when your response solely discusses PYTHONPATH without even mentioning the better alternative I offered (i.e. using -m and just switching your current working directory in a command shell), it's hard to assess your actual use case. As far as documentation goes, I personally find it incredibly difficult to write simple, user-oriented documentation of the import system, as I've long since lost my perspective on what's simple and what's complicated when it comes to the import system. I'd love for someone to tackle the task of writing clear end-user documentation of the whole PEP 302, 328, 338, 366, imp, runpy, importlib, etc arrangement (more comprehensive documentation of the zipfile and directory execution from issue 1739468 wouldn't hurt either). The problem is that the people who already know enough to write such documentation don't need it, and anyone else that sets out to learn enough to be able to write it discovers that there are so many backwards compatibility hacks that complicate an otherwise reasonably clean design that they throw their hands up in despair. (That said, I did attempt to write such a piece of documentation a few years back [1] so anyone that wanted to try it could at least consider using that as a starting point instead of starting with a blank screen) Cheers, Nick. [1] http://svn.python.org/view/sandbox/trunk/userref/ODF/Chapter07_ModulesAndApplications.odt -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ioan.ferencik at tkk.fi Mon Oct 11 14:06:48 2010 From: ioan.ferencik at tkk.fi (Ioan Ferencik) Date: Mon, 11 Oct 2010 15:06:48 +0300 Subject: [Python-Dev] PyArg_ParseTuple Message-ID: <20101011150648.u20sjrq42s0kksgw@webmail3.tkk.fi> I would like to ask where can I find more detailed info on PyArg_ParseTuple function. I find the doc limited on the matter. Mainly I am curious why the function requires an address of a pointer. I have issues in the following case: in python int jmax = 16 print type(jmax) which is just all right but following C code seems to be working PyObject *jmax_o = NULL; if(!PyArg_ParseTuple(args, "i", &jmax_o)){ goto error; } but PyInt_Check(jmax_o) fails. I tried to debug and this is what i could see Program received signal SIGSEGV, Segmentation fault. 0x00007ffff67a75bd in fprintf (self=, args=(16,)) at /usr/include/bits/stdio2.h:98 98 return __fprintf_chk (__stream, __USE_FORTIFY_LEVEL - 1, __fmt, so for some reason the jmax_o can not be converted to int. I use a x86_64 ubuntu and i suspect it could be because of 64 bits arch. Cheers Ioan Ferencik PhD student Aalto University School of Science and Technology Faculty Of Civil and Env. Engineering Lahti Center Tel: +358505122707 From benjamin at python.org Mon Oct 11 15:09:14 2010 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 11 Oct 2010 08:09:14 -0500 Subject: [Python-Dev] PyArg_ParseTuple In-Reply-To: <20101011150648.u20sjrq42s0kksgw@webmail3.tkk.fi> References: <20101011150648.u20sjrq42s0kksgw@webmail3.tkk.fi> Message-ID: 2010/10/11 Ioan Ferencik : > I ?would like to ask where can I find more detailed info on > PyArg_ParseTuple function. See python-list. > > I find the doc limited on the matter. > Mainly I am curious why the function requires an address of a pointer. So it can change the pointer. -- Regards, Benjamin From benjamin at python.org Mon Oct 11 15:09:14 2010 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 11 Oct 2010 08:09:14 -0500 Subject: [Python-Dev] PyArg_ParseTuple In-Reply-To: <20101011150648.u20sjrq42s0kksgw@webmail3.tkk.fi> References: <20101011150648.u20sjrq42s0kksgw@webmail3.tkk.fi> Message-ID: 2010/10/11 Ioan Ferencik : > I ?would like to ask where can I find more detailed info on > PyArg_ParseTuple function. See python-list. > > I find the doc limited on the matter. > Mainly I am curious why the function requires an address of a pointer. So it can change the pointer. -- Regards, Benjamin From benjamin at python.org Mon Oct 11 15:09:14 2010 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 11 Oct 2010 08:09:14 -0500 Subject: [Python-Dev] PyArg_ParseTuple In-Reply-To: <20101011150648.u20sjrq42s0kksgw@webmail3.tkk.fi> References: <20101011150648.u20sjrq42s0kksgw@webmail3.tkk.fi> Message-ID: 2010/10/11 Ioan Ferencik : > I ?would like to ask where can I find more detailed info on > PyArg_ParseTuple function. See python-list. > > I find the doc limited on the matter. > Mainly I am curious why the function requires an address of a pointer. So it can change the pointer. -- Regards, Benjamin From lomax at pumpichank.com Mon Oct 11 16:52:35 2010 From: lomax at pumpichank.com (Frank Lomax) Date: Mon, 11 Oct 2010 10:52:35 -0400 Subject: [Python-Dev] Cheeseshop (was Re: Distutils2 scripts) In-Reply-To: References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008171244.75fc2afa@pitrou.net> <20101008155342.GD10153@unaka.lan> Message-ID: <20101011105235.5c1caaa5@mission> On Oct 08, 2010, at 03:40 PM, Brett Cannon wrote: >Richard Jones is the authority on the story, but from what I can >remember from the discussion it was decided that managers would have >had issues with using a service called the Cheeseshop. So basically >the idea of professional-sounding name won out. I still use >cheeseshop.python.org to access the package index. IOW, the Anti-Humor Subcommittee of the PSU (which emphatically does not exist), a rough outfit which works to promote snakes over British comedy, quashed the Happy Laughing Working Group and actively (even though they do not exist) promotes the boring PHB name in all its propaganda. The HLWG, having been defunded and thrown out of office (if there was an office, which there isn't), toils in exile and hopes to one day reclaim the right and one true name. Join the revolt to take back Pythonland! May the Pythonistas defeat the Pythoneers! May the Pythoneers vanquish the Pythonistas! Long live the Cheeseshop! it's-very-runny-actual-ly y'rs, -frank From martin at v.loewis.de Mon Oct 11 18:17:50 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 11 Oct 2010 18:17:50 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CAF36EF.10306@voidspace.org.uk> References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008110753.6cdaf13b@mission> <4CAF36EF.10306@voidspace.org.uk> Message-ID: <4CB338AE.8050103@v.loewis.de> Am 08.10.2010 17:21, schrieb Michael Foord: > On 08/10/2010 16:07, Barry Warsaw wrote: >> On Oct 08, 2010, at 11:04 AM, Toshio Kuratomi wrote: >> >>> python-setup is a lot like python setup.py >>> pysetup is shorter >>> pyegg is even shorter :-) >> wfm! > > To avoid any potential confusion, wfm is a common tla for the following > phrases: > > Whole Foods Market > Western Federation of Miners > Window Fitters Mate > Workforce Management > > Although wfm is possibly being used here as "works for me" given the > context... ;-) Not being a native speaker (of English), I had actually assumed that Barry is proposing to call the script "wfm". Although, in retrospect, "/usr/bin/wfm!" might work as well :-) Regards, Martin From daniel at stutzbachenterprises.com Mon Oct 11 19:42:05 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 11 Oct 2010 12:42:05 -0500 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <20101008150435.GC10153@unaka.lan> <20101008171244.75fc2afa@pitrou.net> <20101008155342.GD10153@unaka.lan> Message-ID: On Fri, Oct 8, 2010 at 4:44 PM, Tarek Ziad? wrote: > Hehe. What's the story behind changing the name from Cheeseshop to PyPI btw > ? > I found the first one much nicer A through investigation revealed that the Cheeseshop did not in fact have any cheese at all. Not even Wensleydale. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From rrr at ronadam.com Mon Oct 11 20:01:06 2010 From: rrr at ronadam.com (Ron Adam) Date: Mon, 11 Oct 2010 13:01:06 -0500 Subject: [Python-Dev] Relative imports in Py3k In-Reply-To: References: Message-ID: On 10/11/2010 07:27 AM, Nick Coghlan wrote: > On Mon, Oct 11, 2010 at 1:54 AM, anatoly techtonik wrote: >> On Sun, Sep 26, 2010 at 2:32 PM, Nick Coghlan wrote: >>> This is almost certainly failing because the directory containing the >>> spyderlib package isn't on sys.path anywhere (instead, whichever >>> directory contains the script you executed directly will be in there, >>> which will be somewhere inside the package instead of outside it). Put >>> the appropriate directory in PYTHONPATH and these tests should start >>> working. >> >> This is a hack. I use relative imports, because I don't want to care >> about PYTHONPATH issues. I work with two clones of spyderlib >> (reference point and feature branch). You propose to switch PYTHONPATH >> every time I want to execute debug code in the __main__ section from >> either of them. > > Anatoly, unconstructive responses like this are why people often react > negatively to your attempts to be "helpful". > > I specifically mentioned 2 things you could do: > - modify PYTHONPATH > - use -m to execute your modules and just switch your current working > directory depending on which version of spyderlib you want to execute I don't recall Anatoly saying which p3k version and revision he was using. Relative imports was broken for while in 3.2. It's fixed now and I presume he is using a fairly current revision of 3.2. When you do a "make install" for 3.2 on Ubuntu, the current directory path "", isn't perpended to sys.path. I don't know if that is an over site or not, but it could be a factor. A few more suggestions ... Make A test runner script which modifies sys.path. It also could be considered a hack, but it doesn't require modifying PYTHONPATH, so it wouldn't have any potential to have side effects on other modules/programs. One of my personal choices when writing large applications (rather than scripts), is to make a local "lib" directory and prepend that to sys.path in the main application file before any local imports. # Add a local lib to the search path. lib = os.path.abspath(os.path.join(__file__, '..', 'lib')) sys.path.insert(0, lib) [Appliction dir not in PYthon path] main_app_file.py test.py [lib] [test package] ... #test modules ... #other local modules and packages I then add a -test option to the main_app_file.py or a create test.py file at the same level as the main_app_file. The test runner also needs to add lib to sys.path, but after that it can import and find any/all tests you want to run. The test modules can use relative imports as long as they aren't circular. * The error message in the case of circular imports could be much better! Cheers, Ron From g.rodola at gmail.com Mon Oct 11 23:17:30 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Mon, 11 Oct 2010 23:17:30 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CAF2F6B.3030903@trueblade.com> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> Message-ID: 2010/10/8 Eric Smith : > On 10/8/10 10:26 AM, Barry Warsaw wrote: >> In any case, these could be a simple shell script wrapping 'python -m >> setup'. >> It could even take a --use-python-version option to select the pythonX.Y >> it >> used, without having to encode the Python version number in the script >> name. > > On Windows it can't be a shell script or batch file, but needs to be an > executable. setuptools already deals with this. If that's the case what would I type in the command prompt in order to install a module? "C:\PythonXX\pysetup.exe"? If so I would strongly miss old "setup.py install". --- Giampaolo From g.rodola at gmail.com Tue Oct 12 01:11:24 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Tue, 12 Oct 2010 01:11:24 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CB38F6E.4050202@trueblade.com> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> Message-ID: Wouldn't be kinda weird that one can open the command prompt and run "pysetup" but not "python" on Windows? I recall an old issue on the bug tracker in which the latter proposal was widely discussed and finally rejected for reasons I can't remember (and it seems I can't even find the bug right now). I think it's likely that those same reasons are valid for "pysetup" in the same manner. For the record, I would be more than happy to be able to open the command prompt and type "pysetup" and "python" with success, one day. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2010/10/12 Eric Smith : > On 10/11/2010 5:17 PM, Giampaolo Rodol? wrote: >> >> 2010/10/8 Eric Smith: >>> >>> On 10/8/10 10:26 AM, Barry Warsaw wrote: >>>> >>>> In any case, these could be a simple shell script wrapping 'python -m >>>> setup'. >>>> It could even take a --use-python-version option to select the pythonX.Y >>>> it >>>> used, without having to encode the Python version number in the script >>>> name. >>> >>> On Windows it can't be a shell script or batch file, but needs to be an >>> executable. setuptools already deals with this. >> >> If that's the case what would I type in the command prompt in order to >> install a module? >> "C:\PythonXX\pysetup.exe"? >> If so I would strongly miss old "setup.py install". > > Same thing you would type at a shell prompt. Presumably we're talking about > "pysetup install" (which you'll note is one character shorter!). You could > fully qualify the path if need be, on any platform, using its conventions. > > Eric. > From solipsis at pitrou.net Tue Oct 12 01:20:32 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 12 Oct 2010 01:20:32 +0200 Subject: [Python-Dev] Distutils2 scripts References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> Message-ID: <20101012012032.4175d945@pitrou.net> On Tue, 12 Oct 2010 01:11:24 +0200 Giampaolo Rodol? wrote: > Wouldn't be kinda weird that one can open the command prompt and run > "pysetup" but not "python" on Windows? If you add C:\PythonXY to your path, you can run "python". Regards Antoine. From greg.ewing at canterbury.ac.nz Tue Oct 12 01:24:56 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 12 Oct 2010 12:24:56 +1300 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> Message-ID: <4CB39CC8.9050602@canterbury.ac.nz> Giampaolo Rodol? wrote: > If that's the case what would I type in the command prompt in order to > install a module? > "C:\PythonXX\pysetup.exe"? > If so I would strongly miss old "setup.py install". Another thing bothers me about this. With the current scheme, if you have multiple Pythons available, it's easy to be sure that you're installing into the right one, because it's the one that you use to run setup.py. Whereas if installation is performed by a different executable, there's a possibility of them being out of sync. So I think I'd prefer some scheme involving 'python -m ...' or some other option to Python itself, rather than a separate executable. -- Greg From greg.ewing at canterbury.ac.nz Tue Oct 12 01:30:19 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 12 Oct 2010 12:30:19 +1300 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> Message-ID: <4CB39E0B.5040003@canterbury.ac.nz> Giampaolo Rodol? wrote: > Wouldn't be kinda weird that one can open the command prompt and run > "pysetup" but not "python" on Windows? > I recall an old issue on the bug tracker in which the latter proposal > was widely discussed and finally rejected for reasons I can't remember On Windows I think it's easier and more reliable to set things up so that you can invoke a .py file directly as a command. -- Greg From g.rodola at gmail.com Tue Oct 12 01:42:03 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Tue, 12 Oct 2010 01:42:03 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101012012032.4175d945@pitrou.net> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> <20101012012032.4175d945@pitrou.net> Message-ID: 2010/10/12 Antoine Pitrou : > On Tue, 12 Oct 2010 01:11:24 +0200 > Giampaolo Rodol? wrote: >> Wouldn't be kinda weird that one can open the command prompt and run >> "pysetup" but not "python" on Windows? > > If you add C:\PythonXY to your path, you can run "python". I know. My point was you can't do it by default and installing a module is something even a less experienced user usually does. Typing "C:\PythonXX\pysetup" is harder compared to "setup.py install" and solving this problem by modifying your environment paths so that you can just type "pysetup" is something I would expect to be done by the MSI installer, not the user. --- Giampaolo http://code.google.com/p/pyft/ http://code.google.com/p/psutil/ From eric at trueblade.com Tue Oct 12 00:27:58 2010 From: eric at trueblade.com (Eric Smith) Date: Mon, 11 Oct 2010 18:27:58 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> Message-ID: <4CB38F6E.4050202@trueblade.com> On 10/11/2010 5:17 PM, Giampaolo Rodol? wrote: > 2010/10/8 Eric Smith: >> On 10/8/10 10:26 AM, Barry Warsaw wrote: >>> In any case, these could be a simple shell script wrapping 'python -m >>> setup'. >>> It could even take a --use-python-version option to select the pythonX.Y >>> it >>> used, without having to encode the Python version number in the script >>> name. >> >> On Windows it can't be a shell script or batch file, but needs to be an >> executable. setuptools already deals with this. > > If that's the case what would I type in the command prompt in order to > install a module? > "C:\PythonXX\pysetup.exe"? > If so I would strongly miss old "setup.py install". Same thing you would type at a shell prompt. Presumably we're talking about "pysetup install" (which you'll note is one character shorter!). You could fully qualify the path if need be, on any platform, using its conventions. Eric. From merwok at netwok.org Tue Oct 12 06:55:22 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 12 Oct 2010 06:55:22 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> Message-ID: <4CB3EA3A.7020902@netwok.org> Le 12/10/2010 01:11, Giampaolo Rodol? a ?crit : > Wouldn't be kinda weird that one can open the command prompt and run > "pysetup" but not "python" on Windows? > I recall an old issue on the bug tracker in which the latter proposal > was widely discussed and finally rejected for reasons I can't remember > (and it seems I can't even find the bug right now). > I think it's likely that those same reasons are valid for "pysetup" in > the same manner. Arguments for rejection by MvL in http://bugs.python.org/issue3561 : ?Adding something to PATH is nearly unacceptable clutter even during installation, and completely unacceptable (to me) after uninstallation. If you install Python several times, will the path get longer and longer?? ?To me, any change to PATH is a problem... (I really think that software installation should never ever touch it - this aspect of the operating system completely belongs to the user resp. the system administrator)? ?If such a solution was designed, there would still be many questions, such as: - what is the actual problem being solved? Is it real? Could there be other solutions to that problem (such as installing things into system32, or somewhere else that is on the PATH already)? - if the path is modified, should the Python directory be added to the beginning or the end? - IMO, this feature needs to be customizable, and IMO, it must be turned off by default. So how should such customization be offered?? Regarding pysetup, would it be acceptable to require programmers on Windows to edit their PATH or copy the script to a directory on PATH? Regards From fuzzyman at voidspace.org.uk Tue Oct 12 13:19:08 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 12 Oct 2010 12:19:08 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> Message-ID: <4CB4442C.8030308@voidspace.org.uk> On 12/10/2010 00:11, Giampaolo Rodol? wrote: > Wouldn't be kinda weird that one can open the command prompt and run > "pysetup" but not "python" on Windows? > I recall an old issue on the bug tracker in which the latter proposal > was widely discussed and finally rejected for reasons I can't remember > (and it seems I can't even find the bug right now). > I think it's likely that those same reasons are valid for "pysetup" in > the same manner. > > For the record, I would be more than happy to be able to open the > command prompt and type "pysetup" and "python" with success, one day. Well ditto, but not everyone feels the same. If you have multiple python versions having them all modify the path on installation would be annoying - the other issue is that it is hard to deterministically undo a change on uninstallation. (And leaving cruft behind is bad, although plenty of other applications don't have as much of a conscience about it as we do.) For the record I wrote about the PATH settings (etc) useful for Python on Windows: http://www.voidspace.org.uk/python/articles/command_line.shtml#environment-variables Probably too basic for *you* but maybe helpful for others. What would be *really* nice was if we had a versioned script / exe as well (same for python.exe on Windows) so that you could unconditionally add the Python directory and its Scripts directory to the PATH and then be able to specify which version to run. Currently you have to setup aliases yourself to do this. So as well as pysetup.py/.exe I would like pysetup-3.2.py / .exe on Windows please. (I'd really like a python-3.2.exe as well.) All the best, Michael Foord > > --- Giampaolo > http://code.google.com/p/pyftpdlib/ > http://code.google.com/p/psutil/ > > > 2010/10/12 Eric Smith: >> On 10/11/2010 5:17 PM, Giampaolo Rodol? wrote: >>> 2010/10/8 Eric Smith: >>>> On 10/8/10 10:26 AM, Barry Warsaw wrote: >>>>> In any case, these could be a simple shell script wrapping 'python -m >>>>> setup'. >>>>> It could even take a --use-python-version option to select the pythonX.Y >>>>> it >>>>> used, without having to encode the Python version number in the script >>>>> name. >>>> On Windows it can't be a shell script or batch file, but needs to be an >>>> executable. setuptools already deals with this. >>> If that's the case what would I type in the command prompt in order to >>> install a module? >>> "C:\PythonXX\pysetup.exe"? >>> If so I would strongly miss old "setup.py install". >> Same thing you would type at a shell prompt. Presumably we're talking about >> "pysetup install" (which you'll note is one character shorter!). You could >> fully qualify the path if need be, on any platform, using its conventions. >> >> Eric. >> > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From p.f.moore at gmail.com Tue Oct 12 13:55:28 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 12 Oct 2010 12:55:28 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> <20101012012032.4175d945@pitrou.net> Message-ID: On 12 October 2010 00:42, Giampaolo Rodol? wrote: > I know. My point was you can't do it by default and installing a > module is something even a less experienced user usually does. > Typing "C:\PythonXX\pysetup" is harder compared to "setup.py install" > and solving this problem by modifying your environment paths so that > you can just type "pysetup" is something I would expect to be done by > the MSI installer, not the user. I would assume (am I wrong?) that the canonical way of installing modules on Windows for "non-advanced" users under distutils2 would still be to download and run a binary installer. Assuming that's the case, modifying paths to make sure pysetup is available as a command is no harder than making Python itself available. (Having said that, I'd still personally prefer to have the distutils2 command be invoked by some form of python -m invocation). Paul. From fuzzyman at voidspace.org.uk Tue Oct 12 14:04:56 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 12 Oct 2010 13:04:56 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> <20101012012032.4175d945@pitrou.net> Message-ID: <4CB44EE8.7030204@voidspace.org.uk> On 12/10/2010 12:55, Paul Moore wrote: > On 12 October 2010 00:42, Giampaolo Rodol? wrote: >> I know. My point was you can't do it by default and installing a >> module is something even a less experienced user usually does. >> Typing "C:\PythonXX\pysetup" is harder compared to "setup.py install" >> and solving this problem by modifying your environment paths so that >> you can just type "pysetup" is something I would expect to be done by >> the MSI installer, not the user. > I would assume (am I wrong?) that the canonical way of installing > modules on Windows for "non-advanced" users under distutils2 would > still be to download and run a binary installer. > > Assuming that's the case, modifying paths to make sure pysetup is > available as a command is no harder than making Python itself > available. (Having said that, I'd still personally prefer to have the > distutils2 command be invoked by some form of python -m invocation). Sure, scripts like pysetup are typically installed into C:\PythonXY\Scripts on Windows. Adding this to the path is no harder than adding C:\PythonXY to the path - in fact it is *exactly* as hard. Some people have an issue that they have to do this *at all* though. Having the script invoked by "python -m ..." is no easier from this point of view, for it to work from the command line you still have to modify your path to be able to do it. Personally I would prefer a separate script, "pysetup install foo" is less annoying to type than "python -m distutils2.install foo" or "python -m setup install foo". All the best, Michael > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From georg at python.org Tue Oct 12 14:40:52 2010 From: georg at python.org (Georg Brandl) Date: Tue, 12 Oct 2010 14:40:52 +0200 Subject: [Python-Dev] [RELEASED] Python 3.2 alpha 3 Message-ID: <4CB45754.10600@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team, I'm happy to announce the third and final alpha preview release of Python 3.2. Python 3.2 is a continuation of the efforts to improve and stabilize the Python 3.x line. Since the final release of Python 2.7, the 2.x line will only receive bugfixes, and new features are developed for 3.x only. Since PEP 3003, the Moratorium on Language Changes, is in effect, there are no changes in Python's syntax and built-in types in Python 3.2. Development efforts concentrated on the standard library and support for porting code to Python 3. Highlights are: * numerous improvements to the unittest module * PEP 3147, support for .pyc repository directories * PEP 3149, support for version tagged dynamic libraries * an overhauled GIL implementation that reduces contention * many consistency and behavior fixes for numeric operations * countless fixes regarding string/unicode issues; among them full support for a bytes environment (filenames, environment variables) * a sysconfig module to access configuration information * a pure-Python implementation of the datetime module * additions to the shutil module, among them archive file support * improvements to pdb, the Python debugger For an extensive list of changes in 3.2, see Misc/NEWS in the Python distribution. To download Python 3.2 visit: http://www.python.org/download/releases/3.2/ 3.2 documentation can be found at: http://docs.python.org/3.2/ Please consider trying Python 3.2 with your code and reporting any bugs you may notice to: http://bugs.python.org/ Enjoy! - -- Georg Brandl, Release Manager georg at python.org (on behalf of the entire python-dev team and 3.2's contributors) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) iEYEARECAAYFAky0V1QACgkQN9GcIYhpnLCm3wCeJ5EUcv8lYu4Yj/obNvOsIpvC kXAAnR9znSCZwMoEvWwzernXWIxrhojM =UE9Z -----END PGP SIGNATURE----- From ziade.tarek at gmail.com Tue Oct 12 16:42:16 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 12 Oct 2010 16:42:16 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> <20101012012032.4175d945@pitrou.net> Message-ID: On Tue, Oct 12, 2010 at 1:55 PM, Paul Moore wrote: ... > I would assume (am I wrong?) that the canonical way of installing > modules on Windows for "non-advanced" users under distutils2 would > still be to download and run a binary installer. Yes this won't change. Regards Tarek From barry at python.org Tue Oct 12 16:59:51 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 12 Oct 2010 10:59:51 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CB39CC8.9050602@canterbury.ac.nz> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB39CC8.9050602@canterbury.ac.nz> Message-ID: <20101012105951.3ed845b5@mission> On Oct 12, 2010, at 12:24 PM, Greg Ewing wrote: >Giampaolo Rodol? wrote: > >> If that's the case what would I type in the command prompt in order to >> install a module? >> "C:\PythonXX\pysetup.exe"? >> If so I would strongly miss old "setup.py install". > >Another thing bothers me about this. With the current scheme, >if you have multiple Pythons available, it's easy to be sure >that you're installing into the right one, because it's the >one that you use to run setup.py. Whereas if installation is >performed by a different executable, there's a possibility >of them being out of sync. > >So I think I'd prefer some scheme involving 'python -m ...' >or some other option to Python itself, rather than a separate >executable. This is why I suggested that 'setup.sh' (or whatever) take a --python-version option to select the python executable to use. Whatever solution is implemented definitely needs to take the multiple-installed pythons into account. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From lutz at rmi.net Tue Oct 12 18:01:57 2010 From: lutz at rmi.net (lutz at rmi.net) Date: Tue, 12 Oct 2010 16:01:57 -0000 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes Message-ID: barry at python.org wrote in the full post below: > I'm reminded of a survey Guido conducted at some long past > Python conference. He asked (paraphrasing): raise your hand > if you think Python is changing too fast. Lots of hands went > up. Then he asked, raise your hand if you have a feature you > want to get in the next version. Lots of hands went up. When? I doubt that you'd get the same reaction today given the schism that 3.X has created. Regardless, this underscores much of what I'm trying to get across here. Python conference attendees are hardly representative of the user base at large. Even today, they are probably just 0.1% of the whole. This list's readership is an order of magnitude smaller still. Open doesn't mean all that much to those outside the 0.01% whose preferences set the agenda. I appreciate that some people here do indeed weigh compatibility carefully, and realize that there are multiple valid viewpoints on this issue. And regrettably, I have neither solutions nor time to give this thread the further attention it deserves. So my point is just this: Change for change's sake is truly not what most Python users want. If Python core developers want 3.X to become as popular as 2.X, they should be less concerned with posts on this list or hands at a conference, than with the feet of the masses whose votes will ultimately decide 3.X's fate. --Mark Lutz (http://learning-python.com, http://rmi.net/~lutz) > Date: Fri, 8 Oct 2010 14:20:32 -0400 > From: Barry Warsaw > To: python-dev at python.org > Subject: Re: [Python-Dev] Patch making the current email package (mostly) support bytes > > On Oct 08, 2010, at 03:44 PM, lutz at rmi.net wrote: > > >Ultimately, development in the open source world is driven by the > >very few with time to show up, rather than by the very many who > >depend on it. This can unfortunately lead to the perception > >of thrashing by end users. Some even come to see the net effect > >as not that much different from closed models. I have no solution > >to offer, except to underscore again that changes made here affect > >very many people who are too busy using Python to participate here. > >Especially given the still tentative state of 3.X, stability matters. > > I'm reminded of a survey Guido conducted at some long past Python conference. > He asked (paraphrasing): raise your hand if you think Python is changing too > fast. Lots of hands went up. Then he asked, raise your hand if you have a > feature you want to get in the next version. Lots of hands went up. > > I'm sympathetic to the view that changes in Python can be disruptive to end > users. The Python community itself takes this seriously too, as evidenced by > the language moratorium[*]. But OTOH, Python cannot stagnate and even fixing > things means changing things. The reality too is that Python releases come > out approximately every 18 months, and a year and a half can either seem like > an excruciatingly long time, or blink of the eye depending on which side of > the fence you stand on. > > Yes, stability matters, but Python 3 is still a new snakeling and I suspect > that as the pace of porting picks up, more changes will be necessary. Adding > new modules named like distutils2 or unittest2 is less than satisfying but > useful for keeping older APIs around. > > I'm sad to hear that some people think that our development model differs > little from closed source development. To me, nothing could be further from > the truth. But the adage does go "(s)he who does the work, decides", and this > is the forum for those who are doing the work. I think everyone here welcomes > advocates for under-represented Python communities, and their concerns should > be taken in consideration when changes are discussed. But ultimately, Python > must evolve to stay relevant or it will die. This is where competing design > trade-offs must be discussed. If not here, by us, then where and by whom? > > -Barry > > [*] Mostly instituted to allow alternative implementations to catch up, it > does necessarily slow the pace of changes visible to end users. From martin at v.loewis.de Tue Oct 12 19:33:52 2010 From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 12 Oct 2010 19:33:52 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CB4442C.8030308@voidspace.org.uk> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> <4CB4442C.8030308@voidspace.org.uk> Message-ID: <4CB49C00.2080804@v.loewis.de> > So as well as pysetup.py/.exe I would like pysetup-3.2.py / .exe on > Windows please. (I'd really like a python-3.2.exe as well.) Please submit a patch to the installer, then. I'm still skeptical about adding PATH, because a) I find that fairly invasive, and despise long paths myself (it hurts my eyes to see the list of directories that VS adds to MY path) b) it's actually fairly tricky to implement; in particular, removal on uninstallation is difficult. On the other hand, adding a versioned executable in some directory that is known to be on the path already is more easy, at least for a per-machine installation: I guess \windows\system32 would be the right location for such an executable. Placing files into system32 is fairly easy with MSI (except for possible WoW64 problems - what if you install both the 32-bit and the 64-bit Python 3.2 on the same machine). Regards, Martin From rdmurray at bitdance.com Tue Oct 12 20:17:54 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 12 Oct 2010 14:17:54 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CB49C00.2080804@v.loewis.de> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> <4CB4442C.8030308@voidspace.org.uk> <4CB49C00.2080804@v.loewis.de> Message-ID: <20101012181754.DBF6022B3B5@kimball.webabinitio.net> On Tue, 12 Oct 2010 19:33:52 +0200, =?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?= wrote: > > So as well as pysetup.py/.exe I would like pysetup-3.2.py / .exe on > > Windows please. (I'd really like a python-3.2.exe as well.) > > Please submit a patch to the installer, then. > > I'm still skeptical about adding PATH, because > a) I find that fairly invasive, and despise long paths myself > (it hurts my eyes to see the list of directories that VS adds to MY > path) I get annoyed by that on Gentoo, too. Gentoo, though, is using the path entries to control *which* version of various things run when I type a command at the prompt, IIUC, and it updates them when I change the active version of a program, and removes them correctly when a package is uninstalled. Gentoo doesn't use path for that for everything, though...for Python, when I change which version is active ("eselect python python2.5") it updates the symlinks rather than the path, and I like that much better. It's less fragile, too, since it means I don't have to do 'source /etc/profile' in every open shell after switching the active version of Python. I don't use Windows much, but on balance I think I'm with Martin here :) --David From mail at timgolden.me.uk Tue Oct 12 20:41:04 2010 From: mail at timgolden.me.uk (Tim Golden) Date: Tue, 12 Oct 2010 19:41:04 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101012181754.DBF6022B3B5@kimball.webabinitio.net> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> <4CB4442C.8030308@voidspace.org.uk> <4CB49C00.2080804@v.loewis.de> <20101012181754.DBF6022B3B5@kimball.webabinitio.net> Message-ID: <4CB4ABC0.6020406@timgolden.me.uk> On 12/10/2010 7:17 PM, R. David Murray wrote: > On Tue, 12 Oct 2010 19:33:52 +0200, =?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?= wrote: >>> So as well as pysetup.py/.exe I would like pysetup-3.2.py / .exe on >>> Windows please. (I'd really like a python-3.2.exe as well.) >> >> Please submit a patch to the installer, then. >> >> I'm still skeptical about adding PATH, because >> a) I find that fairly invasive, and despise long paths myself >> (it hurts my eyes to see the list of directories that VS adds to MY >> path) Assume, for the sake of the argument, that we patched the MSI so it (optionally) added the installing version of Python (and, optionally ./scripts) to the PATH. What, then, do we do with existing PATH entries which point to older/other Python installations? Option (a) says: clear them all out, because it's meaningless having more than one entry with a python.exe on it and the one we want must be this one because we've just ticked a box to say so. Option (b) says: don't mess with other entries on the PATH; it's not done. That said, the current installer switches an APPPATH entry and changes -- optionally -- the file associations to point to the installing version, so there is a precedent for ditching previous data. I'm actually +0 on the idea. An expert user who's trying to juggle different Python versions should be able to sort himself out. A naive user can use Start > Run > Python to get the current version (thanks to the APPPATH) and can use "program.py arg1 arg2" on the console to run program.py with the associated version. (Notwithstanding the bug which doesn't correctly redirect output via file associations) But all this is pie in the sky until someone actually integrates such a change to the MSI. Martin's clearly not going to since he doesn't like the idea. I'm actually +0.5 on including a script in tools\scripts (or wherever) which, when run, would set as current the version of Python which ran it. I have a roughly working version of such a thing; the problem is getting it to work with all the different Python versions and all the different Windows versions we support. TJG From martin at v.loewis.de Tue Oct 12 22:36:30 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 12 Oct 2010 22:36:30 +0200 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CB4ABC0.6020406@timgolden.me.uk> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB38F6E.4050202@trueblade.com> <4CB4442C.8030308@voidspace.org.uk> <4CB49C00.2080804@v.loewis.de> <20101012181754.DBF6022B3B5@kimball.webabinitio.net> <4CB4ABC0.6020406@timgolden.me.uk> Message-ID: <4CB4C6CE.3080700@v.loewis.de> > Assume, for the sake of the argument, that we patched the > MSI so it (optionally) added the installing version of Python > (and, optionally ./scripts) to the PATH. What, then, do we > do with existing PATH entries which point to older/other Python > installations? Option (a) says: clear them > all out, because it's meaningless having more than one entry > with a python.exe on it and the one we want must be this one > because we've just ticked a box to say so. Option (b) says: > don't mess with other entries on the PATH; it's not done. That, too. For the registry settings, overwriting them is an easy choice: it was the other installer that wrote the original entries (*very* likely so), so we "own" them and we can overwrite them. Running a repair installation on the original installer will revert that. With the PATH entry, it's not such an easy choice: the old entry may have been manually added, or it may have been added by the previous installer. Replacing it in the second case is again a straight-forward choice, but we don't know (unless we record somewhere - in the registry - that we added a PATH entry - perhaps an MSI Feature entry could tell us). Regards, Martin From steve at pearwood.info Tue Oct 12 23:22:07 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 13 Oct 2010 08:22:07 +1100 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: References: Message-ID: <201010130822.07629.steve@pearwood.info> On Wed, 13 Oct 2010 03:01:57 am lutz at rmi.net wrote: > So my point is just this: Change for change's sake is truly not > what most Python users want. ?If Python core developers want 3.X > to become as popular as 2.X, they should be less concerned with > posts on this list or hands at a conference, than with the feet > of the masses whose votes will ultimately decide 3.X's fate. I don't think anyone has ever suggested change for change's sake. If they have, I'd love to read the PEP for it. -- Steven D'Aprano From janssen at parc.com Wed Oct 13 04:00:00 2010 From: janssen at parc.com (Bill Janssen) Date: Tue, 12 Oct 2010 19:00:00 PDT Subject: [Python-Dev] PPC Tiger buildbot going down for an upgrade Message-ID: <15332.1286935200@parc.com> I've found a dual-processor G4 to run the PPC Tiger buildbot on (it's currently an old e Mac), and I plan to take this buildbot down tomorrow, Wednesday, to upgrade. Bill From stephen at xemacs.org Wed Oct 13 11:09:28 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 13 Oct 2010 18:09:28 +0900 Subject: [Python-Dev] Patch making the current email package (mostly) support bytes In-Reply-To: <201010130822.07629.steve@pearwood.info> References: <201010130822.07629.steve@pearwood.info> Message-ID: <87eibu31dj.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > I don't think anyone has ever suggested change for change's sake. If > they have, I'd love to read the PEP for it. Not to mention the BDFL's pronouncement message! From techtonik at gmail.com Wed Oct 13 15:55:03 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 13 Oct 2010 16:55:03 +0300 Subject: [Python-Dev] It is always loo late (Was: Relative imports in Py3k) Message-ID: On Mon, Oct 11, 2010 at 2:22 AM, R. David Murray wrote: >> >> I don't see why such awkward way should be necessary in Python 3k, >> which breaks backwards compatibility. Why it can't "just work" for my >> user story? > > Because you weren't around advocating and implementing a change when > Python 3 was developed? I will never have the time to advocate the changes for products I don't use (do you?), and I still don't use Python 3 - I am just looking forward to it. Is there a public list of annoyances for Python 3 that I can check to see if my change is already scheduled for Python 4 and vote for it? > It's too late now to arbitrarily break backward compatibility, so you'll > have to advocate and develop any change inside the parameters of Python's > normal backward compatibility policy. I use Python a lot and I still can't change anything I dislike, because it is always too late. > In other words, you should move this discussion to python-ideas. It is more about the development process than Python itself. Should I move to ideas? It is yet another ML I am not subscribed. -- anatoly t. From solipsis at pitrou.net Wed Oct 13 16:24:46 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 13 Oct 2010 16:24:46 +0200 Subject: [Python-Dev] It is always loo late (Was: Relative imports in Py3k) References: Message-ID: <20101013162446.2d4a8215@pitrou.net> On Wed, 13 Oct 2010 16:55:03 +0300 anatoly techtonik wrote: > > Is there a public list of annoyances for Python 3 that I can check to > see if my change is already scheduled for Python 4 and vote for it? No, there isn't. If you want to know what Python 3 is about, you can : - read the docs - read the changelogs - download it and try it (especially alpha/beta versions) - read python-dev and python-checkins - read the source code - follow the bug tracker If you aren't willing to do anything of that, then I don't think your complaints belong here. comp.lang.python (also named python-list, or gmane.comp.python.general) is a more appropriate venue for your messages. > I use Python a lot and I still can't change anything I dislike, > because it is always too late. And that's a good thing, because Python is not your toy project and development of Python is not about satisfying your personal tastes. Antoine. From janssen at parc.com Wed Oct 13 18:01:58 2010 From: janssen at parc.com (Bill Janssen) Date: Wed, 13 Oct 2010 09:01:58 PDT Subject: [Python-Dev] sad state of OS X Python testing... In-Reply-To: <42516.1286045405@parc.com> References: <42516.1286045405@parc.com> Message-ID: <16926.1286985718@parc.com> Sorry to drop out like that -- I've been having some email issues. Stephen, great job; thanks for providing that Snow Leopard buildbot. I think what we're missing now is an Intel Leopard buildbot. 35% of Mac users are still running Leopard. I'm running it myself on some machines, due to the NIS issues in Snow Leopard. And, of course, we'd like to get some of these Mac buildbots into the "stable" category, so that they are consulted for releases. The PPC Leopard and x86 Snow Leopard buildbots look like good candidates. Bill From martin at v.loewis.de Wed Oct 13 22:55:21 2010 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 13 Oct 2010 22:55:21 +0200 Subject: [Python-Dev] Stable build slaves authority Message-ID: <4CB61CB9.1060702@v.loewis.de> I have appointed Antoine Pitrou as the authority/manager for which build slave are considered stable. If you want to get a certain slave elevated or demoted, you have to convince him. I would also like to ask release managers to take the stable list serious: test failures in a stable slave should be considered as release blockers (demoting a slave to unstable may be a resolution, though). Of course, release managers can deliberately chose to ignore specific blockers, anyway. Regards, Martin From martin at v.loewis.de Wed Oct 13 22:55:56 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 13 Oct 2010 22:55:56 +0200 Subject: [Python-Dev] sad state of OS X Python testing... In-Reply-To: <16926.1286985718@parc.com> References: <42516.1286045405@parc.com> <16926.1286985718@parc.com> Message-ID: <4CB61CDC.3080403@v.loewis.de> > And, of course, we'd like to get some of these Mac buildbots into the > "stable" category, so that they are consulted for releases. The > PPC Leopard and x86 Snow Leopard buildbots look like good candidates. See the message I just sent: you need to convince Antoine, and he'll arrange it. Regards, Martin From solipsis at pitrou.net Wed Oct 13 23:47:10 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 13 Oct 2010 23:47:10 +0200 Subject: [Python-Dev] Stable build slaves authority References: <4CB61CB9.1060702@v.loewis.de> Message-ID: <20101013234710.2ea1d91b@pitrou.net> On Wed, 13 Oct 2010 22:55:21 +0200 "Martin v. L?wis" wrote: > I have appointed Antoine Pitrou as the authority/manager > for which build slave are considered stable. If you want > to get a certain slave elevated or demoted, you have to > convince him. Thank you Martin! I've added a couple of build slaves to the stable list: - x86 Windows7 (from David Bolen) - x86 Ubuntu (8.04.3 LTS, from David Bolen) - AMD64 Gentoo (from Dirkjan Ochtman) - x86 Snow Leopard (from Stephen Hansen) This is in hope that the stable list represents a greater diversity of systems and architectures. If this turns out to be too ambitious, I'll have to shrink the stable list again. (you'll notice that we have currently no 64-bit Windows machine although 64-bit support under Windows has specific issues) I would like to stress that help from the buildbot owners is welcome when some platform-specific issues need investigating and solving. Being proactive is even better! Regards Antoine. From me+python at ixokai.io Thu Oct 14 00:08:24 2010 From: me+python at ixokai.io (Stephen Hansen) Date: Wed, 13 Oct 2010 15:08:24 -0700 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: <20101013234710.2ea1d91b@pitrou.net> References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> Message-ID: <4CB62DD8.8080307@ixokai.io> On 10/13/10 2:47 PM, Antoine Pitrou wrote: > (you'll notice that we have currently no 64-bit Windows machine although > 64-bit support under Windows has specific issues) Provided its not a problem that its a VM, I have a hefty 64-bit Win7 Professional instance that I can put a buildslave on. Despite being a VM it gets ownership of two cores and 4 gigs of RAM, so should be plenty fast to handle the load. And I do run it 24/7. -- Stephen Hansen ... Also: Ixokai ... Mail: me+python (AT) ixokai (DOT) io ... Blog: http://meh.ixokai.io/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: OpenPGP digital signature URL: From martin at v.loewis.de Thu Oct 14 00:14:21 2010 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 14 Oct 2010 00:14:21 +0200 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: <4CB62DD8.8080307@ixokai.io> References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> Message-ID: <4CB62F3D.50801@v.loewis.de> Am 14.10.2010 00:08, schrieb Stephen Hansen: > On 10/13/10 2:47 PM, Antoine Pitrou wrote: >> (you'll notice that we have currently no 64-bit Windows machine although >> 64-bit support under Windows has specific issues) > > Provided its not a problem that its a VM, I have a hefty 64-bit Win7 > Professional instance that I can put a buildslave on. Despite being a VM > it gets ownership of two cores and 4 gigs of RAM, so should be plenty > fast to handle the load. And I do run it 24/7. So far, we didn't have problems with VMs. Please be aware that Windows poses its own challenges. Often, builds or testsuite runs end up with popup windows, which then hang subsequent builds. You often get dozens of them to click away. So operating a Windows slave is much more tedious than a Unix one. Regards, Martin From solipsis at pitrou.net Thu Oct 14 00:16:01 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 14 Oct 2010 00:16:01 +0200 Subject: [Python-Dev] Stable build slaves authority References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> Message-ID: <20101014001601.4452af07@pitrou.net> On Wed, 13 Oct 2010 15:08:24 -0700 Stephen Hansen wrote: > On 10/13/10 2:47 PM, Antoine Pitrou wrote: > > (you'll notice that we have currently no 64-bit Windows machine although > > 64-bit support under Windows has specific issues) > > Provided its not a problem that its a VM, I have a hefty 64-bit Win7 > Professional instance that I can put a buildslave on. Despite being a VM > it gets ownership of two cores and 4 gigs of RAM, so should be plenty > fast to handle the load. And I do run it 24/7. Sounds good. Now of course you have to setup a buildslave on it, and ask Martin to register the slave on the master (unless I'm supposed to do it myself now?). Regards Antoine. From me+python at ixokai.io Thu Oct 14 00:25:32 2010 From: me+python at ixokai.io (Stephen Hansen) Date: Wed, 13 Oct 2010 15:25:32 -0700 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: <4CB62F3D.50801@v.loewis.de> References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> <4CB62F3D.50801@v.loewis.de> Message-ID: <4CB631DC.4020102@ixokai.io> On 10/13/10 3:14 PM, "Martin v. L?wis" wrote: > Am 14.10.2010 00:08, schrieb Stephen Hansen: >> On 10/13/10 2:47 PM, Antoine Pitrou wrote: >>> (you'll notice that we have currently no 64-bit Windows machine although >>> 64-bit support under Windows has specific issues) >> >> Provided its not a problem that its a VM, I have a hefty 64-bit Win7 >> Professional instance that I can put a buildslave on. Despite being a VM >> it gets ownership of two cores and 4 gigs of RAM, so should be plenty >> fast to handle the load. And I do run it 24/7. > > So far, we didn't have problems with VMs. > > Please be aware that Windows poses its own challenges. Often, builds > or testsuite runs end up with popup windows, which then hang subsequent > builds. You often get dozens of them to click away. So operating a > Windows slave is much more tedious than a Unix one. Windows always poses its own challenges. :) That's why I have the VM (and three others for older versions of windows, that just aren't on all the time like that one is) to begin with*, for testing out my day-job work. I'll give it a go; I have all the software needed to run the buildbot on it already besides VC Express, which I'm installing now. If ultimately it becomes too much of a pain, I'll go back to just providing the mac. But, I actually have a vested interest in upgrading our Python to 64-bit in the next few months, so! I'm motivated. I'll let you know when I have everything installed so you can add a buildslave account. -- Stephen Hansen ... Also: Ixokai ... Mail: me+python (AT) ixokai (DOT) io ... Blog: http://meh.ixokai.io/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: OpenPGP digital signature URL: From alexander.belopolsky at gmail.com Thu Oct 14 00:34:27 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 13 Oct 2010 18:34:27 -0400 Subject: [Python-Dev] Add aware local time support to datetime module In-Reply-To: References: Message-ID: It looks like this response has slipped under my radar. Sorry. On Sun, Aug 8, 2010 at 4:37 AM, Lennart Regebro wrote: [skipped description of two alternative solutions] .. > For all of the reasons you give above, I think it's a bad idea. :-) > I did not give any reason for not having access to the information that is readily available through reasonably cross-platform C API. Can you elaborate on what you think is a bad idea? > You need a proper time zone database to solve these issues, like pytz. Pytz is more ambitious than what I propose. I don't propose adding a timezone database to python or even a functionality to access system timezone database. All I want to add is a method to get aware local time in the system timezone. > Which incidentally solves the ambiguity problem as well. so the > solution is pytz. :-) No, it does not. In order to resolve the ambiguity, pytz adds a non-standard argument to tzinfo methods, but datetime objects don't know how to pass this argument and don't have data to determine its value to begin with. > What pytz doesn't have (but dateutil.tz does) is a timezone object > that uses the operating systems local timezone data, like > /etc/localzone on unix. That could be interesting to include, > possibly. Having a fixed time zone offset object for the localtime > seems a bad idea. The problem it solves is easy to get around, but the > problems created are not. This sounds like an argument against my "second alternative." As I explained, my preference is the same. Do you have an opinion on the localtime([t]) alternative? From martin at v.loewis.de Thu Oct 14 00:42:23 2010 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 14 Oct 2010 00:42:23 +0200 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: <4CB631DC.4020102@ixokai.io> References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> <4CB62F3D.50801@v.loewis.de> <4CB631DC.4020102@ixokai.io> Message-ID: <4CB635CF.7060801@v.loewis.de> > I'll give it a go; I have all the software needed to run the buildbot on > it already besides VC Express, which I'm installing now. If ultimately > it becomes too much of a pain, I'll go back to just providing the mac. > But, I actually have a vested interest in upgrading our Python to 64-bit > in the next few months, so! I'm motivated. That won't work, will it? VC Express doesn't come with an AMD64 compiler (I *think* it's possible to use the SDK one, but this again is more complicated). Regards, Martin From me+python at ixokai.io Thu Oct 14 00:46:50 2010 From: me+python at ixokai.io (Stephen Hansen) Date: Wed, 13 Oct 2010 15:46:50 -0700 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: <4CB635CF.7060801@v.loewis.de> References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> <4CB62F3D.50801@v.loewis.de> <4CB631DC.4020102@ixokai.io> <4CB635CF.7060801@v.loewis.de> Message-ID: <4CB636DA.1020300@ixokai.io> On 10/13/10 3:42 PM, "Martin v. L?wis" wrote: >> I'll give it a go; I have all the software needed to run the buildbot on >> it already besides VC Express, which I'm installing now. If ultimately >> it becomes too much of a pain, I'll go back to just providing the mac. >> But, I actually have a vested interest in upgrading our Python to 64-bit >> in the next few months, so! I'm motivated. > > That won't work, will it? VC Express doesn't come with an AMD64 > compiler (I *think* it's possible to use the SDK one, but this again > is more complicated). Oh! Well if it takes a paid version of VS, then I won't be able to do it. I'll experiment with getting the SDK and using that and seeing if I can make it work. -- Stephen Hansen ... Also: Ixokai ... Mail: me+python (AT) ixokai (DOT) io ... Blog: http://meh.ixokai.io/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: OpenPGP digital signature URL: From brian.curtin at gmail.com Thu Oct 14 00:50:49 2010 From: brian.curtin at gmail.com (Brian Curtin) Date: Wed, 13 Oct 2010 17:50:49 -0500 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: <4CB635CF.7060801@v.loewis.de> References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> <4CB62F3D.50801@v.loewis.de> <4CB631DC.4020102@ixokai.io> <4CB635CF.7060801@v.loewis.de> Message-ID: On Wed, Oct 13, 2010 at 17:42, "Martin v. L?wis" wrote: > > I'll give it a go; I have all the software needed to run the buildbot on > > it already besides VC Express, which I'm installing now. If ultimately > > it becomes too much of a pain, I'll go back to just providing the mac. > > But, I actually have a vested interest in upgrading our Python to 64-bit > > in the next few months, so! I'm motivated. > > That won't work, will it? VC Express doesn't come with an AMD64 > compiler (I *think* it's possible to use the SDK one, but this again > is more complicated). > > Regards, > Martin > Correct. There are a few hacky ways to get Express to use the x64 SDK, or so I read. I have a Server 2008 R2 x64 box with the full Visual Studio that I could add to the buildbot fleet. It's a dual core with 4 GB of RAM, plenty of disk space, and it runs 24/7. I'll see about getting that setup as a build slave. -------------- next part -------------- An HTML attachment was scrubbed... URL: From db3l.net at gmail.com Thu Oct 14 00:59:19 2010 From: db3l.net at gmail.com (David Bolen) Date: Wed, 13 Oct 2010 18:59:19 -0400 Subject: [Python-Dev] Stable build slaves authority References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> <4CB62F3D.50801@v.loewis.de> Message-ID: "Martin v. L?wis" writes: > Please be aware that Windows poses its own challenges. Often, builds > or testsuite runs end up with popup windows, which then hang subsequent > builds. You often get dozens of them to click away. So operating a > Windows slave is much more tedious than a Unix one. For anyone else considering a Windows buildbot, recently I started using an AutoIt script in the background to automatically clear any RTL dialogs (which seems to come and go as an issue but got really bad recently in one of the branches, and I lost track of when Python itself stopped disabling them during tests), and it's been working well so far. I also have a tweak in the local buildbot code to disable all Windows-based (non-RTL) dialogs during test runs. I'd be happy to provide both bits to anyone else starting a Windows buildbot. There's still a regular problem of stranded python_d processes in the background (something shared with my OSX tiger slave, but there I can run a script to detect processes owned by init and kill them). So periodic manual checks/cleanup is still definitely needed. -- David From asmodai at in-nomine.org Thu Oct 14 07:28:22 2010 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 14 Oct 2010 07:28:22 +0200 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> <4CB62F3D.50801@v.loewis.de> <4CB631DC.4020102@ixokai.io> <4CB635CF.7060801@v.loewis.de> Message-ID: <20101014052822.GI75788@nexus.in-nomine.org> -On [20101014 00:55], Brian Curtin (brian.curtin at gmail.com) wrote: >Correct. There are a few hacky ways to get Express to use the x64 SDK, or so I >read. I think Martin meant that you wouldn't need VS Express if you install the Windows SDK, since it provides all the tools in the SDK to build Python. Sorry Martin, haven't had the time at work to test the latest trunks out with just the SDK. I'll see if I can free some time for that in the coming days. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Seize from every moment its unique novelty and do not prepare your joys... From me+python at ixokai.io Thu Oct 14 08:00:05 2010 From: me+python at ixokai.io (Stephen Hansen) Date: Wed, 13 Oct 2010 23:00:05 -0700 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: <20101014052822.GI75788@nexus.in-nomine.org> References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> <4CB62F3D.50801@v.loewis.de> <4CB631DC.4020102@ixokai.io> <4CB635CF.7060801@v.loewis.de> <20101014052822.GI75788@nexus.in-nomine.org> Message-ID: <4CB69C65.6000403@ixokai.io> On 10/13/10 10:28 PM, Jeroen Ruigrok van der Werven wrote: > -On [20101014 00:55], Brian Curtin (brian.curtin at gmail.com) wrote: >> Correct. There are a few hacky ways to get Express to use the x64 SDK, or so I >> read. > > I think Martin meant that you wouldn't need VS Express if you install the > Windows SDK, since it provides all the tools in the SDK to build Python. There's mixed signals here, and I'm not sure what they all mean. I have a Win7-64bit box that I am willing to use to run a buildslave, if its possible to do so. #python-dev thought that VS express was all that was needed; then here, it seemed to me that Martin said that you needed the full version of VS or perhaps a complex setup with the SDK compiler; but you seem to be interpreting Martin that the SDK provides everything and nothing else is needed. Then again on top of that, my offer may be mooted-- if Brian Curtin is going to host a x86_64 windows slave then I don't need to worry about this because its being provided otherwise. I'm willing to put up with the particular windows-specific difficulties that go with running a buildslave (especially with David Bolen's AutoIt scripts which may ease things): but I'm not entirely sure from these varied results if its even possible or needed. So, my questions are: 1. Is someone else (Hi, Brian) providing a 64-bit windows slave, so there isn't actually any need for me to go through the effort of it? 2. If not, is all that's needed is the SDK to build 64-bit Python? 3. Or, does one have to use a combination of VS-Express + the SDK in a "hack"y way (as some seem to claim, but this last mail seems to indicate otherwise) to get it done? Basically, it comes down to: 'it' being a 64-bit windows slave, is it actually needed from me (i.e., is a more apt expert not providing it), and can anyone actually say what the requirements are for making it happen? At the moment I'm uncertain if its even needed or worthwhile to go through the effort to get the whole visual studio environment set up. I have computing resources, cycles, and time that's free to offer up: but the differing responses here makes me unsure if I'm being useful or not in trying here :) -- Stephen Hansen ... Also: Ixokai ... Mail: me+python (AT) ixokai (DOT) io ... Blog: http://meh.ixokai.io/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: OpenPGP digital signature URL: From g.brandl at gmx.net Thu Oct 14 08:07:46 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 14 Oct 2010 08:07:46 +0200 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: <4CB61CB9.1060702@v.loewis.de> References: <4CB61CB9.1060702@v.loewis.de> Message-ID: Am 13.10.2010 22:55, schrieb "Martin v. L?wis": > I have appointed Antoine Pitrou as the authority/manager > for which build slave are considered stable. If you want > to get a certain slave elevated or demoted, you have to > convince him. > > I would also like to ask release managers to take the > stable list serious: test failures in a stable slave should > be considered as release blockers (demoting a slave > to unstable may be a resolution, though). Of course, > release managers can deliberately chose to ignore specific > blockers, anyway. Very nice. http://www.python.org/dev/buildbot/stable/ is completely green at the moment -- which means that I can now indeed take failures seriously in the future. Previously, two of four "stables" for py3k were not even connected for ages (and yes, I should have complained earlier. As we progress into the beta stage, I will take buildbot failures and release blockers more seriously). cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From martin at v.loewis.de Thu Oct 14 09:37:17 2010 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 14 Oct 2010 09:37:17 +0200 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: <4CB69C65.6000403@ixokai.io> References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> <4CB62F3D.50801@v.loewis.de> <4CB631DC.4020102@ixokai.io> <4CB635CF.7060801@v.loewis.de> <20101014052822.GI75788@nexus.in-nomine.org> <4CB69C65.6000403@ixokai.io> Message-ID: <4CB6B32D.5060506@v.loewis.de> > #python-dev thought that VS express was all that was needed; then here, > it seemed to me that Martin said that you needed the full version of VS > or perhaps a complex setup with the SDK compiler; but you seem to be > interpreting Martin that the SDK provides everything and nothing else is > needed. I think I forgot earlier discussion on how to use the SDK. It may well be that it can be used to build Python with no VS installed, but it's not what I meant - I meant that I actually don't know. > 1. Is someone else (Hi, Brian) providing a 64-bit windows slave, so > there isn't actually any need for me to go through the effort of it? With the Windows build slaves: they come and go. So having two of them might be a good idea. > 2. If not, is all that's needed is the SDK to build 64-bit Python? I think that you will need to find out on your own. And, even if it's possible, the next question is whether what the buildbot scripts do actually works without VS (but we could adjust them). > Basically, it comes down to: 'it' being a 64-bit windows slave, is it > actually needed from me (i.e., is a more apt expert not providing it), > and can anyone actually say what the requirements are for making it > happen? As for "needed", well, we a are all volunteers. This project will certainly survive without a 64-bit Windows build slave (it did so in the last months). As for saying what the requirements are: probably nobody can. > I have computing resources, cycles, and time that's free to offer up: > but the differing responses here makes me unsure if I'm being useful or > not in trying here :) Well, the differing responses really demonstrate that it's also expertise that is lacking throughout. If you are motivated, please try to find out how to build Python on AMD64 with free-as-in-beer Microsoft tools, and provide a patch to PCbuild/readme.txt. Having such a setup then tested regularly is certainly worthwhile. Regards, Martin From kalman.gergely at duodecad.hu Thu Oct 14 10:45:25 2010 From: kalman.gergely at duodecad.hu (=?ISO-8859-1?Q?K=E1lm=E1n_Gergely?=) Date: Thu, 14 Oct 2010 10:45:25 +0200 Subject: [Python-Dev] socket.fromfd() documentation problem In-Reply-To: <201010061412.48549.victor.stinner@haypocalc.com> References: <4CAC266D.7040105@duodecad.hu> <201010061412.48549.victor.stinner@haypocalc.com> Message-ID: <4CB6C325.7040104@duodecad.hu> On 10/06/10 14:12, Victor Stinner wrote: > Le mercredi 06 octobre 2010 09:34:05, K?lm?n Gergely a ?crit : > >> Nevertheless what are your thoughts on this? Should I file a bug report >> for it? >> > It will be fixed faster if you open an issue and attach a patch ;-) > > Just did: http://bugs.python.org/issue10099 Kalman Gergely From solipsis at pitrou.net Thu Oct 14 11:25:39 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 14 Oct 2010 11:25:39 +0200 Subject: [Python-Dev] A new warning category? Message-ID: <20101014112539.70cc54c7@pitrou.net> Hello, In the http://bugs.python.org/issue10093 discussion, I proposed to add a specific warning category for unclosed files. The rationale is that these warnings will happen in destructors and therefore filtering by line number and filename doesn't make sense. So a new category would be useful in order to allow defining specific rules. Do you think it would go against the moratorium? As for the category name, I would suggest ResourceWarning if we use it specifically for resource-consumption warnings. Or perhaps DebugWarning if we want to put all kinds of debugging helpers in it. Regards Antoine. From solipsis at pitrou.net Thu Oct 14 11:53:47 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 14 Oct 2010 11:53:47 +0200 Subject: [Python-Dev] A new warning category? References: <20101014112539.70cc54c7@pitrou.net> Message-ID: <20101014115347.5528db61@pitrou.net> On Thu, 14 Oct 2010 11:25:39 +0200 Antoine Pitrou wrote: > > Hello, > > In the http://bugs.python.org/issue10093 discussion, I proposed to add a > specific warning category for unclosed files. The rationale is that > these warnings will happen in destructors and therefore filtering by > line number and filename doesn't make sense. So a new category would be > useful in order to allow defining specific rules. > Do you think it would go against the moratorium? Come to think of it, another reason for a separate warning category is that we problably want those disabled by default (unless we're running in pydebug mode perhaps). Regards Antoine. From steve at pearwood.info Thu Oct 14 12:33:34 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 14 Oct 2010 21:33:34 +1100 Subject: [Python-Dev] A new warning category? In-Reply-To: <20101014112539.70cc54c7@pitrou.net> References: <20101014112539.70cc54c7@pitrou.net> Message-ID: <201010142133.35798.steve@pearwood.info> On Thu, 14 Oct 2010 08:25:39 pm Antoine Pitrou wrote: > Hello, > > In the http://bugs.python.org/issue10093 discussion, I proposed to > add a specific warning category for unclosed files. The rationale is > that these warnings will happen in destructors and therefore > filtering by line number and filename doesn't make sense. So a new > category would be useful in order to allow defining specific rules. Sounds like a reasonable suggestion to me. > Do you think it would go against the moratorium? It's not clear to me one way or the other, but I suspect yes. -- Steven D'Aprano From barry at python.org Thu Oct 14 16:03:33 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 14 Oct 2010 10:03:33 -0400 Subject: [Python-Dev] [Python-checkins] r85463 - python/branches/py3k/Lib/test/test_import.py In-Reply-To: <20101014073456.A58BEDFCB@mail.python.org> References: <20101014073456.A58BEDFCB@mail.python.org> Message-ID: <20101014100333.649ac0e2@mission> On Oct 14, 2010, at 09:34 AM, georg.brandl wrote: >Author: georg.brandl >Date: Thu Oct 14 09:34:56 2010 >New Revision: 85463 > >Log: >Better check for "any optimize option given". > >Modified: > python/branches/py3k/Lib/test/test_import.py > >Modified: python/branches/py3k/Lib/test/test_import.py >============================================================================== >--- python/branches/py3k/Lib/test/test_import.py (original) >+++ python/branches/py3k/Lib/test/test_import.py Thu Oct 14 09:34:56 2010 >@@ -521,7 +521,7 @@ > self.assertTrue(os.path.exists('__pycache__')) > self.assertTrue(os.path.exists(os.path.join( > '__pycache__', '{}.{}.py{}'.format( >- TESTFN, self.tag, sys.flags.optimize and 'o' or 'c')))) >+ TESTFN, self.tag, __debug__ and 'c' or 'o')))) You might want to use imp.cache_from_source() instead for consistency. > > @unittest.skipUnless(os.name == 'posix', > "test meaningful only on posix systems") -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From brian.curtin at gmail.com Thu Oct 14 17:08:12 2010 From: brian.curtin at gmail.com (Brian Curtin) Date: Thu, 14 Oct 2010 10:08:12 -0500 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: <4CB69C65.6000403@ixokai.io> References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> <4CB62F3D.50801@v.loewis.de> <4CB631DC.4020102@ixokai.io> <4CB635CF.7060801@v.loewis.de> <20101014052822.GI75788@nexus.in-nomine.org> <4CB69C65.6000403@ixokai.io> Message-ID: On Thu, Oct 14, 2010 at 01:00, Stephen Hansen > wrote: > On 10/13/10 10:28 PM, Jeroen Ruigrok van der Werven wrote: > > -On [20101014 00:55], Brian Curtin (brian.curtin at gmail.com) wrote: > >> Correct. There are a few hacky ways to get Express to use the x64 SDK, > or so I > >> read. > > > > I think Martin meant that you wouldn't need VS Express if you install the > > Windows SDK, since it provides all the tools in the SDK to build Python. > > There's mixed signals here, and I'm not sure what they all mean. I have > a Win7-64bit box that I am willing to use to run a buildslave, if its > possible to do so. > > #python-dev thought that VS express was all that was needed; then here, > it seemed to me that Martin said that you needed the full version of VS > or perhaps a complex setup with the SDK compiler; but you seem to be > interpreting Martin that the SDK provides everything and nothing else is > needed. > > Then again on top of that, my offer may be mooted-- if Brian Curtin is > going to host a x86_64 windows slave then I don't need to worry about > this because its being provided otherwise. > I'm willing to put up with the particular windows-specific difficulties > that go with running a buildslave (especially with David Bolen's AutoIt > scripts which may ease things): but I'm not entirely sure from these > varied results if its even possible or needed. > > So, my questions are: > 1. Is someone else (Hi, Brian) providing a 64-bit windows slave, so > there isn't actually any need for me to go through the effort of it? > I'm planning to -- just need a little time to sit down and get the box ready to go. I would say that one build slave is good, two is better. If you feel up to the task, go right ahead. As one of the few Windows users around here, I'd certainly appreciate more Windows testing. One of my motivations in providing a build slave is execution of the newly added os.symlink support starting in Windows Vista. Since os.symlink requires a certain privilege, it's not tested on any of the current build slaves. I plan to work out a way for my build slave to get these tests executed, rather than just running them on my desktop boxes. > 2. If not, is all that's needed is the SDK to build 64-bit Python? Sorry for my confusion earlier - I was focused on the VS Express part, rather than the minimum requirement to build Python (which doesn't *require* VS at all, it's just a nice tool to build it). Supposedly the 64-bit SDK by itself should build a 64-bit Python -- I say "supposedly" because I haven't done this myself. I've read that only using the 32-bit SDK does this fine -- also have not done this myself. I understand that doesn't really answer your question, but hopefully my previous involvement seems more clear. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Thu Oct 14 18:44:42 2010 From: brett at python.org (Brett Cannon) Date: Thu, 14 Oct 2010 09:44:42 -0700 Subject: [Python-Dev] [Python-checkins] r85481 - in python/branches/py3k: Misc/NEWS configure.in In-Reply-To: <20101014152422.3FB61EE996@mail.python.org> References: <20101014152422.3FB61EE996@mail.python.org> Message-ID: Doesn't autoconf need to be run to regenerate configure? On Thu, Oct 14, 2010 at 08:24, matthias.klose wrote: > Author: matthias.klose > Date: Thu Oct 14 17:24:22 2010 > New Revision: 85481 > > Log: > - Issue #10094: Use versioned .so files on GNU/kfreeBSD and the GNU Hurd. > > > Modified: > ? python/branches/py3k/Misc/NEWS > ? python/branches/py3k/configure.in > > Modified: python/branches/py3k/Misc/NEWS > ============================================================================== > --- python/branches/py3k/Misc/NEWS ? ? ?(original) > +++ python/branches/py3k/Misc/NEWS ? ? ?Thu Oct 14 17:24:22 2010 > @@ -61,6 +61,11 @@ > > ?- Issue #7287: Demo/imputil/knee.py was removed. > > +Build > +----- > + > +- Issue #10094: Use versioned .so files on GNU/kfreeBSD and the GNU Hurd. > + > > ?What's New in Python 3.2 Alpha 3? > ?================================= > > Modified: python/branches/py3k/configure.in > ============================================================================== > --- python/branches/py3k/configure.in ? (original) > +++ python/branches/py3k/configure.in ? Thu Oct 14 17:24:22 2010 > @@ -3652,7 +3652,8 @@ > ? ? ? ? ? ? ? ?esac > ? ? ? ? ? ? ? ?;; > ? ? ? ?CYGWIN*) ? SO=.dll;; > - ? ? ? Linux*) ? ?SO=.${SOABI}.so;; > + ? ? ? Linux*|GNU*) > + ? ? ? ? ? ? ? ? ?SO=.${SOABI}.so;; > ? ? ? ?*) ? ? ? ? SO=.so;; > ? ? ? ?esac > ?else > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From brett at python.org Thu Oct 14 18:47:25 2010 From: brett at python.org (Brett Cannon) Date: Thu, 14 Oct 2010 09:47:25 -0700 Subject: [Python-Dev] A new warning category? In-Reply-To: <20101014112539.70cc54c7@pitrou.net> References: <20101014112539.70cc54c7@pitrou.net> Message-ID: On Thu, Oct 14, 2010 at 02:25, Antoine Pitrou wrote: > > Hello, > > In the http://bugs.python.org/issue10093 discussion, I proposed to add a > specific warning category for unclosed files. The rationale is that > these warnings will happen in destructors and therefore filtering by > line number and filename doesn't make sense. So a new category would be > useful in order to allow defining specific rules. > Do you think it would go against the moratorium? As one of the co-authors of the PEP I say no. > > As for the category name, I would suggest ResourceWarning if we use it > specifically for resource-consumption warnings. Or perhaps DebugWarning > if we want to put all kinds of debugging helpers in it. I say start with ResourceWarning and if we decide to generalize we can make ResourceWarning subclass DebugWarning without breaking code. From solipsis at pitrou.net Thu Oct 14 18:55:36 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 14 Oct 2010 18:55:36 +0200 Subject: [Python-Dev] Stable build slaves authority References: <4CB61CB9.1060702@v.loewis.de> Message-ID: <20101014185536.702dbf3c@pitrou.net> On Thu, 14 Oct 2010 08:07:46 +0200 Georg Brandl wrote: > > Very nice. http://www.python.org/dev/buildbot/stable/ is completely > green at the moment -- which means that I can now indeed take failures > seriously in the future. Previously, two of four "stables" for py3k > were not even connected for ages (and yes, I should have complained > earlier. As we progress into the beta stage, I will take buildbot > failures and release blockers more seriously). For the record, I've setup one of the stable buildbots ("AMD64 Gentoo Wide") to test wide unicode buils. Also, one of the unstable buildbots ("x86 debian parallel") now runs its tests with -OO, at Georg's request. Regards Antoine. From barry at python.org Thu Oct 14 19:04:01 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 14 Oct 2010 13:04:01 -0400 Subject: [Python-Dev] Issue 10094 Message-ID: <20101014130401.3a2e487b@mission> Posting this here first, though it's looking less like a Python bug and more like an environment problem, or issue with something in Ubuntu. I'm running the regular test suite for the py3k branch and seeing this failure on Ubuntu 10.10: http://bugs.python.org/issue10094 test_urllib.py fails with RuntimeError on Ubuntu 10.10 It's completely consistent. It happens with the regular test suite, or just running test_urllib.py, and it does not happen with any other test afaict. I've gone way back in svn, pulled down earlier alphas that I know worked and they all exhibit the bug. This only happens on Ubuntu 10.10 machines (afaict so far), both i386 and x86_64. It does not happen on Ubuntu 10.04 64bit VMs, nor does it happen on Debian squeeze. I'll continue to investigate, but since it's damn perplexing I just wanted to see if anybody had any thoughts on the problem. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From guido at python.org Thu Oct 14 19:29:12 2010 From: guido at python.org (Guido van Rossum) Date: Thu, 14 Oct 2010 10:29:12 -0700 Subject: [Python-Dev] Issue 10094 In-Reply-To: <20101014130401.3a2e487b@mission> References: <20101014130401.3a2e487b@mission> Message-ID: Could it be IPv6? On Thu, Oct 14, 2010 at 10:04 AM, Barry Warsaw wrote: > Posting this here first, though it's looking less like a Python bug and more > like an environment problem, or issue with something in Ubuntu. > > I'm running the regular test suite for the py3k branch and seeing this failure > on Ubuntu 10.10: > > http://bugs.python.org/issue10094 > test_urllib.py fails with RuntimeError on Ubuntu 10.10 > > It's completely consistent. ?It happens with the regular test suite, or just > running test_urllib.py, and it does not happen with any other test afaict. > I've gone way back in svn, pulled down earlier alphas that I know worked and > they all exhibit the bug. > > This only happens on Ubuntu 10.10 machines (afaict so far), both i386 and > x86_64. ?It does not happen on Ubuntu 10.04 64bit VMs, nor does it happen on > Debian squeeze. > > I'll continue to investigate, but since it's damn perplexing I just wanted to > see if anybody had any thoughts on the problem. > > Cheers, > -Barry > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > > -- --Guido van Rossum (python.org/~guido) From orsenthil at gmail.com Thu Oct 14 19:37:01 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 14 Oct 2010 23:07:01 +0530 Subject: [Python-Dev] Issue 10094 In-Reply-To: References: <20101014130401.3a2e487b@mission> Message-ID: <20101014173701.GB13994@remy> On Thu, Oct 14, 2010 at 10:29:12AM -0700, Guido van Rossum wrote: > Could it be IPv6? The error message says, File "Lib/test/test_urllib.py", line 121, in setUp for k in os.environ.keys(): File "/home/barry/projects/python/py3k/Lib/_abcoll.py", line 410, in __iter__ for key in self._mapping: File "/home/barry/projects/python/py3k/Lib/os.py", line 441, in __iter__ for key in self._data: RuntimeError: dictionary changed size during iteration I doubt if it is related to underlying IP. I thought it was with test runner (or threads) and the way we use EnvironGuard in the tests. I have observed in a very inconsistent manner earlier, surprising that is always failing on Ubuntu 10.10. -- Senthil From barry at python.org Thu Oct 14 19:37:20 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 14 Oct 2010 13:37:20 -0400 Subject: [Python-Dev] Issue 10094 In-Reply-To: References: <20101014130401.3a2e487b@mission> Message-ID: <20101014133720.39de27c6@mission> On Oct 14, 2010, at 10:29 AM, Guido van Rossum wrote: >Could it be IPv6? I don't think so. I have IPv6 disabled on at least one of the machines. Also, I'm sure this failure did not occur before Ubuntu 10.10 final. It also fails on Python 3.1. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From solipsis at pitrou.net Thu Oct 14 19:38:58 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 14 Oct 2010 19:38:58 +0200 Subject: [Python-Dev] Issue 10094 References: <20101014130401.3a2e487b@mission> Message-ID: <20101014193858.3a8231f4@pitrou.net> On Thu, 14 Oct 2010 13:04:01 -0400 Barry Warsaw wrote: > Posting this here first, though it's looking less like a Python bug and more > like an environment problem, or issue with something in Ubuntu. > > I'm running the regular test suite for the py3k branch and seeing this failure > on Ubuntu 10.10: > > http://bugs.python.org/issue10094 > test_urllib.py fails with RuntimeError on Ubuntu 10.10 > > It's completely consistent. It happens with the regular test suite, or just > running test_urllib.py, and it does not happen with any other test afaict. An easy way to reproduce is to have an environment variable named "PROXY": $ PROXY=toto ./python -m test.regrtest -F test_urllib [ 1] test_urllib Warning -- os.environ was modified by test_urllib test test_urllib failed -- Traceback (most recent call last): File "/home/antoine/py3k/debug/Lib/test/test_urllib.py", line 121, in setUp for k in os.environ.keys(): File "/home/antoine/py3k/debug/Lib/_abcoll.py", line 410, in __iter__ for key in self._mapping: File "/home/antoine/py3k/debug/Lib/os.py", line 441, in __iter__ for key in self._data: RuntimeError: dictionary changed size during iteration There doesn't seem to be anything really mysterious, actually. The exception message says it all :) Regards Antoine. From orsenthil at gmail.com Thu Oct 14 19:48:47 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 14 Oct 2010 23:18:47 +0530 Subject: [Python-Dev] Issue 10094 In-Reply-To: <20101014193858.3a8231f4@pitrou.net> References: <20101014130401.3a2e487b@mission> <20101014193858.3a8231f4@pitrou.net> Message-ID: <20101014174847.GD13994@remy> On Thu, Oct 14, 2010 at 07:38:58PM +0200, Antoine Pitrou wrote: > An easy way to reproduce is to have an environment variable named > "PROXY": > > $ PROXY=toto ./python -m test.regrtest -F test_urllib > [ 1] test_urllib > Warning -- os.environ was modified by test_urllib > test test_urllib failed -- Traceback (most recent call last): > File "/home/antoine/py3k/debug/Lib/test/test_urllib.py", line 121, in > setUp for k in os.environ.keys(): > File "/home/antoine/py3k/debug/Lib/_abcoll.py", line 410, in __iter__ > for key in self._mapping: > File "/home/antoine/py3k/debug/Lib/os.py", line 441, in __iter__ > for key in self._data: > RuntimeError: dictionary changed size during iteration > > There doesn't seem to be anything really mysterious, actually. The > exception message says it all :) That is indeed the case. It a bug to be fixed. -- Senthil From daniel at stutzbachenterprises.com Thu Oct 14 19:57:48 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 14 Oct 2010 12:57:48 -0500 Subject: [Python-Dev] [Python-checkins] r85486 - python/branches/py3k/configure In-Reply-To: <20101014173847.348EBEE99C@mail.python.org> References: <20101014173847.348EBEE99C@mail.python.org> Message-ID: On Thu, Oct 14, 2010 at 12:38 PM, barry.warsaw wrote: > -# Generated by GNU Autoconf 2.65 for python 3.2. > +# Generated by GNU Autoconf 2.67 for python 3.2. > Was the change in autoconf versions intentional and/or is it a problem? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Thu Oct 14 19:58:22 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 14 Oct 2010 13:58:22 -0400 Subject: [Python-Dev] Issue 10094 In-Reply-To: <20101014193858.3a8231f4@pitrou.net> References: <20101014130401.3a2e487b@mission> <20101014193858.3a8231f4@pitrou.net> Message-ID: <20101014135822.1908e927@mission> On Oct 14, 2010, at 07:38 PM, Antoine Pitrou wrote: >There doesn't seem to be anything really mysterious, actually. The >exception message says it all :) Yep. Looks like Ubuntu 10.10 added UBUNTU_MENUPROXY to the default environment and that's what's killing it. I'll bet those Ubuntu buildbots are not running Maverick. Oh well, mystery solved at least. I'll work on a patch. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From martin at v.loewis.de Thu Oct 14 20:23:32 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 14 Oct 2010 20:23:32 +0200 Subject: [Python-Dev] A new warning category? In-Reply-To: <20101014112539.70cc54c7@pitrou.net> References: <20101014112539.70cc54c7@pitrou.net> Message-ID: <4CB74AA4.8040306@v.loewis.de> Am 14.10.2010 11:25, schrieb Antoine Pitrou: > > Hello, > > In the http://bugs.python.org/issue10093 discussion, I proposed to add a > specific warning category for unclosed files. The rationale is that > these warnings will happen in destructors and therefore filtering by > line number and filename doesn't make sense. So a new category would be > useful in order to allow defining specific rules. > Do you think it would go against the moratorium? I think that warning categories are a library feature: they are mentioned in the library documentation, and not mentioned in the reference manual (in fact, only a single warning *is* mentioned in the reference manual, namely that exceptions in __del__ are printed - which actually doesn't use the warnings module). If it is the library feature, then it is exempt from the moratorium: Allowed to Change The standard library As the standard library is not directly tied to the language definition it is not covered by this moratorium. Regards, Martin From martin at v.loewis.de Thu Oct 14 20:27:04 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 14 Oct 2010 20:27:04 +0200 Subject: [Python-Dev] [Python-checkins] r85486 - python/branches/py3k/configure In-Reply-To: References: <20101014173847.348EBEE99C@mail.python.org> Message-ID: <4CB74B78.60804@v.loewis.de> Am 14.10.2010 19:57, schrieb Daniel Stutzbach: > On Thu, Oct 14, 2010 at 12:38 PM, barry.warsaw > > wrote: > > -# Generated by GNU Autoconf 2.65 for python 3.2. > +# Generated by GNU Autoconf 2.67 for python 3.2. > > > Was the change in autoconf versions intentional and/or is it a problem? I think it was intentional (at least deliberate), but I think it is a problem and should be reverted. There is, at any point, the official version that Python uses for autoconf, which at the moment is 2.65. The rationale is that with changing autoconf versions, the actual configure script will change forth and back, confusing attributions (svn blame). Regards, Martin From benjamin at python.org Thu Oct 14 20:38:38 2010 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 14 Oct 2010 13:38:38 -0500 Subject: [Python-Dev] [Python-checkins] r85486 - python/branches/py3k/configure In-Reply-To: <4CB74B78.60804@v.loewis.de> References: <20101014173847.348EBEE99C@mail.python.org> <4CB74B78.60804@v.loewis.de> Message-ID: 2010/10/14 "Martin v. L?wis" : > Am 14.10.2010 19:57, schrieb Daniel Stutzbach: >> On Thu, Oct 14, 2010 at 12:38 PM, barry.warsaw >> > wrote: >> >> ? ? -# Generated by GNU Autoconf 2.65 for python 3.2. >> ? ? +# Generated by GNU Autoconf 2.67 for python 3.2. >> >> >> Was the change in autoconf versions intentional and/or is it a problem? > > I think it was intentional (at least deliberate), but I think it is a > problem and should be reverted. There is, at any point, the official > version that Python uses for autoconf, which at the moment is 2.65. > The rationale is that with changing autoconf versions, the actual > configure script will change forth and back, confusing attributions > (svn blame). Why would anyone annotate configure? configure.in is stable wrt to autoconf versions. -- Regards, Benjamin From barry at python.org Thu Oct 14 20:53:59 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 14 Oct 2010 14:53:59 -0400 Subject: [Python-Dev] [Python-checkins] r85486 - python/branches/py3k/configure In-Reply-To: <4CB74B78.60804@v.loewis.de> References: <20101014173847.348EBEE99C@mail.python.org> <4CB74B78.60804@v.loewis.de> Message-ID: <20101014145359.7ec278e5@mission> On Oct 14, 2010, at 08:27 PM, Martin v. L?wis wrote: >I think it was intentional (at least deliberate), but I think it is a >problem and should be reverted. There is, at any point, the official >version that Python uses for autoconf, which at the moment is 2.65. >The rationale is that with changing autoconf versions, the actual >configure script will change forth and back, confusing attributions >(svn blame). While it's true that we use AC_PREREQ(2.65), that specifies only the minimum version of autoconf required. Any time configure.in changes, configure has to change too, and since it's a generated file, I think svn blame on configure is kind of useless. Should we appoint an autoconf BDFL who can commit changes after configure.in is changed? Is it possible to pin the autoconf version (not just floor it)? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From martin at v.loewis.de Thu Oct 14 20:58:30 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 14 Oct 2010 20:58:30 +0200 Subject: [Python-Dev] [Python-checkins] r85486 - python/branches/py3k/configure In-Reply-To: References: <20101014173847.348EBEE99C@mail.python.org> <4CB74B78.60804@v.loewis.de> Message-ID: <4CB752D6.5040302@v.loewis.de> >> I think it was intentional (at least deliberate), but I think it is a >> problem and should be reverted. There is, at any point, the official >> version that Python uses for autoconf, which at the moment is 2.65. >> The rationale is that with changing autoconf versions, the actual >> configure script will change forth and back, confusing attributions >> (svn blame). > > Why would anyone annotate configure? configure.in is stable wrt to > autoconf versions. Ok, it's more an issue with aesthetics, and also reproducibility (what if somebody tests a configure change correctly, but it then breaks with an older autoconf version?) However, if people don't see this as a problem, we can also give up the strictness of requiring an exact autoconf version (and autoconf will already check for a minimum). Regards, Martin From benjamin at python.org Thu Oct 14 21:05:42 2010 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 14 Oct 2010 14:05:42 -0500 Subject: [Python-Dev] [Python-checkins] r85486 - python/branches/py3k/configure In-Reply-To: <4CB752D6.5040302@v.loewis.de> References: <20101014173847.348EBEE99C@mail.python.org> <4CB74B78.60804@v.loewis.de> <4CB752D6.5040302@v.loewis.de> Message-ID: 2010/10/14 "Martin v. L?wis" : >>> I think it was intentional (at least deliberate), but I think it is a >>> problem and should be reverted. There is, at any point, the official >>> version that Python uses for autoconf, which at the moment is 2.65. >>> The rationale is that with changing autoconf versions, the actual >>> configure script will change forth and back, confusing attributions >>> (svn blame). >> >> Why would anyone annotate configure? configure.in is stable wrt to >> autoconf versions. > > Ok, it's more an issue with aesthetics, and also reproducibility > (what if somebody tests a configure change correctly, but it then > breaks with an older autoconf version?) > > However, if people don't see this as a problem, we can also give > up the strictness of requiring an exact autoconf version (and > autoconf will already check for a minimum). I don't see it as any more of a problem than upgrading against other dependencies (like gcc?). -- Regards, Benjamin From martin at v.loewis.de Thu Oct 14 21:11:31 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 14 Oct 2010 21:11:31 +0200 Subject: [Python-Dev] [Python-checkins] r85486 - python/branches/py3k/configure In-Reply-To: References: <20101014173847.348EBEE99C@mail.python.org> <4CB74B78.60804@v.loewis.de> <4CB752D6.5040302@v.loewis.de> Message-ID: <4CB755E3.60606@v.loewis.de> > I don't see it as any more of a problem than upgrading against other > dependencies (like gcc?). Ok, so let's drop the requirement then. Regards, Martin From barry at python.org Thu Oct 14 21:47:17 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 14 Oct 2010 15:47:17 -0400 Subject: [Python-Dev] [Python-checkins] r85486 - python/branches/py3k/configure In-Reply-To: <4CB755E3.60606@v.loewis.de> References: <20101014173847.348EBEE99C@mail.python.org> <4CB74B78.60804@v.loewis.de> <4CB752D6.5040302@v.loewis.de> <4CB755E3.60606@v.loewis.de> Message-ID: <20101014154717.20e8494f@mission> On Oct 14, 2010, at 09:11 PM, Martin v. L?wis wrote: >> I don't see it as any more of a problem than upgrading against other >> dependencies (like gcc?). > >Ok, so let's drop the requirement then. Good for me. Is there a place where this requirement is documented? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From martin at v.loewis.de Thu Oct 14 22:05:05 2010 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 14 Oct 2010 22:05:05 +0200 Subject: [Python-Dev] [Python-checkins] r85486 - python/branches/py3k/configure In-Reply-To: <20101014145359.7ec278e5@mission> References: <20101014173847.348EBEE99C@mail.python.org> <4CB74B78.60804@v.loewis.de> <20101014145359.7ec278e5@mission> Message-ID: <4CB76271.9070604@v.loewis.de> > Is it possible to pin the autoconf version (not just floor it)? Not that I know of, no. > Should we appoint an autoconf BDFL who can commit changes after configure.in > is changed? Most recently, it was between me and Benjamin most of the time, and that seems to have worked fine. Now Benjamin has ruled that we stop pinning it. I still have my reservations, but no real objective objection (i.e. -0) Regards, Martin From stephen at xemacs.org Fri Oct 15 09:22:08 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 15 Oct 2010 16:22:08 +0900 Subject: [Python-Dev] A new warning category? In-Reply-To: References: <20101014112539.70cc54c7@pitrou.net> Message-ID: <87y69zzzrz.fsf@uwakimon.sk.tsukuba.ac.jp> Brett Cannon writes: > As one of the co-authors of the PEP I say no. Procedural question: Doesn't authorship disqualify you from BDF1P, so you can't pronounce? (Benevolent Dictator for One PEP) From victor.stinner at haypocalc.com Fri Oct 15 11:19:44 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Fri, 15 Oct 2010 11:19:44 +0200 Subject: [Python-Dev] Issue 10094 In-Reply-To: <20101014135822.1908e927@mission> References: <20101014130401.3a2e487b@mission> <20101014193858.3a8231f4@pitrou.net> <20101014135822.1908e927@mission> Message-ID: <201010151119.44714.victor.stinner@haypocalc.com> Le jeudi 14 octobre 2010 19:58:22, Barry Warsaw a ?crit : > On Oct 14, 2010, at 07:38 PM, Antoine Pitrou wrote: > >There doesn't seem to be anything really mysterious, actually. The > >exception message says it all :) > > Yep. Looks like Ubuntu 10.10 added UBUNTU_MENUPROXY to the default > environment and that's what's killing it. I'll bet those Ubuntu buildbots > are not running Maverick. Why does urllib test unset this variable? Is it related to an HTTP proxy? -- Victor Stinner http://www.haypocalc.com/ From orsenthil at gmail.com Fri Oct 15 11:41:44 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Fri, 15 Oct 2010 15:11:44 +0530 Subject: [Python-Dev] Issue 10094 In-Reply-To: <201010151119.44714.victor.stinner@haypocalc.com> References: <20101014130401.3a2e487b@mission> <20101014193858.3a8231f4@pitrou.net> <20101014135822.1908e927@mission> <201010151119.44714.victor.stinner@haypocalc.com> Message-ID: <20101015094143.GJ14818@remy> On Fri, Oct 15, 2010 at 11:19:44AM +0200, Victor Stinner wrote: > Why does urllib test unset this variable? Is it related to an HTTP proxy? It does that *temporarily* using EnvironVarGuard to see if the PROXY related environment variables are correctly retrieved by a method. -- Senthil Drawing on my fine command of language, I said nothing. From fuzzyman at voidspace.org.uk Fri Oct 15 11:44:25 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 15 Oct 2010 10:44:25 +0100 Subject: [Python-Dev] A new warning category? In-Reply-To: <87y69zzzrz.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20101014112539.70cc54c7@pitrou.net> <87y69zzzrz.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4CB82279.1070309@voidspace.org.uk> On 15/10/2010 08:22, Stephen J. Turnbull wrote: > Brett Cannon writes: > > > As one of the co-authors of the PEP I say no. > > Procedural question: Doesn't authorship disqualify you from BDF1P, so > you can't pronounce? I think Brett meant co-authorship of the *moratorium* PEP (which is already accepted). At least that was how I read what Brett wrote. All the best, Michael > (Benevolent Dictator for One PEP) > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From ncoghlan at gmail.com Fri Oct 15 12:27:26 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 15 Oct 2010 20:27:26 +1000 Subject: [Python-Dev] A new warning category? In-Reply-To: <4CB82279.1070309@voidspace.org.uk> References: <20101014112539.70cc54c7@pitrou.net> <87y69zzzrz.fsf@uwakimon.sk.tsukuba.ac.jp> <4CB82279.1070309@voidspace.org.uk> Message-ID: On Fri, Oct 15, 2010 at 7:44 PM, Michael Foord wrote: > ?On 15/10/2010 08:22, Stephen J. Turnbull wrote: >> >> Brett Cannon writes: >> >> ?> ?As one of the co-authors of the PEP I say no. >> >> Procedural question: Doesn't authorship disqualify you from BDF1P, so >> you can't pronounce? > > I think Brett meant co-authorship of the *moratorium* PEP (which is already > accepted). That's the way I took it as well. The other factor here is that the moratorium was largely to help other implementations get back up to speed with CPython as they dealt with the transition to 3.x. Adding a new warning type isn't the kind of moving target that something like PEP 380 would be. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From g.brandl at gmx.net Fri Oct 15 13:05:39 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 15 Oct 2010 13:05:39 +0200 Subject: [Python-Dev] Stable build slaves authority In-Reply-To: <20101014185536.702dbf3c@pitrou.net> References: <4CB61CB9.1060702@v.loewis.de> <20101014185536.702dbf3c@pitrou.net> Message-ID: Am 14.10.2010 18:55, schrieb Antoine Pitrou: > On Thu, 14 Oct 2010 08:07:46 +0200 > Georg Brandl wrote: >> >> Very nice. http://www.python.org/dev/buildbot/stable/ is completely >> green at the moment -- which means that I can now indeed take failures >> seriously in the future. Previously, two of four "stables" for py3k >> were not even connected for ages (and yes, I should have complained >> earlier. As we progress into the beta stage, I will take buildbot >> failures and release blockers more seriously). > > For the record, I've setup one of the stable buildbots ("AMD64 Gentoo > Wide") to test wide unicode buils. > Also, one of the unstable buildbots ("x86 debian parallel") now runs > its tests with -OO, at Georg's request. Excellent, thanks! Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From status at bugs.python.org Fri Oct 15 18:08:02 2010 From: status at bugs.python.org (Python tracker) Date: Fri, 15 Oct 2010 18:08:02 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20101015160802.9AD4F781EC@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2010-10-08 - 2010-10-15) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues stats: open 2554 (+34) closed 19344 (+63) total 21898 (+65) Open issues with patches: 1065 Issues opened (34) ================== #7140: imp.new_module does not function correctly if the module is re http://bugs.python.org/issue7140 reopened by benjamin.peterson #10052: Python/dtoa.c:158: #error "Failed to find an exact-width 32-bi http://bugs.python.org/issue10052 opened by akitada #10053: Don???t close fd when FileIO.__init__ fails http://bugs.python.org/issue10053 opened by ocean-city #10058: Unclear PyString_AsStringAndSize return value http://bugs.python.org/issue10058 opened by Nikratio #10060: python.exe crashes or hangs on help() modules when bad modules http://bugs.python.org/issue10060 opened by devplayer #10064: link to page with documentation bugs http://bugs.python.org/issue10064 opened by techtonik #10066: xmlrpclib does not handle some non-printable characters proper http://bugs.python.org/issue10066 opened by gyorkop #10070: 2to3 wishes for already-2to3'ed files http://bugs.python.org/issue10070 opened by hfuru #10071: Should not release GIL while running RegEnumValue http://bugs.python.org/issue10071 opened by ocean-city #10072: ftplib documentation is unclear http://bugs.python.org/issue10072 opened by rafe.kettler #10073: calendar.isleap() not checking parameter type http://bugs.python.org/issue10073 opened by shadikka #10076: Regex objects became uncopyable in 2.5 http://bugs.python.org/issue10076 opened by Michael.Shields #10077: Python 3.1: site error is not logged http://bugs.python.org/issue10077 opened by haypo #10079: idlelib for Python 3 with Guilherme Polo GSoC enhancements http://bugs.python.org/issue10079 opened by Bruce.Sherwood #10080: Py_Initialize crashes when stdout has been redirected with fre http://bugs.python.org/issue10080 opened by Mitchell.Stokes #10084: SSL support for asyncore http://bugs.python.org/issue10084 opened by pitrou #10086: test_sysconfig failure with site-packages http://bugs.python.org/issue10086 opened by hfuru #10087: HTML calendar is broken http://bugs.python.org/issue10087 opened by belopolsky #10089: Add support for arbitrary -X options http://bugs.python.org/issue10089 opened by pitrou #10090: python -m locale fails on OSX http://bugs.python.org/issue10090 opened by belopolsky #10092: calendar does not restore locale properly http://bugs.python.org/issue10092 opened by belopolsky #10093: Warn when files are not explicitly closed http://bugs.python.org/issue10093 opened by pitrou #10094: test_urllib.py fails in py3k r85440 with RuntimeError http://bugs.python.org/issue10094 opened by barry #10098: intermittent failure in test_os http://bugs.python.org/issue10098 opened by pitrou #10104: test_socket failures on Debian unstable http://bugs.python.org/issue10104 opened by pitrou #10107: Quitting IDLE on Mac doesn't save unsaved code http://bugs.python.org/issue10107 opened by Bruce.Sherwood #10108: ExpatError not property wrapped http://bugs.python.org/issue10108 opened by leos #10109: itertools.product with infinite iterator cause MemoryError. http://bugs.python.org/issue10109 opened by falsetru #10110: Queue doesn't recognize it is full after shrinking maxsize http://bugs.python.org/issue10110 opened by jaraco #10111: Minor problems with PyFileObject documentation (Doc/c-api/file http://bugs.python.org/issue10111 opened by pebolle #10112: Use -Wl,--dynamic-list=x.list, not -Xlinker -export-dynamic http://bugs.python.org/issue10112 opened by jankratochvil #10114: compile() doesn't support the PEP 383 (surrogates) http://bugs.python.org/issue10114 opened by haypo #10115: accept4 can fail with errno 90 http://bugs.python.org/issue10115 opened by pitrou #10116: Sporadic failures in test_urllibnet http://bugs.python.org/issue10116 opened by ixokai Most recent 15 issues with no replies (15) ========================================== #10114: compile() doesn't support the PEP 383 (surrogates) http://bugs.python.org/issue10114 #10112: Use -Wl,--dynamic-list=x.list, not -Xlinker -export-dynamic http://bugs.python.org/issue10112 #10111: Minor problems with PyFileObject documentation (Doc/c-api/file http://bugs.python.org/issue10111 #10110: Queue doesn't recognize it is full after shrinking maxsize http://bugs.python.org/issue10110 #10109: itertools.product with infinite iterator cause MemoryError. http://bugs.python.org/issue10109 #10092: calendar does not restore locale properly http://bugs.python.org/issue10092 #10090: python -m locale fails on OSX http://bugs.python.org/issue10090 #10077: Python 3.1: site error is not logged http://bugs.python.org/issue10077 #10064: link to page with documentation bugs http://bugs.python.org/issue10064 #10058: Unclear PyString_AsStringAndSize return value http://bugs.python.org/issue10058 #10051: distutils fail to install unicode-encoded files with POSIX loc http://bugs.python.org/issue10051 #10050: urllib.request still has old 2.x urllib primitives http://bugs.python.org/issue10050 #10048: urllib.request documentation confusing http://bugs.python.org/issue10048 #10037: multiprocessing.pool processes started by worker handler stops http://bugs.python.org/issue10037 #10036: compiler warnings for various modules on Ubuntu x86 http://bugs.python.org/issue10036 Most recent 15 issues waiting for review (15) ============================================= #10114: compile() doesn't support the PEP 383 (surrogates) http://bugs.python.org/issue10114 #10111: Minor problems with PyFileObject documentation (Doc/c-api/file http://bugs.python.org/issue10111 #10110: Queue doesn't recognize it is full after shrinking maxsize http://bugs.python.org/issue10110 #10093: Warn when files are not explicitly closed http://bugs.python.org/issue10093 #10089: Add support for arbitrary -X options http://bugs.python.org/issue10089 #10087: HTML calendar is broken http://bugs.python.org/issue10087 #10079: idlelib for Python 3 with Guilherme Polo GSoC enhancements http://bugs.python.org/issue10079 #10077: Python 3.1: site error is not logged http://bugs.python.org/issue10077 #10076: Regex objects became uncopyable in 2.5 http://bugs.python.org/issue10076 #10072: ftplib documentation is unclear http://bugs.python.org/issue10072 #10071: Should not release GIL while running RegEnumValue http://bugs.python.org/issue10071 #10066: xmlrpclib does not handle some non-printable characters proper http://bugs.python.org/issue10066 #10053: Don???t close fd when FileIO.__init__ fails http://bugs.python.org/issue10053 #10049: Add a "no-op" (null) context manager to contextlib http://bugs.python.org/issue10049 #10044: small int optimization http://bugs.python.org/issue10044 Top 10 most discussed issues (10) ================================= #8863: Display Python backtrace on SIGSEGV, SIGFPE and fatal error http://bugs.python.org/issue8863 16 msgs #10093: Warn when files are not explicitly closed http://bugs.python.org/issue10093 14 msgs #1553375: Add traceback.print_full_exception() http://bugs.python.org/issue1553375 14 msgs #10070: 2to3 wishes for already-2to3'ed files http://bugs.python.org/issue10070 11 msgs #10049: Add a "no-op" (null) context manager to contextlib http://bugs.python.org/issue10049 9 msgs #4388: test_cmd_line fails on MacOS X http://bugs.python.org/issue4388 8 msgs #10008: Two links point to same place http://bugs.python.org/issue10008 8 msgs #10089: Add support for arbitrary -X options http://bugs.python.org/issue10089 8 msgs #7944: Use the 'with' statement in conjunction with 'open' throughout http://bugs.python.org/issue7944 7 msgs #8998: add crypto routines to stdlib http://bugs.python.org/issue8998 7 msgs Issues closed (63) ================== #1051: certificate in Lib/test/test_ssl.py expires in February 2013 http://bugs.python.org/issue1051 closed by pitrou #1708: improvements for linecache http://bugs.python.org/issue1708 closed by r.david.murray #2830: Copy cgi.escape() to html http://bugs.python.org/issue2830 closed by georg.brandl #3865: explain that profilers should be used for profiling, not bench http://bugs.python.org/issue3865 closed by georg.brandl #5355: Expat parser error constants are string descriptions http://bugs.python.org/issue5355 closed by georg.brandl #6417: multiprocessing Process examples: print and getppid http://bugs.python.org/issue6417 closed by georg.brandl #6825: Minor documentation bug with os.path.split http://bugs.python.org/issue6825 closed by georg.brandl #7285: multiprocessing module, example code error http://bugs.python.org/issue7285 closed by orsenthil #7287: import hook demo does not work http://bugs.python.org/issue7287 closed by benjamin.peterson #7516: Flag "-3" is silently ignored when running "regrtest.py -j2" http://bugs.python.org/issue7516 closed by pitrou #7523: add SOCK_NONBLOCK and SOCK_CLOEXEC to socket module http://bugs.python.org/issue7523 closed by pitrou #7642: Minor improvement to os.system doc http://bugs.python.org/issue7642 closed by georg.brandl #7771: dict view comparison methods are not documented http://bugs.python.org/issue7771 closed by georg.brandl #8267: Tutorial section on dictionary keys recommends sort instead of http://bugs.python.org/issue8267 closed by georg.brandl #9003: urllib.request and http.client should allow certificate checki http://bugs.python.org/issue9003 closed by pitrou #9013: Implement tzinfo.dst() method in timezone http://bugs.python.org/issue9013 closed by belopolsky #9093: Tools/README is out of date http://bugs.python.org/issue9093 closed by georg.brandl #9183: Intern UTC timezone http://bugs.python.org/issue9183 closed by belopolsky #9409: doctest in python2.7 can't handle non-ascii characters http://bugs.python.org/issue9409 closed by flox #9774: test_smtpnet fails with "[110] Connection timed out" on AMD64 http://bugs.python.org/issue9774 closed by pitrou #9964: Test failures with -OO http://bugs.python.org/issue9964 closed by georg.brandl #9988: test_warnings fails with PYTHONFSENCODING=latin-1 on UNIX/BSD http://bugs.python.org/issue9988 closed by haypo #9992: Command-line arguments are not correctly decoded if locale and http://bugs.python.org/issue9992 closed by haypo #10011: `except` doesn't use `isinstance` http://bugs.python.org/issue10011 closed by terry.reedy #10014: sys.path[0] is incorrect if PYTHONFSENCODING is used http://bugs.python.org/issue10014 closed by haypo #10018: IDLE not loading in xp pro due to tcl issue http://bugs.python.org/issue10018 closed by terry.reedy #10029: "Equivalent to" code for zip is wrong in Python 3 http://bugs.python.org/issue10029 closed by rhettinger #10039: python ??.py fails with UnicodeEncodeError if PYTHONFSENCODING http://bugs.python.org/issue10039 closed by haypo #10045: poor cStringIO.StringO seek performance http://bugs.python.org/issue10045 closed by fdrake #10046: Correction to atexit documentation http://bugs.python.org/issue10046 closed by georg.brandl #10054: select extension does not build on platforms where uintptr_t i http://bugs.python.org/issue10054 closed by pitrou #10055: C99 code in _json.c http://bugs.python.org/issue10055 closed by pitrou #10056: Can't build tkinter with libtk 8.6 http://bugs.python.org/issue10056 closed by pitrou #10057: Unclear if exception should be set http://bugs.python.org/issue10057 closed by jankratochvil #10059: add the method `index` to collections.deque http://bugs.python.org/issue10059 closed by Simon.Liedtke #10061: ** operator yielding wrong result for negative numbers http://bugs.python.org/issue10061 closed by orsenthil #10062: Python 3.2 does not build on systems which do not have sem_tim http://bugs.python.org/issue10062 closed by pitrou #10065: future_builtins' docstring lacks some functions http://bugs.python.org/issue10065 closed by orsenthil #10067: itertools' docs put izip_longest in the "terminating on the sh http://bugs.python.org/issue10067 closed by rhettinger #10068: global objects created in some module are not destroyed when l http://bugs.python.org/issue10068 closed by benjamin.peterson #10069: 2to3 crash in fix_urllib.py http://bugs.python.org/issue10069 closed by benjamin.peterson #10074: dictobject.c: crash in Py_XDECREF http://bugs.python.org/issue10074 closed by benjamin.peterson #10075: Get session cache stats from SSLContext http://bugs.python.org/issue10075 closed by pitrou #10078: Documentation: 'Postive' should be 'Positive' http://bugs.python.org/issue10078 closed by benjamin.peterson #10081: 'import' operator doesn't work with unicode non 7-bit ASCII mo http://bugs.python.org/issue10081 closed by brett.cannon #10082: PyRun_SimpleFile crashes application http://bugs.python.org/issue10082 closed by loewis #10083: locale.currency() uses different formatting than system locale http://bugs.python.org/issue10083 closed by eric.smith #10085: Memory allocation for primitive types http://bugs.python.org/issue10085 closed by pitrou #10088: pdb can't re-set variables http://bugs.python.org/issue10088 closed by giampaolo.rodola #10091: ast.literal_eval does not handled new set literals http://bugs.python.org/issue10091 closed by georg.brandl #10095: Support undecodable filenames in the parser API http://bugs.python.org/issue10095 closed by haypo #10096: Question regarding python migration http://bugs.python.org/issue10096 closed by orsenthil #10097: platform.uname gives UnicodeError for non-ASCII computer names http://bugs.python.org/issue10097 closed by lemburg #10099: socket.fromfd() documentation problem http://bugs.python.org/issue10099 closed by orsenthil #10100: fromfd is now available on all platforms http://bugs.python.org/issue10100 closed by orsenthil #10101: a bug in built-in function round() ? http://bugs.python.org/issue10101 closed by benjamin.peterson #10102: mktime adding an hour in April (naive struct)? http://bugs.python.org/issue10102 closed by belopolsky #10103: use the SOABI for GNU/kfreeBSD and the GNU Hurd http://bugs.python.org/issue10103 closed by doko #10105: Strings failing to print http://bugs.python.org/issue10105 closed by eric.araujo #10106: missing packages http://bugs.python.org/issue10106 closed by brett.cannon #10113: UnicodeDecodeError in mimetypes.guess_type on Windows http://bugs.python.org/issue10113 closed by r.david.murray #1710703: zipfile.ZipFile behavior inconsistent. http://bugs.python.org/issue1710703 closed by georg.brandl #10063: file:// scheme will stop accessing via ftp protocol http://bugs.python.org/issue10063 closed by orsenthil From sable at users.sourceforge.net Fri Oct 15 17:38:47 2010 From: sable at users.sourceforge.net (=?UTF-8?B?U8OpYmFzdGllbiBTYWJsw6k=?=) Date: Fri, 15 Oct 2010 17:38:47 +0200 Subject: [Python-Dev] Buildbot for AIX In-Reply-To: <4C97B422.4040606@v.loewis.de> References: <1284586127.79.0.767793857364.issue1633863@psf.upfronthosting.co.za> <4C93377C.4040300@users.sourceforge.net> <4C936225.1000500@v.loewis.de> <4C9770FB.1010103@users.sourceforge.net> <4C97B422.4040606@v.loewis.de> Message-ID: <4CB87587.9020100@users.sourceforge.net> Hi Martin, I finally got the authorization to run some buildbot slaves on our AIX servers. I made some tests with a buildbot script as close as possible to what you already use. Here is a patch that show the kind of modifications I had to do in order to get the buildbot slave to correctly run on AIX. It is also necessary to apply the patch provided in issue 9862 so that the tests won't hang forever. It also would help if the corrections provided in the following issues could be commited: 4499, 678250, 730467. Could you please take a look at those modifications in master.cfg, provide me some password for the bot slaves and apply the corrections in those issues? Once this is done, I can run 2 buildbot slaves for AIX 6.1 and AIX 5.3 with build plans for gcc and xlc. I can't guarantee that those bots will run forever, and I may have to ask to schedule them only at night if the activity it generates on the server appears to be too high. But I think it could greatly help to improve the state of Python on AIX (which is far from perfect at the moment). regards -- S?bastien Sabl? Le 20/09/2010 21:21, "Martin v. L?wis" a ?crit : >> Also could you provide me the master.cfg file (with obfuscated >> passwords) that is used by the Python buildbot master or tell me if it >> is in subversion somewhere? > > Attached! > > Regards, > Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: buildbot_aix.diff Type: text/x-patch Size: 7582 bytes Desc: not available URL: From sridharr at activestate.com Fri Oct 15 18:50:04 2010 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Fri, 15 Oct 2010 09:50:04 -0700 Subject: [Python-Dev] Buildbot for AIX In-Reply-To: <4C9772A6.2070908@users.sourceforge.net> References: <1284586127.79.0.767793857364.issue1633863@psf.upfronthosting.co.za> <4C93377C.4040300@users.sourceforge.net> <20100917150551.099113e4@pitrou.net> <4C9772A6.2070908@users.sourceforge.net> Message-ID: On 2010-09-20, at 7:41 AM, S?bastien Sabl? wrote: > Le 17/09/2010 15:05, Antoine Pitrou a ?crit : >> Following on Martin's comments, you might also want to share things >> with the ActiveState guys who, AFAIK, maintain an AIX version of Python >> (but you have been the most active AIX user on the bug tracker lately; >> perhaps they are keeping their patches to themselves). > > I tried to find some source package or subversion repository on their web site, but they only distribute binary versions. > Also it is only the "Business Edition" that supports AIX and it is clearly sold with a proprietary license. > > So I doubt they would share their patches related to AIX, but I can always ask. Hi S?bastien, We definitely like to share our core Python patches for AIX 5.1/5.2 and other platforms. We have, in the past, submitted a few of them already in the Python bug tracker. As not all of them have been submitted (for lack of time), I uploaded them all here for your access, http://gist.github.com/628519 Please do send us any patches you may have for building Python 3 on AIX. -srid From brett at python.org Fri Oct 15 19:23:11 2010 From: brett at python.org (Brett Cannon) Date: Fri, 15 Oct 2010 10:23:11 -0700 Subject: [Python-Dev] A new warning category? In-Reply-To: References: <20101014112539.70cc54c7@pitrou.net> <87y69zzzrz.fsf@uwakimon.sk.tsukuba.ac.jp> <4CB82279.1070309@voidspace.org.uk> Message-ID: On Fri, Oct 15, 2010 at 03:27, Nick Coghlan wrote: > On Fri, Oct 15, 2010 at 7:44 PM, Michael Foord > wrote: >> ?On 15/10/2010 08:22, Stephen J. Turnbull wrote: >>> >>> Brett Cannon writes: >>> >>> ?> ?As one of the co-authors of the PEP I say no. >>> >>> Procedural question: Doesn't authorship disqualify you from BDF1P, so >>> you can't pronounce? >> >> I think Brett meant co-authorship of the *moratorium* PEP (which is already >> accepted). > > That's the way I took it as well. And that is the way to take it. =) > > The other factor here is that the moratorium was largely to help other > implementations get back up to speed with CPython as they dealt with > the transition to 3.x. Adding a new warning type isn't the kind of > moving target that something like PEP 380 would be. Exactly. This is especially true if the other VMs use warnings.py which will take care of all the details for them. -Brett > > Cheers, > Nick. > > -- > Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From raymond.hettinger at gmail.com Fri Oct 15 19:37:18 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Fri, 15 Oct 2010 10:37:18 -0700 Subject: [Python-Dev] Locked-in defect? 32-bit hash values on 64-bit builds Message-ID: After rereading http://bugs.python.org/issue9778 , I'm growing concerned about an impending ABI freeze before the core devs find time to fix the 32-bit hash limitation. ISTM, the use of 64-bit builds is growing in popularity. It was be a bummer to have a locked-in an effective size limit for dictionaries and sets because the API only supports 32-bit hash values. The thread seems to show agreement that the hash values should be Py_ssize_t but the chance to fix it will be lost unless core devs get more time to work on the problem or unless the ABI freeze is deferred. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Fri Oct 15 19:40:33 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 15 Oct 2010 12:40:33 -0500 Subject: [Python-Dev] Locked-in defect? 32-bit hash values on 64-bit builds In-Reply-To: References: Message-ID: 2010/10/15 Raymond Hettinger : > After rereading?http://bugs.python.org/issue9778?, I'm growing concerned > about an impending ABI freeze before the core devs find time to fix the > 32-bit hash limitation. > ISTM, the use of 64-bit builds is growing in popularity. ?It was be a bummer > to have a locked-in an effective size limit for dictionaries and sets > because > the API only supports 32-bit hash values. > The thread seems to show agreement that the hash values should be > Py_ssize_t but the chance to fix it will be lost unless core devs?get > more time to work on the problem or unless the ABI freeze is deferred. I think the panic is a bit of an overreaction. PEP 384 has still not been accepted, and I haven't seen a final decision about freezing the ABI in 3.2. -- Regards, Benjamin From skip at pobox.com Fri Oct 15 19:44:07 2010 From: skip at pobox.com (skip at pobox.com) Date: Fri, 15 Oct 2010 12:44:07 -0500 Subject: [Python-Dev] Locked-in defect? 32-bit hash values on 64-bit builds In-Reply-To: References: Message-ID: <19640.37607.153217.40557@montanaro.dyndns.org> Raymond> ... but the chance to fix it will be lost unless core devs get Raymond> more time to work on the problem or unless the ABI freeze is Raymond> deferred. +1 Skip From barry at python.org Fri Oct 15 20:07:24 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 15 Oct 2010 14:07:24 -0400 Subject: [Python-Dev] Locked-in defect? 32-bit hash values on 64-bit builds In-Reply-To: References: Message-ID: <20101015140724.63980509@mission> On Oct 15, 2010, at 12:40 PM, Benjamin Peterson wrote: >2010/10/15 Raymond Hettinger : >> After rereading?http://bugs.python.org/issue9778?, I'm growing concerned >> about an impending ABI freeze before the core devs find time to fix the >> 32-bit hash limitation. >> ISTM, the use of 64-bit builds is growing in popularity. ?It was be a bummer >> to have a locked-in an effective size limit for dictionaries and sets >> because >> the API only supports 32-bit hash values. >> The thread seems to show agreement that the hash values should be >> Py_ssize_t but the chance to fix it will be lost unless core devs?get >> more time to work on the problem or unless the ABI freeze is deferred. > >I think the panic is a bit of an overreaction. PEP 384 has still not >been accepted, and I haven't seen a final decision about freezing the >ABI in 3.2. Yes, but there's less than a month left before 3.2 beta 1, and then it *will* be too late, at least for this release. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From raymond.hettinger at gmail.com Fri Oct 15 22:00:16 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Fri, 15 Oct 2010 13:00:16 -0700 Subject: [Python-Dev] Fwd: [Python-ideas] stats module Was: minmax() function ... References: Message-ID: <1F769A68-3C17-482B-A252-7DB6BFF7F37B@gmail.com> Hello guys. If you don't mind, I would like to hijack your thread :-) ISTM, that the minmax() idea is really just an optimization request. A single-pass minmax() is easily coded in simple, pure-python, so really the discussion is about how to remove the loop overhead (there isn't much you can do about the cost of the two compares which is where most of the time would be spent anyway). My suggestion is to aim higher. There is no reason a single pass couldn't also return min/max/len/sum and perhaps even other summary statistics like sum(x**2) so that you can compute standard deviation and variance. A few years ago, Guido and other python devvers supported a proposal I made to create a stats module, but I didn't have time to develop it. The basic idea was that python's batteries should include most of the functionality available on advanced student calculators. Another idea behind it was that we could invisibility do-the-right-thing under the hood to help users avoid numerical problems (i.e. math.fsum(s)/len(s) is a more accurate way to compute an average because it doesn't lose precision when building-up the intermediate sums). I think the creativity and energy of this group is much better directed at building a quality stats module (perhaps with some R-like capabilities). That would likely be a better use of energy than bike-shedding about ways to speed-up a trivial piece of code that is ultimately constrained by the cost of the compares per item. my-two-cents, Raymond From raymond.hettinger at gmail.com Fri Oct 15 22:10:37 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Fri, 15 Oct 2010 13:10:37 -0700 Subject: [Python-Dev] Locked-in defect? 32-bit hash values on 64-bit builds In-Reply-To: References: Message-ID: <431D9390-F7CF-46C9-A31D-F7D7403190B0@gmail.com> On Oct 15, 2010, at 10:40 AM, Benjamin Peterson wrote: > I think the panic is a bit of an overreaction. PEP 384 has still not > been accepted, and I haven't seen a final decision about freezing the > ABI in 3.2. Not sure where the "panic" seems to be. I just want to make sure the ABI doesn't get frozen before hash functions are converted to Py_ssize_t. Even if the ABI is nor frozen at 3.2 as Martin has proposed, it would still be great to get this in for 3.2 Fortunately, this doesn't affect everyday users, it only arises for very large datasets. When it does kick-in though (around 2**32 entries), the degradation is not small, it is close to catastrophic, making dicts/set unusable where O(1) lookups become O(n) with a *very* large n. Raymond From bostjan.mejak at gmail.com Fri Oct 15 23:02:51 2010 From: bostjan.mejak at gmail.com (=?UTF-8?Q?Bo=C5=A1tjan_Mejak?=) Date: Fri, 15 Oct 2010 23:02:51 +0200 Subject: [Python-Dev] Fixing zipfile.BadZipfile to zipfile.BadZipFile Message-ID: I am very glad you're reorganizing the Standard Library. Thumbs up! I hope everything will comply to PEP 8 after you're done. Since you're reorganizing, I have my own contribution. I have attached a patch. The issue7351 was not accepted at the time, so I hope you'll accept this fix now. My point is that every class name in the module zipfile is like this: - LargeZipFile - ZipFile - PyZipFile . . . So apply my patch to make the class name BadZipfile consistent to other class names in the zipfile module and let it be named BadZipFile. Thank you. Best regards, Bo?tjan Mejak -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: zipfile-patch.diff Type: application/octet-stream Size: 3786 bytes Desc: not available URL: From reid.kleckner at gmail.com Fri Oct 15 23:35:25 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Fri, 15 Oct 2010 17:35:25 -0400 Subject: [Python-Dev] Locked-in defect? 32-bit hash values on 64-bit builds In-Reply-To: <431D9390-F7CF-46C9-A31D-F7D7403190B0@gmail.com> References: <431D9390-F7CF-46C9-A31D-F7D7403190B0@gmail.com> Message-ID: On Fri, Oct 15, 2010 at 4:10 PM, Raymond Hettinger wrote: > > On Oct 15, 2010, at 10:40 AM, Benjamin Peterson wrote: > >> I think the panic is a bit of an overreaction. PEP 384 has still not >> been accepted, and I haven't seen a final decision about freezing the >> ABI in 3.2. > > Not sure where the "panic" seems to be. > I just want to make sure the ABI doesn't get frozen > before hash functions are converted to Py_ssize_t. > > Even if the ABI is nor frozen at 3.2 as Martin has proposed, > it would still be great to get this in for 3.2 > > Fortunately, this doesn't affect everyday users, it only > arises for very large datasets. ?When it does kick-in though > (around 2**32 entries), the degradation is not small, it > is close to catastrophic, making dicts/set unusable > where O(1) lookups become O(n) with a *very* large n. Just to be clear, hashing right now just uses the C long type. The only major platform where sizeof(long) < sizeof(Py_ssize_t) is 64-bit Windows, right? And the change being proposed is to make tp_hash return a Py_ssize_t instead of a long, and then make all the clients of tp_hash compute with Py_ssize_t instead of long? Reid From debatem1 at gmail.com Sat Oct 16 02:05:26 2010 From: debatem1 at gmail.com (geremy condra) Date: Fri, 15 Oct 2010 17:05:26 -0700 Subject: [Python-Dev] Fwd: [Python-ideas] stats module Was: minmax() function ... In-Reply-To: <1F769A68-3C17-482B-A252-7DB6BFF7F37B@gmail.com> References: <1F769A68-3C17-482B-A252-7DB6BFF7F37B@gmail.com> Message-ID: On Fri, Oct 15, 2010 at 1:00 PM, Raymond Hettinger wrote: > Hello guys. ?If you don't mind, I would like to hijack your thread :-) > > ISTM, that the minmax() idea is really just an optimization request. > A single-pass minmax() is easily coded in simple, pure-python, > so really the discussion is about how to remove the loop overhead > (there isn't much you can do about the cost of the two compares > which is where most of the time would be spent anyway). > > My suggestion is to aim higher. ? There is no reason a single pass > couldn't also return min/max/len/sum and perhaps even other summary > statistics like sum(x**2) so that you can compute standard deviation > and variance. +1 from me. Here's a normal cdf and chi squared cdf approximation I use for randomness testing. They may need to refined for inclusion, but you're welcome to use them if you'd like. from math import sqrt, erf def normal_cdf(x, mu=0, sigma=1): """Approximates the normal cumulative distribution""" return (1/2) * (1 + erf((x+mu)/(sigma*sqrt(2)))) def chi_squared_cdf(x, k): """Approximates the cumulative chi-squared statistic with k degrees of freedom.""" numerator = 1 - (2/(9*k)) - ((x/k)**(1/3)) denominator = (1/3) * sqrt(2/k) return normal_cdf(numerator/denominator) > A few years ago, Guido and other python devvers supported a > proposal I made to create a stats module, but I didn't have time > to develop it. ?The basic idea was that python's batteries should > include most of the functionality available on advanced student > calculators. ?Another idea behind it was that we could invisibility > do-the-right-thing under the hood to help users avoid numerical > problems (i.e. math.fsum(s)/len(s) is a more accurate way to > compute an average because it doesn't lose precision when > building-up the intermediate sums). Can you give some other examples? Sage does some of this and I frequently find it annoying, actually, but I'm not sure if you're referring to the same things there. Geremy Condra From steve at pearwood.info Sat Oct 16 02:06:44 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 16 Oct 2010 11:06:44 +1100 Subject: [Python-Dev] Fwd: [Python-ideas] stats module Was: minmax() function ... In-Reply-To: <1F769A68-3C17-482B-A252-7DB6BFF7F37B@gmail.com> References: <1F769A68-3C17-482B-A252-7DB6BFF7F37B@gmail.com> Message-ID: <201010161106.44621.steve@pearwood.info> On Sat, 16 Oct 2010 07:00:16 am Raymond Hettinger wrote: > Hello guys. If you don't mind, I would like to hijack your thread > :-) Please do :) > A few years ago, Guido and other python devvers supported a > proposal I made to create a stats module, but I didn't have time > to develop it. [...] > I think the creativity and energy of this group is much better > directed at building a quality stats module (perhaps with some R-like > capabilities). +1 Are you still interested in working on it, or is this a subtle hint that somebody else should do so? -- Steven D'Aprano From sunqiang at gmail.com Sat Oct 16 02:49:15 2010 From: sunqiang at gmail.com (sunqiang) Date: Sat, 16 Oct 2010 08:49:15 +0800 Subject: [Python-Dev] Fwd: [Python-ideas] stats module Was: minmax() function ... In-Reply-To: References: <1F769A68-3C17-482B-A252-7DB6BFF7F37B@gmail.com> Message-ID: On Sat, Oct 16, 2010 at 8:05 AM, geremy condra wrote: > On Fri, Oct 15, 2010 at 1:00 PM, Raymond Hettinger > wrote: >> Hello guys. ?If you don't mind, I would like to hijack your thread :-) >> >> ISTM, that the minmax() idea is really just an optimization request. >> A single-pass minmax() is easily coded in simple, pure-python, >> so really the discussion is about how to remove the loop overhead >> (there isn't much you can do about the cost of the two compares >> which is where most of the time would be spent anyway). >> >> My suggestion is to aim higher. ? There is no reason a single pass >> couldn't also return min/max/len/sum and perhaps even other summary >> statistics like sum(x**2) so that you can compute standard deviation >> and variance. > > +1 from me. Here's a normal cdf and chi squared cdf approximation I > use for randomness testing. They may need to refined for inclusion, > but you're welcome to use them if you'd like. > > from math import sqrt, erf > > def normal_cdf(x, mu=0, sigma=1): > ? ? ? ?"""Approximates the normal cumulative distribution""" > ? ? ? ?return (1/2) * (1 + erf((x+mu)/(sigma*sqrt(2)))) > > def chi_squared_cdf(x, k): > ? ? ? ?"""Approximates the cumulative chi-squared statistic with k degrees > of freedom.""" > ? ? ? ?numerator = 1 - (2/(9*k)) - ((x/k)**(1/3)) > ? ? ? ?denominator = (1/3) * sqrt(2/k) > ? ? ? ?return normal_cdf(numerator/denominator) > >> A few years ago, Guido and other python devvers supported a >> proposal I made to create a stats module, but I didn't have time >> to develop it. ?The basic idea was that python's batteries should >> include most of the functionality available on advanced student >> calculators. ?Another idea behind it was that we could invisibility >> do-the-right-thing under the hood to help users avoid numerical >> problems (i.e. math.fsum(s)/len(s) is a more accurate way to >> compute an average because it doesn't lose precision when >> building-up the intermediate sums). > > Can you give some other examples? Sage does some of this and I > frequently find it annoying, actually, but I'm not sure if you're > referring to the same things there. have seen a blog post[1] several months ago from reddit[2], maybe it worth a reading. [1]: http://www.johndcook.com/blog/2010/06/07/math-library-functions-that-seem-unnecessary/ [2]: http://www.reddit.com/r/programming/comments/ccbja/math_library_functions_that_seem_unnecessary/ > Geremy Condra > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/sunqiang%40gmail.com > From janssen at parc.com Sat Oct 16 02:57:24 2010 From: janssen at parc.com (Bill Janssen) Date: Fri, 15 Oct 2010 17:57:24 PDT Subject: [Python-Dev] PPC Tiger is back up Message-ID: <348.1287190644@parc.com> I've moved the OS X buildbot parc-tiger-1 (PPC Tiger) to new hardware, a MDD dual-G4 PowerMac with more memory. Should move considerably faster. Enjoy! Bill From g.brandl at gmx.net Sat Oct 16 13:50:30 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 16 Oct 2010 13:50:30 +0200 Subject: [Python-Dev] r85559 - in python/branches/py3k: Doc/library/sys.rst Lib/distutils/command/build_ext.py Lib/distutils/tests/test_build_ext.py Lib/test/test_sys.py Makefile.pre.in Misc/python-config.in Python/sysmodule.c configure configure.in In-Reply-To: <20101016010409.7F965EE9D9@mail.python.org> References: <20101016010409.7F965EE9D9@mail.python.org> Message-ID: Am 16.10.2010 03:04, schrieb barry.warsaw: > Author: barry.warsaw > Date: Sat Oct 16 03:04:07 2010 > New Revision: 85559 > > Log: > First (uncontroversial) part of issue 9807. > > * Expose the build flags to Python as sys.abiflags > * Shared library libpythonX.Y.so > * python-config --abiflags > * Make two distutils tests that failed with --enable-shared (even before this > patch) succeed. > * Fix a few small style issues. > > > Modified: > python/branches/py3k/Doc/library/sys.rst > python/branches/py3k/Lib/distutils/command/build_ext.py > python/branches/py3k/Lib/distutils/tests/test_build_ext.py > python/branches/py3k/Lib/test/test_sys.py > python/branches/py3k/Makefile.pre.in > python/branches/py3k/Misc/python-config.in > python/branches/py3k/Python/sysmodule.c > python/branches/py3k/configure > python/branches/py3k/configure.in > > Modified: python/branches/py3k/Doc/library/sys.rst > ============================================================================== > --- python/branches/py3k/Doc/library/sys.rst (original) > +++ python/branches/py3k/Doc/library/sys.rst Sat Oct 16 03:04:07 2010 > @@ -955,6 +955,11 @@ > module for informational purposes; modifying this value has no effect on the > registry keys used by Python. Availability: Windows. > > +.. data:: abiflags > + > + On POSIX systems where Python is build with the standard ``configure`` > + script, this contains the ABI flags as specified by :pep:`3149`. > + Needs a versionadded. Also, I think we at least tried to maintain a rough alphabetical order of sys members... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From barry at python.org Sat Oct 16 16:18:41 2010 From: barry at python.org (Barry Warsaw) Date: Sat, 16 Oct 2010 10:18:41 -0400 Subject: [Python-Dev] r85559 - in python/branches/py3k: Doc/library/sys.rst Lib/distutils/command/build_ext.py Lib/distutils/tests/test_build_ext.py Lib/test/test_sys.py Makefile.pre.in Misc/python-config.in Python/sysmodule.c configure configure.in In-Reply-To: References: <20101016010409.7F965EE9D9@mail.python.org> Message-ID: <20101016101841.2ce33f9c@mission> On Oct 16, 2010, at 01:50 PM, Georg Brandl wrote: >> --- python/branches/py3k/Doc/library/sys.rst (original) >> +++ python/branches/py3k/Doc/library/sys.rst Sat Oct 16 03:04:07 2010 >> @@ -955,6 +955,11 @@ >> module for informational purposes; modifying this value has no effect on the >> registry keys used by Python. Availability: Windows. >> >> +.. data:: abiflags >> + >> + On POSIX systems where Python is build with the standard ``configure`` >> + script, this contains the ABI flags as specified by :pep:`3149`. >> + > >Needs a versionadded. Also, I think we at least tried to maintain a rough >alphabetical order of sys members... Thanks, good catch. r85571 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From solipsis at pitrou.net Sat Oct 16 18:26:31 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 16 Oct 2010 18:26:31 +0200 Subject: [Python-Dev] r85569 - in python/branches/py3k: Misc/NEWS Parser/tokenizer.h Python/ast.c Python/bltinmodule.c Python/compile.c Python/pythonrun.c Python/traceback.c References: <20101016131410.D3DA2EE982@mail.python.org> Message-ID: <20101016182631.47e139b8@pitrou.net> On Sat, 16 Oct 2010 15:14:10 +0200 (CEST) victor.stinner wrote: > Author: victor.stinner > Date: Sat Oct 16 15:14:10 2010 > New Revision: 85569 > > Log: > Issue #9713, #10114: Parser functions (eg. PyParser_ASTFromFile) expects > filenames encoded to the filesystem encoding with surrogateescape error handler > (to support undecodable bytes), instead of UTF-8 in strict mode. This change broke at least two buildbots: http://www.python.org/dev/buildbot/builders/AMD64%20Gentoo%20Wide%203.x http://www.python.org/dev/buildbot/builders/sparc%20solaris10%20gcc%203.x cheers Antoine. From janssen at parc.com Sat Oct 16 22:14:20 2010 From: janssen at parc.com (Bill Janssen) Date: Sat, 16 Oct 2010 13:14:20 PDT Subject: [Python-Dev] PPC Leopard buildbot failing sqlite test for Python 3.2 Message-ID: <88545.1287260060@parc.com> There was a test added to the sqlite suite between 3.1 and the current 3.x trunk, TestInTransaction. It's failing pretty consistently on the PPC Leopard buildbot (2.7 and 3.1 are testing OK). Calling sqlite3.sqlite_version returns 3.4.0. A couple of things come to mind: * Does this require a different flavor of the sqlite libraries? I looked through the 3.x docs and couldn't find any mention of a specific sqlite version. * Is this a big-endian bug? Might fail on SPARC, too, then. What should I look at? Bill From rdmurray at bitdance.com Sat Oct 16 22:51:09 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Sat, 16 Oct 2010 16:51:09 -0400 Subject: [Python-Dev] PPC Leopard buildbot failing sqlite test for Python 3.2 In-Reply-To: <88545.1287260060@parc.com> References: <88545.1287260060@parc.com> Message-ID: <20101016205109.D307721B6E5@kimball.webabinitio.net> On Sat, 16 Oct 2010 13:14:20 -0700, Bill Janssen wrote: > There was a test added to the sqlite suite between 3.1 and the current > 3.x trunk, TestInTransaction. It's failing pretty consistently on the > PPC Leopard buildbot (2.7 and 3.1 are testing OK). > Calling sqlite3.sqlite_version returns 3.4.0. > > A couple of things come to mind: > > * Does this require a different flavor of the sqlite libraries? I > looked through the 3.x docs and couldn't find any mention of a > specific sqlite version. That's what I was wondering (I see you found the relevant issue...), but it doesn't *seem* to be a version issue, but it would be cool if you could check it against 3.7 just for confirmation. > * Is this a big-endian bug? Might fail on SPARC, too, then. It might be, except for two things: it passes on the Sparc Solaris 10 buildbot, and the io module also makes use of a T_BOOL field and its tests pass on all buildbots. On the flip side, the only other failure I've see was on the Debian Sparc buildbot. Which claims it is running sqlite 3.7.2-1. It works for me using 3.7.2 on Gentoo i86. > What should I look at? Good question :) When I added this it seemed like such a simple thing. Maybe the endianness issue is the one to look at, despite the pass on Solaris. -- R. David Murray www.bitdance.com From victor.stinner at haypocalc.com Sun Oct 17 00:00:15 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sun, 17 Oct 2010 00:00:15 +0200 Subject: [Python-Dev] r85569 - in python/branches/py3k: Misc/NEWS Parser/tokenizer.h Python/ast.c Python/bltinmodule.c Python/compile.c Python/pythonrun.c Python/traceback.c In-Reply-To: <20101016182631.47e139b8@pitrou.net> References: <20101016131410.D3DA2EE982@mail.python.org> <20101016182631.47e139b8@pitrou.net> Message-ID: <201010170000.16014.victor.stinner@haypocalc.com> Hi, Le samedi 16 octobre 2010 18:26:31, Antoine Pitrou a ?crit : > On Sat, 16 Oct 2010 15:14:10 +0200 (CEST) > > victor.stinner wrote: > > Author: victor.stinner > > Date: Sat Oct 16 15:14:10 2010 > > New Revision: 85569 > > > > Log: > > Issue #9713, #10114: Parser functions (eg. PyParser_ASTFromFile) expects > > filenames encoded to the filesystem encoding with surrogateescape error > > handler (to support undecodable bytes), instead of UTF-8 in strict mode. > > This change broke at least two buildbots: > http://www.python.org/dev/buildbot/builders/AMD64%20Gentoo%20Wide%203.x > http://www.python.org/dev/buildbot/builders/sparc%20solaris10%20gcc%203.x Yeah, I already noticed that. I added comments to #10114, and then opened issue #10123. #10123 should be fixed by r85578. -- Victor Stinner http://www.haypocalc.com/ From janssen at parc.com Sun Oct 17 02:23:30 2010 From: janssen at parc.com (Bill Janssen) Date: Sat, 16 Oct 2010 17:23:30 PDT Subject: [Python-Dev] PPC Leopard buildbot failing sqlite test for Python 3.2 In-Reply-To: <20101016205109.D307721B6E5@kimball.webabinitio.net> References: <88545.1287260060@parc.com> <20101016205109.D307721B6E5@kimball.webabinitio.net> Message-ID: <90695.1287275010@parc.com> R. David Murray wrote: > On Sat, 16 Oct 2010 13:14:20 -0700, Bill Janssen wrote: > > There was a test added to the sqlite suite between 3.1 and the current > > 3.x trunk, TestInTransaction. It's failing pretty consistently on the > > PPC Leopard buildbot (2.7 and 3.1 are testing OK). > > Calling sqlite3.sqlite_version returns 3.4.0. > > > > A couple of things come to mind: > > > > * Does this require a different flavor of the sqlite libraries? I > > looked through the 3.x docs and couldn't find any mention of a > > specific sqlite version. > > That's what I was wondering (I see you found the relevant issue...), > but it doesn't *seem* to be a version issue, but it would be > cool if you could check it against 3.7 just for confirmation. > > > * Is this a big-endian bug? Might fail on SPARC, too, then. > > It might be, except for two things: it passes on the Sparc Solaris 10 > buildbot, For 3.x, the "SPARC Solaris gcc" buildbot says, [ 87/349] test_sqlite test_sqlite skipped -- No module named _sqlite3 > and the io module also makes use of a T_BOOL field and > its tests pass on all buildbots. On the flip side, the only other > failure I've see was on the Debian Sparc buildbot. Which claims > it is running sqlite 3.7.2-1. It works for me using 3.7.2 on Gentoo > i86. > > > What should I look at? > > Good question :) > > When I added this it seemed like such a simple thing. Maybe the > endianness issue is the one to look at, despite the pass on Solaris. The SPARC architecture supports either big-endian and little-endian use (as does the PowerPC -- actually, I think any program can switch endianness on the PowerPC, except...); the OS gets to decide. I'd expect Solaris on SPARC to run big-endian, like OS X on PPC, and Wikipedia says Ubuntu on SPARC runs big-endian, too. Except: PPC Leopard is running on a G5 process, a PowerPC 970. As far as I know, that's the only PowerPC chip that couldn't run either-endian -- it only runs big-endian. So that code has to be running big-endian on PPC Leopard. Looks like it's not passing on any big-endian architecture. Bill From martin at v.loewis.de Sun Oct 17 20:10:21 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 17 Oct 2010 20:10:21 +0200 Subject: [Python-Dev] Locked-in defect? 32-bit hash values on 64-bit builds In-Reply-To: References: Message-ID: <4CBB3C0D.5000405@v.loewis.de> > ISTM, the use of 64-bit builds is growing in popularity. It was be a bummer > to have a locked-in an effective size limit for dictionaries and sets > because the API only supports 32-bit hash values. If the ABI is frozen with 3.2, it would still be possible to introduce "wide" hashes - it just will have to be a compatible change. > The thread seems to show agreement that the hash values should be > Py_ssize_t but the chance to fix it will be lost unless core devs get > more time to work on the problem or unless the ABI freeze is deferred. I agree in principle, also, but would like to see some analysis on existing third-party modules. Regards, Martin From ncoghlan at gmail.com Mon Oct 18 00:03:40 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 18 Oct 2010 08:03:40 +1000 Subject: [Python-Dev] Exposing pkguitl's import emulation (was Re: [Python-checkins] r85538 - python/branches/py3k/Doc/library/pkgutil.rst) Message-ID: I'm a little dubious about exposing these officially. They're mainly a hack to get some parts of the standard library working (e.g. runpy) in the absence of full PEP 302 support in the imp module, not really something we want to encourage anyone else to use (and yes, they should probably have underscores in their names, but we missed that when the various private implementations scattered around the stdlib were consolidated in pkgutil). That said, who knows when we'll actually have it done right, so in the meantime maybe having an official workaround is better than nothing... Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From rdmurray at bitdance.com Mon Oct 18 03:25:37 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 17 Oct 2010 21:25:37 -0400 Subject: [Python-Dev] Buildbot for AIX In-Reply-To: <4CB87587.9020100@users.sourceforge.net> References: <1284586127.79.0.767793857364.issue1633863@psf.upfronthosting.co.za> <4C93377C.4040300@users.sourceforge.net> <4C936225.1000500@v.loewis.de> <4C9770FB.1010103@users.sourceforge.net> <4C97B422.4040606@v.loewis.de> <4CB87587.9020100@users.sourceforge.net> Message-ID: <20101018012537.3F66C1FD3C1@kimball.webabinitio.net> On Fri, 15 Oct 2010 17:38:47 +0200, =?UTF-8?B?U8OpYmFzdGllbiBTYWJsw6k=?= wrote: > I finally got the authorization to run some buildbot slaves on our AIX > servers. > > I made some tests with a buildbot script as close as possible to what > you already use. Here is a patch that show the kind of modifications I > had to do in order to get the buildbot slave to correctly run on AIX. > > It is also necessary to apply the patch provided in issue 9862 so that > the tests won't hang forever. > > It also would help if the corrections provided in the following issues > could be commited: 4499, 678250, 730467. I've committed #9862, #4499 and #678250. I'd like a confirming opinion on #730467, though it sounds reasonable to me. -- R. David Murray www.bitdance.com From sable at users.sourceforge.net Mon Oct 18 12:19:55 2010 From: sable at users.sourceforge.net (=?ISO-8859-1?Q?S=E9bastien_Sabl=E9?=) Date: Mon, 18 Oct 2010 12:19:55 +0200 Subject: [Python-Dev] Buildbot for AIX In-Reply-To: References: <1284586127.79.0.767793857364.issue1633863@psf.upfronthosting.co.za> <4C93377C.4040300@users.sourceforge.net> <20100917150551.099113e4@pitrou.net> <4C9772A6.2070908@users.sourceforge.net> Message-ID: <4CBC1F4B.3090307@users.sourceforge.net> Hi Sridhar, Le 15/10/2010 18:50, Sridhar Ratnakumar a ?crit : > We definitely like to share our core Python patches for AIX 5.1/5.2 and other platforms. Great to hear that ActiveState shares their improvements for Python on AIX! Thanks for the patch in git, I will check it and try to open some issues in the Python bug tracker in order to get those modifications integrated. > Please do send us any patches you may have for building Python 3 on AIX. Here is a list of issues concerning AIX on which I have been working: http://bugs.python.org/issue678250 = OK (test_mmap error) http://bugs.python.org/issue678264 = WAIT (test_resource error) http://bugs.python.org/issue730467 = WAIT (AIX_GENUINE_CPLUSPLUS error) http://bugs.python.org/issue713169 (test_pty error) http://bugs.python.org/issue941346 = OK (shared library) http://bugs.python.org/issue1542544 = OK (shared library - duplicate) http://bugs.python.org/issue1563807 (compilation ctypes) http://bugs.python.org/issue1633863 = OK (bad default compiler) http://bugs.python.org/issue1745108 (curses error) http://bugs.python.org/issue3526 (custom memory allocation system - dlmalloc) http://bugs.python.org/issue4026 = OK (fcntl extension) http://bugs.python.org/issue4499 = OK (tilde macro) http://bugs.python.org/issue5718 (ctypes compilation) http://bugs.python.org/issue6006 (ctypes compilation) http://bugs.python.org/issue6164 (flag -blibpath) http://bugs.python.org/issue7657 (test_ctypes error) http://bugs.python.org/issue8695 = OK (crash on Python 2.6.4 at compilation) http://bugs.python.org/issue8882 (pb getsockaddrarg) http://bugs.python.org/issue9700 = OK (semwait) http://bugs.python.org/issue9799 = (xlc requires --without-computed-gotos) => OK add some doc http://bugs.python.org/issue9862 = OK (broken PIPE_BUF on AIX) The 2 most interesting patches I think are: * in issue 941346, a patch to be able to compile Python as a shared library (already applied in svn) * in issue 3526, a patch to use a custom memory allocation system (dlmalloc) in Python in order to greatly reduce memory consumption on AIX => not applied yet; I need to formally discuss it on Python-dev, but I am waiting for the AIX buildbot to be setup before that. best regards -- S?bastien Sabl? From merwok at netwok.org Mon Oct 18 14:45:49 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Mon, 18 Oct 2010 14:45:49 +0200 Subject: [Python-Dev] Fixing zipfile.BadZipfile to zipfile.BadZipFile In-Reply-To: References: Message-ID: <4CBC417D.1050005@netwok.org> Hello [Sorry if this comes twice, connection errors here] (A bit of context: The original message comes from bug #2775, ?Implement PEP 3108?, a meta-bug tracking stdlib reorganization for py3k.) > I am very glad you're reorganizing the Standard Library. Thumbs up! I > hope everything will comply to PEP 8 after you're done. You may have missed the timeline: Most of the PEP 3108 changes have been done before the first 3.x release went out. Now that we have 3.1 out as a stable and supported, we cannot reorganize and break compatibility anymore. (A note about PEP 8 compliance: Module names have been mostly fixed, but not all function/method names, for example in logging and unittest. If I recall correctly, readability did not seem to make all the rewrites worth it.) > Since you're reorganizing, I have my own contribution. I have attached > a patch. The issue7351 was not > accepted at the time, so I hope you'll accept this fix now. I?ve just re-read the answers there and they are still valid. Ezio and me: ?Your patch need to include an alias (BadZipfile = BadZipFile) to preserve compatibility with old pickles, as explains msg95477.? Antoine: ?I don't think changing it for the sake of aesthetics is a good deal given that many existing programs will have to be converted to the new spelling.? Regards From ncoghlan at gmail.com Mon Oct 18 15:13:44 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 18 Oct 2010 23:13:44 +1000 Subject: [Python-Dev] Fixing zipfile.BadZipfile to zipfile.BadZipFile In-Reply-To: <4CBC417D.1050005@netwok.org> References: <4CBC417D.1050005@netwok.org> Message-ID: On Mon, Oct 18, 2010 at 10:45 PM, ?ric Araujo wrote: > (A note about PEP 8 compliance: Module names have been mostly fixed, but > not all function/method names, for example in logging and unittest. ?If > I recall correctly, readability did not seem to make all the rewrites > worth it.) Correct. We went through this for one module that I recall (threading) and that was annoying enough that we mostly left things alone after that unless they were truly obnoxious. For threading we were able to clean a lot of things up in the process (such as adding properties where appropriate), but even so, we still made sure all the old names continued to work. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From merwok at netwok.org Mon Oct 18 17:20:13 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Mon, 18 Oct 2010 17:20:13 +0200 Subject: [Python-Dev] =?utf-8?q?About_resolution_=E2=80=9Caccepted?= =?utf-8?q?=E2=80=9D_on_the_tracker?= Message-ID: <4CBC65AD.1060401@netwok.org> Hi everyone [Sorry if this comes twice, connection errors here] Raymond Hettinger noticed on the tracker that there are different interpretations of the ?accepted? resolution: > Traditionally it denotes an approved patch, not a agreement that the > bug is valid. Daniel Stutzbach and I are (were) two users of the second meaning. It?s more useful to follow Raymond?s meaning, so that a simple query can find all accepted patches that are awaiting commit. I don?t remember if I took up this use from R. David Murray or someone else, so I thought I?d redirect the message to python-dev so that all triagers can see it. Forward to python-list as you see fit, I?m not subscribed there. Regards From chkneo at gmail.com Mon Oct 18 18:35:56 2010 From: chkneo at gmail.com (chandru) Date: Mon, 18 Oct 2010 22:05:56 +0530 Subject: [Python-Dev] How to get commit access Message-ID: I am waiting for the bug Issue5111 (httplib: wrong Host header when connecting to IPv6 litteral URL) to be fixed for a very long. Even I attached patches, test patches. How to get commit access so that I can fix such issues ( HTTP lib ) - Chandrasekar -------------- next part -------------- An HTML attachment was scrubbed... URL: From merwok at netwok.org Sun Oct 17 23:00:42 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sun, 17 Oct 2010 23:00:42 +0200 Subject: [Python-Dev] Fixing zipfile.BadZipfile to zipfile.BadZipFile In-Reply-To: References: Message-ID: <4CBB63FA.8080108@netwok.org> Hello (A bit of context: The original message comes from bug #2775, ?Implement PEP 3108?, a meta-bug tracking stdlib reorganization for py3k.) > I am very glad you're reorganizing the Standard Library. Thumbs up! I > hope everything will comply to PEP 8 after you're done. You may have missed the timeline: Most of the PEP 3108 changes have been done before the first 3.x release went out. Now that we have 3.1 out as a stable and supported, we cannot reorganize and break compatibility anymore. (A note about PEP 8 compliance: Module names have been mostly fixed, but not all function/method names, for example in logging and unittest. If I recall correctly, readability did not seem to make all the rewrites worth it.) > Since you're reorganizing, I have my own contribution. I have attached > a patch. The issue7351 was not > accepted at the time, so I hope you'll accept this fix now. I?ve just re-read the answers there and they are still valid. Ezio and me: ?Your patch need to include an alias (BadZipfile = BadZipFile) to preserve compatibility with old pickles, as explains msg95477.? Antoine: ?I don't think changing it for the sake of aesthetics is a good deal given that many existing programs will have to be converted to the new spelling.? Regards From orsenthil at gmail.com Mon Oct 18 19:24:01 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Mon, 18 Oct 2010 22:54:01 +0530 Subject: [Python-Dev] How to get commit access In-Reply-To: References: Message-ID: <20101018172401.GA2065@remy> Hello Chandrasekar, On Mon, Oct 18, 2010 at 10:05:56PM +0530, chandru wrote: > I am waiting for the bug?Issue5111 (httplib: wrong Host header when connecting > to IPv6 litteral URL) ?to be fixed for a very long. I just had a look at the bug. Looks like a minor change and tests are there too. I shall check that it does not break of any of existing changes and commit it. > > Even I attached patches, test patches. How to get commit access so that I can > fix such issues ( HTTP lib ) The details on "How to contribute to Python" are here: http://www.python.org/dev/contributing/ -- Senthil Blutarsky's Axiom: Nothing is impossible for the man who will not listen to reason. From merwok at netwok.org Mon Oct 18 16:04:06 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Mon, 18 Oct 2010 16:04:06 +0200 Subject: [Python-Dev] =?utf-8?q?About_resolution_=E2=80=9Caccepted?= =?utf-8?q?=E2=80=9D_on_the_tracker?= Message-ID: <4CBC53D6.7070602@netwok.org> Hi everyone Raymond Hettinger noticed on the tracker that there are different interpretations of the ?accepted? resolution: > Traditionally it denotes an approved patch, not a agreement that the > bug is valid. Daniel Stutzbach and I are (were) two users of the second meaning. It?s more useful to follow Raymond?s meaning, so that a simple query can find all accepted patches that are awaiting commit. I don?t remember if I took up this use from R. David Murray or someone else, so I thought I?d redirect the message to python-dev so that all triagers can see it. Forward to python-list as you see fit, I?m not subscribed there. Regards From barry at python.org Mon Oct 18 20:11:37 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 18 Oct 2010 14:11:37 -0400 Subject: [Python-Dev] =?utf-8?q?About_resolution_=E2=80=9Caccepted?= =?utf-8?q?=E2=80=9D__on_the_tracker?= In-Reply-To: <4CBC53D6.7070602@netwok.org> References: <4CBC53D6.7070602@netwok.org> Message-ID: <20101018141137.6946e9e9@mission> On Oct 18, 2010, at 04:04 PM, ?ric Araujo wrote: >Raymond Hettinger noticed on the tracker that there are different >interpretations of the ?accepted? resolution: > >> Traditionally it denotes an approved patch, not a agreement that the >> bug is valid. I'm with Raymond; I've always used 'accepted' to mean an approved patch. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From g.brandl at gmx.net Mon Oct 18 20:18:53 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 18 Oct 2010 20:18:53 +0200 Subject: [Python-Dev] =?windows-1252?q?About_resolution_=93accepted=94__on?= =?windows-1252?q?_the_tracker?= In-Reply-To: <20101018141137.6946e9e9@mission> References: <4CBC53D6.7070602@netwok.org> <20101018141137.6946e9e9@mission> Message-ID: Am 18.10.2010 20:11, schrieb Barry Warsaw: > On Oct 18, 2010, at 04:04 PM, ?ric Araujo wrote: > >>Raymond Hettinger noticed on the tracker that there are different >>interpretations of the ?accepted? resolution: >> >>> Traditionally it denotes an approved patch, not a agreement that the >>> bug is valid. > > I'm with Raymond; I've always used 'accepted' to mean an approved patch. Same here. I think of the resolution as only relevant when the issue status is closed (or pending, of course). Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From fuzzyman at voidspace.org.uk Mon Oct 18 21:04:49 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 18 Oct 2010 20:04:49 +0100 Subject: [Python-Dev] =?windows-1252?q?About_resolution_=93accepted=94__on?= =?windows-1252?q?_the_tracker?= In-Reply-To: References: <4CBC53D6.7070602@netwok.org> <20101018141137.6946e9e9@mission> Message-ID: <4CBC9A51.6090706@voidspace.org.uk> On 18/10/2010 19:18, Georg Brandl wrote: > Am 18.10.2010 20:11, schrieb Barry Warsaw: >> On Oct 18, 2010, at 04:04 PM, ?ric Araujo wrote: >> >>> Raymond Hettinger noticed on the tracker that there are different >>> interpretations of the ?accepted? resolution: >>> >>>> Traditionally it denotes an approved patch, not a agreement that the >>>> bug is valid. >> I'm with Raymond; I've always used 'accepted' to mean an approved patch. > Same here. I think of the resolution as only relevant when the issue > status is closed (or pending, of course). Whilst I agree we lack a "status" (resolution is probably not right) that says either this bug has been verified as genuine and needs fixing or for a feature request that it has been agreed that the feature can be added but not yet got as far as a completed patch. All the best, Michael > Georg > -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From g.brandl at gmx.net Mon Oct 18 21:24:39 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 18 Oct 2010 21:24:39 +0200 Subject: [Python-Dev] =?windows-1252?q?About_resolution_=93accepted=94__on?= =?windows-1252?q?_the_tracker?= In-Reply-To: <4CBC9A51.6090706@voidspace.org.uk> References: <4CBC53D6.7070602@netwok.org> <20101018141137.6946e9e9@mission> <4CBC9A51.6090706@voidspace.org.uk> Message-ID: Am 18.10.2010 21:04, schrieb Michael Foord: > On 18/10/2010 19:18, Georg Brandl wrote: >> Am 18.10.2010 20:11, schrieb Barry Warsaw: >>> On Oct 18, 2010, at 04:04 PM, ?ric Araujo wrote: >>> >>>> Raymond Hettinger noticed on the tracker that there are different >>>> interpretations of the ?accepted? resolution: >>>> >>>>> Traditionally it denotes an approved patch, not a agreement that the >>>>> bug is valid. >>> I'm with Raymond; I've always used 'accepted' to mean an approved patch. >> Same here. I think of the resolution as only relevant when the issue >> status is closed (or pending, of course). > > Whilst I agree we lack a "status" (resolution is probably not right) > that says either this bug has been verified as genuine and needs fixing > or for a feature request that it has been agreed that the feature can be > added but not yet got as far as a completed patch. Don't we have "stage" for that (at the moment at least)? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From fuzzyman at voidspace.org.uk Mon Oct 18 21:28:53 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 18 Oct 2010 20:28:53 +0100 Subject: [Python-Dev] =?windows-1252?q?About_resolution_=93accepted=94__on?= =?windows-1252?q?_the_tracker?= In-Reply-To: References: <4CBC53D6.7070602@netwok.org> <20101018141137.6946e9e9@mission> <4CBC9A51.6090706@voidspace.org.uk> Message-ID: <4CBC9FF5.3030502@voidspace.org.uk> On 18/10/2010 20:24, Georg Brandl wrote: > Am 18.10.2010 21:04, schrieb Michael Foord: >> On 18/10/2010 19:18, Georg Brandl wrote: >>> Am 18.10.2010 20:11, schrieb Barry Warsaw: >>>> On Oct 18, 2010, at 04:04 PM, ?ric Araujo wrote: >>>> >>>>> Raymond Hettinger noticed on the tracker that there are different >>>>> interpretations of the ?accepted? resolution: >>>>> >>>>>> Traditionally it denotes an approved patch, not a agreement that the >>>>>> bug is valid. >>>> I'm with Raymond; I've always used 'accepted' to mean an approved patch. >>> Same here. I think of the resolution as only relevant when the issue >>> status is closed (or pending, of course). >> Whilst I agree we lack a "status" (resolution is probably not right) >> that says either this bug has been verified as genuine and needs fixing >> or for a feature request that it has been agreed that the feature can be >> added but not yet got as far as a completed patch. > Don't we have "stage" for that (at the moment at least)? Which stage do you have in mind, "patch needed"? This is currently used for any issue that lacks a patch and doesn't mean that an issue has been accepted as "valid" in any way. All the best, Michael Foord > Georg > -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From g.brandl at gmx.net Mon Oct 18 21:31:24 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 18 Oct 2010 21:31:24 +0200 Subject: [Python-Dev] =?windows-1252?q?About_resolution_=93accepted=94__on?= =?windows-1252?q?_the_tracker?= In-Reply-To: <4CBC9FF5.3030502@voidspace.org.uk> References: <4CBC53D6.7070602@netwok.org> <20101018141137.6946e9e9@mission> <4CBC9A51.6090706@voidspace.org.uk> <4CBC9FF5.3030502@voidspace.org.uk> Message-ID: Am 18.10.2010 21:28, schrieb Michael Foord: > On 18/10/2010 20:24, Georg Brandl wrote: >> Am 18.10.2010 21:04, schrieb Michael Foord: >>> On 18/10/2010 19:18, Georg Brandl wrote: >>>> Am 18.10.2010 20:11, schrieb Barry Warsaw: >>>>> On Oct 18, 2010, at 04:04 PM, ?ric Araujo wrote: >>>>> >>>>>> Raymond Hettinger noticed on the tracker that there are different >>>>>> interpretations of the ?accepted? resolution: >>>>>> >>>>>>> Traditionally it denotes an approved patch, not a agreement that the >>>>>>> bug is valid. >>>>> I'm with Raymond; I've always used 'accepted' to mean an approved patch. >>>> Same here. I think of the resolution as only relevant when the issue >>>> status is closed (or pending, of course). >>> Whilst I agree we lack a "status" (resolution is probably not right) >>> that says either this bug has been verified as genuine and needs fixing >>> or for a feature request that it has been agreed that the feature can be >>> added but not yet got as far as a completed patch. >> Don't we have "stage" for that (at the moment at least)? > > Which stage do you have in mind, "patch needed"? > > This is currently used for any issue that lacks a patch and doesn't mean > that an issue has been accepted as "valid" in any way. This is probably sophistry, but if an issue is invalid, it doesn't need a patch :) The first stage seems to be "unit test needed" anyway, which sounds to me a bit like "needs to be checked for reproducibility/validity". Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From janzert at janzert.com Mon Oct 18 21:33:09 2010 From: janzert at janzert.com (Janzert) Date: Mon, 18 Oct 2010 15:33:09 -0400 Subject: [Python-Dev] Digital video basics tutorial Message-ID: http://xiph.org/video/vid1.shtml From janzert at janzert.com Mon Oct 18 21:37:05 2010 From: janzert at janzert.com (Janzert) Date: Mon, 18 Oct 2010 15:37:05 -0400 Subject: [Python-Dev] Digital video basics tutorial In-Reply-To: References: Message-ID: On 10/18/2010 3:33 PM, Janzert wrote: > http://xiph.org/video/vid1.shtml > Sorry, sent to the wrong place. Janzert From solipsis at pitrou.net Mon Oct 18 21:42:08 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 18 Oct 2010 21:42:08 +0200 Subject: [Python-Dev] =?windows-1252?q?About_resolution_=93accepted=94_on_?= =?windows-1252?q?the_tracker?= References: <4CBC53D6.7070602@netwok.org> <20101018141137.6946e9e9@mission> <4CBC9A51.6090706@voidspace.org.uk> <4CBC9FF5.3030502@voidspace.org.uk> Message-ID: <20101018214208.467b3b2f@pitrou.net> On Mon, 18 Oct 2010 21:31:24 +0200 Georg Brandl wrote: > > This is probably sophistry, but if an issue is invalid, it doesn't need > a patch :) Not only, but it generally gets closed too. > The first stage seems to be "unit test needed" anyway, which > sounds to me a bit like "needs to be checked for reproducibility/validity". I don't like this first stage, it makes it look like we mandate a proper unit test to proceed with actually writing patches, which is really not true. Regards Antoine. From rdmurray at bitdance.com Tue Oct 19 01:57:04 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 18 Oct 2010 19:57:04 -0400 Subject: [Python-Dev] =?utf-8?q?About_resolution_=E2=80=9Caccepted?= =?utf-8?q?=E2=80=9D_on_the_tracker?= In-Reply-To: <4CBC65AD.1060401@netwok.org> References: <4CBC65AD.1060401@netwok.org> Message-ID: <20101018235704.2F99C222AE0@kimball.webabinitio.net> On Mon, 18 Oct 2010 17:20:13 +0200, wrote: > Raymond Hettinger noticed on the tracker that there are different > interpretations of the ???accepted??? resolution: > > > Traditionally it denotes an approved patch, not a agreement that the > > bug is valid. > > Daniel Stutzbach and I are (were) two users of the second meaning. It???s > more useful to follow Raymond???s meaning, so that a simple query can find > all accepted patches that are awaiting commit. > > I don???t remember if I took up this use from R. David Murray or someone > else, so I thought I???d redirect the message to python-dev so that all > triagers can see it. Forward to python-list as you see fit, I???m not > subscribed there. 'twasn't me, I figured the resolution field was only for resolved bugs. I did notice a number of people using for accepted feature requests or 'valid bugs', or sometimes patches accepted in principle that were nonetheless not quite complete. Clearly (some) people would *like* a toggle that indicates such a status :) -- R. David Murray www.bitdance.com From rdmurray at bitdance.com Tue Oct 19 02:07:34 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 18 Oct 2010 20:07:34 -0400 Subject: [Python-Dev] =?windows-1252?q?About_resolution_=93accepted=94_on_?= =?windows-1252?q?the_tracker?= In-Reply-To: <20101018214208.467b3b2f@pitrou.net> References: <4CBC53D6.7070602@netwok.org> <20101018141137.6946e9e9@mission> <4CBC9A51.6090706@voidspace.org.uk> <4CBC9FF5.3030502@voidspace.org.uk> <20101018214208.467b3b2f@pitrou.net> Message-ID: <20101019000734.4ECFD218A05@kimball.webabinitio.net> On Mon, 18 Oct 2010 21:42:08 +0200, Antoine Pitrou wrote: > On Mon, 18 Oct 2010 21:31:24 +0200 > Georg Brandl wrote: > > > > This is probably sophistry, but if an issue is invalid, it doesn't need > > a patch :) > > Not only, but it generally gets closed too. > > > The first stage seems to be "unit test needed" anyway, which > > sounds to me a bit like "needs to be checked for reproducibility/validity". > > I don't like this first stage, it makes it look like we mandate a > proper unit test to proceed with actually writing patches, which is > really not true. Why isn't it? :) Seriously, though, what it indicates is indicates is that we need a unit test for the patch to be complete. We have a number of issues with patches but no tests, I believe. Which order 'unit test' and 'fix' occur in is arbitrary in practice. I certainly prefer to have the unit tests first myself, though. The problem is that the stage field really isn't all that useful. I'd prefer a set of check boxes, as I've suggested in the wiki. I was the one who advocated labeling it 'unit test needed', but if people would rather change it back to just 'test needed', I will raise no objection, since in practice trying to squeeze the meaning I wanted into the stage field doesn't really work. -- R. David Murray www.bitdance.com From victor.stinner at haypocalc.com Tue Oct 19 03:53:34 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 19 Oct 2010 03:53:34 +0200 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done Message-ID: <201010190353.34796.victor.stinner@haypocalc.com> Hi, Seven months after my first commit related to this issue, the full test suite of Python 3.2 pass with ASCII, ISO-8859-1 and UTF-8 locale encodings in a non- ascii source directory. It means that Python 3.2 now process correctly filenames in all modules, build scripts and other utilities, with any locale encoding. General changes: * Encode/decode filenames with the locale encoding, instead of utf-8, until the filesystem is set * mbcs encoding (Windows filesystem encoding) is now strict by default, whereas it ignores unencodable characters and replace undecodable bytes in Python 3.1. Old behaviour can still be used using the right error handler: 'ignore' to encode, 'replace' to decode. * tarfile uses utf-8 encoding on Windows (instead of mbcs), and the surrogateescape error handler on all OSes * sys.getfilesystemencoding() cannot be None anymore * Don't accept bytearray as filenames anymore Changes of the Python API: * Add os.environb: bytes version of os.environ, os.getenvb() function and os.supports_bytes_environ constant * Add os.fsencode() and os.fsdecode() functions * Remove sys.setfilesystemencoding() function Changes of the C API: * Add PyUnicode_EncodeFSDefault() function * Add PyUnicode_FSDecoder() ParseTuple converter * Add PySys_FormatStdout(), PySys_FormatStderr() and PyErr_WarnFormat() functions * Add PyUnicode_AsWideCharString() function: don't need a buffer size. * Add Py_UNICODE_strrchr(), Py_UNICODE_strcat(), PyUnicode_AsUnicodeCopy() and Py_UNICODE_strncmp() functions * PyUnicode_DecodeFSDefault() and PyUnicode_DecodeFSDefaultAndSize() use the surrogateescape error handler * File utilities: add _Py_wchar2char() (reverse of Py_char2wchar()), _Py_stat() and _Py_fopen() functions; move all file utilities to Python/fileutils.c * The format string of PyUnicode_FromFormat() and PyErr_Format() is now pure ASCII: raise an error on non-ascii character * PyUnicode_FSConverter() doesn't accept bytearray anymore Bugfixes: * Fix modules: tarfile, pickle, pickletools, ctypes, subprocess, bz2, ssl, profile, xmlrpclib, platform, libpython (gdb plugin), sqlite, distutils.log, locale, _warnings, zipimport, imp * Fix functions: os.exec*(), os.system(), ctypes.dlopen(), os.getenv(), os.get_exec_path() * Fix tests: test_gdb, test_httpservers, test_cmd_line, test_size, test_generic_path, test_subprocess, test_doctest, test_cmd_line_script * Fix utf-8 encoder to support error handlers producing unicode string (eg. 'backslashreplace') * Fix conversion from unicode to a wide character string if Py_UNICODE and wchar_t have different sizes: UTF-16 => UTF-32 or UTF-32 => UTF-16 * Fix Python command line parser if the the command line contains surrogates * Avoid _PyUnicode_AsString() because it returns NULL if the string contains surrogates, or catch the error * Fix regrtest.py to support surrogate characters in the current working directory and in the tracebacks I wrote also some tests and documentation. The most difficult part was to debug Python initialization (Py_InitializeEx and calculate_path) and the import machinery (import.c, zipimport.c), because gdb does sometimes crash (for various reasons) and because the import machinery is fragile and difficult to understand. A special thanks to Marc-Andre Lemburg, Martin v. L?wis, Antoine Pitrou and Amaury Forgeot d'Arc for their help, useful advices and code reviews! -- Bonus: short story of PYTHONFSENCODING --- In the middle of August, I created the PYTHONFSENCODING environment variable, as suggested by Marc-Andre Lemburg. Because of this variable and because Python used utf-8 until the filesystem encoding is known, I had to write ugly and fragile "redecode" functions to redecode all filenames of all objects (sys.path, sys.meta_path, sys.executable, sys.modules, all code objects, etc.). Then I found 4 issues related to PYTHONFSENCODING, inconsistencies between the filesystem encoding and the locale encoding. It was not easy to decide how to fix these issues, but at the end, we choosed to drop PYTHONFSENCODING variable, use the locale encoding as the filesystem encoding, and always use utf-8 as the filesystem encoding on Mac OS X. -- Victor Stinner http://www.haypocalc.com/ From rrr at ronadam.com Tue Oct 19 05:27:06 2010 From: rrr at ronadam.com (Ron Adam) Date: Mon, 18 Oct 2010 22:27:06 -0500 Subject: [Python-Dev] =?windows-1252?q?About_resolution_=93accepted=94_on_?= =?windows-1252?q?the_tracker?= In-Reply-To: <20101019000734.4ECFD218A05@kimball.webabinitio.net> References: <4CBC53D6.7070602@netwok.org> <20101018141137.6946e9e9@mission> <4CBC9A51.6090706@voidspace.org.uk> <4CBC9FF5.3030502@voidspace.org.uk> <20101018214208.467b3b2f@pitrou.net> <20101019000734.4ECFD218A05@kimball.webabinitio.net> Message-ID: <4CBD100A.4000004@ronadam.com> On 10/18/2010 07:07 PM, R. David Murray wrote: > Seriously, though, what it indicates is indicates is that we need a unit > test for the patch to be complete. We have a number of issues with > patches but no tests, I believe. Which order 'unit test' and 'fix' > occur in is arbitrary in practice. I certainly prefer to have the unit > tests first myself, though. > > The problem is that the stage field really isn't all that useful. I'd > prefer a set of check boxes, as I've suggested in the wiki. > > I was the one who advocated labeling it 'unit test needed', but if > people would rather change it back to just 'test needed', I will raise > no objection, since in practice trying to squeeze the meaning I wanted > into the stage field doesn't really work. This is about communicating both content and quality, with several decisions at certain points. It's not a straight line step by step process, which is why it gets confusing if you try to treat it as such. Check boxes work well for things like this. Could something like (or parts of) the following work? It would have assignment and module keywords items as well. [] boxes can be set or unset by both the bug/feature author and those with tracker/commit privileges. {} boxes settable only by those with tracker and commit privileges? Title [________________________________] Description [_______________________...] TYPE: [] Bug Versions: [] 2.5 [] 2.6 [] 2.7 [] 3.0 [] 3.1 [] 3.2 ... {} Verified [] Feature Request For Version [_____] *Drop down list. {} Initial approval *May be rejected later. Requires PEP: {} yes PEP Number [______] {} no STATUS: Components: [] Code Patch {} approved [] Test patch {} approved [] Document Patch {} approved [] News Entry Patch {} approved [] Final Commit Approval {} approved {} Committed Revision #(s) {____________________} Notes:{____________________________} {} Rejected Reason:{___________} *Drop down list here. {} Closed ATTENTION REQUESTS: [] Request bug verification *Auto set when new bug. [] Request feature initial approval *Auto set when new. [] Request author reassignment *Current author unable to work on this. Request test patch [] review [] approval Request code patch [] review [] approval Request document patch [] review [] approval Request news patch [] review [] approval Request final commit [] review [] approval *More than one attention request can be set at a time. *Reviewer clears request if revision is needed or item is ok. Most changes to any of these should also be documented in the discussion area as well. Ron From rrr at ronadam.com Tue Oct 19 05:27:06 2010 From: rrr at ronadam.com (Ron Adam) Date: Mon, 18 Oct 2010 22:27:06 -0500 Subject: [Python-Dev] =?windows-1252?q?About_resolution_=93accepted=94_on_?= =?windows-1252?q?the_tracker?= In-Reply-To: <20101019000734.4ECFD218A05@kimball.webabinitio.net> References: <4CBC53D6.7070602@netwok.org> <20101018141137.6946e9e9@mission> <4CBC9A51.6090706@voidspace.org.uk> <4CBC9FF5.3030502@voidspace.org.uk> <20101018214208.467b3b2f@pitrou.net> <20101019000734.4ECFD218A05@kimball.webabinitio.net> Message-ID: <4CBD100A.4000004@ronadam.com> On 10/18/2010 07:07 PM, R. David Murray wrote: > Seriously, though, what it indicates is indicates is that we need a unit > test for the patch to be complete. We have a number of issues with > patches but no tests, I believe. Which order 'unit test' and 'fix' > occur in is arbitrary in practice. I certainly prefer to have the unit > tests first myself, though. > > The problem is that the stage field really isn't all that useful. I'd > prefer a set of check boxes, as I've suggested in the wiki. > > I was the one who advocated labeling it 'unit test needed', but if > people would rather change it back to just 'test needed', I will raise > no objection, since in practice trying to squeeze the meaning I wanted > into the stage field doesn't really work. This is about communicating both content and quality, with several decisions at certain points. It's not a straight line step by step process, which is why it gets confusing if you try to treat it as such. Check boxes work well for things like this. Could something like (or parts of) the following work? It would have assignment and module keywords items as well. [] boxes can be set or unset by both the bug/feature author and those with tracker/commit privileges. {} boxes settable only by those with tracker and commit privileges? Title [________________________________] Description [_______________________...] TYPE: [] Bug Versions: [] 2.5 [] 2.6 [] 2.7 [] 3.0 [] 3.1 [] 3.2 ... {} Verified [] Feature Request For Version [_____] *Drop down list. {} Initial approval *May be rejected later. Requires PEP: {} yes PEP Number [______] {} no STATUS: Components: [] Code Patch {} approved [] Test patch {} approved [] Document Patch {} approved [] News Entry Patch {} approved [] Final Commit Approval {} approved {} Committed Revision #(s) {____________________} Notes:{____________________________} {} Rejected Reason:{___________} *Drop down list here. {} Closed ATTENTION REQUESTS: [] Request bug verification *Auto set when new bug. [] Request feature initial approval *Auto set when new. [] Request author reassignment *Current author unable to work on this. Request test patch [] review [] approval Request code patch [] review [] approval Request document patch [] review [] approval Request news patch [] review [] approval Request final commit [] review [] approval *More than one attention request can be set at a time. *Reviewer clears request if revision is needed or item is ok. Most changes to any of these should also be documented in the discussion area as well. Ron From rrr at ronadam.com Tue Oct 19 06:21:32 2010 From: rrr at ronadam.com (Ron Adam) Date: Mon, 18 Oct 2010 23:21:32 -0500 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <201010190353.34796.victor.stinner@haypocalc.com> References: <201010190353.34796.victor.stinner@haypocalc.com> Message-ID: <4CBD1CCC.9030606@ronadam.com> On 10/18/2010 08:53 PM, Victor Stinner wrote: > Hi, > > Seven months after my first commit related to this issue, the full test suite > of Python 3.2 pass with ASCII, ISO-8859-1 and UTF-8 locale encodings in a non- > ascii source directory. It means that Python 3.2 now process correctly > filenames in all modules, build scripts and other utilities, with any locale > encoding. [...] Congratulations Victor, From what I saw it looked like it was a lot of work. I don't suppose you could take a look at this issue also? http://bugs.python.org/issue9319 (If not, maybe someone else can.) When pydoc uses imp to search for modules it runs across a test file with a bad BOM. Which then causes a segfault. ra at Gutsy:~/svn/py3k$ ./python Python 3.2a3+ (py3k:85719, Oct 18 2010, 22:32:47) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> help('modules ""') Here is a list of matching modules. Enter any module name to get more help. Segmentation fault Or more directly... >>> import imp >>> imp.find_module('test/badsyntax_pep3120') Segmentation fault I believe it should issue a SyntaxError instead. Thanks, Ron Adam From rrr at ronadam.com Tue Oct 19 06:21:32 2010 From: rrr at ronadam.com (Ron Adam) Date: Mon, 18 Oct 2010 23:21:32 -0500 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <201010190353.34796.victor.stinner@haypocalc.com> References: <201010190353.34796.victor.stinner@haypocalc.com> Message-ID: <4CBD1CCC.9030606@ronadam.com> On 10/18/2010 08:53 PM, Victor Stinner wrote: > Hi, > > Seven months after my first commit related to this issue, the full test suite > of Python 3.2 pass with ASCII, ISO-8859-1 and UTF-8 locale encodings in a non- > ascii source directory. It means that Python 3.2 now process correctly > filenames in all modules, build scripts and other utilities, with any locale > encoding. [...] Congratulations Victor, From what I saw it looked like it was a lot of work. I don't suppose you could take a look at this issue also? http://bugs.python.org/issue9319 (If not, maybe someone else can.) When pydoc uses imp to search for modules it runs across a test file with a bad BOM. Which then causes a segfault. ra at Gutsy:~/svn/py3k$ ./python Python 3.2a3+ (py3k:85719, Oct 18 2010, 22:32:47) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> help('modules ""') Here is a list of matching modules. Enter any module name to get more help. Segmentation fault Or more directly... >>> import imp >>> imp.find_module('test/badsyntax_pep3120') Segmentation fault I believe it should issue a SyntaxError instead. Thanks, Ron Adam From merwok at netwok.org Tue Oct 19 06:52:31 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 19 Oct 2010 06:52:31 +0200 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <201010190353.34796.victor.stinner@haypocalc.com> References: <201010190353.34796.victor.stinner@haypocalc.com> Message-ID: <4CBD240F.6090401@netwok.org> Congratulations Victor! This is not a small feat. The PSU should send you cookies to thank you, but they won?t since they don?t exist and From p.f.moore at gmail.com Tue Oct 19 09:23:41 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 19 Oct 2010 08:23:41 +0100 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <4CBD240F.6090401@netwok.org> References: <201010190353.34796.victor.stinner@haypocalc.com> <4CBD240F.6090401@netwok.org> Message-ID: On 19 October 2010 05:52, ?ric Araujo wrote: > Congratulations Victor! ?This is not a small feat. ?The PSU should send > you cookies to thank you, but they won?t since they don?t exist and What? Cookies don't exist??? Paul. From solipsis at pitrou.net Tue Oct 19 10:15:20 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 19 Oct 2010 10:15:20 +0200 Subject: [Python-Dev] =?utf-8?q?About_resolution_=E2=80=9Caccepted?= =?utf-8?q?=E2=80=9D_on_the_tracker?= In-Reply-To: <4CBD100A.4000004@ronadam.com> References: <4CBC53D6.7070602@netwok.org> <20101018141137.6946e9e9@mission> <4CBC9A51.6090706@voidspace.org.uk> <4CBC9FF5.3030502@voidspace.org.uk> <20101018214208.467b3b2f@pitrou.net> <20101019000734.4ECFD218A05@kimball.webabinitio.net> <4CBD100A.4000004@ronadam.com> Message-ID: <20101019101520.02391488@pitrou.net> On Mon, 18 Oct 2010 22:27:06 -0500 Ron Adam wrote: > > Could something like (or parts of) the following work? It would have > assignment and module keywords items as well. Well, this is very nice, except that the more complicated the form is, the less likely people are to fill it (and even less, fill it properly). +1 for relabelling "unit test needed" as simply "test needed". Regards Antoine. From mal at egenix.com Tue Oct 19 10:29:20 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 19 Oct 2010 10:29:20 +0200 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <201010190353.34796.victor.stinner@haypocalc.com> References: <201010190353.34796.victor.stinner@haypocalc.com> Message-ID: <4CBD56E0.2060008@egenix.com> Victor Stinner wrote: > Hi, > > Seven months after my first commit related to this issue, the full test suite > of Python 3.2 pass with ASCII, ISO-8859-1 and UTF-8 locale encodings in a non- > ascii source directory. It means that Python 3.2 now process correctly > filenames in all modules, build scripts and other utilities, with any locale > encoding. > > > General changes: > > * Encode/decode filenames with the locale encoding, instead of utf-8, > until the filesystem encoding is set > * mbcs encoding (Windows filesystem encoding) is now strict by default, > whereas it ignores unencodable characters and replace undecodable bytes > in Python 3.1. Old behaviour can still be used using the right error > handler: 'ignore' to encode, 'replace' to decode. > * tarfile uses utf-8 encoding on Windows (instead of mbcs), and the > surrogateescape error handler on all OSes > * sys.getfilesystemencoding() cannot be None anymore > * Don't accept bytearray as filenames anymore > > > Changes of the Python API: > > * Add os.environb: bytes version of os.environ, os.getenvb() function > and os.supports_bytes_environ constant > * Add os.fsencode() and os.fsdecode() functions > * Remove sys.setfilesystemencoding() function > > > Changes of the C API: > > * Add PyUnicode_EncodeFSDefault() function > * Add PyUnicode_FSDecoder() ParseTuple converter > * Add PySys_FormatStdout(), PySys_FormatStderr() and PyErr_WarnFormat() > functions > * Add PyUnicode_AsWideCharString() function: don't need a buffer size. > * Add Py_UNICODE_strrchr(), Py_UNICODE_strcat(), PyUnicode_AsUnicodeCopy() > and Py_UNICODE_strncmp() functions > * PyUnicode_DecodeFSDefault() and PyUnicode_DecodeFSDefaultAndSize() use the > surrogateescape error handler > * File utilities: add _Py_wchar2char() (reverse of Py_char2wchar()), > _Py_stat() and _Py_fopen() functions; move all file utilities to > Python/fileutils.c > * The format string of PyUnicode_FromFormat() and PyErr_Format() is now > pure ASCII: raise an error on non-ascii character > * PyUnicode_FSConverter() doesn't accept bytearray anymore > > > Bugfixes: > > * Fix modules: tarfile, pickle, pickletools, ctypes, subprocess, bz2, ssl, > profile, xmlrpclib, platform, libpython (gdb plugin), sqlite, > distutils.log, locale, _warnings, zipimport, imp > * Fix functions: os.exec*(), os.system(), ctypes.dlopen(), os.getenv(), > os.get_exec_path() > * Fix tests: test_gdb, test_httpservers, test_cmd_line, test_size, > test_generic_path, test_subprocess, test_doctest, test_cmd_line_script > * Fix utf-8 encoder to support error handlers producing unicode string > (eg. 'backslashreplace') > * Fix conversion from unicode to a wide character string if Py_UNICODE > and wchar_t have different sizes: UTF-16 => UTF-32 or UTF-32 => UTF-16 > * Fix Python command line parser if the the command line contains surrogates > * Avoid _PyUnicode_AsString() because it returns NULL if the string contains > surrogates, or catch the error > * Fix regrtest.py to support surrogate characters in the current working > directory and in the tracebacks > > > I wrote also some tests and documentation. > > The most difficult part was to debug Python initialization (Py_InitializeEx > and calculate_path) and the import machinery (import.c, zipimport.c), because > gdb does sometimes crash (for various reasons) and because the import > machinery is fragile and difficult to understand. > > A special thanks to Marc-Andre Lemburg, Martin v. L?wis, Antoine Pitrou and > Amaury Forgeot d'Arc for their help, useful advices and code reviews! Many thanks to you for opening up this can of worms and fighting through all the issues ! > -- Bonus: short story of PYTHONFSENCODING --- > > In the middle of August, I created the PYTHONFSENCODING environment variable, > as suggested by Marc-Andre Lemburg. Because of this variable and because > Python used utf-8 until the filesystem encoding is known, I had to write ugly > and fragile "redecode" functions to redecode all filenames of all objects > (sys.path, sys.meta_path, sys.executable, sys.modules, all code objects, > etc.). > > Then I found 4 issues related to PYTHONFSENCODING, inconsistencies between the > filesystem encoding and the locale encoding. It was not easy to decide how to > fix these issues, but at the end, we choosed to drop PYTHONFSENCODING > variable, use the locale encoding as the filesystem encoding, and always use > utf-8 as the filesystem encoding on Mac OS X. Time will tell whether we can manage without some logic to tell Python what to use as file system encoding without having to rely on the locale settings. Anything is better than having Python stop with a fatal error, though :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 19 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ncoghlan at gmail.com Tue Oct 19 12:02:43 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 19 Oct 2010 20:02:43 +1000 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <201010190353.34796.victor.stinner@haypocalc.com> References: <201010190353.34796.victor.stinner@haypocalc.com> Message-ID: On Tue, Oct 19, 2010 at 11:53 AM, Victor Stinner wrote: > Hi, > > Seven months after my first commit related to this issue, the full test suite > of Python 3.2 pass with ASCII, ISO-8859-1 and UTF-8 locale encodings in a non- > ascii source directory. It means that Python 3.2 now process correctly > filenames in all modules, build scripts and other utilities, with any locale > encoding. Well done on facing up to that particular dragon and slaying it. Especially this part: > the import machinery (import.c, zipimport.c), because > gdb does sometimes crash (for various reasons) and because ?the import > machinery is fragile and difficult to understand. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From barry at python.org Tue Oct 19 16:12:56 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 19 Oct 2010 10:12:56 -0400 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <201010190353.34796.victor.stinner@haypocalc.com> References: <201010190353.34796.victor.stinner@haypocalc.com> Message-ID: <20101019101256.73afa7c2@mission> On Oct 19, 2010, at 03:53 AM, Victor Stinner wrote: >Seven months after my first commit related to this issue, the full test suite >of Python 3.2 pass with ASCII, ISO-8859-1 and UTF-8 locale encodings in a non- >ascii source directory. It means that Python 3.2 now process correctly >filenames in all modules, build scripts and other utilities, with any locale >encoding. Very impressive. Congratulations, and thanks for following through on what must have been some tricky work in many of Python's deep dark corners. Going forward, is there adequate documentation, guidelines, and safeguards for future coders so that they Do The Right Thing with new code? Perhaps a short How To in the standard documentation would be helpful, with links to it from any old/bad API calls? >The most difficult part was to debug Python initialization (Py_InitializeEx >and calculate_path) and the import machinery (import.c, zipimport.c), because >gdb does sometimes crash (for various reasons) and because the import >machinery is fragile and difficult to understand. You are also a master of the understatement! :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From pje at telecommunity.com Tue Oct 19 17:24:59 2010 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 19 Oct 2010 11:24:59 -0400 Subject: [Python-Dev] Exposing pkguitl's import emulation (was Re: [Python-checkins] r85538 - python/branches/py3k/Doc/library/pkgutil.rst) In-Reply-To: References: Message-ID: <20101019152452.354EC3A40DF@sparrow.telecommunity.com> At 08:03 AM 10/18/2010 +1000, Nick Coghlan wrote: >I'm a little dubious about exposing these officially. They're mainly a >hack to get some parts of the standard library working (e.g. runpy) in >the absence of full PEP 302 support in the imp module, not really >something we want to encourage anyone else to use (and yes, they >should probably have underscores in their names, but we missed that >when the various private implementations scattered around the stdlib >were consolidated in pkgutil). Well, my intention at least was that they should be documented and released; it's the documenting part I didn't get around to. ;-) Of course, this was also pre-importlib; were we starting the work today, the obvious thing to do would be to expose the Python implementations of the relevant objects. >That said, who knows when we'll actually have it done right, so in the >meantime maybe having an official workaround is better than nothing... > >Cheers, >Nick. > >-- >Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia >_______________________________________________ >Python-Dev mailing list >Python-Dev at python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: >http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com From jcea at jcea.es Tue Oct 19 18:50:26 2010 From: jcea at jcea.es (Jesus Cea) Date: Tue, 19 Oct 2010 18:50:26 +0200 Subject: [Python-Dev] Support for async read/write Message-ID: <4CBDCC52.8080209@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Current Python lacks support for "aio_*" syscalls to do async IO. I think this could be a nice addition for python 3.3. If you agree, I will create an issue in the tracker. If you think the idea is of no value, please say so for me to move on. Maybe an 3th party module, but I think this functionality sould be available in core python. Thanks!. PS: The function calls are: aio_cancel, aio_error, aio_fsync, aio_read, aio_return, aio_write. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBTL3MUplgi5GaxT1NAQKv5QQAnDan88kXJ67fucz2rT/mZze+065lm9E4 +XJ2JfqGMVE1/qMsXwg81l19RHSYReBgBjd7zyXWE9Fk/1Rfq4gjOZEtoG0IpGZG E3CH+Ab5/o/PjJZNKQaPpe0cwGSXFPKkPpgepKS1d8ZXyf6VXMc8UTSWjMI5jIVe 4M+yvw5m4I0= =nsdO -----END PGP SIGNATURE----- From peter.a.portante at gmail.com Tue Oct 19 19:16:43 2010 From: peter.a.portante at gmail.com (Peter Portante) Date: Tue, 19 Oct 2010 13:16:43 -0400 Subject: [Python-Dev] Support for async read/write In-Reply-To: <4CBDCC52.8080209@jcea.es> Message-ID: Yes, that would be a nice addition. -peter On 10/19/10 12:50 PM, "Jesus Cea" wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Current Python lacks support for "aio_*" syscalls to do async IO. I > think this could be a nice addition for python 3.3. > > If you agree, I will create an issue in the tracker. If you think the > idea is of no value, please say so for me to move on. Maybe an 3th party > module, but I think this functionality sould be available in core python. > > Thanks!. > > PS: The function calls are: aio_cancel, aio_error, aio_fsync, aio_read, > aio_return, aio_write. > > - -- > Jesus Cea Avion _/_/ _/_/_/ _/_/_/ > jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ > jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ > . _/_/ _/_/ _/_/ _/_/ _/_/ > "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ > "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ > "El amor es poner tu felicidad en la felicidad de otro" - Leibniz > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQCVAwUBTL3MUplgi5GaxT1NAQKv5QQAnDan88kXJ67fucz2rT/mZze+065lm9E4 > +XJ2JfqGMVE1/qMsXwg81l19RHSYReBgBjd7zyXWE9Fk/1Rfq4gjOZEtoG0IpGZG > E3CH+Ab5/o/PjJZNKQaPpe0cwGSXFPKkPpgepKS1d8ZXyf6VXMc8UTSWjMI5jIVe > 4M+yvw5m4I0= > =nsdO > -----END PGP SIGNATURE----- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/peter.a.portante%40gmail.com From exarkun at twistedmatrix.com Tue Oct 19 19:47:39 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Tue, 19 Oct 2010 17:47:39 -0000 Subject: [Python-Dev] Support for async read/write In-Reply-To: <4CBDCC52.8080209@jcea.es> References: <4CBDCC52.8080209@jcea.es> Message-ID: <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> On 04:50 pm, jcea at jcea.es wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >Current Python lacks support for "aio_*" syscalls to do async IO. I >think this could be a nice addition for python 3.3. Adding more platform wrappers is always nice. Keep in mind that the quality of most (all?) aio_* implementations is spotty at best, though. On Linux, they will sometimes block (for example, if you fail to align buffers properly, or open a file without O_DIRECT, or if there are too many other aio operations active on the system at the time, etc). Thus, these APIs are finicky at best, and the Python bindings will be similarly fragile. Jean-Paul >If you agree, I will create an issue in the tracker. If you think the >idea is of no value, please say so for me to move on. Do you have an application in mind? >Maybe an 3th party >module, but I think this functionality sould be available in core >python. Starting off as a 3rd party library to try to develop some interest and users (and thus experience) before adding it to the standard library might make sense (as it frequently does). >Thanks!. > >PS: The function calls are: aio_cancel, aio_error, aio_fsync, aio_read, >aio_return, aio_write. Jean-Paul From g.rodola at gmail.com Tue Oct 19 19:57:09 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Tue, 19 Oct 2010 19:57:09 +0200 Subject: [Python-Dev] Support for async read/write In-Reply-To: <4CBDCC52.8080209@jcea.es> References: <4CBDCC52.8080209@jcea.es> Message-ID: You should file a new issue on the bug tracker but unless you have a patch to propose it's unlikely that someone else is gonna implement it. Regards --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2010/10/19 Jesus Cea : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Current Python lacks support for "aio_*" syscalls to do async IO. I > think this could be a nice addition for python 3.3. > > If you agree, I will create an issue in the tracker. If you think the > idea is of no value, please say so for me to move on. Maybe an 3th party > module, but I think this functionality sould be available in core python. > > Thanks!. > > PS: The function calls are: aio_cancel, aio_error, aio_fsync, aio_read, > aio_return, aio_write. > > - -- > Jesus Cea Avion ? ? ? ? ? ? ? ? ? ? ? ? _/_/ ? ? ?_/_/_/ ? ? ? ?_/_/_/ > jcea at jcea.es - http://www.jcea.es/ ? ? _/_/ ? ?_/_/ ?_/_/ ? ?_/_/ ?_/_/ > jabber / xmpp:jcea at jabber.org ? ? ? ? _/_/ ? ?_/_/ ? ? ? ? ?_/_/_/_/_/ > . ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?_/_/ ?_/_/ ? ?_/_/ ? ? ? ? ?_/_/ ?_/_/ > "Things are not so easy" ? ? ?_/_/ ?_/_/ ? ?_/_/ ?_/_/ ? ?_/_/ ?_/_/ > "My name is Dump, Core Dump" ? _/_/_/ ? ? ? ?_/_/_/ ? ? ?_/_/ ?_/_/ > "El amor es poner tu felicidad en la felicidad de otro" - Leibniz > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQCVAwUBTL3MUplgi5GaxT1NAQKv5QQAnDan88kXJ67fucz2rT/mZze+065lm9E4 > +XJ2JfqGMVE1/qMsXwg81l19RHSYReBgBjd7zyXWE9Fk/1Rfq4gjOZEtoG0IpGZG > E3CH+Ab5/o/PjJZNKQaPpe0cwGSXFPKkPpgepKS1d8ZXyf6VXMc8UTSWjMI5jIVe > 4M+yvw5m4I0= > =nsdO > -----END PGP SIGNATURE----- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > From g.brandl at gmx.net Tue Oct 19 20:26:36 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 19 Oct 2010 20:26:36 +0200 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <20101019101256.73afa7c2@mission> References: <201010190353.34796.victor.stinner@haypocalc.com> <20101019101256.73afa7c2@mission> Message-ID: Am 19.10.2010 16:12, schrieb Barry Warsaw: > On Oct 19, 2010, at 03:53 AM, Victor Stinner wrote: > >>Seven months after my first commit related to this issue, the full test suite >>of Python 3.2 pass with ASCII, ISO-8859-1 and UTF-8 locale encodings in a non- >>ascii source directory. It means that Python 3.2 now process correctly >>filenames in all modules, build scripts and other utilities, with any locale >>encoding. > > Very impressive. Congratulations, and thanks for following through on what > must have been some tricky work in many of Python's deep dark corners. I'd like to join that statement. It must be like coming back to the surface after a long trip to deep underground caves with lava and sticky air :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From foom at fuhm.net Tue Oct 19 21:01:39 2010 From: foom at fuhm.net (James Y Knight) Date: Tue, 19 Oct 2010 15:01:39 -0400 Subject: [Python-Dev] Support for async read/write In-Reply-To: <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> References: <4CBDCC52.8080209@jcea.es> <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> Message-ID: <7C2AC3C7-5D87-4780-99F3-7B15D63F955E@fuhm.net> On Oct 19, 2010, at 1:47 PM, exarkun at twistedmatrix.com wrote: > Adding more platform wrappers is always nice. Keep in mind that the quality of most (all?) aio_* implementations is spotty at best, though. On Linux, they will sometimes block (for example, if you fail to align buffers properly, or open a file without O_DIRECT, or if there are too many other aio operations active on the system at the time, etc). You're thinking of the linux-specific AIO calls. Those have the properties you're describing (which makes them pretty useless for most code too), but they're completely different from the aio_* functions. The POSIX aio_* calls don't do any of that. They aren't syscalls implemented in the kernel, they're implemented in glibc. They "simply" create a threadpool in your process to call the standard synchronous operations, and make it difficult to reliably get completion notification (completion notification takes place via Real-Time signals (SIGEV_SIGNAL), which can be dropped if linux runs out of space in its RT-signal-queue, and when that happens you get no indication that that has occurred. You can also do completion notification via calling a function on a thread (SIGEV_THREAD), but, for that, glibc will always spawns a brand new thread for each notification, which is quite slow.) Basically: you shouldn't ever use those APIs. Especially on linux, but probably everywhere else. So, in conclusion, I disagree that adding wrappers for these would be nice. It wouldn't. It would cause some people to think they would be useful things to call, and they would always be wrong. James From g.brandl at gmx.net Tue Oct 19 23:04:21 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 19 Oct 2010 23:04:21 +0200 Subject: [Python-Dev] r85724 - in python/branches/py3k: Lib/logging/__init__.py Misc/NEWS In-Reply-To: <20101019152625.29097EEAA8@mail.python.org> References: <20101019152625.29097EEAA8@mail.python.org> Message-ID: Am 19.10.2010 17:26, schrieb vinay.sajip: > Author: vinay.sajip > Date: Tue Oct 19 17:26:24 2010 > New Revision: 85724 > > Log: > logging: Added _logRecordClass, getLogRecordClass, setLogRecordClass to increase flexibility of LogRecord creation. Will this be documented? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Tue Oct 19 23:56:27 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 19 Oct 2010 23:56:27 +0200 Subject: [Python-Dev] Support for async read/write References: <4CBDCC52.8080209@jcea.es> Message-ID: <20101019235627.3812bd24@pitrou.net> Hello Jesus, > Current Python lacks support for "aio_*" syscalls to do async IO. I > think this could be a nice addition for python 3.3. > > If you agree, I will create an issue in the tracker. If you think the > idea is of no value, please say so for me to move on. Maybe an 3th party > module, but I think this functionality sould be available in core python. Well, if you think this is interesting, it would be nice if you could do a bit more research instead of just filing issues for us to solve. After all, you probably already have an idea about why this is useful, how to interface it into the stdlib, etc. Just saying "we should expose these functions" doesn't bring a lot of added value IMO (there are many system functions which could be exposed and currently aren't). Also, the canonical way to do file I/O in Python 3 is the `io` lib, therefore it would be a bit of a shame to have separate, non-integrated `aio_*` functions. Also, since the `io` lib is already supposed to support non-blocking IO, perhaps it would be valuable to stress this support and propose any interesting patches for fixing and/or improving it. Regards Antoine. From martin at v.loewis.de Wed Oct 20 00:39:21 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 20 Oct 2010 00:39:21 +0200 Subject: [Python-Dev] Support for async read/write In-Reply-To: <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> References: <4CBDCC52.8080209@jcea.es> <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> Message-ID: <4CBE1E19.6010801@v.loewis.de> Am 19.10.2010 19:47, schrieb exarkun at twistedmatrix.com: > On 04:50 pm, jcea at jcea.es wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Current Python lacks support for "aio_*" syscalls to do async IO. I >> think this could be a nice addition for python 3.3. > > Adding more platform wrappers is always nice. Exactly so. Exposing platform APIs as-is really doesn't need much discussion, except when the platform wrapper is going to deviate significantly from the platform API. There is a long tradition of exposing even obscure unixish APIs in Python, making it the #1 language to write exotic scripts in. We definitely want to continue this tradition. Please refrain from trying to come up with "abstractions", though; this should be outside of Python for the moment. Covering both Unix AIO and Windows completion ports (say) may be quite a challenge (but then, JP may have some view on this). Regards, Martin From martin at v.loewis.de Wed Oct 20 00:44:12 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 20 Oct 2010 00:44:12 +0200 Subject: [Python-Dev] Support for async read/write In-Reply-To: <7C2AC3C7-5D87-4780-99F3-7B15D63F955E@fuhm.net> References: <4CBDCC52.8080209@jcea.es> <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> <7C2AC3C7-5D87-4780-99F3-7B15D63F955E@fuhm.net> Message-ID: <4CBE1F3C.7040209@v.loewis.de> > So, in conclusion, I disagree that adding wrappers for these would be > nice. It wouldn't. It would cause some people to think they would be > useful things to call, and they would always be wrong. We are all consenting adults. If people want to shoot themselves in their feet, we let them. For example, we have os.open, even though there is no garbage collection for file handles, and we have os._exit, even though it doesn't call finalizers. People should trust that the APIs we expose go *literally* into the library, so we can blame any consequences that this has onto the library (and possibly the kernel, in turn). People readily accept that explanation. Regards, Martin From martin at v.loewis.de Wed Oct 20 00:48:14 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 20 Oct 2010 00:48:14 +0200 Subject: [Python-Dev] Support for async read/write In-Reply-To: <20101019235627.3812bd24@pitrou.net> References: <4CBDCC52.8080209@jcea.es> <20101019235627.3812bd24@pitrou.net> Message-ID: <4CBE202E.8060409@v.loewis.de> > Also, the canonical way to do file I/O in Python 3 is the `io` lib, > therefore it would be a bit of a shame to have separate, non-integrated > `aio_*` functions. I disagree. We also have posix.open, posix.dup, etc. We always expose POSIX functions in the posix module (except for the socket functions, unfortunately, and a few other exceptions), and I see no reason to break with this tradition. The io module should be thought of as sitting on top of the posix module (and it initially was). > Also, since the `io` lib is already supposed to support non-blocking > IO, perhaps it would be valuable to stress this support and propose any > interesting patches for fixing and/or improving it. I think that's entirely independent. Regards, Martin From solipsis at pitrou.net Wed Oct 20 00:58:42 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 20 Oct 2010 00:58:42 +0200 Subject: [Python-Dev] Support for async read/write In-Reply-To: <4CBE202E.8060409@v.loewis.de> References: <4CBDCC52.8080209@jcea.es> <20101019235627.3812bd24@pitrou.net> <4CBE202E.8060409@v.loewis.de> Message-ID: <1287529122.3488.9.camel@localhost.localdomain> Le mercredi 20 octobre 2010 ? 00:48 +0200, "Martin v. L?wis" a ?crit : > > Also, the canonical way to do file I/O in Python 3 is the `io` lib, > > therefore it would be a bit of a shame to have separate, non-integrated > > `aio_*` functions. > > I disagree. We also have posix.open, posix.dup, etc. We always expose > POSIX functions in the posix module (except for the socket functions, > unfortunately, and a few other exceptions), and I see no reason to break > with this tradition. The io module should be thought of as sitting > on top of the posix module (and it initially was). I'm not suggesting to break with any "tradition", but that building duplicate APIs to do file I/O has little value and only makes things more cumbersome. > > Also, since the `io` lib is already supposed to support non-blocking > > IO, perhaps it would be valuable to stress this support and propose any > > interesting patches for fixing and/or improving it. > > I think that's entirely independent. I don't think it is. If the goal is to give ways to improve file I/O performance for specialized use cases, then it certainly deserves integration with the current I/O stack. If the goal is not that, then I don't really know what it is. It would be nice to know about the use case Jesus has in mind. Regards Antoine. From foom at fuhm.net Wed Oct 20 02:09:29 2010 From: foom at fuhm.net (James Y Knight) Date: Tue, 19 Oct 2010 20:09:29 -0400 Subject: [Python-Dev] Support for async read/write In-Reply-To: <4CBE1F3C.7040209@v.loewis.de> References: <4CBDCC52.8080209@jcea.es> <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> <7C2AC3C7-5D87-4780-99F3-7B15D63F955E@fuhm.net> <4CBE1F3C.7040209@v.loewis.de> Message-ID: <569E82E3-FA36-468E-8220-8D60C5BE4EE9@fuhm.net> On Oct 19, 2010, at 6:44 PM, Martin v. L?wis wrote: >> So, in conclusion, I disagree that adding wrappers for these would be >> nice. It wouldn't. It would cause some people to think they would be >> useful things to call, and they would always be wrong. > > We are all consenting adults. If people want to shoot themselves in > their feet, we let them. For example, we have os.open, even though > there is no garbage collection for file handles, and we have > os._exit, even though it doesn't call finalizers. There's a difference. os._exit is useful. os.open is useful. aio_* are *not* useful. For anything. If there's anything you think you want to use them for, you're wrong. It either won't work properly or it will worse performing than the simpler alternatives. It would absolutely be a waste of time (of both the implementor of the wrapper and the poor users who stumble across them in documentation and try to use them) to bother adding wrappers to these functions for python. James From victor.stinner at haypocalc.com Wed Oct 20 02:11:35 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Wed, 20 Oct 2010 02:11:35 +0200 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <20101019101256.73afa7c2@mission> References: <201010190353.34796.victor.stinner@haypocalc.com> <20101019101256.73afa7c2@mission> Message-ID: <201010200211.35608.victor.stinner@haypocalc.com> Le mardi 19 octobre 2010 16:12:56, Barry Warsaw a ?crit : > Going forward, is there adequate documentation, guidelines, and safeguards > for future coders so that they Do The Right Thing with new code? Perhaps > a short How To in the standard documentation would be helpful, with links > to it from any old/bad API calls? Hum, as usual, I suggest to decode all inputs to unicode as early as possible, and encode back to bytes (or other native format) at the last moment. For filenames, it means that PyUnicode_FSDecoder() is better than PyUnicode_FSConverter(), because it gives an unicode object (instead of byte string) and so the function will support unencodable characters. Use PyUnicode_EncodeFSDefault() / PyUnicode_DecodeFSDefault() and os.fsencode() / os.fsdecode() to encode/decode filenames instead of your own function, to support the PEP 383 (undecodable bytes <=> surrogate characters). Be also careful to support undecodable bytes (on OSes other than Windows), eg. try a filename with a non-ASCII character with the C locale (ASCII locale encoding). Even with utf-8 filesystem encoding, this problem may occurs with a system not correclty configured (eg. USB key with the FAT fileystem using the "wrong" encoding). If you would like to avoid all encoding issues on filenames on UNIX/BSD, use bytes: os.environb, os.listdir(b'.'), os.getcwdb(), etc. Be careful with the utf-8 codec: its default mode (strict error handler) refuses to encode surrogate characters. Eg. print(filename) may raise a UnicodeEncodeError. Use repr(filename) to escape surrogate characters. -- I plan to fix Python documentation: specify the encoding used to decode all byte string arguments of the C API. I already wrote a draft patch: issue #9738. This lack of documentation was a big problem for me, because I had to follow the function calls to get the encoding. -- Victor Stinner http://www.haypocalc.com/ From ndbecker2 at gmail.com Wed Oct 20 02:57:22 2010 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 19 Oct 2010 20:57:22 -0400 Subject: [Python-Dev] [patch] fpconst for python3 Message-ID: Where should I send this patch? diff -u fpconst-0.7.2/fpconst.py fpconst-0.7.2.new/fpconst.py --- fpconst-0.7.2/fpconst.py 2005-02-24 12:42:03.000000000 -0500 +++ fpconst-0.7.2.new/fpconst.py 2010-10-19 20:55:07.407765664 -0400 @@ -40,18 +40,18 @@ ident = "$Id: fpconst.py,v 1.16 2005/02/24 17:42:03 warnes Exp $" import struct, operator +from functools import reduce # check endianess -_big_endian = struct.pack('i',1)[0] != '\x01' - +_big_endian = struct.pack('i',1)[0] != 1 # and define appropriate constants if(_big_endian): - NaN = struct.unpack('d', '\x7F\xF8\x00\x00\x00\x00\x00\x00')[0] - PosInf = struct.unpack('d', '\x7F\xF0\x00\x00\x00\x00\x00\x00')[0] + NaN = struct.unpack('d', b'\x7F\xF8\x00\x00\x00\x00\x00\x00')[0] + PosInf = struct.unpack('d', b'\x7F\xF0\x00\x00\x00\x00\x00\x00')[0] NegInf = -PosInf else: - NaN = struct.unpack('d', '\x00\x00\x00\x00\x00\x00\xf8\xff')[0] - PosInf = struct.unpack('d', '\x00\x00\x00\x00\x00\x00\xf0\x7f')[0] + NaN = struct.unpack('d', b'\x00\x00\x00\x00\x00\x00\xf8\xff')[0] + PosInf = struct.unpack('d', b'\x00\x00\x00\x00\x00\x00\xf0\x7f')[0] NegInf = -PosInf def _double_as_bytes(dval): From benjamin at python.org Wed Oct 20 03:07:44 2010 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 19 Oct 2010 20:07:44 -0500 Subject: [Python-Dev] [patch] fpconst for python3 In-Reply-To: References: Message-ID: fpconst developers? 2010/10/19 Neal Becker : > Where should I send this patch? > > diff -u fpconst-0.7.2/fpconst.py fpconst-0.7.2.new/fpconst.py > --- fpconst-0.7.2/fpconst.py ? ?2005-02-24 12:42:03.000000000 -0500 > +++ fpconst-0.7.2.new/fpconst.py ? ? ? ?2010-10-19 20:55:07.407765664 -0400 > @@ -40,18 +40,18 @@ > ?ident = "$Id: fpconst.py,v 1.16 2005/02/24 17:42:03 warnes Exp $" > > ?import struct, operator > +from functools import reduce > > ?# check endianess > -_big_endian = struct.pack('i',1)[0] != '\x01' > - > +_big_endian = struct.pack('i',1)[0] != 1 > ?# and define appropriate constants > ?if(_big_endian): > - ? ?NaN ? ?= struct.unpack('d', '\x7F\xF8\x00\x00\x00\x00\x00\x00')[0] > - ? ?PosInf = struct.unpack('d', '\x7F\xF0\x00\x00\x00\x00\x00\x00')[0] > + ? ?NaN ? ?= struct.unpack('d', b'\x7F\xF8\x00\x00\x00\x00\x00\x00')[0] > + ? ?PosInf = struct.unpack('d', b'\x7F\xF0\x00\x00\x00\x00\x00\x00')[0] > ? ? NegInf = -PosInf > ?else: > - ? ?NaN ? ?= struct.unpack('d', '\x00\x00\x00\x00\x00\x00\xf8\xff')[0] > - ? ?PosInf = struct.unpack('d', '\x00\x00\x00\x00\x00\x00\xf0\x7f')[0] > + ? ?NaN ? ?= struct.unpack('d', b'\x00\x00\x00\x00\x00\x00\xf8\xff')[0] > + ? ?PosInf = struct.unpack('d', b'\x00\x00\x00\x00\x00\x00\xf0\x7f')[0] > ? ? NegInf = -PosInf > > ?def _double_as_bytes(dval): > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/benjamin%40python.org > -- Regards, Benjamin From glyph at twistedmatrix.com Wed Oct 20 03:37:10 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Tue, 19 Oct 2010 21:37:10 -0400 Subject: [Python-Dev] Support for async read/write In-Reply-To: <569E82E3-FA36-468E-8220-8D60C5BE4EE9@fuhm.net> References: <4CBDCC52.8080209@jcea.es> <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> <7C2AC3C7-5D87-4780-99F3-7B15D63F955E@fuhm.net> <4CBE1F3C.7040209@v.loewis.de> <569E82E3-FA36-468E-8220-8D60C5BE4EE9@fuhm.net> Message-ID: <843390F5-3774-4357-AF24-9F57CD11B7E5@twistedmatrix.com> On Oct 19, 2010, at 8:09 PM, James Y Knight wrote: > There's a difference. > > os._exit is useful. os.open is useful. aio_* are *not* useful. For anything. If there's anything you think you want to use them for, you're wrong. It either won't work properly or it will worse performing than the simpler alternatives. I'd like to echo this sentiment. This is not about providing a 'safe' wrapper to hide some powerful feature of these APIs: the POSIX aio_* functions are really completely useless. To quote the relevant standard : APPLICATION USAGE None. RATIONALE None. FUTURE DIRECTIONS None. Not only is the performance usually worse than expected, the behavior of aio_* functions require all kinds of subtle and mysterious coordination with signal handling, which I'm not entirely sure Python would even be able to pull off without some modifications to the signal module. (And, as Jean-Paul mentioned, if your OS kernel runs out of space in a queue somewhere, completion notifications might just never be delivered at all.) I would love for someone to prove me wrong. In particular, I would really love for there to be a solution to asynchronous filesystem I/O better than "start a thread, read until you block". But, as far as I know, there isn't, and wrapping these functions will just confuse and upset anyone who attempts to use them in any way. -------------- next part -------------- An HTML attachment was scrubbed... URL: From exarkun at twistedmatrix.com Wed Oct 20 03:55:20 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 20 Oct 2010 01:55:20 -0000 Subject: [Python-Dev] Support for async read/write In-Reply-To: <843390F5-3774-4357-AF24-9F57CD11B7E5@twistedmatrix.com> References: <4CBDCC52.8080209@jcea.es> <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> <7C2AC3C7-5D87-4780-99F3-7B15D63F955E@fuhm.net> <4CBE1F3C.7040209@v.loewis.de> <569E82E3-FA36-468E-8220-8D60C5BE4EE9@fuhm.net> <843390F5-3774-4357-AF24-9F57CD11B7E5@twistedmatrix.com> Message-ID: <20101020015520.2414.482393106.divmod.xquotient.645@localhost.localdomain> On 01:37 am, glyph at twistedmatrix.com wrote: > >On Oct 19, 2010, at 8:09 PM, James Y Knight wrote: >>There's a difference. >> >>os._exit is useful. os.open is useful. aio_* are *not* useful. For >>anything. If there's anything you think you want to use them for, >>you're wrong. It either won't work properly or it will worse >>performing than the simpler alternatives. > > >I'd like to echo this sentiment. This is not about providing a 'safe' >wrapper to hide some powerful feature of these APIs: the POSIX aio_* >functions are really completely useless. > >To quote the relevant standard >: >APPLICATION USAGE > >None. > >RATIONALE > >None. > >FUTURE DIRECTIONS > >None. > >Not only is the performance usually worse than expected, the behavior >of aio_* functions require all kinds of subtle and mysterious >coordination with signal handling, which I'm not entirely sure Python >would even be able to pull off without some modifications to the signal >module. (And, as Jean-Paul mentioned, if your OS kernel runs out of >space in a queue somewhere, completion notifications might just never >be delivered at all.) Just to be clear, James corrected me there. I thought Jesus was talking about the mostly useless Linux AIO APIs, which have the problems I described. He was actually talking about the POSIX AIO APIs, which have a different set of problems making them a waste of time. Jean-Paul From jyasskin at gmail.com Wed Oct 20 06:31:40 2010 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Tue, 19 Oct 2010 23:31:40 -0500 Subject: [Python-Dev] Support for async read/write In-Reply-To: <843390F5-3774-4357-AF24-9F57CD11B7E5@twistedmatrix.com> References: <4CBDCC52.8080209@jcea.es> <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> <7C2AC3C7-5D87-4780-99F3-7B15D63F955E@fuhm.net> <4CBE1F3C.7040209@v.loewis.de> <569E82E3-FA36-468E-8220-8D60C5BE4EE9@fuhm.net> <843390F5-3774-4357-AF24-9F57CD11B7E5@twistedmatrix.com> Message-ID: On Tue, Oct 19, 2010 at 8:37 PM, Glyph Lefkowitz wrote: > I'd like to echo this sentiment. ?This is not about providing a 'safe' > wrapper to hide some powerful feature of these APIs: the POSIX aio_* > functions are really completely useless. > To quote the relevant standard > : > > APPLICATION USAGE > None. > RATIONALE > None. > FUTURE DIRECTIONS > None. No comment on the rest of your claim, but this is a silly argument. The standard says the same thing about at least fcntl.h, signal.h, pthread.h, and ucontext.h, which clearly are useful. Jeffrey From ncoghlan at gmail.com Wed Oct 20 15:47:25 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Oct 2010 23:47:25 +1000 Subject: [Python-Dev] [Python-checkins] r85739 - in python/branches/issue4388/Lib/test: script_helper.py test_cmd_line.py In-Reply-To: <20101020120646.9333DEEAAE@mail.python.org> References: <20101020120646.9333DEEAAE@mail.python.org> Message-ID: On Wed, Oct 20, 2010 at 10:06 PM, victor.stinner wrote: > Modified: python/branches/issue4388/Lib/test/test_cmd_line.py > ============================================================================== > --- python/branches/issue4388/Lib/test/test_cmd_line.py (original) > +++ python/branches/issue4388/Lib/test/test_cmd_line.py Wed Oct 20 14:06:46 2010 > @@ -103,8 +103,15 @@ > ? ? ? ? if sys.getfilesystemencoding() != 'ascii': > ? ? ? ? ? ? if test.support.verbose: > ? ? ? ? ? ? ? ? import locale > + ? ? ? ? ? ? ? ?env = os.environ.copy() > + ? ? ? ? ? ? ? ?for key in ('LC_ALL', 'LC_CTYPE', 'LANG'): > + ? ? ? ? ? ? ? ? ? ?try: > + ? ? ? ? ? ? ? ? ? ? ? ?del env[key] > + ? ? ? ? ? ? ? ? ? ?except KeyError: > + ? ? ? ? ? ? ? ? ? ? ? ?pass > ? ? ? ? ? ? ? ? print('locale encoding = %s, filesystem encoding = %s' > - ? ? ? ? ? ? ? ? ? ? ?% (locale.getpreferredencoding(), sys.getfilesystemencoding())) > + ? ? ? ? ? ? ? ? ? ? ?% (locale.getpreferredencoding(), sys.getfilesystemencoding()), > + ? ? ? ? ? ? ? ? ? ? ?env=env) > ? ? ? ? ? ? command = "assert(ord('\xe9') == 0xe9)" > ? ? ? ? ? ? assert_python_ok('-c', command) Isn't that "env=env" argument meant to be down in the call to assert_python_ok, not in the print call? (your next checkin didn't change this) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From dmalcolm at redhat.com Wed Oct 20 18:43:23 2010 From: dmalcolm at redhat.com (David Malcolm) Date: Wed, 20 Oct 2010 12:43:23 -0400 Subject: [Python-Dev] [patch] fpconst for python3 In-Reply-To: References: Message-ID: <1287593003.13108.32.camel@radiator.bos.redhat.com> (my apologies, if necessary, for top-posting) FWIW Neal asked about this on Fedora's development mailing list as well: http://lists.fedoraproject.org/pipermail/devel/2010-October/144535.html If I'm reading: http://pypi.python.org/pypi/fpconst/ correctly, that project hasn't had an upstream update in over four years, and the upstream home page seems to be down. I believe that there are a number of python 2 modules that are either mature or "good enough", but need porting to work with python 3, and in some of these cases, upstream may have either disappeared, or lost interest. I'm guessing that Neal has already made an effort to contact the maintainer of fpconst. IIRC, fpconst is a relatively simple module, and the fixes to port to python 3 seem simple. So IMHO this is on-topic for python.org (if not necessarily this list), in that we have an interest in timely porting of Python 2 code to Python 3, and there has been discussion here on how to encourage people to port their code. We don't want each Linux distribution to do different patches to port the code to Python 3: there ought to be some kind of clearing-house for this kind of simple porting, so that python 3 stacks will work consistently across the different distributions. Whether or not that's python-dev, I'm not sure. Perhaps: http://mail.python.org/mailman/listinfo/python-porting is a better list. FWIW you can see the current status of Python 3 porting within Fedora here: https://fedoraproject.org/wiki/Python3#Porting_status Hope this is helpful Dave On Tue, 2010-10-19 at 20:07 -0500, Benjamin Peterson wrote: > fpconst developers? > > 2010/10/19 Neal Becker : > > Where should I send this patch? > > > > diff -u fpconst-0.7.2/fpconst.py fpconst-0.7.2.new/fpconst.py > > --- fpconst-0.7.2/fpconst.py 2005-02-24 12:42:03.000000000 -0500 > > +++ fpconst-0.7.2.new/fpconst.py 2010-10-19 20:55:07.407765664 -0400 > > @@ -40,18 +40,18 @@ > > ident = "$Id: fpconst.py,v 1.16 2005/02/24 17:42:03 warnes Exp $" > > > > import struct, operator > > +from functools import reduce > > > > # check endianess > > -_big_endian = struct.pack('i',1)[0] != '\x01' > > - > > +_big_endian = struct.pack('i',1)[0] != 1 > > # and define appropriate constants > > if(_big_endian): > > - NaN = struct.unpack('d', '\x7F\xF8\x00\x00\x00\x00\x00\x00')[0] > > - PosInf = struct.unpack('d', '\x7F\xF0\x00\x00\x00\x00\x00\x00')[0] > > + NaN = struct.unpack('d', b'\x7F\xF8\x00\x00\x00\x00\x00\x00')[0] > > + PosInf = struct.unpack('d', b'\x7F\xF0\x00\x00\x00\x00\x00\x00')[0] > > NegInf = -PosInf > > else: > > - NaN = struct.unpack('d', '\x00\x00\x00\x00\x00\x00\xf8\xff')[0] > > - PosInf = struct.unpack('d', '\x00\x00\x00\x00\x00\x00\xf0\x7f')[0] > > + NaN = struct.unpack('d', b'\x00\x00\x00\x00\x00\x00\xf8\xff')[0] > > + PosInf = struct.unpack('d', b'\x00\x00\x00\x00\x00\x00\xf0\x7f')[0] > > NegInf = -PosInf > > > > def _double_as_bytes(dval): > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/benjamin%40python.org > > > > > From fuzzyman at voidspace.org.uk Wed Oct 20 18:50:52 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 20 Oct 2010 17:50:52 +0100 Subject: [Python-Dev] [patch] fpconst for python3 In-Reply-To: <1287593003.13108.32.camel@radiator.bos.redhat.com> References: <1287593003.13108.32.camel@radiator.bos.redhat.com> Message-ID: <4CBF1DEC.4000801@voidspace.org.uk> On 20/10/2010 17:43, David Malcolm wrote: > (my apologies, if necessary, for top-posting) > > FWIW Neal asked about this on Fedora's development mailing list as well: > http://lists.fedoraproject.org/pipermail/devel/2010-October/144535.html > > If I'm reading: > http://pypi.python.org/pypi/fpconst/ > correctly, that project hasn't had an upstream update in over four > years, and the upstream home page seems to be down. > > I believe that there are a number of python 2 modules that are either > mature or "good enough", but need porting to work with python 3, and in > some of these cases, upstream may have either disappeared, or lost > interest. > > I'm guessing that Neal has already made an effort to contact the > maintainer of fpconst. > > IIRC, fpconst is a relatively simple module, and the fixes to port to > python 3 seem simple. > > So IMHO this is on-topic for python.org (if not necessarily this list), > in that we have an interest in timely porting of Python 2 code to Python > 3, and there has been discussion here on how to encourage people to port > their code. > > We don't want each Linux distribution to do different patches to port > the code to Python 3: there ought to be some kind of clearing-house for > this kind of simple porting, so that python 3 stacks will work > consistently across the different distributions. > > Whether or not that's python-dev, I'm not sure. Perhaps: > http://mail.python.org/mailman/listinfo/python-porting > is a better list. > > FWIW you can see the current status of Python 3 porting within Fedora > here: > https://fedoraproject.org/wiki/Python3#Porting_status Wow, that's very interesting. Whilst a lot of people on this list will have an interest in knowing about this it still isn't the right list for discussing it. The python-porting list *is* an entirely appropriate list and it would be great to see more traffic there. What to do about "abandoned" PyPI packages that are still used, and defacto maintained by downstream packagers, is another (interesting/important/difficult) topic that is also off-topic for this list. catalog-sig is possibly the right place for that. Creating a new PyPI package for the port and forking the project is probably the best approach. Michael Foord > Hope this is helpful > Dave > > On Tue, 2010-10-19 at 20:07 -0500, Benjamin Peterson wrote: >> fpconst developers? >> >> 2010/10/19 Neal Becker: >>> Where should I send this patch? >>> >>> diff -u fpconst-0.7.2/fpconst.py fpconst-0.7.2.new/fpconst.py >>> --- fpconst-0.7.2/fpconst.py 2005-02-24 12:42:03.000000000 -0500 >>> +++ fpconst-0.7.2.new/fpconst.py 2010-10-19 20:55:07.407765664 -0400 >>> @@ -40,18 +40,18 @@ >>> ident = "$Id: fpconst.py,v 1.16 2005/02/24 17:42:03 warnes Exp $" >>> >>> import struct, operator >>> +from functools import reduce >>> >>> # check endianess >>> -_big_endian = struct.pack('i',1)[0] != '\x01' >>> - >>> +_big_endian = struct.pack('i',1)[0] != 1 >>> # and define appropriate constants >>> if(_big_endian): >>> - NaN = struct.unpack('d', '\x7F\xF8\x00\x00\x00\x00\x00\x00')[0] >>> - PosInf = struct.unpack('d', '\x7F\xF0\x00\x00\x00\x00\x00\x00')[0] >>> + NaN = struct.unpack('d', b'\x7F\xF8\x00\x00\x00\x00\x00\x00')[0] >>> + PosInf = struct.unpack('d', b'\x7F\xF0\x00\x00\x00\x00\x00\x00')[0] >>> NegInf = -PosInf >>> else: >>> - NaN = struct.unpack('d', '\x00\x00\x00\x00\x00\x00\xf8\xff')[0] >>> - PosInf = struct.unpack('d', '\x00\x00\x00\x00\x00\x00\xf0\x7f')[0] >>> + NaN = struct.unpack('d', b'\x00\x00\x00\x00\x00\x00\xf8\xff')[0] >>> + PosInf = struct.unpack('d', b'\x00\x00\x00\x00\x00\x00\xf0\x7f')[0] >>> NegInf = -PosInf >>> >>> def _double_as_bytes(dval): >>> >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/benjamin%40python.org >>> >> >> > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From barry at python.org Wed Oct 20 20:35:43 2010 From: barry at python.org (Barry Warsaw) Date: Wed, 20 Oct 2010 14:35:43 -0400 Subject: [Python-Dev] [patch] fpconst for python3 In-Reply-To: <4CBF1DEC.4000801@voidspace.org.uk> References: <1287593003.13108.32.camel@radiator.bos.redhat.com> <4CBF1DEC.4000801@voidspace.org.uk> Message-ID: <20101020143543.5d7ccc40@mission> On Oct 20, 2010, at 05:50 PM, Michael Foord wrote: >Wow, that's very interesting. Whilst a lot of people on this list will have >an interest in knowing about this it still isn't the right list for >discussing it. The python-porting list *is* an entirely appropriate list and >it would be great to see more traffic there. Agreed. python-porting (which I'd forgotten about but just subscribed to) seems like the right place for cross-distro and other porting issues. We also have this top page in the wiki: http://wiki.python.org/moin/PortingToPy3k Right now it contains guides to porting Python code and C code, but I think it should also contain pages that coordinate porting efforts for packages on Cheeseshop and in distros. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From glyph at twistedmatrix.com Wed Oct 20 21:12:25 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Wed, 20 Oct 2010 15:12:25 -0400 Subject: [Python-Dev] Support for async read/write In-Reply-To: <20101020015520.2414.482393106.divmod.xquotient.645@localhost.localdomain> References: <4CBDCC52.8080209@jcea.es> <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> <7C2AC3C7-5D87-4780-99F3-7B15D63F955E@fuhm.net> <4CBE1F3C.7040209@v.loewis.de> <569E82E3-FA36-468E-8220-8D60C5BE4EE9@fuhm.net> <843390F5-3774-4357-AF24-9F57CD11B7E5@twistedmatrix.com> <20101020015520.2414.482393106.divmod.xquotient.645@localhost.localdomain> Message-ID: <1B73D748-835A-40A9-A154-9F2C291FE164@twistedmatrix.com> On Oct 19, 2010, at 9:55 PM, exarkun at twistedmatrix.com wrote: >> Not only is the performance usually worse than expected, the behavior of aio_* functions require all kinds of subtle and mysterious coordination with signal handling, which I'm not entirely sure Python would even be able to pull off without some modifications to the signal module. (And, as Jean-Paul mentioned, if your OS kernel runs out of space in a queue somewhere, completion notifications might just never be delivered at all.) > > Just to be clear, James corrected me there. I thought Jesus was talking about the mostly useless Linux AIO APIs, which have the problems I described. He was actually talking about the POSIX AIO APIs, which have a different set of problems making them a waste of time. I know, I'm referring to the behavior of POSIX AIO. Perhaps I'm overstating the case with 'subtle and mysterious', then, but the POISX 'aiocb' structure still includes an 'aio_sigevent' member which is the way to find out about I/O event completion. If you're writing an application that uses AIO, basically all of your logic ends up living in the context of a signal handler, and as puts it, "When signal-catching functions are invoked asynchronously with process execution, the behavior of some of the functions defined by this volume of IEEE Std 1003.1-2001 is unspecified if they are called from a signal-catching function." Of course, you could try using signalfd(), but that's not in POSIX. (Or, you could use SIGEV_THREAD, but that would be functionally equivalent to running read() in a thread, except much more difficult.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From glyph at twistedmatrix.com Wed Oct 20 21:55:28 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Wed, 20 Oct 2010 15:55:28 -0400 Subject: [Python-Dev] Support for async read/write In-Reply-To: References: <4CBDCC52.8080209@jcea.es> <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> <7C2AC3C7-5D87-4780-99F3-7B15D63F955E@fuhm.net> <4CBE1F3C.7040209@v.loewis.de> <569E82E3-FA36-468E-8220-8D60C5BE4EE9@fuhm.net> <843390F5-3774-4357-AF24-9F57CD11B7E5@twistedmatrix.com> Message-ID: On Oct 20, 2010, at 12:31 AM, Jeffrey Yasskin wrote: > No comment on the rest of your claim, but this is a silly argument. > The standard says the same thing about at least fcntl.h, signal.h, > pthread.h, and ucontext.h, which clearly are useful. It was meant to be tongue-in-cheek :). Perhaps I should not have assumed that everyone else was as familiar with the POSIX documentation; I figured that most readers would know that most pages say that. But, that was the result of a string of many different searches attempting to find someone explaining why this was a good idea or why anyone would want to use it. I think in this case, it's accurate. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jyasskin at gmail.com Wed Oct 20 22:04:17 2010 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 20 Oct 2010 15:04:17 -0500 Subject: [Python-Dev] Support for async read/write In-Reply-To: References: <4CBDCC52.8080209@jcea.es> <20101019174739.2414.2072677591.divmod.xquotient.640@localhost.localdomain> <7C2AC3C7-5D87-4780-99F3-7B15D63F955E@fuhm.net> <4CBE1F3C.7040209@v.loewis.de> <569E82E3-FA36-468E-8220-8D60C5BE4EE9@fuhm.net> <843390F5-3774-4357-AF24-9F57CD11B7E5@twistedmatrix.com> Message-ID: On Wed, Oct 20, 2010 at 2:55 PM, Glyph Lefkowitz wrote: > > On Oct 20, 2010, at 12:31 AM, Jeffrey Yasskin wrote: > > No comment on the rest of your claim, but this is a silly argument. > The standard says the same thing about at least fcntl.h, signal.h, > pthread.h, and ucontext.h, which clearly are useful. > > It was meant to be tongue-in-cheek :). ?Perhaps I should not have assumed > that everyone else was as familiar with the POSIX documentation; I figured > that most readers would know that most pages say that. Oops, sorry. My joke-detector failed. From flub at devork.be Thu Oct 21 01:33:40 2010 From: flub at devork.be (Floris Bruynooghe) Date: Thu, 21 Oct 2010 00:33:40 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: Hi Sorry for the late response On 8 October 2010 13:02, Fred Drake wrote: > I'm in favor of add a top-level setup module that can be invoked using > "python -m setup ...". I'd say +1 for this option. It has the advantage that it's very clear which python environment you're installing (or whatever other valid action) the package into. For a stand-alone script this might not always be as clear. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From fuzzyman at voidspace.org.uk Thu Oct 21 01:45:23 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 21 Oct 2010 00:45:23 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: Message-ID: <4CBF7F13.80005@voidspace.org.uk> On 21/10/2010 00:33, Floris Bruynooghe wrote: > Hi > > Sorry for the late response > > On 8 October 2010 13:02, Fred Drake wrote: >> I'm in favor of add a top-level setup module that can be invoked using >> "python -m setup ...". > I'd say +1 for this option. It has the advantage that it's very clear > which python environment you're installing (or whatever other valid > action) the package into. For a stand-alone script this might not > always be as clear. > Versioned scripts would also allow that without requiring the extra typing... Michael > Regards > Floris > -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From rrr at ronadam.com Thu Oct 21 03:01:56 2010 From: rrr at ronadam.com (Ron Adam) Date: Wed, 20 Oct 2010 20:01:56 -0500 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101012105951.3ed845b5@mission> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB39CC8.9050602@canterbury.ac.nz> <20101012105951.3ed845b5@mission> Message-ID: <4CBF9104.6080400@ronadam.com> On 10/12/2010 09:59 AM, Barry Warsaw wrote: > On Oct 12, 2010, at 12:24 PM, Greg Ewing wrote: > >> Giampaolo Rodol? wrote: >> >>> If that's the case what would I type in the command prompt in order to >>> install a module? >>> "C:\PythonXX\pysetup.exe"? >>> If so I would strongly miss old "setup.py install". >> >> Another thing bothers me about this. With the current scheme, >> if you have multiple Pythons available, it's easy to be sure >> that you're installing into the right one, because it's the >> one that you use to run setup.py. Whereas if installation is >> performed by a different executable, there's a possibility >> of them being out of sync. >> >> So I think I'd prefer some scheme involving 'python -m ...' >> or some other option to Python itself, rather than a separate >> executable. > > This is why I suggested that 'setup.sh' (or whatever) take a --python-version > option to select the python executable to use. > > Whatever solution is implemented definitely needs to take the > multiple-installed pythons into account. On Ubuntu, I use python, python2.7, python3.1, python3.2 and that is what I type to use that particular version. The -m option seems to me to be the easiest to do and works with all of these. python2.7 -m setup python3.2 -m setup I don't see why that isn't an acceptable solution to this? It's not any different than doing ... python3.2 -m test.regrtest python3.2 -m pydoc -g python3.2 -m idlelib.idle python3.2 -m this python3.2 -m turtle python3.2 -m timeit -h python3.2 -m trace --help python3.2 -m dis filename.py python3.2 -m zipfile There are probably others I don't remember or know about. The point is, without the handy '-m', you have to know where the file is, or set environment variables, or create .bat and/or .sh files, and those takes a lot more work. So why not just embrace it and move on? Ron From rrr at ronadam.com Thu Oct 21 03:01:56 2010 From: rrr at ronadam.com (Ron Adam) Date: Wed, 20 Oct 2010 20:01:56 -0500 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101012105951.3ed845b5@mission> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB39CC8.9050602@canterbury.ac.nz> <20101012105951.3ed845b5@mission> Message-ID: <4CBF9104.6080400@ronadam.com> On 10/12/2010 09:59 AM, Barry Warsaw wrote: > On Oct 12, 2010, at 12:24 PM, Greg Ewing wrote: > >> Giampaolo Rodol? wrote: >> >>> If that's the case what would I type in the command prompt in order to >>> install a module? >>> "C:\PythonXX\pysetup.exe"? >>> If so I would strongly miss old "setup.py install". >> >> Another thing bothers me about this. With the current scheme, >> if you have multiple Pythons available, it's easy to be sure >> that you're installing into the right one, because it's the >> one that you use to run setup.py. Whereas if installation is >> performed by a different executable, there's a possibility >> of them being out of sync. >> >> So I think I'd prefer some scheme involving 'python -m ...' >> or some other option to Python itself, rather than a separate >> executable. > > This is why I suggested that 'setup.sh' (or whatever) take a --python-version > option to select the python executable to use. > > Whatever solution is implemented definitely needs to take the > multiple-installed pythons into account. On Ubuntu, I use python, python2.7, python3.1, python3.2 and that is what I type to use that particular version. The -m option seems to me to be the easiest to do and works with all of these. python2.7 -m setup python3.2 -m setup I don't see why that isn't an acceptable solution to this? It's not any different than doing ... python3.2 -m test.regrtest python3.2 -m pydoc -g python3.2 -m idlelib.idle python3.2 -m this python3.2 -m turtle python3.2 -m timeit -h python3.2 -m trace --help python3.2 -m dis filename.py python3.2 -m zipfile There are probably others I don't remember or know about. The point is, without the handy '-m', you have to know where the file is, or set environment variables, or create .bat and/or .sh files, and those takes a lot more work. So why not just embrace it and move on? Ron From ncoghlan at gmail.com Thu Oct 21 04:17:48 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 21 Oct 2010 12:17:48 +1000 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CBF9104.6080400@ronadam.com> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB39CC8.9050602@canterbury.ac.nz> <20101012105951.3ed845b5@mission> <4CBF9104.6080400@ronadam.com> Message-ID: On Thu, Oct 21, 2010 at 11:01 AM, Ron Adam wrote: > There are probably others I don't remember or know about. "python -m site" is another handy one if you're trying to debug sys.path issues Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From solipsis at pitrou.net Thu Oct 21 10:44:54 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 21 Oct 2010 10:44:54 +0200 Subject: [Python-Dev] Distutils2 scripts References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB39CC8.9050602@canterbury.ac.nz> <20101012105951.3ed845b5@mission> <4CBF9104.6080400@ronadam.com> Message-ID: <20101021104454.4ba48afe@pitrou.net> On Wed, 20 Oct 2010 20:01:56 -0500 Ron Adam wrote: > > On Ubuntu, I use python, python2.7, python3.1, python3.2 and that is what I > type to use that particular version. The -m option seems to me to be the > easiest to do and works with all of these. > > python2.7 -m setup > python3.2 -m setup Having pysetup or pysetupX.Y executables would be much more practical with shell auto-completion, though. Regards Antoine. From ab at rdprojekt.pl Thu Oct 21 13:20:12 2010 From: ab at rdprojekt.pl (=?UTF-8?B?QWRhbSBCaWVsYcWEc2tp?=) Date: Thu, 21 Oct 2010 13:20:12 +0200 Subject: [Python-Dev] maximum recursion depth exceeded in lib\pstats.py Message-ID: <4CC021EC.1030302@rdprojekt.pl> Hi all, There's a bug in Python 2.6, module lib\pstats.py, line 150. I'm pretty sure, however that it also exists in other versions, as I don't think that profiler differs very much between them. Let me paste a little piece of surrounding code: class Stats: (....) def add(self, *arg_list): if not arg_list: return self if len(arg_list) > 1: self.add(*arg_list[1:]) #Line 150, the problem is right here! other = arg_list[0] (... do some processing with first item from arg_list ...) This is the code for adding profiling stats from multiple files (names are on arg_list) so that they can be displayed in cumulated and readable form. As you can see it chops off the first item from the list and then uses recursion to process the rest of it. It's no wonder then that when you profiled a little too many function calls, you're bound to see RuntimeError with 'maximum recursion depth exceeded' message when you try using this. IMO this should be done with iteration instead, like: def add(self, *arg_list): if not arg_list: return self for other in arg_list: #Preserved variable name only for sake of comparison with previous snippet (... do some processing with each item from arg_list, merging results in some rightful manner ...) Kind regards, Adam Bielanski. From fuzzyman at voidspace.org.uk Thu Oct 21 14:08:21 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 21 Oct 2010 13:08:21 +0100 Subject: [Python-Dev] maximum recursion depth exceeded in lib\pstats.py In-Reply-To: <4CC021EC.1030302@rdprojekt.pl> References: <4CC021EC.1030302@rdprojekt.pl> Message-ID: <4CC02D35.4040305@voidspace.org.uk> On 21/10/2010 12:20, Adam Biela?ski wrote: > Hi all, > > There's a bug in Python 2.6, module lib\pstats.py, line 150. I'm > pretty sure, however that it also exists in other versions, as I don't > think that profiler differs very much between them. > > Let me paste a little piece of surrounding code: > > class Stats: > (....) > def add(self, *arg_list): > if not arg_list: return self > if len(arg_list) > 1: self.add(*arg_list[1:]) #Line 150, > the problem is right here! > other = arg_list[0] > (... do some processing with first item from arg_list ...) > > This is the code for adding profiling stats from multiple files (names > are on arg_list) so that they can be displayed in cumulated and > readable form. > > As you can see it chops off the first item from the list and then uses > recursion to process the rest of it. It's no wonder then that when you > profiled a little too many function calls, you're bound to see > RuntimeError with 'maximum recursion depth exceeded' message when you > try using this. IMO this should be done with iteration instead, like: > > def add(self, *arg_list): > if not arg_list: return self > for other in arg_list: #Preserved variable name only for > sake of comparison with previous snippet > (... do some processing with each item from arg_list, > merging results in some rightful manner ...) > Without looking at the details your diagnosis and suggested fix seem good. Please raise an issue on the Python bug tracker - preferably with patch and test. http://bugs.python.org/ All the best, Michael Foord > > Kind regards, > Adam Bielanski. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From eric at trueblade.com Thu Oct 21 14:01:27 2010 From: eric at trueblade.com (Eric Smith) Date: Thu, 21 Oct 2010 08:01:27 -0400 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <20101021104454.4ba48afe@pitrou.net> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB39CC8.9050602@canterbury.ac.nz> <20101012105951.3ed845b5@mission> <4CBF9104.6080400@ronadam.com> <20101021104454.4ba48afe@pitrou.net> Message-ID: <4CC02B97.9000608@trueblade.com> On 10/21/2010 4:44 AM, Antoine Pitrou wrote: > On Wed, 20 Oct 2010 20:01:56 -0500 > Ron Adam wrote: >> >> On Ubuntu, I use python, python2.7, python3.1, python3.2 and that is what I >> type to use that particular version. The -m option seems to me to be the >> easiest to do and works with all of these. >> >> python2.7 -m setup >> python3.2 -m setup > > Having pysetup or pysetupX.Y executables would be much more practical > with shell auto-completion, though. I agree. I also don't see any confusion about which python a "pysetupX.Y" would use. Or for that matter a plain "pysetup". It would be the one that a plain "python" would get you. Eric. From dirkjan at ochtman.nl Thu Oct 21 15:32:07 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 21 Oct 2010 15:32:07 +0200 Subject: [Python-Dev] hg migration schedule Message-ID: Okay, we're getting somewhere here. I've been conferring with Georg on when we can do the conversion. He's done some testing with the buildbot setup, which seems to be mostly done. He also completed preliminary ports of the hooks used in the SVN repository. He managed to convince me that we're at a point where it would be good to post some kind of schedule, so here we go. 2010-11-13: availability of a test repo I'm working towards finally having a test repository that's close to what we'll have for the final repository. I hope to have it done sooner than by Nov 13, so it seems like a fairly realistic date. I'm not promising we'll also have docs done by then, but good documentation will be my focus as soon as I'm done with the test repository. I should also mention that I'll be visiting the US east cost (Boston and NYC) from Nov 13 to Nov 21, so I probably won't be spending much time on this that week. 2010-12-12: final conversion (tentative) Georg would really like the conversion to be done by the end of the year, and it seems that this is within reach. The aim here is not to disturb the 3.2 release process too much, of course, while still giving Georg more powerful tools to manage what's left of the release process. We're currently tracking progress in a todo.txt file in the pymigr repo [1] (and in the #python-dev IRC channel), so if you want to help out, you're more than welcome! We're also going to need at least one more big fight about workflow/organizational issues, but I'd like to hold off on that until after we have a test repo that can actually demonstrate the issues that are relevant to that discussion. Cheers, Dirkjan [1] http://hg.python.org/pymigr/file/tip/todo.txt From guido at python.org Thu Oct 21 16:56:03 2010 From: guido at python.org (Guido van Rossum) Date: Thu, 21 Oct 2010 07:56:03 -0700 Subject: [Python-Dev] hg migration schedule In-Reply-To: References: Message-ID: Yay! I am so looking forward to dissing SVN... On Thu, Oct 21, 2010 at 6:32 AM, Dirkjan Ochtman wrote: > Okay, we're getting somewhere here. > > I've been conferring with Georg on when we can do the conversion. He's > done some testing with the buildbot setup, which seems to be mostly > done. He also completed preliminary ports of the hooks used in the SVN > repository. He managed to convince me that we're at a point where it > would be good to post some kind of schedule, so here we go. > > 2010-11-13: availability of a test repo > > I'm working towards finally having a test repository that's close to > what we'll have for the final repository. I hope to have it done > sooner than by Nov 13, so it seems like a fairly realistic date. I'm > not promising we'll also have docs done by then, but good > documentation will be my focus as soon as I'm done with the test > repository. I should also mention that I'll be visiting the US east > cost (Boston and NYC) from Nov 13 to Nov 21, so I probably won't be > spending much time on this that week. > > 2010-12-12: final conversion (tentative) > > Georg would really like the conversion to be done by the end of the > year, and it seems that this is within reach. The aim here is not to > disturb the 3.2 release process too much, of course, while still > giving Georg more powerful tools to manage what's left of the release > process. > > We're currently tracking progress in a todo.txt file in the pymigr > repo [1] (and in the #python-dev IRC channel), so if you want to help > out, you're more than welcome! We're also going to need at least one > more big fight about workflow/organizational issues, but I'd like to > hold off on that until after we have a test repo that can actually > demonstrate the issues that are relevant to that discussion. > > Cheers, > > Dirkjan > > [1] http://hg.python.org/pymigr/file/tip/todo.txt > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) From barry at python.org Thu Oct 21 18:00:40 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 21 Oct 2010 12:00:40 -0400 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <201010200211.35608.victor.stinner@haypocalc.com> References: <201010190353.34796.victor.stinner@haypocalc.com> <20101019101256.73afa7c2@mission> <201010200211.35608.victor.stinner@haypocalc.com> Message-ID: <20101021120040.66b613a0@mission> On Oct 20, 2010, at 02:11 AM, Victor Stinner wrote: >I plan to fix Python documentation: specify the encoding used to decode all >byte string arguments of the C API. I already wrote a draft patch: issue >#9738. This lack of documentation was a big problem for me, because I had to >follow the function calls to get the encoding. That's exactly what I was looking for! Thanks. I think you've learned a huge amount of good information that's difficult to find, so writing it up in a more permanent and easy to find location will really help future Python developers! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From brett at python.org Thu Oct 21 19:01:15 2010 From: brett at python.org (Brett Cannon) Date: Thu, 21 Oct 2010 10:01:15 -0700 Subject: [Python-Dev] hg migration schedule In-Reply-To: References: Message-ID: On Thu, Oct 21, 2010 at 06:32, Dirkjan Ochtman wrote: > Okay, we're getting somewhere here. > > I've been conferring with Georg on when we can do the conversion. He's > done some testing with the buildbot setup, which seems to be mostly > done. He also completed preliminary ports of the hooks used in the SVN > repository. He managed to convince me that we're at a point where it > would be good to post some kind of schedule, so here we go. > > 2010-11-13: availability of a test repo > > I'm working towards finally having a test repository that's close to > what we'll have for the final repository. I hope to have it done > sooner than by Nov 13, so it seems like a fairly realistic date. I'm > not promising we'll also have docs done by then, but good > documentation will be my focus as soon as I'm done with the test > repository. I should also mention that I'll be visiting the US east > cost (Boston and NYC) from Nov 13 to Nov 21, so I probably won't be > spending much time on this that week. I am currently slated to start my PSF grant in January when I will rewrite the dev docs, so I can help with the workflow documentation starting then. -Brett > > 2010-12-12: final conversion (tentative) > > Georg would really like the conversion to be done by the end of the > year, and it seems that this is within reach. The aim here is not to > disturb the 3.2 release process too much, of course, while still > giving Georg more powerful tools to manage what's left of the release > process. > > We're currently tracking progress in a todo.txt file in the pymigr > repo [1] (and in the #python-dev IRC channel), so if you want to help > out, you're more than welcome! We're also going to need at least one > more big fight about workflow/organizational issues, but I'd like to > hold off on that until after we have a test repo that can actually > demonstrate the issues that are relevant to that discussion. > > Cheers, > > Dirkjan > > [1] http://hg.python.org/pymigr/file/tip/todo.txt > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From merwok at netwok.org Thu Oct 21 20:22:35 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 21 Oct 2010 20:22:35 +0200 Subject: [Python-Dev] [Python-checkins] r85733 - in python/branches/py3k: Doc/library/logging.rst Lib/logging/__init__.py In-Reply-To: <20101019211349.C7237EE982@mail.python.org> References: <20101019211349.C7237EE982@mail.python.org> Message-ID: <4CC084EB.4000803@netwok.org> +``Filters` can be used by ``Handlers`` and ``Loggers`` for more ? missing ` here. Regards From a.badger at gmail.com Thu Oct 21 21:14:55 2010 From: a.badger at gmail.com (Toshio Kuratomi) Date: Thu, 21 Oct 2010 12:14:55 -0700 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <20101021120040.66b613a0@mission> References: <201010190353.34796.victor.stinner@haypocalc.com> <20101019101256.73afa7c2@mission> <201010200211.35608.victor.stinner@haypocalc.com> <20101021120040.66b613a0@mission> Message-ID: <20101021191455.GQ3652@unaka.lan> On Thu, Oct 21, 2010 at 12:00:40PM -0400, Barry Warsaw wrote: > On Oct 20, 2010, at 02:11 AM, Victor Stinner wrote: > > >I plan to fix Python documentation: specify the encoding used to decode all > >byte string arguments of the C API. I already wrote a draft patch: issue > >#9738. This lack of documentation was a big problem for me, because I had to > >follow the function calls to get the encoding. > This will be truly excellent! > That's exactly what I was looking for! Thanks. I think you've learned a huge > amount of good information that's difficult to find, so writing it up in a > more permanent and easy to find location will really help future Python > developers! > One further thing I'd be interested in is if you could document any best practices from this experience. Things like, "surrogateescape is a good/bad default in these cases", When is parallel functions for bytes and str better than a single polymorphic function? That way when other modules are added to the stdlib, things can be more consistent. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From benjamin at python.org Fri Oct 22 00:57:05 2010 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 21 Oct 2010 17:57:05 -0500 Subject: [Python-Dev] 3.1.3 and 2.7.1 release schedule Message-ID: In the interest of getting 3.1.3 and 2.7.1 out by next year, here's a tentative release schedule: November 13th - RC1 November 27th - RC2 December 11th - Final I'll jump for it if there are no objections. -- Regards, Benjamin From martin at v.loewis.de Fri Oct 22 01:12:55 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 22 Oct 2010 01:12:55 +0200 Subject: [Python-Dev] python-2.6.6 coredump running newspipe In-Reply-To: <20101007121752.GB15070@danbala.tuwien.ac.at> References: <20101007121752.GB15070@danbala.tuwien.ac.at> Message-ID: <4CC0C8F7.3040505@v.loewis.de> > Any ideas what could cause this or how to fix this? It could be a compiler bug; try disabling optimization. Regards, Martin From greg.ewing at canterbury.ac.nz Fri Oct 22 02:13:36 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 22 Oct 2010 13:13:36 +1300 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CC02B97.9000608@trueblade.com> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB39CC8.9050602@canterbury.ac.nz> <20101012105951.3ed845b5@mission> <4CBF9104.6080400@ronadam.com> <20101021104454.4ba48afe@pitrou.net> <4CC02B97.9000608@trueblade.com> Message-ID: <4CC0D730.5070909@canterbury.ac.nz> Eric Smith wrote: > Or for that matter a plain "pysetup". It would > be the one that a plain "python" would get you. If 'pysetup' is simply a shell script that invokes 'python -m setup' using the current search path, I guess that's true. On Windows, however, it seems to me that the current 'python setup.py' scheme has advantages, since it lets you simply invoke 'setup.py' and rely on file associations to get you the current python. Supporting either 'python -m setup' or 'pysetup' out of the box would require install-time path hacking of the sort that some people are uncomfortable about. -- Greg From barry at python.org Fri Oct 22 03:57:06 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 21 Oct 2010 21:57:06 -0400 Subject: [Python-Dev] 3.1.3 and 2.7.1 release schedule In-Reply-To: References: Message-ID: <20101021215706.21f13340@mission> On Oct 21, 2010, at 05:57 PM, Benjamin Peterson wrote: >In the interest of getting 3.1.3 and 2.7.1 out by next year, here's a >tentative release schedule: > >November 13th - RC1 >November 27th - RC2 >December 11th - Final Sounds like you're planning to get finals out this year, not next :). +1 from me! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From rrr at ronadam.com Fri Oct 22 05:31:11 2010 From: rrr at ronadam.com (Ron Adam) Date: Thu, 21 Oct 2010 22:31:11 -0500 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CC0D730.5070909@canterbury.ac.nz> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB39CC8.9050602@canterbury.ac.nz> <20101012105951.3ed845b5@mission> <4CBF9104.6080400@ronadam.com> <20101021104454.4ba48afe@pitrou.net> <4CC02B97.9000608@trueblade.com> <4CC0D730.5070909@canterbury.ac.nz> Message-ID: <4CC1057F.6070208@ronadam.com> On 10/21/2010 07:13 PM, Greg Ewing wrote: > Eric Smith wrote: >> Or for that matter a plain "pysetup". It would be the one that a plain >> "python" would get you. > > If 'pysetup' is simply a shell script that invokes 'python -m setup' > using the current search path, I guess that's true. > > On Windows, however, it seems to me that the current 'python setup.py' > scheme has advantages, since it lets you simply invoke 'setup.py' and > rely on file associations to get you the current python. Supporting > either 'python -m setup' or 'pysetup' out of the box would require > install-time path hacking of the sort that some people are uncomfortable > about. Terek said this in the first post of this thread... > I just wanted to make sure that once distutils2 is back in the stdlib, > it's OK for us to add that script in the distribution. When it's in the stdlib, the -m option should work just like any other script run from the stdlib. What path hacking are you thinking of? Ron From rrr at ronadam.com Fri Oct 22 05:31:11 2010 From: rrr at ronadam.com (Ron Adam) Date: Thu, 21 Oct 2010 22:31:11 -0500 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CC0D730.5070909@canterbury.ac.nz> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB39CC8.9050602@canterbury.ac.nz> <20101012105951.3ed845b5@mission> <4CBF9104.6080400@ronadam.com> <20101021104454.4ba48afe@pitrou.net> <4CC02B97.9000608@trueblade.com> <4CC0D730.5070909@canterbury.ac.nz> Message-ID: <4CC1057F.6070208@ronadam.com> On 10/21/2010 07:13 PM, Greg Ewing wrote: > Eric Smith wrote: >> Or for that matter a plain "pysetup". It would be the one that a plain >> "python" would get you. > > If 'pysetup' is simply a shell script that invokes 'python -m setup' > using the current search path, I guess that's true. > > On Windows, however, it seems to me that the current 'python setup.py' > scheme has advantages, since it lets you simply invoke 'setup.py' and > rely on file associations to get you the current python. Supporting > either 'python -m setup' or 'pysetup' out of the box would require > install-time path hacking of the sort that some people are uncomfortable > about. Terek said this in the first post of this thread... > I just wanted to make sure that once distutils2 is back in the stdlib, > it's OK for us to add that script in the distribution. When it's in the stdlib, the -m option should work just like any other script run from the stdlib. What path hacking are you thinking of? Ron From p.f.moore at gmail.com Fri Oct 22 09:20:19 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 22 Oct 2010 08:20:19 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: <4CC1057F.6070208@ronadam.com> References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB39CC8.9050602@canterbury.ac.nz> <20101012105951.3ed845b5@mission> <4CBF9104.6080400@ronadam.com> <20101021104454.4ba48afe@pitrou.net> <4CC02B97.9000608@trueblade.com> <4CC0D730.5070909@canterbury.ac.nz> <4CC1057F.6070208@ronadam.com> Message-ID: On 22 October 2010 04:31, Ron Adam wrote: > When it's in the stdlib, the -m option should work just like any other > script run from the stdlib. > > What path hacking are you thinking of? On Windows, neither the "python" executable nor scripts in C:\Pythonxx\Scripts are in the PATH by default. On the other hand, .py files will run automatically via the registered file extension. Manipulating PATH at install time (to add C:\PythonXX and/or C:\PythonXX\Scripts) is not done - it is (rightly, in my view) considered too difficult to get right, particularly when it comes to uninstalling. Many Windows users will, I guess, manually add python to PATH (so that python-m works). Some people also add C:\PythonXX\Scripts. Personally, I don't - so for me a pysetup script in that location would be no use. So my personal vote is +1 for a python -m approach, and -0 for a pysetup executable. I'm -1 on a pysetup.bat batch file - bat files have other issues which IMO make them effectively useless. Paul. From dirkjan at ochtman.nl Fri Oct 22 09:36:00 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 22 Oct 2010 09:36:00 +0200 Subject: [Python-Dev] 3.1.3 and 2.7.1 release schedule In-Reply-To: References: Message-ID: On Fri, Oct 22, 2010 at 00:57, Benjamin Peterson wrote: > In the interest of getting 3.1.3 and 2.7.1 out by next year, here's a > tentative release schedule: > > November 13th - RC1 > November 27th - RC2 > December 11th - Final The last one might clash with the hg migration a bit, do we need to worry about that? Or did you purposely pick the day before the planned hg migration? Cheers, Dirkjan From g.brandl at gmx.net Fri Oct 22 11:06:52 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 22 Oct 2010 11:06:52 +0200 Subject: [Python-Dev] 3.1.3 and 2.7.1 release schedule In-Reply-To: References: Message-ID: Am 22.10.2010 09:36, schrieb Dirkjan Ochtman: > On Fri, Oct 22, 2010 at 00:57, Benjamin Peterson wrote: >> In the interest of getting 3.1.3 and 2.7.1 out by next year, here's a >> tentative release schedule: >> >> November 13th - RC1 >> November 27th - RC2 >> December 11th - Final > > The last one might clash with the hg migration a bit, do we need to > worry about that? Or did you purposely pick the day before the planned > hg migration? If everything goes as planned, there won't be many commits between RC2 and final, so it should be fine. The svn repos won't be removed anyway, so making a release from them is still possible. (Side question: do we want to move the svn repos to a slightly different URL so that people tracking the repo know something's happened?) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From dirkjan at ochtman.nl Fri Oct 22 11:41:44 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 22 Oct 2010 11:41:44 +0200 Subject: [Python-Dev] 3.1.3 and 2.7.1 release schedule In-Reply-To: References: Message-ID: On Fri, Oct 22, 2010 at 11:06, Georg Brandl wrote: > If everything goes as planned, there won't be many commits between RC2 and > final, so it should be fine. ?The svn repos won't be removed anyway, so > making a release from them is still possible. Okay, but accepting commits in both SVN and hg at the same time is potentially messy. > (Side question: do we want to move the svn repos to a slightly different > URL so that people tracking the repo know something's happened?) I would be in favor of that, yes. Cheers, Dirkjan From g.brandl at gmx.net Fri Oct 22 11:51:02 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 22 Oct 2010 11:51:02 +0200 Subject: [Python-Dev] 3.1.3 and 2.7.1 release schedule In-Reply-To: References: Message-ID: Am 22.10.2010 11:41, schrieb Dirkjan Ochtman: > On Fri, Oct 22, 2010 at 11:06, Georg Brandl wrote: >> If everything goes as planned, there won't be many commits between RC2 and >> final, so it should be fine. The svn repos won't be removed anyway, so >> making a release from them is still possible. > > Okay, but accepting commits in both SVN and hg at the same time is > potentially messy. I don't think Benjamin would want anyone but him committing to these branches between rc2 and final anyway :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From victor.stinner at haypocalc.com Fri Oct 22 14:01:44 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Fri, 22 Oct 2010 14:01:44 +0200 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <20101021191455.GQ3652@unaka.lan> References: <201010190353.34796.victor.stinner@haypocalc.com> <20101021120040.66b613a0@mission> <20101021191455.GQ3652@unaka.lan> Message-ID: <201010221401.44549.victor.stinner@haypocalc.com> Le jeudi 21 octobre 2010 21:14:55, Toshio Kuratomi a ?crit : > > That's exactly what I was looking for! Thanks. I think you've learned a > > huge amount of good information that's difficult to find, so writing it > > up in a more permanent and easy to find location will really help future > > Python developers! > > One further thing I'd be interested in is if you could document any best > practices from this experience. Things like, "surrogateescape is a > good/bad default in these cases", I advice to use the PEP 383 (surrogateescape) when the *native* data type is bytes. Some examples: - filenames on UNIX/BSD - environment variables on UNIX/BSD - well, most data send/received from the system on UNIX/BSD :-) For network protocols, I don't know. It looks like the new email modules will offer two API levels: low level (native type) using bytes, high level using str (unicode). I don't know if the high level API uses the PEP 383 or not. PEP 383 can be used to avoid UnicodeDecodeError. But sometimes it's better to raise an error to warn the user that the encoding is incorrect or the input data is invalid (well, at least not correctly according to the encoding). I don't use strict rules. Each problem is different. Eg. it looks like not everybody agrees to use the PEP 383 for the host/domain name (issue #9377, I didn't read the whole issue, just few lines). > When is parallel functions for bytes and str better than a single > polymorphic function? If you cannot decide the output type depending on the inputs, it's better to have two functions. Examples: - 2 functions; os.getcwd() / os.getcwdb(). - polymorphic: os.path.*() But you should never accept mixed types, eg. os.path.join(b'bytes', 'unicode) have to raise a TypeError. -- Victor Stinner http://www.haypocalc.com/ From stefan_ml at behnel.de Fri Oct 22 15:54:39 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 22 Oct 2010 15:54:39 +0200 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k Message-ID: Hi, since SVN rev. 85392, Cython's installation fails on the py3k branch with a weird globals error. I think it is related to some sys.modules magic that we do in order to support running Cython in Python 3 using lib2to3. Basically, what we do is, we import some parts of Cython at the beginning that are Py3 clean, specifically some distutils build_ext replacement for building Cython modules. Then we start up distutils, which first runs lib2to3 on Cython's sources to convert them into Py3 code. When it then gets to building the binary modules, we remove all Cython modules and packages from sys.modules and reimport their 2to3-ed sources so that we can run the complete compiler during the installation (to bootstrap parts of Cython into binary modules). Since the above revision, this process bails out with an error when accessing "os.path" because "os" is None. The "os" module is imported globally in our early-imported build_ext module, more or less like this: import os from distutils.command import build_ext as _build_ext class build_ext(_build_ext.build_ext): def build_extensions(self): print(os) # prints None! I suspect that the fact that we remove the modules from sys.modules somehow triggers the cleanup of these modules while there are still objects from these modules alive that refer to their globals. So, what I think is happening is that the module cleanup sets the module's globals to None before the objects from that module that refer to these globals have actually gone out of scope. Could someone (benjamin?) please look into this? Thanks, Stefan From benjamin at python.org Fri Oct 22 16:03:21 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 22 Oct 2010 09:03:21 -0500 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k In-Reply-To: References: Message-ID: 2010/10/22 Stefan Behnel : > Hi, > > since SVN rev. 85392, Cython's installation fails on the py3k branch with a > weird globals error. I think it is related to some sys.modules magic that we > do in order to support running Cython in Python 3 using lib2to3. > > Basically, what we do is, we import some parts of Cython at the beginning > that are Py3 clean, specifically some distutils build_ext replacement for > building Cython modules. Then we start up distutils, which first runs > lib2to3 on Cython's sources to convert them into Py3 code. When it then gets > to building the binary modules, we remove all Cython modules and packages > from sys.modules and reimport their 2to3-ed sources so that we can run the > complete compiler during the installation (to bootstrap parts of Cython into > binary modules). > > Since the above revision, this process bails out with an error when > accessing "os.path" because "os" is None. The "os" module is imported > globally in our early-imported build_ext module, more or less like this: > > ? ?import os > > ? ?from distutils.command import build_ext as _build_ext > > ? ?class build_ext(_build_ext.build_ext): > > ? ? ? ?def build_extensions(self): > ? ? ? ? ? ?print(os) # prints None! > > I suspect that the fact that we remove the modules from sys.modules somehow > triggers the cleanup of these modules while there are still objects from > these modules alive that refer to their globals. So, what I think is > happening is that the module cleanup sets the module's globals to None > before the objects from that module that refer to these globals have > actually gone out of scope. > > Could someone (benjamin?) please look into this? Is this broken before 2.7, ie 2.6 and 2.6? -- Regards, Benjamin From benjamin at python.org Fri Oct 22 16:09:01 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 22 Oct 2010 09:09:01 -0500 Subject: [Python-Dev] 3.1.3 and 2.7.1 release schedule In-Reply-To: References: Message-ID: 2010/10/22 Dirkjan Ochtman : > On Fri, Oct 22, 2010 at 00:57, Benjamin Peterson wrote: >> In the interest of getting 3.1.3 and 2.7.1 out by next year, here's a >> tentative release schedule: >> >> November 13th - RC1 >> November 27th - RC2 >> December 11th - Final > > The last one might clash with the hg migration a bit, do we need to > worry about that? Or did you purposely pick the day before the planned > hg migration? I'm not too worried. Commits should be at a minimum, and changesets can be tagged post-transition if needed. -- Regards, Benjamin From stefan_ml at behnel.de Fri Oct 22 16:13:17 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 22 Oct 2010 16:13:17 +0200 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k In-Reply-To: References: Message-ID: Benjamin Peterson, 22.10.2010 16:03: > 2010/10/22 Stefan Behnel: >> since SVN rev. 85392, Cython's installation fails on the py3k branch with a >> weird globals error. I think it is related to some sys.modules magic that we >> do in order to support running Cython in Python 3 using lib2to3. >> >> Basically, what we do is, we import some parts of Cython at the beginning >> that are Py3 clean, specifically some distutils build_ext replacement for >> building Cython modules. Then we start up distutils, which first runs >> lib2to3 on Cython's sources to convert them into Py3 code. When it then gets >> to building the binary modules, we remove all Cython modules and packages >> from sys.modules and reimport their 2to3-ed sources so that we can run the >> complete compiler during the installation (to bootstrap parts of Cython into >> binary modules). >> >> Since the above revision, this process bails out with an error when >> accessing "os.path" because "os" is None. The "os" module is imported >> globally in our early-imported build_ext module, more or less like this: >> >> import os >> >> from distutils.command import build_ext as _build_ext >> >> class build_ext(_build_ext.build_ext): >> >> def build_extensions(self): >> print(os) # prints None! >> >> I suspect that the fact that we remove the modules from sys.modules somehow >> triggers the cleanup of these modules while there are still objects from >> these modules alive that refer to their globals. So, what I think is >> happening is that the module cleanup sets the module's globals to None >> before the objects from that module that refer to these globals have >> actually gone out of scope. >> >> Could someone (benjamin?) please look into this? > > Is this broken before 2.7, ie 2.6 and 2.6? I can't tell. Py2 doesn't need 2to3, so we don't unload the modules there. Stefan From benjamin at python.org Fri Oct 22 16:17:44 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 22 Oct 2010 09:17:44 -0500 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k In-Reply-To: References: Message-ID: 2010/10/22 Stefan Behnel : > Benjamin Peterson, 22.10.2010 16:03: >> >> 2010/10/22 Stefan Behnel: >>> >>> since SVN rev. 85392, Cython's installation fails on the py3k branch with >>> a >>> weird globals error. I think it is related to some sys.modules magic that >>> we >>> do in order to support running Cython in Python 3 using lib2to3. >>> >>> Basically, what we do is, we import some parts of Cython at the beginning >>> that are Py3 clean, specifically some distutils build_ext replacement for >>> building Cython modules. Then we start up distutils, which first runs >>> lib2to3 on Cython's sources to convert them into Py3 code. When it then >>> gets >>> to building the binary modules, we remove all Cython modules and packages >>> from sys.modules and reimport their 2to3-ed sources so that we can run >>> the >>> complete compiler during the installation (to bootstrap parts of Cython >>> into >>> binary modules). >>> >>> Since the above revision, this process bails out with an error when >>> accessing "os.path" because "os" is None. The "os" module is imported >>> globally in our early-imported build_ext module, more or less like this: >>> >>> ? ?import os >>> >>> ? ?from distutils.command import build_ext as _build_ext >>> >>> ? ?class build_ext(_build_ext.build_ext): >>> >>> ? ? ? ?def build_extensions(self): >>> ? ? ? ? ? ?print(os) # prints None! >>> >>> I suspect that the fact that we remove the modules from sys.modules >>> somehow >>> triggers the cleanup of these modules while there are still objects from >>> these modules alive that refer to their globals. So, what I think is >>> happening is that the module cleanup sets the module's globals to None >>> before the objects from that module that refer to these globals have >>> actually gone out of scope. >>> >>> Could someone (benjamin?) please look into this? >> >> Is this broken before 2.7, ie 2.6 and 2.6? > > I can't tell. Py2 doesn't need 2to3, so we don't unload the modules there. What about 3.1.0 then? -- Regards, Benjamin From exarkun at twistedmatrix.com Fri Oct 22 16:32:25 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 22 Oct 2010 14:32:25 -0000 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k In-Reply-To: References: Message-ID: <20101022143225.1927.912284794.divmod.xquotient.35@localhost.localdomain> On 02:13 pm, stefan_ml at behnel.de wrote: >Benjamin Peterson, 22.10.2010 16:03: >>2010/10/22 Stefan Behnel: >>>since SVN rev. 85392, Cython's installation fails on the py3k branch >>>with a >>>weird globals error. I think it is related to some sys.modules magic >>>that we >>>do in order to support running Cython in Python 3 using lib2to3. >>> >>>Basically, what we do is, we import some parts of Cython at the >>>beginning >>>that are Py3 clean, specifically some distutils build_ext replacement >>>for >>>building Cython modules. Then we start up distutils, which first runs >>>lib2to3 on Cython's sources to convert them into Py3 code. When it >>>then gets >>>to building the binary modules, we remove all Cython modules and >>>packages >>>from sys.modules and reimport their 2to3-ed sources so that we can >>>run the >>>complete compiler during the installation (to bootstrap parts of >>>Cython into >>>binary modules). >>> >>>Since the above revision, this process bails out with an error when >>>accessing "os.path" because "os" is None. The "os" module is imported >>>globally in our early-imported build_ext module, more or less like >>>this: >>> >>> import os >>> >>> from distutils.command import build_ext as _build_ext >>> >>> class build_ext(_build_ext.build_ext): >>> >>> def build_extensions(self): >>> print(os) # prints None! >>> >>>I suspect that the fact that we remove the modules from sys.modules >>>somehow >>>triggers the cleanup of these modules while there are still objects >>>from >>>these modules alive that refer to their globals. So, what I think is >>>happening is that the module cleanup sets the module's globals to >>>None >>>before the objects from that module that refer to these globals have >>>actually gone out of scope. Instances of classes don't refer to the module their class is defined in. It seems more likely that the reason the module is garbage collected is that there really is nothing which refers to it anymore. The behavior of setting the attributes of a module being freed to None has been in place for a long time, r85392 only restored it after a brief absence. Perhaps Cython itself should keep the modules alive that it wants kept alive. Alternatively, if Cython owns the code that's running into the zapped global, you could change it to not use globals. Jean-Paul From stefan_ml at behnel.de Fri Oct 22 16:33:06 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 22 Oct 2010 16:33:06 +0200 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k In-Reply-To: References: Message-ID: Benjamin Peterson, 22.10.2010 16:17: > 2010/10/22 Stefan Behnel: >> Benjamin Peterson, 22.10.2010 16:03: >>> 2010/10/22 Stefan Behnel: >>>> >>>> since SVN rev. 85392, Cython's installation fails on the py3k branch with >>>> a >>>> weird globals error. I think it is related to some sys.modules magic that >>>> we >>>> do in order to support running Cython in Python 3 using lib2to3. >>>> >>>> Basically, what we do is, we import some parts of Cython at the beginning >>>> that are Py3 clean, specifically some distutils build_ext replacement for >>>> building Cython modules. Then we start up distutils, which first runs >>>> lib2to3 on Cython's sources to convert them into Py3 code. When it then >>>> gets >>>> to building the binary modules, we remove all Cython modules and packages >>>> from sys.modules and reimport their 2to3-ed sources so that we can run >>>> the >>>> complete compiler during the installation (to bootstrap parts of Cython >>>> into binary modules). >>>> >>>> Since the above revision, this process bails out with an error when >>>> accessing "os.path" because "os" is None. The "os" module is imported >>>> globally in our early-imported build_ext module, more or less like this: >>>> >>>> import os >>>> >>>> from distutils.command import build_ext as _build_ext >>>> >>>> class build_ext(_build_ext.build_ext): >>>> >>>> def build_extensions(self): >>>> print(os) # prints None! >>>> >>>> I suspect that the fact that we remove the modules from sys.modules >>>> somehow >>>> triggers the cleanup of these modules while there are still objects from >>>> these modules alive that refer to their globals. So, what I think is >>>> happening is that the module cleanup sets the module's globals to None >>>> before the objects from that module that refer to these globals have >>>> actually gone out of scope. >>>> >>>> Could someone (benjamin?) please look into this? >>> >>> Is this broken before 2.7, ie 2.6 and 2.6? >> >> I can't tell. Py2 doesn't need 2to3, so we don't unload the modules there. > > What about 3.1.0 then? The 3.1.2 release was fine, but the current 3.1 SVN branch is not. I assume you have applied the change there, too? py3.1 branch: https://sage.math.washington.edu:8091/hudson/job/cython-devel-build-py31/524/console py3k branch: https://sage.math.washington.edu:8091/hudson/job/cython-devel-build-py3k/613/console We have our CI tests running against both branches, as well as all Py2 branches starting from 2.3. That's how I spotted it. Stefan From benjamin at python.org Fri Oct 22 16:41:09 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 22 Oct 2010 09:41:09 -0500 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k In-Reply-To: <20101022143225.1927.912284794.divmod.xquotient.35@localhost.localdomain> References: <20101022143225.1927.912284794.divmod.xquotient.35@localhost.localdomain> Message-ID: 2010/10/22 : > Instances of classes don't refer to the module their class is defined in. > ?It seems more likely that the reason the module is garbage collected is > that there really is nothing which refers to it anymore. Indeed, this is really a Python bug, but there's no good way to deal with it unless dictionaries can know they are module globals. -- Regards, Benjamin From benjamin at python.org Fri Oct 22 16:43:56 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 22 Oct 2010 09:43:56 -0500 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k In-Reply-To: References: Message-ID: 2010/10/22 Stefan Behnel : > Benjamin Peterson, 22.10.2010 16:17: >> What about 3.1.0 then? > > The 3.1.2 release was fine, but the current 3.1 SVN branch is not. I assume > you have applied the change there, too? Yes. Unfortunately, this behavior is more "correct" for most cases because it results in module globals being finalized. You could run 2to3 in a subprocess or hang onto modules you to keep the globals of while you're using them. -- Regards, Benjamin From solipsis at pitrou.net Fri Oct 22 16:55:52 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 22 Oct 2010 16:55:52 +0200 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k References: <20101022143225.1927.912284794.divmod.xquotient.35@localhost.localdomain> Message-ID: <20101022165552.286af03e@pitrou.net> On Fri, 22 Oct 2010 09:41:09 -0500 Benjamin Peterson wrote: > 2010/10/22 : > > Instances of classes don't refer to the module their class is defined in. > > ?It seems more likely that the reason the module is garbage collected is > > that there really is nothing which refers to it anymore. > > Indeed, this is really a Python bug, but there's no good way to deal > with it unless dictionaries can know they are module globals. How about making functions keep a reference to the module they're defined in? Is there any reason we shouldn't do that? Regards Antoine. From stefan_ml at behnel.de Fri Oct 22 16:58:00 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 22 Oct 2010 16:58:00 +0200 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k In-Reply-To: <20101022143225.1927.912284794.divmod.xquotient.35@localhost.localdomain> References: <20101022143225.1927.912284794.divmod.xquotient.35@localhost.localdomain> Message-ID: exarkun at twistedmatrix.com, 22.10.2010 16:32: > Instances of classes don't refer to the module their class is defined > in. It seems more likely that the reason the module is garbage collected > is that there really is nothing which refers to it anymore. > > The behavior of setting the attributes of a module being freed to None > has been in place for a long time, r85392 only restored it after a brief > absence. > > Perhaps Cython itself should keep the modules alive that it wants kept > alive. Given that this only happens during an install process, this works for me. Thanks, Stefan From benjamin at python.org Fri Oct 22 17:06:10 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 22 Oct 2010 10:06:10 -0500 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k In-Reply-To: <20101022165552.286af03e@pitrou.net> References: <20101022143225.1927.912284794.divmod.xquotient.35@localhost.localdomain> <20101022165552.286af03e@pitrou.net> Message-ID: 2010/10/22 Antoine Pitrou : > On Fri, 22 Oct 2010 09:41:09 -0500 > Benjamin Peterson wrote: >> 2010/10/22 ?: >> > Instances of classes don't refer to the module their class is defined in. >> > ?It seems more likely that the reason the module is garbage collected is >> > that there really is nothing which refers to it anymore. >> >> Indeed, this is really a Python bug, but there's no good way to deal >> with it unless dictionaries can know they are module globals. > > How about making functions keep a reference to the module they're > defined in? Is there any reason we shouldn't do that? I thought of that, too. It wouldn't be trivial to implement, though, and wouldn't solve the problem of clearing globals to avoid references cycles. -- Regards, Benjamin From stefan_ml at behnel.de Fri Oct 22 17:37:53 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 22 Oct 2010 17:37:53 +0200 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k In-Reply-To: <20101022165552.286af03e@pitrou.net> References: <20101022143225.1927.912284794.divmod.xquotient.35@localhost.localdomain> <20101022165552.286af03e@pitrou.net> Message-ID: Antoine Pitrou, 22.10.2010 16:55: > On Fri, 22 Oct 2010 09:41:09 -0500 > Benjamin Peterson wrote: >> 2010/10/22 exarkun: >>> Instances of classes don't refer to the module their class is defined in. >>> It seems more likely that the reason the module is garbage collected is >>> that there really is nothing which refers to it anymore. >> >> Indeed, this is really a Python bug, but there's no good way to deal >> with it unless dictionaries can know they are module globals. > > How about making functions keep a reference to the module they're > defined in? Is there any reason we shouldn't do that? I think the general problem is that "the module" can be a pretty broad thing, potentially referring to all sorts of stuff such as (usually) several other modules. Think of an extreme case where you build and return a local function in a module function. Now all you use the module for is to get at the outer function and call it once to get an instance of the inner function. But the module must stay alive as long as the inner function lives, even if its closure is completely self-sufficient. You could optimise this specific case, but as soon as as the function refers to any non-local name, you'd have to keep the entire module dict alive in case someone decides to change it. It would also mean that each and every function will end up in a reference cycle by default that will require garbage collection for cleanup. This isn't currently a common case, as only modules refer to their content and rarely the other way round. Augmenting the function closure to include (indirections to) module globals won't work very well either, as it would either mean you still keep the module dict alive or it would complicate modifications to the module dict that would need to be reflected in the closures in some way. That gets us back to the usual space-time tradeoff, out of which we currently prefer both at the cost of less safe behaviour in the (rather rare) case of module unloading. Stefan From status at bugs.python.org Fri Oct 22 18:08:20 2010 From: status at bugs.python.org (Python tracker) Date: Fri, 22 Oct 2010 18:08:20 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20101022160820.C7498782F2@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2010-10-15 - 2010-10-22) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues stats: open 2494 (+32) closed 19460 (+110) total 21954 (+56) Open issues with patches: 1039 Issues opened (32) ================== #3367: Uninitialized value read in parsetok.c http://bugs.python.org/issue3367 reopened by skrah #10117: Tools/scripts/reindent.py fails on non-UTF-8 encodings http://bugs.python.org/issue10117 opened by belopolsky #10118: Tkinter does not find font http://bugs.python.org/issue10118 opened by mark.saaltink #10121: test_multiprocessing stuck in test_make_pool if run in a loop http://bugs.python.org/issue10121 opened by sandro.tosi #10126: test_distutils failure with --enable-shared http://bugs.python.org/issue10126 opened by pitrou #10128: multiprocessing.Pool throws exception with __main__.py http://bugs.python.org/issue10128 opened by Michael.Olson #10130: Create epub format docs and offer them on the download page http://bugs.python.org/issue10130 opened by georg.brandl #10131: deepcopying an xml.dom.minidom.Document generates an invalid X http://bugs.python.org/issue10131 opened by flox #10133: multiprocessing: conn_recv_string() broken error handling http://bugs.python.org/issue10133 opened by hfuru #10134: test_email failures on Windows: end of line issue? http://bugs.python.org/issue10134 opened by haypo #10136: kill_python doesn't work with short path http://bugs.python.org/issue10136 opened by ocean-city #10138: calendar module does not support years outside [1, 9999] range http://bugs.python.org/issue10138 opened by belopolsky #10141: SocketCan support http://bugs.python.org/issue10141 opened by instigatorirc #10142: Support for SEEK_HOLE/SEEK_DATA http://bugs.python.org/issue10142 opened by jcea #10143: Update "os.pathconf" values http://bugs.python.org/issue10143 opened by jcea #10145: float.is_integer is undocumented http://bugs.python.org/issue10145 opened by ralph.corderoy #10148: st_mtime differs after shutil.copy2 http://bugs.python.org/issue10148 opened by shaurz #10149: Data truncation in expat parser http://bugs.python.org/issue10149 opened by Maciek.J #10153: Memory leak in pythonrun.c http://bugs.python.org/issue10153 opened by skrah #10154: locale.normalize strips "-" from UTF-8, which fails on Mac http://bugs.python.org/issue10154 opened by ixokai #10155: Add fixups for encoding problems to wsgiref http://bugs.python.org/issue10155 opened by aclover #10156: Initialization of globals in unicodeobject.c http://bugs.python.org/issue10156 opened by skrah #10157: Memory leak (r70152) http://bugs.python.org/issue10157 opened by skrah #10158: BadInternalCall running test_multiprocessing http://bugs.python.org/issue10158 opened by rthalley #10160: operator.attrgetter slower than lambda after adding dotted nam http://bugs.python.org/issue10160 opened by tzot #10161: unittest: use ascii() instead of repr() to display values on e http://bugs.python.org/issue10161 opened by haypo #10164: Add an assertBytesEqual to unittest and use it for bytes asser http://bugs.python.org/issue10164 opened by r.david.murray #10167: ESET Trojan Alert [python-3.1.2.amd64 ON Win7-64] http://bugs.python.org/issue10167 opened by Mr.Collins #10169: socket.sendto raises incorrect exception when passed incorrect http://bugs.python.org/issue10169 opened by exarkun #10170: Relationship between turtle speed setting and actual speed is http://bugs.python.org/issue10170 opened by belopolsky #10171: Ugly buttons in some Tkinter objects in Windows http://bugs.python.org/issue10171 opened by rafe.kettler #10172: code block has no syntax coloring http://bugs.python.org/issue10172 opened by wcyang Most recent 15 issues with no replies (15) ========================================== #10172: code block has no syntax coloring http://bugs.python.org/issue10172 #10171: Ugly buttons in some Tkinter objects in Windows http://bugs.python.org/issue10171 #10170: Relationship between turtle speed setting and actual speed is http://bugs.python.org/issue10170 #10169: socket.sendto raises incorrect exception when passed incorrect http://bugs.python.org/issue10169 #10161: unittest: use ascii() instead of repr() to display values on e http://bugs.python.org/issue10161 #10138: calendar module does not support years outside [1, 9999] range http://bugs.python.org/issue10138 #10136: kill_python doesn't work with short path http://bugs.python.org/issue10136 #10133: multiprocessing: conn_recv_string() broken error handling http://bugs.python.org/issue10133 #10130: Create epub format docs and offer them on the download page http://bugs.python.org/issue10130 #10118: Tkinter does not find font http://bugs.python.org/issue10118 #10112: Use -Wl,--dynamic-list=x.list, not -Xlinker -export-dynamic http://bugs.python.org/issue10112 #10090: python -m locale fails on OSX http://bugs.python.org/issue10090 #10077: Python 3.1: site error is not logged http://bugs.python.org/issue10077 #10050: urllib.request still has old 2.x urllib primitives http://bugs.python.org/issue10050 #10048: urllib.request documentation confusing http://bugs.python.org/issue10048 Most recent 15 issues waiting for review (15) ============================================= #10164: Add an assertBytesEqual to unittest and use it for bytes asser http://bugs.python.org/issue10164 #10161: unittest: use ascii() instead of repr() to display values on e http://bugs.python.org/issue10161 #10160: operator.attrgetter slower than lambda after adding dotted nam http://bugs.python.org/issue10160 #10157: Memory leak (r70152) http://bugs.python.org/issue10157 #10156: Initialization of globals in unicodeobject.c http://bugs.python.org/issue10156 #10155: Add fixups for encoding problems to wsgiref http://bugs.python.org/issue10155 #10153: Memory leak in pythonrun.c http://bugs.python.org/issue10153 #10149: Data truncation in expat parser http://bugs.python.org/issue10149 #10141: SocketCan support http://bugs.python.org/issue10141 #10136: kill_python doesn't work with short path http://bugs.python.org/issue10136 #10134: test_email failures on Windows: end of line issue? http://bugs.python.org/issue10134 #10133: multiprocessing: conn_recv_string() broken error handling http://bugs.python.org/issue10133 #10128: multiprocessing.Pool throws exception with __main__.py http://bugs.python.org/issue10128 #10117: Tools/scripts/reindent.py fails on non-UTF-8 encodings http://bugs.python.org/issue10117 #10115: accept4 can fail with errno 90 http://bugs.python.org/issue10115 Top 10 most discussed issues (10) ================================= #9807: deriving configuration information for different builds with t http://bugs.python.org/issue9807 14 msgs #10126: test_distutils failure with --enable-shared http://bugs.python.org/issue10126 13 msgs #9377: socket, PEP 383: Mishandling of non-ASCII bytes in host/domain http://bugs.python.org/issue9377 11 msgs #7061: Improve 24.5. turtle doc http://bugs.python.org/issue7061 9 msgs #6011: python doesn't build if prefix contains non-ascii characters http://bugs.python.org/issue6011 6 msgs #10027: os.lstat/os.stat don't set st_nlink on Windows http://bugs.python.org/issue10027 6 msgs #10134: test_email failures on Windows: end of line issue? http://bugs.python.org/issue10134 6 msgs #10156: Initialization of globals in unicodeobject.c http://bugs.python.org/issue10156 6 msgs #10160: operator.attrgetter slower than lambda after adding dotted nam http://bugs.python.org/issue10160 6 msgs #9670: Exceed Recursion Limit in Thread http://bugs.python.org/issue9670 5 msgs Issues closed (110) =================== #1343: XMLGenerator: nice elements http://bugs.python.org/issue1343 closed by r.david.murray #1945: Document back ported C functions http://bugs.python.org/issue1945 closed by georg.brandl #3077: h2py char literal doesn't work http://bugs.python.org/issue3077 closed by georg.brandl #3356: some tests fail with 'make EXTRA_CFLAGS="-DPy_DEBUG"' (test_di http://bugs.python.org/issue3356 closed by pitrou #3482: re.split, re.sub and re.subn should support flags http://bugs.python.org/issue3482 closed by r.david.murray #3486: bytes.join does not accept a sequence of bytearrays http://bugs.python.org/issue3486 closed by georg.brandl #3964: quiet the freeze makefile http://bugs.python.org/issue3964 closed by georg.brandl #4086: support %z format in time.strftime and _strptime? http://bugs.python.org/issue4086 closed by georg.brandl #4117: missing update() in _Screen.setup() of Lib/turtle.py http://bugs.python.org/issue4117 closed by belopolsky #4352: imp.find_module() fails with a UnicodeDecodeError when called http://bugs.python.org/issue4352 closed by haypo #4388: test_cmd_line fails on MacOS X http://bugs.python.org/issue4388 closed by haypo #4499: redefinition of TILDE macro on AIX platform http://bugs.python.org/issue4499 closed by r.david.murray #4785: json.JSONDecoder() strict argument undocumented and potentiall http://bugs.python.org/issue4785 closed by georg.brandl #4829: confusing error for file("foo", "w++") http://bugs.python.org/issue4829 closed by georg.brandl #4968: Clarify inspect.is method docs http://bugs.python.org/issue4968 closed by georg.brandl #5117: os.path.relpath problem with root directory http://bugs.python.org/issue5117 closed by ocean-city #5121: PyRun_InteractiveLoop disagrees with documentation? http://bugs.python.org/issue5121 closed by georg.brandl #5212: Incorrect note about md5 in hmac module documentation http://bugs.python.org/issue5212 closed by georg.brandl #5510: patches for Modules/socketmodule.c for NetBSD http://bugs.python.org/issue5510 closed by gregory.p.smith #5762: AttributeError: 'NoneType' object has no attribute 'replace' http://bugs.python.org/issue5762 closed by georg.brandl #5962: Ambiguity about the semantics of sys.exit() and os._exit() in http://bugs.python.org/issue5962 closed by georg.brandl #6014: No shell prompt when a graphics window that was started from I http://bugs.python.org/issue6014 closed by ned.deily #6098: xml.dom.minidom incorrectly claims DOM Level 3 conformance http://bugs.python.org/issue6098 closed by georg.brandl #6364: freeze tool broken in Python 3.x http://bugs.python.org/issue6364 closed by georg.brandl #6460: test failure in test_xmlrpc on Gentoo in trunk http://bugs.python.org/issue6460 closed by r.david.murray #6763: Crash on mac os x leopard in mimetypes.guess_type (or PyObject http://bugs.python.org/issue6763 closed by ronaldoussoren #6798: Argument for sys.settrace() callbacks documented incorrectly http://bugs.python.org/issue6798 closed by georg.brandl #7096: test_curses fails on 3.1 when run under regrtest http://bugs.python.org/issue7096 closed by r.david.murray #7303: pkgutil lacks documentation for useful functions http://bugs.python.org/issue7303 closed by georg.brandl #7450: document that os.chmod accepts an octal digit mode http://bugs.python.org/issue7450 closed by georg.brandl #7473: Compile error when building a 3-way universal framework when a http://bugs.python.org/issue7473 closed by ronaldoussoren #7759: mhlib fails on Btrfs filesystem (test_mhlib failure) http://bugs.python.org/issue7759 closed by nascheme #7790: struct_time documentation entry should point to the table defi http://bugs.python.org/issue7790 closed by georg.brandl #8335: distutils test_build_ext's test_get_outputs fails in bootstrap http://bugs.python.org/issue8335 closed by eric.araujo #8465: re: Backreferences vs. escapes: a silent failure solved http://bugs.python.org/issue8465 closed by r.david.murray #8556: Confusing string formatting examples http://bugs.python.org/issue8556 closed by georg.brandl #8611: Python3 doesn't support locale different than utf8 and an non- http://bugs.python.org/issue8611 closed by haypo #8686: "This isn't defined beyond that" phrase is not friendly to non http://bugs.python.org/issue8686 closed by georg.brandl #8725: Python3: use ASCII for the file system encoding on initfsencod http://bugs.python.org/issue8725 closed by haypo #8811: fixing sqlite3 docs for py3k http://bugs.python.org/issue8811 closed by georg.brandl #8855: Shelve documentation lacks security warning http://bugs.python.org/issue8855 closed by georg.brandl #8968: token type constants are not documented http://bugs.python.org/issue8968 closed by georg.brandl #8999: Add Mercurial support to patchcheck http://bugs.python.org/issue8999 closed by georg.brandl #9054: pyexpat configured with "--with-system-expat" is incompatible http://bugs.python.org/issue9054 closed by georg.brandl #9086: Wrong linking terminology in windows FAQ http://bugs.python.org/issue9086 closed by georg.brandl #9095: patchcheck should handle extraneous whitespace in .rst files http://bugs.python.org/issue9095 closed by georg.brandl #9105: pickle security note should be more prominent http://bugs.python.org/issue9105 closed by georg.brandl #9112: argparse missing documentation for error() method http://bugs.python.org/issue9112 closed by georg.brandl #9117: class syntax not fully documented in reference manual http://bugs.python.org/issue9117 closed by georg.brandl #9138: Tutorial: classes intro paragraph icky http://bugs.python.org/issue9138 closed by georg.brandl #9167: argv double encoding on OSX http://bugs.python.org/issue9167 closed by r.david.murray #9195: Link in docs from "String Formatting Operations" to "Template http://bugs.python.org/issue9195 closed by georg.brandl #9204: The documentation of PyType_Type in py3k mentions types.TypeTy http://bugs.python.org/issue9204 closed by georg.brandl #9237: Add sys.call_tracing to on-line sys module documentation http://bugs.python.org/issue9237 closed by georg.brandl #9289: test_long_key(test_winreg) fails on win2k + py2.x http://bugs.python.org/issue9289 closed by ocean-city #9308: Remove redundant coding cookies from 3.x stdlib http://bugs.python.org/issue9308 closed by belopolsky #9425: Rewrite import machinery to work with unicode paths http://bugs.python.org/issue9425 closed by haypo #9539: python-2.6.4: test failure in test_distutils due to linking to http://bugs.python.org/issue9539 closed by eric.araujo #9683: Dead code in py3k inspect module http://bugs.python.org/issue9683 closed by georg.brandl #9713: Py_CompileString fails on non decode-able paths. http://bugs.python.org/issue9713 closed by haypo #9730: base64 docs refers to strings instead of bytes http://bugs.python.org/issue9730 closed by georg.brandl #9778: Make hash values the same width as a pointer (or Py_ssize_t) http://bugs.python.org/issue9778 closed by benjamin.peterson #9791: nntplib.py lacks a test suite http://bugs.python.org/issue9791 closed by sandro.tosi #9801: Can not use append/extend to lists in a multiprocessing manage http://bugs.python.org/issue9801 closed by georg.brandl #9862: PIPE_BUF is invalid on AIX http://bugs.python.org/issue9862 closed by r.david.murray #9919: gdbinit lineno result is one line in excess http://bugs.python.org/issue9919 closed by georg.brandl #9997: function named 'top' gets unexpected namespace/scope behaviour http://bugs.python.org/issue9997 closed by benjamin.peterson #10024: Outdated advice in C-API tutorial? http://bugs.python.org/issue10024 closed by georg.brandl #10051: distutils fail to install unicode-encoded files with POSIX loc http://bugs.python.org/issue10051 closed by eric.araujo #10058: C-API Intro should be more explicit about error return codes http://bugs.python.org/issue10058 closed by georg.brandl #10064: link to page with documentation bugs http://bugs.python.org/issue10064 closed by georg.brandl #10072: ftplib documentation is unclear http://bugs.python.org/issue10072 closed by georg.brandl #10073: calendar.isleap() not checking parameter type http://bugs.python.org/issue10073 closed by belopolsky #10089: Add support for arbitrary -X options http://bugs.python.org/issue10089 closed by pitrou #10092: calendar does not restore locale properly http://bugs.python.org/issue10092 closed by georg.brandl #10098: intermittent failure in test_os http://bugs.python.org/issue10098 closed by brian.curtin #10109: itertools.product with infinite iterator cause MemoryError. http://bugs.python.org/issue10109 closed by terry.reedy #10111: Minor problems with PyFileObject documentation (Doc/c-api/file http://bugs.python.org/issue10111 closed by georg.brandl #10114: compile() doesn't support the PEP 383 (surrogates) http://bugs.python.org/issue10114 closed by haypo #10119: test_urllibnet failure when using support.transient_internet http://bugs.python.org/issue10119 closed by orsenthil #10120: concurrent.futures module is not installed properly http://bugs.python.org/issue10120 closed by bquinlan #10122: Documentation typo fix and a side question http://bugs.python.org/issue10122 closed by georg.brandl #10123: test_doctest failure with ASCII filesystem encoding http://bugs.python.org/issue10123 closed by haypo #10124: obvious typo in cporting HOWTO http://bugs.python.org/issue10124 closed by georg.brandl #10125: Closing a file within its writelines method causes segfault http://bugs.python.org/issue10125 closed by benjamin.peterson #10127: ssl.SSLSocket().close() does not close the connection http://bugs.python.org/issue10127 closed by pitrou #10129: Curious 'name not defined error' with 'python -m' http://bugs.python.org/issue10129 closed by georg.brandl #10132: mkpkg.py is missing http://bugs.python.org/issue10132 closed by tarek #10135: Format specifier 'n' not working http://bugs.python.org/issue10135 closed by eric.smith #10137: Patch to IDLE for Python 2.7, at Guido's request http://bugs.python.org/issue10137 closed by terry.reedy #10139: regex A|B : both A and B match, but B is wrongly preferred http://bugs.python.org/issue10139 closed by georg.brandl #10140: Tools/scripts/pathfix.py: add option to preserve timestamps http://bugs.python.org/issue10140 closed by orsenthil #10144: Buffering bug after calling curses function http://bugs.python.org/issue10144 closed by ned.deily #10146: Incorrect regex generation in translate_pattern http://bugs.python.org/issue10146 closed by eric.araujo #10147: Python Documentation bugs http://bugs.python.org/issue10147 closed by georg.brandl #10150: Local references not released after exception http://bugs.python.org/issue10150 closed by georg.brandl #10151: Docs: can globals() be updated? http://bugs.python.org/issue10151 closed by georg.brandl #10152: symtable.c: ste_tmpname uninitialized http://bugs.python.org/issue10152 closed by benjamin.peterson #10159: rlcompleter's tests depend on dict order http://bugs.python.org/issue10159 closed by georg.brandl #10162: mimetypes read_windows_registry should tolerate permissions er http://bugs.python.org/issue10162 closed by brian.curtin #10163: 7/200 and 7%200 show weird results http://bugs.python.org/issue10163 closed by georg.brandl #10165: char overwrite mode is by default on in Python console under V http://bugs.python.org/issue10165 closed by brian.curtin #10166: maximum recursion depth exceeded in lib\pstats.py http://bugs.python.org/issue10166 closed by georg.brandl #10168: tkinter.Canvas.coords should return a list, not a map http://bugs.python.org/issue10168 closed by eric.araujo #808129: Change --changelog to accept files http://bugs.python.org/issue808129 closed by eric.araujo #1203650: Allow larger programs to be frozen under Win32 http://bugs.python.org/issue1203650 closed by georg.brandl #459007: Document sys.path on Windows http://bugs.python.org/issue459007 closed by georg.brandl #1528363: forward in turtle module may cause incorrect display http://bugs.python.org/issue1528363 closed by belopolsky #1531775: HTTPSConnection request hangs http://bugs.python.org/issue1531775 closed by orsenthil #1699594: shlex fails to parse strings correctly http://bugs.python.org/issue1699594 closed by georg.brandl From solipsis at pitrou.net Fri Oct 22 18:17:55 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 22 Oct 2010 18:17:55 +0200 Subject: [Python-Dev] SVN rev. 85392 broke module handling in py3k References: <20101022143225.1927.912284794.divmod.xquotient.35@localhost.localdomain> <20101022165552.286af03e@pitrou.net> Message-ID: <20101022181755.70965e25@pitrou.net> On Fri, 22 Oct 2010 17:37:53 +0200 Stefan Behnel wrote: > > I think the general problem is that "the module" can be a pretty broad > thing, potentially referring to all sorts of stuff such as (usually) > several other modules. I wouldn't think of unloading modules as a general problem. We should support it for various reasons (including your use case), but if it takes a further garbage collection pass to get rid of all stale objects, then so be it. > It would also mean that each and every function will end up in a reference > cycle by default that will require garbage collection for cleanup. This is already the case, since most functions are in a reference cycle with their own globals dict (== module dict). It's just that the reference cycle would then include the module object as well, but I don't see it as a big drawback. The module object holds little to no state except its dict, precisely. Regards Antoine. From g.brandl at gmx.net Fri Oct 22 18:26:14 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 22 Oct 2010 18:26:14 +0200 Subject: [Python-Dev] Exposing pkguitl's import emulation (was Re: [Python-checkins] r85538 - python/branches/py3k/Doc/library/pkgutil.rst) In-Reply-To: <20101019152452.354EC3A40DF@sparrow.telecommunity.com> References: <20101019152452.354EC3A40DF@sparrow.telecommunity.com> Message-ID: Am 19.10.2010 17:24, schrieb P.J. Eby: > At 08:03 AM 10/18/2010 +1000, Nick Coghlan wrote: >>I'm a little dubious about exposing these officially. They're mainly a >>hack to get some parts of the standard library working (e.g. runpy) in >>the absence of full PEP 302 support in the imp module, not really >>something we want to encourage anyone else to use (and yes, they >>should probably have underscores in their names, but we missed that >>when the various private implementations scattered around the stdlib >>were consolidated in pkgutil). > > Well, my intention at least was that they should be documented and > released; it's the documenting part I didn't get around to. ;-) > > Of course, this was also pre-importlib; were we starting the work > today, the obvious thing to do would be to expose the Python > implementations of the relevant objects. I don't care much either way; however I don't really like when there are public APIs (i.e. non-underscore-prefixed globals in a non-underscore- prefixed module) that aren't documented, because it is confusing to developers who don't know if they can use it or not. (See re.scanner.) The best thing is probably to let Brett (Hello Brett!) determine how much of it can be replaced by importlib, and add a note to that effect to the pkgutil docs. cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From fuzzyman at voidspace.org.uk Fri Oct 22 18:53:22 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 22 Oct 2010 17:53:22 +0100 Subject: [Python-Dev] Distutils2 scripts In-Reply-To: References: <20101008102636.3ffede1e@mission> <4CAF2F6B.3030903@trueblade.com> <4CB39CC8.9050602@canterbury.ac.nz> <20101012105951.3ed845b5@mission> <4CBF9104.6080400@ronadam.com> <20101021104454.4ba48afe@pitrou.net> <4CC02B97.9000608@trueblade.com> <4CC0D730.5070909@canterbury.ac.nz> <4CC1057F.6070208@ronadam.com> Message-ID: <4CC1C182.1080009@voidspace.org.uk> On 22/10/2010 08:20, Paul Moore wrote: > On 22 October 2010 04:31, Ron Adam wrote: >> When it's in the stdlib, the -m option should work just like any other >> script run from the stdlib. >> >> What path hacking are you thinking of? > On Windows, neither the "python" executable nor scripts in > C:\Pythonxx\Scripts are in the PATH by default. On the other hand, .py > files will run automatically via the registered file extension. > > Manipulating PATH at install time (to add C:\PythonXX and/or > C:\PythonXX\Scripts) is not done - it is (rightly, in my view) > considered too difficult to get right, particularly when it comes to > uninstalling. > > Many Windows users will, I guess, manually add python to PATH (so that > python-m works). Some people also add C:\PythonXX\Scripts. Personally, > I don't - so for me a pysetup script in that location would be no use. > Well, that is where pip and other scripts installed by Python go, so it is the 'right' place for scripts to live. Any reason not to allow both though? (python -m and an explicit script) For what its worth I have the same issue with unittest / unittest2. Test discovery and test running in Python 2.7 / 3.2 is done with: python -m unittest args As unittest2 is a package and supports Python 2.6 (and earlier), python -m unittest2 doesn't work so I provide a "unit2" script for accessing its functionality. I *much* prefer using "unit2 ..." to "python -m unittest ...". Michael > So my personal vote is +1 for a python -m approach, and -0 for a > pysetup executable. I'm -1 on a pysetup.bat batch file - bat files > have other issues which IMO make them effectively useless. > > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From solipsis at pitrou.net Fri Oct 22 19:04:51 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 22 Oct 2010 19:04:51 +0200 Subject: [Python-Dev] Summary of Python tracker Issues References: <20101022160820.C7498782F2@psf.upfronthosting.co.za> Message-ID: <20101022190451.0aeb61b8@pitrou.net> On Fri, 22 Oct 2010 18:08:20 +0200 (CEST) Python tracker wrote: > > ACTIVITY SUMMARY (2010-10-15 - 2010-10-22) > Python tracker at http://bugs.python.org/ > > To view or respond to any of the issues listed below, click on the issue. > Do NOT respond to this message. > > Issues stats: > open 2494 (+32) > closed 19460 (+110) > total 21954 (+56) The figures in parentheses look wrong. Last week, the stats said: > Issues stats: > open 2557 (+37) > closed 19276 (+30) > total 21833 (+44) So, logically, this week's stats should say: open 2494 (-63) closed 19460 (+184) total 21954 (+121) Regards Antoine. From brett at python.org Fri Oct 22 21:11:03 2010 From: brett at python.org (Brett Cannon) Date: Fri, 22 Oct 2010 12:11:03 -0700 Subject: [Python-Dev] Exposing pkguitl's import emulation (was Re: [Python-checkins] r85538 - python/branches/py3k/Doc/library/pkgutil.rst) In-Reply-To: References: <20101019152452.354EC3A40DF@sparrow.telecommunity.com> Message-ID: On Fri, Oct 22, 2010 at 09:26, Georg Brandl wrote: > Am 19.10.2010 17:24, schrieb P.J. Eby: >> At 08:03 AM 10/18/2010 +1000, Nick Coghlan wrote: >>>I'm a little dubious about exposing these officially. They're mainly a >>>hack to get some parts of the standard library working (e.g. runpy) in >>>the absence of full PEP 302 support in the imp module, not really >>>something we want to encourage anyone else to use (and yes, they >>>should probably have underscores in their names, but we missed that >>>when the various private implementations scattered around the stdlib >>>were consolidated in pkgutil). >> >> Well, my intention at least was that they should be documented and >> released; it's the documenting part I didn't get around to. ?;-) >> >> Of course, this was also pre-importlib; were we starting the work >> today, the obvious thing to do would be to expose the Python >> implementations of the relevant objects. > > I don't care much either way; however I don't really like when there are > public APIs (i.e. non-underscore-prefixed globals in a non-underscore- > prefixed module) that aren't documented, because it is confusing to > developers who don't know if they can use it or not. ?(See re.scanner.) > > The best thing is probably to let Brett (Hello Brett!) determine how > much of it can be replaced by importlib, and add a note to that effect > to the pkgutil docs. The pkgutil stuff that was exposed cannot be directly replaced with a public API in Python 3.2, but the plan is that it will be in Python 3.3 when *all* concrete implementations of importers are exposed (because I will be attempting to bootstrap importlib). So if people are willing to wait and take me at my word that this will happen in Python 3.3, then this can come back out. But obviously I cannot make promises as Real Life will *actually* be starting for me when the Python 3.3 development cycle begins. From stephen at xemacs.org Sat Oct 23 06:01:14 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 23 Oct 2010 13:01:14 +0900 Subject: [Python-Dev] My work on Python3 and non-ascii paths is done In-Reply-To: <201010221401.44549.victor.stinner@haypocalc.com> References: <201010190353.34796.victor.stinner@haypocalc.com> <20101021120040.66b613a0@mission> <20101021191455.GQ3652@unaka.lan> <201010221401.44549.victor.stinner@haypocalc.com> Message-ID: <87zku5k16d.fsf@uwakimon.sk.tsukuba.ac.jp> First, let me offer congratulations and heartfelt thanks for your hard work! Victor Stinner writes: > For network protocols, I don't know. It looks like the new email > modules will offer two API levels: low level (native type) using > bytes, high level using str (unicode). I don't know if the high > level API uses the PEP 383 or not. Give that about 0.3% of the mail in my system uses the UNKNOWN encoding in MIME-words, "best effort" to decode headers will require it. (Half-decoded text is a nightmare, of course.) > I don't use strict rules. Each problem is different. I agree. The reason is that the application must be designed to handle PEP 383 strings. If it isn't, you're just postponing the exception (perhaps to an unsuspecting second application). PEP 383 strings are an excellent, consistent way to handle broken text if your application is like email, where the design specifies passing around uninterpreted data, but transforming the representation is desirable. But many applications *should* just raise an exception here. From solipsis at pitrou.net Sat Oct 23 10:54:23 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 23 Oct 2010 10:54:23 +0200 Subject: [Python-Dev] Multiprocessing maintenance Message-ID: <20101023105423.37b264ff@pitrou.net> Hello everybody, Who is doing multiprocessing maintenance these days? I thought Ask Solem had been given commit privs for that, but I haven't seen any activity from him; and Jesse is, mostly, absent. Is anyone working on the multiprocessing issues? (no, I'm not planning to address them :-)) cheers Antoine. From ncoghlan at gmail.com Sat Oct 23 16:28:49 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 24 Oct 2010 00:28:49 +1000 Subject: [Python-Dev] [Python-checkins] Exposing pkguitl's import emulation (was Re: r85538 - python/branches/py3k/Doc/library/pkgutil.rst) In-Reply-To: References: <20101019152452.354EC3A40DF@sparrow.telecommunity.com> Message-ID: On Sat, Oct 23, 2010 at 5:11 AM, Brett Cannon wrote: > On Fri, Oct 22, 2010 at 09:26, Georg Brandl wrote: >> Am 19.10.2010 17:24, schrieb P.J. Eby: >>> Well, my intention at least was that they should be documented and >>> released; it's the documenting part I didn't get around to. ?;-) >>> >>> Of course, this was also pre-importlib; were we starting the work >>> today, the obvious thing to do would be to expose the Python >>> implementations of the relevant objects. >> >> I don't care much either way; however I don't really like when there are >> public APIs (i.e. non-underscore-prefixed globals in a non-underscore- >> prefixed module) that aren't documented, because it is confusing to >> developers who don't know if they can use it or not. ?(See re.scanner.) >> >> The best thing is probably to let Brett (Hello Brett!) determine how >> much of it can be replaced by importlib, and add a note to that effect >> to the pkgutil docs. > > The pkgutil stuff that was exposed cannot be directly replaced with a > public API in Python 3.2, but the plan is that it will be in Python > 3.3 when *all* concrete implementations of importers are exposed > (because I will be attempting to bootstrap importlib). So if people > are willing to wait and take me at my word that this will happen in > Python 3.3, then this can come back out. But obviously I cannot make > promises as Real Life will *actually* be starting for me when the > Python 3.3 development cycle begins. Given the water under this bridge (and the fact PJE actually did intend for these to be public interfaces), I'm happy enough with the idea of having these pkgutil features documented properly. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From solipsis at pitrou.net Sat Oct 23 18:47:21 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 23 Oct 2010 18:47:21 +0200 Subject: [Python-Dev] Windows 64 build slave References: <4CB61CB9.1060702@v.loewis.de> <20101013234710.2ea1d91b@pitrou.net> <4CB62DD8.8080307@ixokai.io> <4CB62F3D.50801@v.loewis.de> <4CB631DC.4020102@ixokai.io> <4CB635CF.7060801@v.loewis.de> Message-ID: <20101023184721.70081694@pitrou.net> On Wed, 13 Oct 2010 17:50:49 -0500 Brian Curtin wrote: [...] > > I have a Server 2008 R2 x64 box with the full Visual Studio that I could add > to the buildbot fleet. It's a dual core with 4 GB of RAM, plenty of disk > space, and it runs 24/7. The adventures of http://bugs.python.org/issue9778 show we would really benefit from a 64-bit Windows build slave. Regards Antoine. From solipsis at pitrou.net Sat Oct 23 19:08:28 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 23 Oct 2010 19:08:28 +0200 Subject: [Python-Dev] Bug week-end on the 20th-21st? Message-ID: <20101023190828.47b7f03e@pitrou.net> Hello, The first 3.2 beta is scheduled by Georg for November 13th. What would you think of scheduling a bug week-end one week later, that is on November 20th and 21st? We would need enough core developers to be available on #python-dev. Regards Antoine. From g.brandl at gmx.net Sat Oct 23 19:50:44 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 23 Oct 2010 19:50:44 +0200 Subject: [Python-Dev] Bug week-end on the 20th-21st? In-Reply-To: <20101023190828.47b7f03e@pitrou.net> References: <20101023190828.47b7f03e@pitrou.net> Message-ID: Am 23.10.2010 19:08, schrieb Antoine Pitrou: > > Hello, > > The first 3.2 beta is scheduled by Georg for November 13th. > What would you think of scheduling a bug week-end one week later, that > is on November 20th and 21st? We would need enough core developers to > be available on #python-dev. I'll be there. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From martin at v.loewis.de Sat Oct 23 20:03:27 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 23 Oct 2010 20:03:27 +0200 Subject: [Python-Dev] 3.1.3 and 2.7.1 release schedule In-Reply-To: References: Message-ID: <4CC3236F.9040401@v.loewis.de> Am 22.10.2010 16:09, schrieb Benjamin Peterson: > 2010/10/22 Dirkjan Ochtman : >> On Fri, Oct 22, 2010 at 00:57, Benjamin Peterson wrote: >>> In the interest of getting 3.1.3 and 2.7.1 out by next year, here's a >>> tentative release schedule: >>> >>> November 13th - RC1 >>> November 27th - RC2 >>> December 11th - Final >> >> The last one might clash with the hg migration a bit, do we need to >> worry about that? Or did you purposely pick the day before the planned >> hg migration? > > I'm not too worried. Commits should be at a minimum, and changesets > can be tagged post-transition if needed. I'm worried about build identification. Either the switchover happens before RC1, or after Final. I expect significant breakage from the Mercurial switchover, so that should all be figured out before or after the release. FWIW, I'm pondering to do all remaining 2.5 release from svn, despite the switchover to Mercurial, just so that the build identification does not get harmed. As a side note - I don't think two release candidates are really necessary. So if it helps, it may be reasonable to drop one of them. OTOH, I don't mind having two RCs, either. Regards, Martin From martin at v.loewis.de Sat Oct 23 20:07:16 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 23 Oct 2010 20:07:16 +0200 Subject: [Python-Dev] Summary of Python tracker Issues In-Reply-To: <20101022190451.0aeb61b8@pitrou.net> References: <20101022160820.C7498782F2@psf.upfronthosting.co.za> <20101022190451.0aeb61b8@pitrou.net> Message-ID: <4CC32454.9010706@v.loewis.de> >> Issues stats: >> open 2494 (+32) >> closed 19460 (+110) >> total 21954 (+56) > > The figures in parentheses look wrong. Last week, the stats said: No: they just mean something different that you think. +32 doesn't mean that there are now 32 more open issues than last week, but that 32 issue have been opened-and-not-closed in the week. You can see these 32 issues further down in the message. Regards, Martin From martin at v.loewis.de Sat Oct 23 20:10:43 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 23 Oct 2010 20:10:43 +0200 Subject: [Python-Dev] Multiprocessing maintenance In-Reply-To: <20101023105423.37b264ff@pitrou.net> References: <20101023105423.37b264ff@pitrou.net> Message-ID: <4CC32523.4030408@v.loewis.de> > Who is doing multiprocessing maintenance these days? I thought Ask > Solem had been given commit privs for that, but I haven't seen any > activity from him; and Jesse is, mostly, absent. Is anyone working on > the multiprocessing issues? > > (no, I'm not planning to address them :-)) You mean: actively feeling responsible for it? I guess nobody - as for many other modules in the standard library. Or do you mean: who is willing to work on it, in principle? The last committers are georg.brandl, gregory.p.smith, martin.v.loewis, and antoine.pitrou. Regards, Martin From solipsis at pitrou.net Sat Oct 23 20:20:41 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 23 Oct 2010 20:20:41 +0200 Subject: [Python-Dev] Multiprocessing maintenance In-Reply-To: <4CC32523.4030408@v.loewis.de> References: <20101023105423.37b264ff@pitrou.net> <4CC32523.4030408@v.loewis.de> Message-ID: <1287858041.3611.7.camel@localhost.localdomain> > You mean: actively feeling responsible for it? I guess nobody - as for > many other modules in the standard library. > > Or do you mean: who is willing to work on it, in principle? Both. Originally the module is/was meant to be officially maintained by Jesse, as far as I understand. But bugs filed against multiprocessing have been lingering in the tracker for quite a long time. Regards Antoine. From benjamin at python.org Sat Oct 23 20:24:26 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 23 Oct 2010 13:24:26 -0500 Subject: [Python-Dev] 3.1.3 and 2.7.1 release schedule In-Reply-To: <4CC3236F.9040401@v.loewis.de> References: <4CC3236F.9040401@v.loewis.de> Message-ID: 2010/10/23 "Martin v. L?wis" : > I'm worried about build identification. Either the switchover happens > before RC1, or after Final. I expect significant breakage from the > Mercurial switchover, so that should all be figured out before or after > the release. I hope that can be well tested before the release. > > FWIW, I'm pondering to do all remaining 2.5 release from svn, despite > the switchover to Mercurial, just so that the build identification > does not get harmed. > > As a side note - I don't think two release candidates are really > necessary. So if it helps, it may be reasonable to drop one of them. > OTOH, I don't mind having two RCs, either. I'm okay with dropping one RC, too. This should give us some wiggle room. -- Regards, Benjamin From martin at v.loewis.de Sat Oct 23 20:43:43 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 23 Oct 2010 20:43:43 +0200 Subject: [Python-Dev] Multiprocessing maintenance In-Reply-To: <1287858041.3611.7.camel@localhost.localdomain> References: <20101023105423.37b264ff@pitrou.net> <4CC32523.4030408@v.loewis.de> <1287858041.3611.7.camel@localhost.localdomain> Message-ID: <4CC32CDF.6060602@v.loewis.de> > Both. Originally the module is/was meant to be officially maintained by > Jesse, as far as I understand. But bugs filed against multiprocessing > have been lingering in the tracker for quite a long time. I personally think we should treat these reports in the same way as all other lingering issues: explain to people that they need to submit patches if they want these issues fixed. Of course, this is then the same situation as with all the other patches that stay unreviewed (i.e. some get committed soon, some after some time, some never). Regards, Martin From barry at python.org Sat Oct 23 20:56:09 2010 From: barry at python.org (Barry Warsaw) Date: Sat, 23 Oct 2010 14:56:09 -0400 Subject: [Python-Dev] 3.1.3 and 2.7.1 release schedule In-Reply-To: <4CC3236F.9040401@v.loewis.de> References: <4CC3236F.9040401@v.loewis.de> Message-ID: <08A156A2-B096-40F6-A39D-D99E4C39C419@python.org> I will also do any future 2.6 release from svn. It does mean that patches for those release need to make it into svn. I propose that only the RM have commit to the svn branches after the switch. Sent from my digital lollipop. On Oct 23, 2010, at 2:03 PM, "Martin v. L?wis" wrote: > Am 22.10.2010 16:09, schrieb Benjamin Peterson: >> 2010/10/22 Dirkjan Ochtman : >>> On Fri, Oct 22, 2010 at 00:57, Benjamin Peterson wrote: >>>> In the interest of getting 3.1.3 and 2.7.1 out by next year, here's a >>>> tentative release schedule: >>>> >>>> November 13th - RC1 >>>> November 27th - RC2 >>>> December 11th - Final >>> >>> The last one might clash with the hg migration a bit, do we need to >>> worry about that? Or did you purposely pick the day before the planned >>> hg migration? >> >> I'm not too worried. Commits should be at a minimum, and changesets >> can be tagged post-transition if needed. > > I'm worried about build identification. Either the switchover happens > before RC1, or after Final. I expect significant breakage from the > Mercurial switchover, so that should all be figured out before or after > the release. > > FWIW, I'm pondering to do all remaining 2.5 release from svn, despite > the switchover to Mercurial, just so that the build identification > does not get harmed. > > As a side note - I don't think two release candidates are really > necessary. So if it helps, it may be reasonable to drop one of them. > OTOH, I don't mind having two RCs, either. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/barry%40python.org From martin at v.loewis.de Sat Oct 23 21:12:12 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 23 Oct 2010 21:12:12 +0200 Subject: [Python-Dev] 3.1.3 and 2.7.1 release schedule In-Reply-To: <08A156A2-B096-40F6-A39D-D99E4C39C419@python.org> References: <4CC3236F.9040401@v.loewis.de> <08A156A2-B096-40F6-A39D-D99E4C39C419@python.org> Message-ID: <4CC3338C.4070503@v.loewis.de> Am 23.10.2010 20:56, schrieb Barry Warsaw: > I will also do any future 2.6 release from svn. It does mean that > patches for those release need to make it into svn. I propose that > only the RM have commit to the svn branches after the switch. This is also my thinking. I would like to see a Mercurial branch for 2.5 established, and will then move any commits to it myself to the svn branch (there shouldn't be that many). Regards, Martin From maker.py at gmail.com Sun Oct 24 07:30:09 2010 From: maker.py at gmail.com (=?UTF-8?Q?Michele_Orr=C3=B9?=) Date: Sun, 24 Oct 2010 07:30:09 +0200 Subject: [Python-Dev] Goole Code-In Contest Message-ID: Hello. I'm a student under 18 years, who really like programming in python. Few days ago I've found Google Code-In contest, and I'm seriously considering it as a good opportunity do get more confident with Python. Although I've fixed a pair of bugs (see issue1100562 and issue9496), I'm still scared of breaking something and making silly things. So two days ago I've asked on #python-dev some infos about this contest, but nobody seemed to know if Python would have been a project in GCI, or give me any new information. Thanks to bitdancencer, I'm writing this mail, hoping some of you may consider this contest and give me the opportinity to be a better python developer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From orsenthil at gmail.com Sun Oct 24 08:31:02 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Sun, 24 Oct 2010 12:01:02 +0530 Subject: [Python-Dev] Goole Code-In Contest In-Reply-To: References: Message-ID: On Sun, Oct 24, 2010 at 11:00 AM, Michele Orr? wrote: > I'm a student under 18 years, who really like programming in python. Few > days ago I've found Google Code-In contest, and I'm seriously considering it > as a good opportunity do get more confident with Python. Although I've fixed > a pair of bugs (see?issue1100562 and issue9496), I'm still scared of > breaking something and making silly things. As far as I see, the Google Code In has just requested the organizations like PSF to register. After that there will be a call for proposal and if you meet the criteria, you should apply for the Google Code In program. Both your patches have been good are committed too, so it is a good sign. > any new information. Thanks to bitdancencer, I'm writing this mail, hoping You surely meant, bitdancer, not bitdancencer :) -- Senthil From askh at opera.com Sun Oct 24 09:33:01 2010 From: askh at opera.com (Ask Solem) Date: Sun, 24 Oct 2010 09:33:01 +0200 Subject: [Python-Dev] Multiprocessing maintenance In-Reply-To: <4CC32523.4030408@v.loewis.de> References: <20101023105423.37b264ff@pitrou.net> <4CC32523.4030408@v.loewis.de> Message-ID: On Oct 23, 2010, at 8:10 PM, Martin v. L?wis wrote: >> Who is doing multiprocessing maintenance these days? I thought Ask >> Solem had been given commit privs for that, but I haven't seen any >> activity from him; and Jesse is, mostly, absent. Is anyone working on >> the multiprocessing issues? >> >> (no, I'm not planning to address them :-)) > > You mean: actively feeling responsible for it? I guess nobody - as for > many other modules in the standard library. > > Or do you mean: who is willing to work on it, in principle? > The last committers are georg.brandl, gregory.p.smith, > martin.v.loewis, and antoine.pitrou. > > Regards, > Martin Hey! I do feel responsible, and I know Jesse do to. I'm working on multiprocessing issues every week, though not as much as I would like to. I have quite a few patches ready, some in the tracker, some not. My idea was to send those to Jesse last week for approval, but for work related reasons I didn't manage to prepare them in time. If you have anything you want me to look at please forward it, and if you already did then don't be afraid to nag me. -- {Ask Solem, +47 98435213 | twitter.com/asksol }. From martin at v.loewis.de Sun Oct 24 10:23:26 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 24 Oct 2010 10:23:26 +0200 Subject: [Python-Dev] Multiprocessing maintenance In-Reply-To: References: <20101023105423.37b264ff@pitrou.net> <4CC32523.4030408@v.loewis.de> Message-ID: <4CC3ECFE.8010205@v.loewis.de> > If you have anything you want me to look at please forward it, > and if you already did then don't be afraid to nag me. Please keep the release schedule in mind. After 3.2b1, no new features can be accepted (until after the 3.2 release). So there might be some urgency for some of the patches. Also, submitting many small changes is preferred over patch bombs. So it might be better to start committing what is complete, rather than first completing all work in progress. Regards, Martin From whitequill.bj at gmail.com Sun Oct 24 18:46:53 2010 From: whitequill.bj at gmail.com (Bj Raz) Date: Sun, 24 Oct 2010 12:46:53 -0400 Subject: [Python-Dev] __setcall__ Message-ID: I was looking for a way to set a function being equal to another function: q(m+1) = q(m)+ ((-1)^m) * exp(lnanswer); I was hoping to use something like this: (q).__setcall__.(m+1) = (q).__setcall__.(m)+ ((-1)^m) * exp(lnanswer); As a work around. But I have not found the `__setcall__' built in being there. Here is the code block I'm working with: indvar = 200; q = 0; lnanswer = 0; for m = 1:150 lnanswer = (3 * m) * log(indvar) - log(factorial(3 * m)) ; q(m+1) = q(m)+ ((-1)^m) * exp(lnanswer); end lnanswer q Any help would be appreciated. Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Sun Oct 24 18:51:55 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 24 Oct 2010 11:51:55 -0500 Subject: [Python-Dev] __setcall__ In-Reply-To: References: Message-ID: 2010/10/24 Bj Raz : > I was looking for a way to set a function being equal to another function: > q(m+1) = q(m)+ ((-1)^m) * exp(lnanswer); > I was hoping to use something like this: > (q).__setcall__.(m+1) = (q).__setcall__.(m)+ ((-1)^m) * exp(lnanswer); > As a work around. > But I have not found the `__setcall__' built in being there. > > Here is the code block I'm working with: > > indvar = 200; > q = 0; > lnanswer = 0; > for m = 1:150 > ? lnanswer = (3 * m) * log(indvar) - log(factorial(3 * m))? ; > q(m+1) = q(m)+ ((-1)^m) * exp(lnanswer); > end > lnanswer > q > > Any help would be appreciated. Please see python-list. This list is for the development of python. The SAGE math package can do something like this. -- Regards, Benjamin From martin at v.loewis.de Sun Oct 24 21:14:34 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 24 Oct 2010 21:14:34 +0200 Subject: [Python-Dev] [Python-checkins] r85822 - python/branches/py3k/Modules/ossaudiodev.c In-Reply-To: <20101024142142.5162FEE985@mail.python.org> References: <20101024142142.5162FEE985@mail.python.org> Message-ID: <4CC4859A.4090208@v.loewis.de> > Add casts (one needed, one for consistency). FWIW, I feel that these casts are misguided. Having function pointer casts means that type errors in the signature get suppressed. E.g. people using such casts for tp_hash will be unable to detect the change in the tp_hash return type that we have implemented for 3.2. It would be better, IMO, if these casts get avoided everywhere, even if that means that the functions get a line longer. Regards, Martin From jnoller at gmail.com Mon Oct 25 16:00:39 2010 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 25 Oct 2010 10:00:39 -0400 Subject: [Python-Dev] Multiprocessing maintenance In-Reply-To: <1287858041.3611.7.camel@localhost.localdomain> References: <20101023105423.37b264ff@pitrou.net> <4CC32523.4030408@v.loewis.de> <1287858041.3611.7.camel@localhost.localdomain> Message-ID: On Sat, Oct 23, 2010 at 2:20 PM, Antoine Pitrou wrote: > >> You mean: actively feeling responsible for it? I guess nobody - as for >> many other modules in the standard library. >> >> Or do you mean: who is willing to work on it, in principle? > > Both. Originally the module is/was meant to be officially maintained by > Jesse, as far as I understand. But bugs filed against multiprocessing > have been lingering in the tracker for quite a long time. > > Regards > > Antoine. Like Ask said, we both are, and still feel responsible. Finding time is another problem altogether, and it's been in short supply. In short: No, neither one of us has walked away. jesse From jnoller at gmail.com Mon Oct 25 16:01:43 2010 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 25 Oct 2010 10:01:43 -0400 Subject: [Python-Dev] Multiprocessing maintenance In-Reply-To: <4CC32523.4030408@v.loewis.de> References: <20101023105423.37b264ff@pitrou.net> <4CC32523.4030408@v.loewis.de> Message-ID: On Sat, Oct 23, 2010 at 2:10 PM, "Martin v. L?wis" wrote: >> Who is doing multiprocessing maintenance these days? I thought Ask >> Solem had been given commit privs for that, but I haven't seen any >> activity from him; and Jesse is, mostly, absent. Is anyone working on >> the multiprocessing issues? >> >> (no, I'm not planning to address them :-)) > > You mean: actively feeling responsible for it? I guess nobody - as for > many other modules in the standard library. > > Or do you mean: who is willing to work on it, in principle? > The last committers are georg.brandl, gregory.p.smith, > martin.v.loewis, and antoine.pitrou. Both Ask and myself feel actively responsible. Antoine could have just as easily asked privately. From rbp at isnomore.net Mon Oct 25 16:22:24 2010 From: rbp at isnomore.net (Rodrigo Bernardo Pimentel) Date: Mon, 25 Oct 2010 12:22:24 -0200 Subject: [Python-Dev] Bug week-end on the 20th-21st? In-Reply-To: References: <20101023190828.47b7f03e@pitrou.net> Message-ID: > Am 23.10.2010 19:08, schrieb Antoine Pitrou: >> >> Hello, >> >> The first 3.2 beta is scheduled by Georg for November 13th. >> What would you think of scheduling a bug week-end one week later, that >> is on November 20th and 21st? We would need enough core developers to >> be available on #python-dev. FWIW, I'm +1, and I'll try to get the Sao Paulo users group to participate. ? ? rbp -- ?http://isnomore.net From vinay_sajip at yahoo.co.uk Mon Oct 25 16:28:49 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 25 Oct 2010 14:28:49 +0000 (UTC) Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles Message-ID: I've just checked in a change to logging into the py3k branch (r85835), including doc changes and tests, for providing slightly more flexibility in alternative format styles for logging. Basically, Formatter.__init__ gets an extra optional keyword arg style=. This is then used to merge the format string with the LogRecord: either fmt % record.__dict__, or fmt.format(**record.dict), or string.Template(fmt).substitute(**record.dict). Backward compatibility is maintained (unless I've missed something). This does not cater for how you combine the logging message + its args when you make a logging call, but should work with any existing library which uses %-formatting in its logging calls. We discussed this here around a year ago in the context of generally encouraging a move from %-formatting to {}-formatting, but there seemed to be no obvious easy path forward. As of now, and even without this change I've just checked in, developers can use {}- or $-formatting in their logging messages using e.g. logger.debug(__('Message with {} {}', 2, 'place-holders')) with a suitable class standing in for __. A bit ugly, I know, with extra parentheses and what not, but hopefully for those who must have {} now it shouldn't be too much of a price to pay and there's no major performance implication, as actual rendering to string is done as late as possible, just as it is now. Comments welcome. Assuming there are no strong objections asking for reversion of this change, I'll publicise to the wider community in a few days. Regards, Vinay Sajip From jnoller at gmail.com Mon Oct 25 16:30:31 2010 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 25 Oct 2010 10:30:31 -0400 Subject: [Python-Dev] Bug week-end on the 20th-21st? In-Reply-To: <20101023190828.47b7f03e@pitrou.net> References: <20101023190828.47b7f03e@pitrou.net> Message-ID: On Sat, Oct 23, 2010 at 1:08 PM, Antoine Pitrou wrote: > > Hello, > > The first 3.2 beta is scheduled by Georg for November 13th. > What would you think of scheduling a bug week-end one week later, that > is on November 20th and 21st? We would need enough core developers to > be available on #python-dev. > > Regards > > Antoine. If anyone wants to spin up local, in person presences for this - namely, small local sprints, let me / the sprints team know. We (The PSF) have money to help fund. This would be the perfect use of the resources - more info http://pythonsprints.com/ jesse From rdmurray at bitdance.com Mon Oct 25 17:32:42 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 25 Oct 2010 11:32:42 -0400 Subject: [Python-Dev] Bug week-end on the 20th-21st? In-Reply-To: References: <20101023190828.47b7f03e@pitrou.net> Message-ID: <20101025153242.2FBEC219F92@kimball.webabinitio.net> On Mon, 25 Oct 2010 12:22:24 -0200, Rodrigo Bernardo Pimentel wrote: >> Am 23.10.2010 19:08, schrieb Antoine Pitrou: >>> The first 3.2 beta is scheduled by Georg for November 13th. >>> What would you think of scheduling a bug week-end one week later, that >>> is on November 20th and 21st? We would need enough core developers to >>> be available on #python-dev. > >FWIW, I'm +1, and I'll try to get the Sao Paulo users group to participate. I think this is a great idea (both Antoine's initial suggestion and the idea of getting users groups to participate). I'll be around and able to participate that weekend except for evening US Eastern time. -- R. David Murray www.bitdance.com From solipsis at pitrou.net Mon Oct 25 19:47:16 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 25 Oct 2010 19:47:16 +0200 Subject: [Python-Dev] r85839 - python/branches/py3k/Python/sysmodule.c References: <20101025173723.BD3F3EE9A4@mail.python.org> Message-ID: <20101025194716.3c06a517@pitrou.net> On Mon, 25 Oct 2010 19:37:23 +0200 (CEST) victor.stinner wrote: > > - if (argc == 0) > - return; > argv0 = argv[0]; Well, are you sure argv[0] is valid when argc is 0? Regards Antoine. From solipsis at pitrou.net Mon Oct 25 22:04:01 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 25 Oct 2010 22:04:01 +0200 Subject: [Python-Dev] Bug week-end on the 20th-21st? References: <20101023190828.47b7f03e@pitrou.net> <20101025153242.2FBEC219F92@kimball.webabinitio.net> Message-ID: <20101025220401.0406722b@pitrou.net> On Mon, 25 Oct 2010 11:32:42 -0400 "R. David Murray" wrote: > On Mon, 25 Oct 2010 12:22:24 -0200, Rodrigo Bernardo Pimentel wrote: > >> Am 23.10.2010 19:08, schrieb Antoine Pitrou: > >>> The first 3.2 beta is scheduled by Georg for November 13th. > >>> What would you think of scheduling a bug week-end one week later, that > >>> is on November 20th and 21st? We would need enough core developers to > >>> be available on #python-dev. > > > >FWIW, I'm +1, and I'll try to get the Sao Paulo users group to participate. > > I think this is a great idea (both Antoine's initial suggestion and the > idea of getting users groups to participate). > > I'll be around and able to participate that weekend except for evening > US Eastern time. Ok, so 20th-21st of November it shall be! Regards Antoine. From solipsis at pitrou.net Mon Oct 25 23:03:37 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 25 Oct 2010 23:03:37 +0200 Subject: [Python-Dev] Python bug week-end : 20-21 November Message-ID: <20101025230337.41aeef12@pitrou.net> Hello, The development team of the Python interpreter (a.k.a python-dev) is organizing a bug week-end on Saturday 20th and Sunday 21st of November. We would like to encourage anyone who feels interested in participating to give it a try. Contributing to Python is much less intimidating than it sounds. You don't need to have previous experience with modifying the Python source; in fact bug days offer a good opportunity to learn the basics by asking questions and working on relatively simple bugs (see "how to get prepared" below). And most core developers are actual human beings! How it happens -------------- The bug week-end will happen on the #python-dev IRC channel on the Freenode network, where several core developers routinely hang out. No physical meeting is scheduled as far as I know, but anyone is encouraged to organize one and announce it on the official Python channels such as this one. Participants (you!) join #python-dev and collaboratively go through the Python issue tracker at http://bugs.python.org . From there, you can provide patches and/or review existing patches. Also, you can help us assess issues on any specific topic you have expertise in (the range of topics touched in the stdlib is quite broad and it is more than likely that the core developers' expertise is lacking in some of them). Or, if you feel shy, you can simply watch other people work and slowly get more confident about participating yourself. Development is public and lurkers are welcome. What you can work on --------------------- Our expectation is that Python 3.2 beta 1 will have been released a couple of days before the bug week-end and, therefore, one primary goal is to polish the 3.2 branch for the following betas and the final release. There are many issues to choose from on the bug tracker; any bug fixes or documentation improvements will do. New features are discouraged: they can't be checked in before the official 3.2 release. How to get prepared ------------------- If you are a beginner with the Python codebase, you may want to read the development guide available here (courtesy of Brian Curtin): http://docs.pythonsprints.com/core_development/beginners.html There's a small practical guide to bug days/week-ends on the wiki: http://wiki.python.org/moin/PythonBugDay And the development FAQ holds answers to generic development questions: http://www.python.org/dev/faq/ You can also do all of the above during the bug week-end, of course. Please, don't hesitate to ask us questions on the #python-dev channel. Regards Antoine. From solipsis at pitrou.net Tue Oct 26 01:19:23 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 26 Oct 2010 01:19:23 +0200 Subject: [Python-Dev] Multiprocessing maintenance In-Reply-To: References: <20101023105423.37b264ff@pitrou.net> <4CC32523.4030408@v.loewis.de> Message-ID: <20101026011923.02c9383d@pitrou.net> On Mon, 25 Oct 2010 10:01:43 -0400 Jesse Noller wrote: > On Sat, Oct 23, 2010 at 2:10 PM, "Martin v. L?wis" wrote: > >> Who is doing multiprocessing maintenance these days? I thought Ask > >> Solem had been given commit privs for that, but I haven't seen any > >> activity from him; and Jesse is, mostly, absent. Is anyone working on > >> the multiprocessing issues? > >> > >> (no, I'm not planning to address them :-)) > > > > You mean: actively feeling responsible for it? I guess nobody - as for > > many other modules in the standard library. > > > > Or do you mean: who is willing to work on it, in principle? > > The last committers are georg.brandl, gregory.p.smith, > > martin.v.loewis, and antoine.pitrou. > > Both Ask and myself feel actively responsible. Antoine could have just > as easily asked privately. Well, the question and its eventual answer were meant for public consumption, not for my personal edification. Thanks for answering anyway. We can then continue to put you and Ask in the nosy lists for multiprocessing issues. Regards Antoine. From jnoller at gmail.com Tue Oct 26 01:24:00 2010 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 25 Oct 2010 19:24:00 -0400 Subject: [Python-Dev] Multiprocessing maintenance In-Reply-To: <20101026011923.02c9383d@pitrou.net> References: <20101023105423.37b264ff@pitrou.net> <4CC32523.4030408@v.loewis.de> <20101026011923.02c9383d@pitrou.net> Message-ID: On Mon, Oct 25, 2010 at 7:19 PM, Antoine Pitrou wrote: > On Mon, 25 Oct 2010 10:01:43 -0400 > Jesse Noller wrote: >> On Sat, Oct 23, 2010 at 2:10 PM, "Martin v. L?wis" wrote: >> >> Who is doing multiprocessing maintenance these days? I thought Ask >> >> Solem had been given commit privs for that, but I haven't seen any >> >> activity from him; and Jesse is, mostly, absent. Is anyone working on >> >> the multiprocessing issues? >> >> >> >> (no, I'm not planning to address them :-)) >> > >> > You mean: actively feeling responsible for it? I guess nobody - as for >> > many other modules in the standard library. >> > >> > Or do you mean: who is willing to work on it, in principle? >> > The last committers are georg.brandl, gregory.p.smith, >> > martin.v.loewis, and antoine.pitrou. >> >> Both Ask and myself feel actively responsible. Antoine could have just >> as easily asked privately. > > Well, the question and its eventual answer were meant for public > consumption, not for my personal edification. > Thanks for answering anyway. We can then continue to put you and Ask in > the nosy lists for multiprocessing issues. > > Regards > > Antoine. I'm not pleased with the current state of affairs and I am working to change this. I apologize for the delay on the bugs. From jnoller at gmail.com Tue Oct 26 02:56:48 2010 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 25 Oct 2010 20:56:48 -0400 Subject: [Python-Dev] PyCon 2011 Reminder: Call for Proposals, Posters and Tutorials - us.pycon.org Message-ID: PyCon 2011 Reminder: Call for Proposals, Posters and Tutorials - us.pycon.org =============================================== Well, it's October 25th! The leaves have turned and the deadline for submitting main-conference talk proposals expires in 7 days (November 1st, 2010)! We are currently accepting main conference talk proposals: http://us.pycon.org/2011/speaker/proposals/ Tutorial Proposals: http://us.pycon.org/2011/speaker/proposals/tutorials/ Poster Proposals: http://us.pycon.org/2011/speaker/posters/cfp/ PyCon 2011 will be held March 9th through the 17th, 2011 in Atlanta, Georgia. (Home of some of the best southern food you can possibly find on Earth!) The PyCon conference days will be March 11-13, preceded by two tutorial days (March 9-10), and followed by four days of development sprints (March 14-17). We are also proud to announce that we have booked our first Keynote speaker - Hilary Mason, her bio: "Hilary is the lead scientist at bit.ly, where she is finding sense in vast data sets. She is a former computer science professor with a background in machine learning and data mining, has published numerous academic papers, and regularly releases code on her personal site, http://www.hilarymason.com/. She has discovered two new species, loves to bake cookies, and asks way too many questions." We're really looking forward to having her this year as a keynote speaker! Remember, we've also added an "Extreme" talk track this year - no introduction, no fluff - only the pure technical meat! For more information on "Extreme Talks" see: http://us.pycon.org/2011/speaker/extreme/ We look forward to seeing you in Atlanta! Please also note - registration for PyCon 2011 will also be capped at a maximum of 1,500 delegates, including speakers. When registration opens (soon), you're going to want to make sure you register early! Speakers with accepted talks will have a guaranteed slot. We have published all registration prices online at: http://us.pycon.org/2011/tickets/ Important Dates November 1st, 2010: Talk proposals due. December 15th, 2010: Acceptance emails sent. January 19th, 2011: Early bird registration closes. March 9-10th, 2011: Tutorial days at PyCon. March 11-13th, 2011: PyCon main conference. March 14-17th, 2011: PyCon sprints days. Contact Emails: Van Lindberg (Conference Chair) - van at python.org Jesse Noller (Co-Chair) - jnoller at python.org PyCon Organizers list: pycon-organizers at python.org From pingebre at yahoo.com Tue Oct 26 07:04:39 2010 From: pingebre at yahoo.com (Peter Ingebretson) Date: Mon, 25 Oct 2010 22:04:39 -0700 (PDT) Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function Message-ID: <588268.3375.qm@web34406.mail.mud.yahoo.com> I have a patch that adds a new function to the gc module.? The gc.remap() function uses the tp_traverse mechanism to find all references to any keys in a provided mapping, and remaps these references in-place to instead point to the value corresponding to each key. The motivation for adding this method is to enable writing a module that provide an enhanced version of imp.reload.? The builtin reload function is very useful for iterating on a single module within the Python interpreter shell, but in more complex situations it very limited. In particular, instances of classes declared in the reloaded module will continue to reference the old versions of the classes, and other modules that imported elements of the old module using the 'from ... import ...' syntax will continue to refer to the stale version of the functions or classes that they imported. The gc.remap() function enables writing a new version of reload which uses imp.reload to reload a module and then replaces all references to stale objects from the old module to instead point to equivalent newly defined objects.? This still has many limitations, for instance if an __init__ function has been changed the new __init__ will not be run on old instances.? On the other hand, in many cases this is sufficient to continue iterating on code without needing to restart the Python environment, which can be a significant time savings. I initially tried to implement this reloading strategy entirely in Python using gc.getreferrers() to find references to objects defined in the old module, but I found it was too difficult to reliably replace references in objects once they had been found.? Since the GC already has a way to find all fields that refer to objects, it seemed fairly straightforward to extend that mechanism to additionally modify references. This reloading strategy is documented in more detail here: http://doublestar.org/in-place-python-reloading/ A potentially controversial aspect of this change is that the signature of the visitproc has been modified to take (PyObject **) as an argument instead of (PyObject *) so that a visitor can modify fields visited with Py_VISIT. A few traverse functions in the standard library also had to be changed to use Py_VISIT on the actual members rather than on aliased pointers. I also have a prototype of an enhanced reload function using gc.remap.? This is only a partial implementation of the proposal, in particular it does not rehash dictionaries that have been invalidated as a result of reloading, and it does not support custom __reload__ hooks.? A link to the code as well as some examples are here: http://doublestar.org/python-hot-loading-prototype/ Please let me know if you have any feedback on the reloading proposal, the hot loading prototype, or on the patch. Thanks, Peter From g.brandl at gmx.net Tue Oct 26 09:31:11 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 26 Oct 2010 09:31:11 +0200 Subject: [Python-Dev] r85838 - python/branches/py3k/.gitignore In-Reply-To: <20101025173718.7CF19F753@mail.python.org> References: <20101025173718.7CF19F753@mail.python.org> Message-ID: Am 25.10.2010 19:37, schrieb victor.stinner: > Author: victor.stinner > Date: Mon Oct 25 19:37:18 2010 > New Revision: 85838 > > Log: > update gitignore > > Added: > python/branches/py3k/.gitignore This looks more like "Add gitignore". Do we really want to check in ignore files for every possible DVCS? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From hrvoje.niksic at avl.com Tue Oct 26 13:07:02 2010 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Tue, 26 Oct 2010 13:07:02 +0200 Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <588268.3375.qm@web34406.mail.mud.yahoo.com> References: <588268.3375.qm@web34406.mail.mud.yahoo.com> Message-ID: <4CC6B656.3050903@avl.com> On 10/26/2010 07:04 AM, Peter Ingebretson wrote: > I have a patch that adds a new function to the gc module. The gc.remap() > function uses the tp_traverse mechanism to find all references to any keys > in a provided mapping, and remaps these references in-place to instead point > to the value corresponding to each key. What about objects that don't implement tp_traverse because they cannot take part in cycles? Changing immutable objects such as tuples and frozensets doesn't exactly sound appealing. > A potentially controversial aspect of this change is that the signature of the > visitproc has been modified to take (PyObject **) as an argument instead of > (PyObject *) so that a visitor can modify fields visited with Py_VISIT. This sounds like a bad idea -- visitproc is not limited to visiting struct members. Visited objects can be stored in data structures where their address cannot be directly obtained. For example, in C++, you could have an std::map with PyObject* keys, and it wouldn't be legal to pass addresses of those. Various C++ bindings also implement smart_ptr-style wrappers over PyObject* that handle Python reference counting, and those will also have problems with visitproc taking PyObject **. And this is not just some oddball C++ thing. Many extensions wrap arbitrary C objects which can reach Python data through other C objects, which expose the PyObject* only through a generic "void *get_user_data()"-style accessor. For such objects to cooperate with the GC, they must be able to visit arbitrary PyObject pointers without knowing their address. PyGTK and pygobject are the obvious examples of this, but I'm sure there are many others. If you want to go this route, rather create an extended visit procedure (visitchangeproc?) that accepts a function that can change the reference. A convenience function or macro could implement this for the common case of struct member or PyObject**. From ncoghlan at gmail.com Tue Oct 26 13:08:24 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Oct 2010 21:08:24 +1000 Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles In-Reply-To: References: Message-ID: On Tue, Oct 26, 2010 at 12:28 AM, Vinay Sajip wrote: > Comments welcome. Assuming there are no strong objections asking for reversion > of this change, I'll publicise to the wider community in a few days. It strikes me as a solid, pragmatic solution to a thorny problem. Looking at your checkin though, I wonder if it might be worth implementing some little formatting style classes to get rid of the if/elif chains from the Formatter code. Something like: # Style objects class PercentStyle(object): default_format = "%(message)" def is_asctime_in_format(fmt): return fmt.find("%(asctime)") >= 0 def format_record(fmt, record): return fmt % record.__dict__ class StrFormatStyle(object): default_format = "{message}" def is_asctime_in_format(fmt): return fmt.find("{asctime}") >= 0 def format_record(fmt, record): return fmt.format(**record.__dict__) from string import Template # Top level, embedding may cause import deadlock issues class StringTemplateStyle(object): def __init__(self): self._fmt = self._tmpl = None # Setup for format_record default_format = "${message}" def is_asctime_in_format(fmt): return fmt.find("$asctime") >= 0 or fmt.find("${asctime}") >= 0 def format_record(fmt, record): if fmt != self._fmt: self._fmt = fmt self._tmpl = Template(fmt) return self._tmpl.substitute(**record.__dict__) # Build style map _STYLE_CODES = tuple("% { $".split()) _STYLES = PercentStyle, StrFormatStyle, StringTemplateStyle _STYLE_MAP = dict(zip(_STYLE_CODES, _STYLES)) # Interpretation of the style parameter in Formatter.__init__ if style not in _STYLE_CODES: # Preserve option to allow arbitrary style objects at some point in the future raise ValueError("Style must be one of {!r}".format(_STYLE_CODES)) self._style = _STYLE_MAP[style]() # self._fmt initialisation if/elif chain replacement self._fmt = self._style.default_format # Time check if/elif chain replacement result = self._style.is_asctime_in_format(self._fmt) # Record formatting if/elif chain replacement s = self._style.format_record(self._fmt, record) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Tue Oct 26 13:15:54 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Oct 2010 21:15:54 +1000 Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles In-Reply-To: References: Message-ID: On Tue, Oct 26, 2010 at 9:08 PM, Nick Coghlan wrote: > On Tue, Oct 26, 2010 at 12:28 AM, Vinay Sajip wrote: >> Comments welcome. Assuming there are no strong objections asking for reversion >> of this change, I'll publicise to the wider community in a few days. > > It strikes me as a solid, pragmatic solution to a thorny problem. > > Looking at your checkin though, I wonder if it might be worth > implementing some little formatting style classes to get rid of the > if/elif chains from the Formatter code. Something like: Some syntax highlighting may make that wall-o'-code a little easier to read: http://pastebin.com/SG0Qr3r5 Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Tue Oct 26 13:19:29 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 26 Oct 2010 21:19:29 +1000 Subject: [Python-Dev] r85838 - python/branches/py3k/.gitignore In-Reply-To: References: <20101025173718.7CF19F753@mail.python.org> Message-ID: On Tue, Oct 26, 2010 at 5:31 PM, Georg Brandl wrote: > This looks more like "Add gitignore". ?Do we really want to check in > ignore files for every possible DVCS? No, but supporting the current big four open source ones (svn, hg, bzr, git) seems reasonable enough. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From bostjan.mejak at gmail.com Tue Oct 26 13:38:38 2010 From: bostjan.mejak at gmail.com (=?UTF-8?Q?Bo=C5=A1tjan_Mejak?=) Date: Tue, 26 Oct 2010 13:38:38 +0200 Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles In-Reply-To: References: Message-ID: Line 31 (in Pastebin): _STYLE_CODES = tuple("% { $".split()) Is this really necessary? Why not _STYLE_CODES = ('%', '{', '$') On Tue, Oct 26, 2010 at 1:15 PM, Nick Coghlan wrote: > On Tue, Oct 26, 2010 at 9:08 PM, Nick Coghlan wrote: > > On Tue, Oct 26, 2010 at 12:28 AM, Vinay Sajip > wrote: > >> Comments welcome. Assuming there are no strong objections asking for > reversion > >> of this change, I'll publicise to the wider community in a few days. > > > > It strikes me as a solid, pragmatic solution to a thorny problem. > > > > Looking at your checkin though, I wonder if it might be worth > > implementing some little formatting style classes to get rid of the > > if/elif chains from the Formatter code. Something like: > > Some syntax highlighting may make that wall-o'-code a little easier to > read: http://pastebin.com/SG0Qr3r5 > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/bostjan.mejak%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinay_sajip at yahoo.co.uk Tue Oct 26 13:55:14 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 26 Oct 2010 11:55:14 +0000 (UTC) Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles References: Message-ID: Nick Coghlan gmail.com> writes: > Looking at your checkin though, I wonder if it might be worth > implementing some little formatting style classes to get rid of the > if/elif chains from the Formatter code. Something like: Fair comment: I did think about the messiness of that if/elif, but considered it OK as it was likely that Python would not grow any more formatting approaches. However, I hadn't thought about user-defined approaches; what you're suggesting is more extensible. I'll incorporate your and Bo?tjan's comments in the next checkin. Thanks, Vinay From barry at python.org Tue Oct 26 14:51:24 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 26 Oct 2010 08:51:24 -0400 Subject: [Python-Dev] r85838 - python/branches/py3k/.gitignore In-Reply-To: References: <20101025173718.7CF19F753@mail.python.org> Message-ID: <20101026085124.4c68459c@mission> On Oct 26, 2010, at 09:19 PM, Nick Coghlan wrote: >On Tue, Oct 26, 2010 at 5:31 PM, Georg Brandl wrote: >> This looks more like "Add gitignore". ?Do we really want to check in >> ignore files for every possible DVCS? > >No, but supporting the current big four open source ones (svn, hg, >bzr, git) seems reasonable enough. +1. A couple of extra dot files never hurt anyone. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From benjamin at python.org Tue Oct 26 18:06:03 2010 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 26 Oct 2010 11:06:03 -0500 Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <588268.3375.qm@web34406.mail.mud.yahoo.com> References: <588268.3375.qm@web34406.mail.mud.yahoo.com> Message-ID: 2010/10/26 Peter Ingebretson : > I have a patch that adds a new function to the gc module.? The gc.remap() > function uses the tp_traverse mechanism to find all references to any keys > in a provided mapping, and remaps these references in-place to instead point > to the value corresponding to each key. > > The motivation for adding this method is to enable writing a module that > provide an enhanced version of imp.reload.? The builtin reload function > is very useful for iterating on a single module within the Python interpreter > shell, but in more complex situations it very limited. Is there any reason that you'd want to do this? ... > http://doublestar.org/python-hot-loading-prototype/ > > Please let me know if you have any feedback on the reloading proposal, the > hot loading prototype, or on the patch. Overall, I think this adds lots of backwards incompatible code for an obscure use-case that will cause subtle and complicated bugs. So, -1. -- Regards, Benjamin From pingebre at yahoo.com Tue Oct 26 19:11:00 2010 From: pingebre at yahoo.com (Peter Ingebretson) Date: Tue, 26 Oct 2010 10:11:00 -0700 (PDT) Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <4CC6B656.3050903@avl.com> Message-ID: <866196.92064.qm@web34402.mail.mud.yahoo.com> --- On Tue, 10/26/10, Hrvoje Niksic wrote: > What about objects that don't implement tp_traverse because > they cannot take part in cycles? A significant majority of objects that can hold references to other objects can take part in cycles and do implement tp_traverse. My original thought was that modifying any references not visible to the cyclic GC would be out of the scope of gc.remap. Even adding a 'tp_extended_traverse' method might not help solve this problem because untracked objects are not in any generation list, so there is no general way to find all of them. > Changing immutable objects such as tuples and frozensets > doesn't exactly sound appealing. My original Python-only approach cloned immutable objects that referenced objects that were to be remapped, and then added the old and new immutable object to the mapping. This worked well, although it was somewhat complicated because it had to happen in dependency order (e.g., to handle tuples of tuples in frozensets). I thought about keeping this, but I am now convinced that as long as you are doing something as drastic as changing references in the heap you may as well change immutable objects. The main argument is that preserving immutable objects increases the complexity of remapping and does not actually solve many problems. The primary reason for objects to be immutable is so that their comparison operators and hash value can remain consistent. Changing, for example, the contents of a tuple that a dictionary key references has the same effect as changing the identity of the tuple -- both modify the hash value of the key and thus invalidate the dictionary. The full reload processs needs to rehash collections invalidated by hash values changing, so we might as well modify the contents of tuples. > > the signature of visitproc has been modified to take (PyObject **) > > instead of (PyObject *) so that a visitor can modify fields > > visited with Py_VISIT. > > This sounds like a bad idea -- visitproc is not limited to > visiting struct members.? Visited objects can be stored > in data structures where their address cannot be directly > obtained. > > If you want to go this route, rather create an extended > visit procedure (visitchangeproc?) that accepts a function > that can change the reference.? A convenience function > or macro could implement this for the common case of struct > member or PyObject**. This is a compelling argument. I considered adding an extended traverse / visit path, but decided against it after not finding any cases in the base distribution that required it. The disadvantage of creating an additional method is that C types will have yet another method to implement for the gc (tp_traverse, tp_clear, and now tp_traverse_modify(?)). On the other hand, you've convinced me that this is necessary in some cases, so it might as well be used in all of them. Jon Parise also pointed out in a private communication that this eliminates the minor performance impact on tp_traverse, which is another advantage over my change. If a 'tp_traverse_modify' function were added, many types could replace their custom tp_clear function with a generic method that makes use of (visitchangeproc), which somewhat mitigates adding another method. From pingebre at yahoo.com Tue Oct 26 19:24:26 2010 From: pingebre at yahoo.com (Peter Ingebretson) Date: Tue, 26 Oct 2010 10:24:26 -0700 (PDT) Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: Message-ID: <934815.49444.qm@web34406.mail.mud.yahoo.com> --- On Tue, 10/26/10, Benjamin Peterson wrote: > Is there any reason that you'd want to do this? > > http://doublestar.org/python-hot-loading-prototype/ I have a relatively large application written in Python, and a specific use case where it will significantly increase our speed of iteration to be able to change and test modules without needing to restart the application. We have experimented with different approaches to reloading and this one seems the most promising by a wide margin. > Overall, I think this adds lots of backwards incompatible > code for an obscure use-case that will cause subtle and > complicated bugs. So, -1. Would you still object to the change if (visitproc), Py_VISIT and tp_traverse were reverted to their previous state, and a separate path was added for modifying references using (visitchangeproc), Py_VISIT_CHANGE, and tp_traverse_change? Thanks, Peter From raymond.hettinger at gmail.com Tue Oct 26 19:39:19 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Tue, 26 Oct 2010 10:39:19 -0700 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> Message-ID: <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> On Oct 26, 2010, at 9:31 AM, Guido van Rossum wrote: > I would like Gregor Lingl's approval of turning turtle.py into a package. It might make some things harder for novices, e.g. trackebacks and just browsing the source code. > > Also many people don't expect to find any code in a file named __init__.py (and most of the time I agree with this). But the alternative isn't so great either, assuming we'll want strict backwards compatibility (I wouldn't want the instructions in Gregor's or anyone's book to start failing because of this). You can't rename turtle to turtle/turtle.py either, because then there'd be horrible confusion between turtle/ and turtle.py. > > IOW, yes, flat still seems better than nested here! Thanks for saying this. It might be a good time to also have a discussion about what may be an emerging trend of wanting to split standard library modules into packages. I understand the urge to split longer modules into multiple source files but would like to mention some of the advantages of keeping modules in a single source file. Taking the decimal module as an example: * Right now, I can pull up the source and entire history from subversion by knowing just the module name: http://svn.python.org/view/python/branches/py3k/Lib/decimal.py?view=markup , and I can search the entire module with nothing more sophisticated than "find" in the web browser. That no longer works with unittest. * I can easily email the file to someone or copy it back to a previous version of Python because only one file is involved. To look at the code, I don't have to guess how someone would have partitioned it into decimal.context.utils.threadlocal.py or somesuch. I don't need a packaging tool to distribute a module (pyparsing and BeautifulSoup are two examples of tools that are easily managed by just being in one file). * Some editors are set up to easily dance across multiple files and directories as if they were a single unit; however, many editors are not. Most file viewers and editors work better with single files. Take two operations in IDLE for example. If I press Alt-M for open module, I can retrieve all of the source for decimal (but this won't work with the logging or unittest packages). Once the source is visible, I press Alt-C to bring up a class browser for quickly reviewing the source (but this also won't work with packages). Several Python editors have this capability (provided by the pyclbr module) but they are defeated by packaging. * The Lib directory is more easily grepped when it is flat. The more we nest the Lib directory, the harder it will be to search the sources (while excluding the test directory). Packaging is not always wrong. Maybe it was the right thing to do for unittest, maybe not. I just wanted to bring up some of the advantages of having a single file for a library module. It would not be a good thing if some of the newer committers embarked on taking modules over 2000 lines and split them into packages. Right now, we have hundreds of source files, but it wouldn't take long for someone on a packaging mission to turn that into thousands. If someone wants to reorganize code for clarity, I would prefer keeping it within one file, bringing related functions together and using comment lines to mark the major sections. ISTM, this is cleaner than introducing a new directory and having multiple files that cross-import one another. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From nad at acm.org Tue Oct 26 19:42:27 2010 From: nad at acm.org (Ned Deily) Date: Tue, 26 Oct 2010 10:42:27 -0700 Subject: [Python-Dev] r85838 - python/branches/py3k/.gitignore References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> Message-ID: In article <20101026085124.4c68459c at mission>, Barry Warsaw wrote: > On Oct 26, 2010, at 09:19 PM, Nick Coghlan wrote: > >On Tue, Oct 26, 2010 at 5:31 PM, Georg Brandl wrote: > >> This looks more like "Add gitignore". ?Do we really want to check in > >> ignore files for every possible DVCS? > > > >No, but supporting the current big four open source ones (svn, hg, > >bzr, git) seems reasonable enough. > > +1. A couple of extra dot files never hurt anyone. :) Actually they have. I got burned a while back by the checked-in .hgignore file when creating my own hg repo (to use mq patch management) on top of a svn checkout. Now that I know they are there, I ensure they get deleted. -- Ned Deily, nad at acm.org From brett at python.org Tue Oct 26 19:53:51 2010 From: brett at python.org (Brett Cannon) Date: Tue, 26 Oct 2010 10:53:51 -0700 Subject: [Python-Dev] Python bug week-end : 20-21 November In-Reply-To: <20101025230337.41aeef12@pitrou.net> References: <20101025230337.41aeef12@pitrou.net> Message-ID: Can whomever has edit access to the Python Google Calendar add this? On Mon, Oct 25, 2010 at 14:03, Antoine Pitrou wrote: > > Hello, > > The development team of the Python interpreter (a.k.a python-dev) is > organizing a bug week-end on Saturday 20th and Sunday 21st of November. > > We would like to encourage anyone who feels interested in participating > to give it a try. Contributing to Python is much less intimidating than > it sounds. You don't need to have previous experience with modifying > the Python source; in fact bug days offer a good opportunity to learn > the basics by asking questions and working on relatively simple bugs > (see "how to get prepared" below). And most core developers are actual > human beings! > > How it happens > -------------- > > The bug week-end will happen on the #python-dev IRC channel on the > Freenode network, where several core developers routinely hang out. No > physical meeting is scheduled as far as I know, but anyone is > encouraged to organize one and announce it on the official Python > channels such as this one. > > Participants (you!) join #python-dev and collaboratively go through the > Python issue tracker at http://bugs.python.org . From there, you can > provide patches and/or review existing patches. Also, you can help us > assess issues on any specific topic you have expertise in (the range of > topics touched in the stdlib is quite broad and it is more than likely > that the core developers' expertise is lacking in some of them). > > Or, if you feel shy, you can simply watch other people work and > slowly get more confident about participating yourself. > Development is public and lurkers are welcome. > > What you can work on > --------------------- > > Our expectation is that Python 3.2 beta 1 will have been released a > couple of days before the bug week-end and, therefore, one primary goal > is to polish the 3.2 branch for the following betas and the final > release. There are many issues to choose from on the bug tracker; any > bug fixes or documentation improvements will do. New features are > discouraged: they can't be checked in before the official 3.2 release. > > How to get prepared > ------------------- > > If you are a beginner with the Python codebase, you may want to read the > development guide available here (courtesy of Brian Curtin): > http://docs.pythonsprints.com/core_development/beginners.html > > There's a small practical guide to bug days/week-ends on the wiki: > http://wiki.python.org/moin/PythonBugDay > > And the development FAQ holds answers to generic development questions: > http://www.python.org/dev/faq/ > > You can also do all of the above during the bug week-end, of course. > Please, don't hesitate to ask us questions on the #python-dev channel. > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From martin at v.loewis.de Tue Oct 26 19:56:36 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 26 Oct 2010 19:56:36 +0200 Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <934815.49444.qm@web34406.mail.mud.yahoo.com> References: <934815.49444.qm@web34406.mail.mud.yahoo.com> Message-ID: <4CC71654.1030307@v.loewis.de> Am 26.10.2010 19:24, schrieb Peter Ingebretson: > --- On Tue, 10/26/10, Benjamin Peterson wrote: >> Is there any reason that you'd want to do this? >>> http://doublestar.org/python-hot-loading-prototype/ > > I have a relatively large application written in Python, and a > specific use case where it will significantly increase our speed > of iteration to be able to change and test modules without needing > to restart the application. We have experimented with different > approaches to reloading and this one seems the most promising by > a wide margin. I think this then mandates a PEP; I'm -1 on the feature also. In the PEP, you should explain what the alternatives are and why they don't work in the use cases of this feature. For example, I wonder why just changing the classes to have the updated methods isn't good enough. Or, if you want to be able to change the class of all instances of that class, why you can't track all instances explicitly. In the PEP, you should then also explain what the limitations of the feature should. I.e. the feature should not be specified by its implementation, but have some abstract specification. Regards, Martin From solipsis at pitrou.net Tue Oct 26 20:02:34 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 26 Oct 2010 20:02:34 +0200 Subject: [Python-Dev] r85838 - python/branches/py3k/.gitignore References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> Message-ID: <20101026200234.5f8e8323@pitrou.net> On Tue, 26 Oct 2010 10:42:27 -0700 Ned Deily wrote: > In article <20101026085124.4c68459c at mission>, > Barry Warsaw wrote: > > On Oct 26, 2010, at 09:19 PM, Nick Coghlan wrote: > > >On Tue, Oct 26, 2010 at 5:31 PM, Georg Brandl wrote: > > >> This looks more like "Add gitignore". ?Do we really want to check in > > >> ignore files for every possible DVCS? > > > > > >No, but supporting the current big four open source ones (svn, hg, > > >bzr, git) seems reasonable enough. > > > > +1. A couple of extra dot files never hurt anyone. :) > > Actually they have. I got burned a while back by the checked-in > .hgignore file when creating my own hg repo (to use mq patch management) > on top of a svn checkout. You could use own of the "official" mirrors at http://code.python.org/hg/ cheers Antoine. From alexander.belopolsky at gmail.com Tue Oct 26 20:11:09 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 26 Oct 2010 14:11:09 -0400 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> Message-ID: On Tue, Oct 26, 2010 at 1:39 PM, Raymond Hettinger wrote: .. > Packaging is not always wrong. ?Maybe it was the right thing to do for > unittest, maybe not. This is an example that I personally find ill-justified. Particularly annoying is the fact that opening __init__.py gives you a list of relative imports and sends you to the next file for everything else. Having both main.py and __main__.py seems redundant. What were the benefits that justified unittest.py split? From rdmurray at bitdance.com Tue Oct 26 21:05:38 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 26 Oct 2010 15:05:38 -0400 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> Message-ID: <20101026190538.7E9791FE543@kimball.webabinitio.net> On Tue, 26 Oct 2010 10:39:19 -0700, Raymond Hettinger wrote: > If someone wants to reorganize code for clarity, I would prefer keeping > it within one file, bringing related functions together and using > comment lines to mark the major sections. ISTM, this is cleaner than > introducing a new directory and having multiple files that cross-import > one another. +1 (or more) -- R. David Murray www.bitdance.com From benjamin at python.org Tue Oct 26 21:18:22 2010 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 26 Oct 2010 14:18:22 -0500 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> Message-ID: 2010/10/26 Alexander Belopolsky : > On Tue, Oct 26, 2010 at 1:39 PM, Raymond Hettinger > wrote: > .. >> Packaging is not always wrong. ?Maybe it was the right thing to do for >> unittest, maybe not. > > This is an example that I personally find ill-justified. ?Particularly > annoying is the fact that opening __init__.py gives you a list of > relative imports and sends you to the next file for everything else. > Having both main.py and __main__.py seems redundant. ? ?What were the > benefits ?that justified unittest.py split? Mostly it was huge. -- Regards, Benjamin From nad at acm.org Tue Oct 26 21:19:52 2010 From: nad at acm.org (Ned Deily) Date: Tue, 26 Oct 2010 12:19:52 -0700 Subject: [Python-Dev] r85838 - python/branches/py3k/.gitignore References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> <20101026200234.5f8e8323@pitrou.net> Message-ID: In article <20101026200234.5f8e8323 at pitrou.net>, Antoine Pitrou wrote: > You could use own of the "official" mirrors at > http://code.python.org/hg/ The "official" hg mirrors work great: in my experience, faster than svn and simpler with all the useful history information retained from the original svn repos. Thanks for that! As an example, for OS X TextMate users out there, with up-to-date Mercurial and Subversion bundles, try doing an Annotate on an hg clone of a Python branch vs a Blame on an svn co. With the hg clones and the hg rebase extension (now a standard part of hg), I've moved away from using hg mq patch queues for patch development although I still use the svn repos (until the transition is closer) and mq for installer build and regression testing. FWIW, my hg workflow these days is to clone a local "master" clone for each of 27, 31, and py3k repos from the official mirrors then just clone working copies from my local masters as needed for each issue. Using a local master speeds things up, potentially saves disk space as hg uses hard links for local clones where possible, and saves on network bandwidth. With work-in-progress committed to a working copy, I can update automatically to the latest tip by a two-step update: update the local master from the official mirror (via "pull") and then update the working copy from the local master and use rebase to manage any new merge conflicts. I couldn't quite get the two-stage pulls to completely work using one standard hg command so I wrote a "pull-clone" script that does the trick for me: import ConfigParser import os.path import subprocess import sys HG_DEFAULTS = '.hg/hgrc' HG_BIN = '/opt/local/bin/hg' if not os.path.exists(HG_DEFAULTS): sys.exit('No hg repository here') else: config = ConfigParser.ConfigParser() config.read(HG_DEFAULTS) try: default = config.get('paths', 'default') except ConfigParser.NoSectionError, ConfigParser.NoOptionError: default = '' if default.startswith('/'): subprocess.check_call([HG_BIN, 'pull', '-R', default], cwd=default) subprocess.check_call([HG_BIN, 'pull', '-u', '--rebase']) subprocess.check_call([HG_BIN, 'summary', '--remote']) -- Ned Deily, nad at acm.org From raymond.hettinger at gmail.com Tue Oct 26 21:34:30 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Tue, 26 Oct 2010 12:34:30 -0700 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> Message-ID: <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> On Oct 26, 2010, at 12:18 PM, Benjamin Peterson wrote: > 2010/10/26 Alexander Belopolsky : >> On Tue, Oct 26, 2010 at 1:39 PM, Raymond Hettinger >> wrote: >> .. >>> Packaging is not always wrong. Maybe it was the right thing to do for >>> unittest, maybe not. >> >> This is an example that I personally find ill-justified. Particularly >> annoying is the fact that opening __init__.py gives you a list of >> relative imports and sends you to the next file for everything else. >> Having both main.py and __main__.py seems redundant. What were the >> benefits that justified unittest.py split? > > Mostly it was huge. Now, it's huge and split across multiple files so it's harder to search, the class browser won't work, and the full source cannot be brought up immediately using just the module name. The svn annotations and history are munged-up. The components were highly interdependent so now every file has to start with a set of cross-imports. Well, at least the __init__.py is not full of code. That's something. FWIW, it wasn't that big (approx 2500 lines). The argparse, difflib, doctest, pickletools, pydoc, tarfile modules are about the same size and the decimal module is even larger. Please don't split those. Raymond From solipsis at pitrou.net Tue Oct 26 21:54:43 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 26 Oct 2010 21:54:43 +0200 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> Message-ID: <20101026215443.0c78fc29@pitrou.net> On Tue, 26 Oct 2010 12:34:30 -0700 Raymond Hettinger wrote: > > FWIW, it wasn't that big (approx 2500 lines). > The argparse, difflib, doctest, pickletools, pydoc, tarfile modules > are about the same size and the decimal module is even larger. > Please don't split those. I think it comes down to the preference of whoever works the most actively on it. Michael is the most active contributor to unittest by far, and I suppose he prefers it to be split into several submodules. Regards Antoine. From g.brandl at gmx.net Tue Oct 26 22:04:23 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 26 Oct 2010 22:04:23 +0200 Subject: [Python-Dev] Python bug week-end : 20-21 November In-Reply-To: References: <20101025230337.41aeef12@pitrou.net> Message-ID: Am 26.10.2010 19:53, schrieb Brett Cannon: > Can whomever has edit access to the Python Google Calendar add this? Done. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From pingebre at yahoo.com Tue Oct 26 22:28:15 2010 From: pingebre at yahoo.com (Peter Ingebretson) Date: Tue, 26 Oct 2010 13:28:15 -0700 (PDT) Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <4CC71654.1030307@v.loewis.de> Message-ID: <95436.62608.qm@web34407.mail.mud.yahoo.com> --- On Tue, 10/26/10, "Martin v. L?wis" wrote: > I think this then mandates a PEP; I'm -1 on the feature also. I am happy to write up a PEP for this feature. I'll start that process now, though if anyone feels that this idea has no chance of acceptance please let me know. Thanks, Peter From fuzzyman at voidspace.org.uk Tue Oct 26 22:46:15 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 26 Oct 2010 16:46:15 -0400 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <20101026190538.7E9791FE543@kimball.webabinitio.net> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <20101026190538.7E9791FE543@kimball.webabinitio.net> Message-ID: <4CC73E17.1000501@voidspace.org.uk> On 26/10/2010 15:05, R. David Murray wrote: > On Tue, 26 Oct 2010 10:39:19 -0700, Raymond Hettinger wrote: >> If someone wants to reorganize code for clarity, I would prefer keeping >> it within one file, bringing related functions together and using >> comment lines to mark the major sections. ISTM, this is cleaner than >> introducing a new directory and having multiple files that cross-import >> one another. > +1 (or more) Really, you prefer to work in single multi-thousand line modules than cleanly split smaller packages? (Is that how you develop your own projects, out of interest?) The cross imports are unfortunate (there are very few actually circular dependencies in the package though I think), but more of an indictment on the design of unittest than the decision to split it into a package. How the splitting happened is that I proposed it on this list many months ago and got one reply in favour and no dissenters. Benjamin actually did the splitting. I find unittest *massively* easier to maintain and navigate now that it is a package. It doesn't take very long to work out where the classes live (no time at all if you look in the module names - where the classes live is *very* obvious except for the TextTestResult which lives alongside the TestRunner). The package split into modules is into conceptual units, where the responsibility of the units is clear. This makes the code not only easier to navigate but easier to *think about*. The structure of unittest is now much clearer. I find the big-ball-of-mud style development, where everything lives inside huge monolithic modules, very painful. I also think that it is an extremely bad example for new developers. There is something to be said for consistency within the standard library, but I don't think it is a price worth paying in this particular case. All the best, Michael Foord > -- > R. David Murray www.bitdance.com > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ From martin at v.loewis.de Tue Oct 26 22:47:39 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 26 Oct 2010 22:47:39 +0200 Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <95436.62608.qm@web34407.mail.mud.yahoo.com> References: <95436.62608.qm@web34407.mail.mud.yahoo.com> Message-ID: <4CC73E6B.20403@v.loewis.de> Am 26.10.2010 22:28, schrieb Peter Ingebretson: > --- On Tue, 10/26/10, "Martin v. L?wis" wrote: >> I think this then mandates a PEP; I'm -1 on the feature also. > > I am happy to write up a PEP for this feature. I'll start that > process now, though if anyone feels that this idea has no chance of > acceptance please let me know. If it could actually work in a reasonable way, I would be +0. If, as I think, it can't possibly work correctly, I'll be -1. In this evaluation, I compare this to Smalltalk's Object>>#become: What you propose should have a similar effect, IMO, although it's probably not necessary to provide the two-way nature of become: Regards, Martin From victor.stinner at haypocalc.com Tue Oct 26 23:10:54 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 26 Oct 2010 23:10:54 +0200 Subject: [Python-Dev] r85838 - python/branches/py3k/.gitignore In-Reply-To: References: <20101025173718.7CF19F753@mail.python.org> Message-ID: <201010262310.55070.victor.stinner@haypocalc.com> Le mardi 26 octobre 2010 09:31:11, Georg Brandl a ?crit : > Am 25.10.2010 19:37, schrieb victor.stinner: > > Author: victor.stinner > > Date: Mon Oct 25 19:37:18 2010 > > New Revision: 85838 > > > > Log: > > update gitignore > > > > Added: > > python/branches/py3k/.gitignore > > This looks more like "Add gitignore". Do we really want to check in > ignore files for every possible DVCS? It's a mistake. I didn't want to commit this file. The changelog is "update" because I merged two or three commits about gitignore using git rebase -i, and I kept the wrong changelog. If it doesn't "hurt", keep it, otherwise you can delete it. As you want. But each time that I create a git-svn repository, I have to recreate this file. -- Victor Stinner http://www.haypocalc.com/ From fdrake at acm.org Tue Oct 26 23:37:41 2010 From: fdrake at acm.org (Fred Drake) Date: Tue, 26 Oct 2010 17:37:41 -0400 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <4CC73E17.1000501@voidspace.org.uk> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <20101026190538.7E9791FE543@kimball.webabinitio.net> <4CC73E17.1000501@voidspace.org.uk> Message-ID: On Tue, Oct 26, 2010 at 4:46 PM, Michael Foord wrote: > I find the big-ball-of-mud style development, where everything lives inside > huge monolithic modules, very painful. I also think that it is an extremely > bad example for new developers. Gadzooks, Michael! Something else we agree on. 2000-line modules are a source of maintenance pain. ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From pingebre at yahoo.com Tue Oct 26 23:45:10 2010 From: pingebre at yahoo.com (Peter Ingebretson) Date: Tue, 26 Oct 2010 14:45:10 -0700 (PDT) Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <4CC73E6B.20403@v.loewis.de> Message-ID: <838696.71149.qm@web34402.mail.mud.yahoo.com> --- On Tue, 10/26/10, "Martin v. L?wis" wrote: >>> I think this then mandates a PEP; I'm -1 on the feature also. >> >> I am happy to write up a PEP for this feature.? I'll start that >> process now, though if anyone feels that this idea has no chance of >> acceptance please let me know. > > If it could actually work in a reasonable way, I would be +0. If, > as I think, it can't possibly work correctly, I'll be -1. > > In this evaluation, I compare this to Smalltalk's > Object>>#become: > What you propose should have a similar effect, IMO, although > it's probably not necessary to provide the two-way nature > of become. Thanks, I didn't know about Object>>#become until now but it is a perfect comparison. The two-way nature of become appears to be due to the implementation detail of swapping two entries in a table, but the current spec for gc.remap can achieve the same effect with: >>> gc.remap({a:b, b:a}) Of course #become and gc.remap also share the same power and danger. I'm retracting the patch in 10194 and will submit a new one later as part of the PEP that uses a parallel traverse mechanism. Still, if you are concerned that this approach cannot work I encourage you to try out the patch associated with 10194 by playing around with gc.remap in the interpreter or looking at the unit tests. I was surprised when I made the change initially by how little code was required and by how well it seemed to work in practice. Thanks, Peter From nas at arctrix.com Tue Oct 26 23:45:19 2010 From: nas at arctrix.com (Neil Schemenauer) Date: Tue, 26 Oct 2010 21:45:19 +0000 (UTC) Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function References: <4CC71654.1030307@v.loewis.de> <95436.62608.qm@web34407.mail.mud.yahoo.com> Message-ID: Peter Ingebretson wrote: > I am happy to write up a PEP for this feature. I'll start that > process now, though if anyone feels that this idea has no chance of > acceptance please let me know. I think a feature that allows modules to be more reliability reloaded could be accepted. Martin's suggestion sounds like it could be useful. I would recommend trying to limit the scope of the feature and clearly define what it intends to achieve (e.g. use cases). The idea of replacing references does not seem to have much hope, IMHO. It presents all kinds of subtle problems. Dictionary hashing is only one of many invariants that could be broken by blindly replacing references. You have no way of knowing what other invariants are expected or if the new objects will satisfy them. Also, there would have to be a very compelling reason to change to the signature of "visitproc". Every Python module that participates in GC would have to be modified as a result of the signature change. Regards, Neil From exarkun at twistedmatrix.com Tue Oct 26 23:53:58 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Tue, 26 Oct 2010 21:53:58 -0000 Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <95436.62608.qm@web34407.mail.mud.yahoo.com> References: <4CC71654.1030307@v.loewis.de> <95436.62608.qm@web34407.mail.mud.yahoo.com> Message-ID: <20101026215358.2040.1902248436.divmod.xquotient.42@localhost.localdomain> On 08:28 pm, pingebre at yahoo.com wrote: >--- On Tue, 10/26/10, "Martin v. L?wis" wrote: >>I think this then mandates a PEP; I'm -1 on the feature also. > >I am happy to write up a PEP for this feature. I'll start that >process now, though if anyone feels that this idea has no chance of >acceptance please let me know. This can be implemented with ctypes right now (I half did it several years ago). Jean-Paul From rrr at ronadam.com Tue Oct 26 23:54:40 2010 From: rrr at ronadam.com (Ron Adam) Date: Tue, 26 Oct 2010 16:54:40 -0500 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> Message-ID: On 10/26/2010 02:34 PM, Raymond Hettinger wrote: > FWIW, it wasn't that big (approx 2500 lines). > The argparse, difflib, doctest, pickletools, pydoc, tarfile modules > are about the same size and the decimal module is even larger. > Please don't split those. Sense you mention this... I've worked on pydoc to make it much nicer to use in a browser. While doing that I needed to reworked the server part. That resulted in a clean server thread object (and supporting parts) with no pydoc specific code in those parts. It can work as a stand alone module quite nicely. It's about 170 lines with around a third of that as documented examples that can also run as doctests. More to the point, it's a simple text/html server wrapped in a thread object. It can work as a starting point to using a browser as a user interface like pydoc does. There is a patch in the bug tracker, I just need to make some minor updates to it and it can go in, but I really need some code organizing/placement review help. I I'm wonder what you may think. Keep it in pydoc or move it to the HTTP package? Document it or not? Ron From whitequill.bj at gmail.com Wed Oct 27 00:05:11 2010 From: whitequill.bj at gmail.com (Bj Raz) Date: Tue, 26 Oct 2010 18:05:11 -0400 Subject: [Python-Dev] __setcall__ In-Reply-To: References: Message-ID: I'll look into SAGE, I am still curious if there is, a way, to write this in native python, cause I'm currently plotting in Autodesk Maya, and using Python. Sent from my iPhone On Oct 24, 2010, at 12:51 PM, Benjamin Peterson wrote: > 2010/10/24 Bj Raz : >> I was looking for a way to set a function being equal to another function: >> q(m+1) = q(m)+ ((-1)^m) * exp(lnanswer); >> I was hoping to use something like this: >> (q).__setcall__.(m+1) = (q).__setcall__.(m)+ ((-1)^m) * exp(lnanswer); >> As a work around. >> But I have not found the `__setcall__' built in being there. >> >> Here is the code block I'm working with: >> >> indvar = 200; >> q = 0; >> lnanswer = 0; >> for m = 1:150 >> lnanswer = (3 * m) * log(indvar) - log(factorial(3 * m)) ; >> q(m+1) = q(m)+ ((-1)^m) * exp(lnanswer); >> end >> lnanswer >> q >> >> Any help would be appreciated. > > Please see python-list. This list is for the development of python. > > The SAGE math package can do something like this. > > > > -- > Regards, > Benjamin From pje at telecommunity.com Wed Oct 27 00:12:07 2010 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 26 Oct 2010 18:12:07 -0400 Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <934815.49444.qm@web34406.mail.mud.yahoo.com> References: <934815.49444.qm@web34406.mail.mud.yahoo.com> Message-ID: <20101026221201.A87FE3A408C@sparrow.telecommunity.com> At 10:24 AM 10/26/2010 -0700, Peter Ingebretson wrote: >I have a relatively large application written in Python, and a >specific use case where it will significantly increase our speed >of iteration to be able to change and test modules without needing >to restart the application. If all you really want this for is reloading, it would probably make more sense to simply modify the existing class and function objects using the reloaded values as a template, then save the modified classes and functions back to the module. Have you tried http://pypi.python.org/pypi/plone.reload or http://svn.python.org/projects/sandbox/trunk/xreload/xreload.py, or any other existing code reloaders, or tried extending them for your specific use case? From pingebre at yahoo.com Wed Oct 27 00:18:27 2010 From: pingebre at yahoo.com (Peter Ingebretson) Date: Tue, 26 Oct 2010 15:18:27 -0700 (PDT) Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: Message-ID: <219058.25307.qm@web34408.mail.mud.yahoo.com> --- On Tue, 10/26/10, Neil Schemenauer wrote: > > I am happy to write up a PEP for this feature.? > > I'll start that process now, though if anyone > > feels that this idea has no chance of > > acceptance please let me know. > > I think a feature that allows modules to be more > reliability reloaded could be accepted.? Martin's > suggestion sounds like it could be useful.? I would > recommend trying to limit the scope of the feature > and clearly define what it intends to achieve (e.g. > use cases). > The idea of replacing references does not seem to > have much hope, IMHO. I agree that the important feature is module reloading, whether it is implemented via remapping references or by replacing the state of existing objects is an implementation detail. I will try to keep the scope of the PEP focused, and if necessary I will split it up into two. Thanks, Peter From robert.kern at gmail.com Wed Oct 27 00:28:22 2010 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 26 Oct 2010 17:28:22 -0500 Subject: [Python-Dev] __setcall__ In-Reply-To: References: Message-ID: On 10/26/10 5:05 PM, Bj Raz wrote: > I'll look into SAGE, I am still curious if there is, a way, to write this in native python, cause I'm currently plotting in Autodesk Maya, and using Python. This thread is off-topic for python-dev, which is intended for the development *of* the Python interpreter, not development *in* Python. We should continue this discussion on python-list, instead: http://mail.python.org/mailman/listinfo/python-list That said, I will answer your question. No, there is no way to do this in Python. However, I don't think that is what the code you are trying to emulate does. It looks like it is building up a list of values, not defining a function. If you take your question over to python-list, I can explain more. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From raymond.hettinger at gmail.com Wed Oct 27 00:35:49 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Tue, 26 Oct 2010 15:35:49 -0700 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> Message-ID: On Oct 26, 2010, at 2:54 PM, Ron Adam wrote: > I've worked on pydoc to make it much nicer to use in a browser. While you're at it. Can you please modernize the html and create a style sheet? Right now, all of formatting is deeply intertwined with content generation. Fixing that would be a *huge* improvement. Raymond From pingebre at yahoo.com Wed Oct 27 01:31:41 2010 From: pingebre at yahoo.com (Peter Ingebretson) Date: Tue, 26 Oct 2010 16:31:41 -0700 (PDT) Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <20101026215358.2040.1902248436.divmod.xquotient.42@localhost.localdomain> Message-ID: <665617.76443.qm@web34408.mail.mud.yahoo.com> --- On Tue, 10/26/10, P.J. Eby wrote: > If all you really want this for is reloading, it would > probably make more sense to simply modify the existing class > and function objects using the reloaded values as a > template, then save the modified classes and functions back > to the module. > > Have you tried http://pypi.python.org/pypi/plone.reload > or http://svn.python.org/projects/sandbox/trunk/xreload/xreload.py, > or any other existing code reloaders, or tried extending > them for your specific use case? I've investigated several reloading frameworks, including the ones you mentions as well as http://code.google.com/p/reimport/ and http://code.google.com/p/livecoding/. The approach of using the gc to remap references seemed to have the fewest overall limitations, but requiring C API changes is a big downside. I'm going to have to do a more detailed comparison of the features offered by each approach. --- On Tue, 10/26/10, exarkun at twistedmatrix.com wrote: > This can be implemented with ctypes right now (I half did > it several years ago). > > Jean-Paul Is there a trick to doing it this way, or are you suggesting building a ctypes wrapper for each C type in the Python library, and then effectively reimplementing tp_traverse in Python? From greg.ewing at canterbury.ac.nz Wed Oct 27 01:44:54 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 27 Oct 2010 12:44:54 +1300 Subject: [Python-Dev] __setcall__ In-Reply-To: References: Message-ID: <4CC767F6.90903@canterbury.ac.nz> Robert Kern wrote: > This thread is off-topic for python-dev, which is intended for the > development *of* the Python interpreter, not development *in* Python. I got the impression that he was asking for a new feature -- i.e. to be allowed to write a call on the left of an assignment, with a corresponding special method. If that's the case, the appropriate place for initial discussion would be python-ideas. -- Greg From rrr at ronadam.com Wed Oct 27 02:55:46 2010 From: rrr at ronadam.com (Ron Adam) Date: Tue, 26 Oct 2010 19:55:46 -0500 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> Message-ID: On 10/26/2010 05:35 PM, Raymond Hettinger wrote: > > On Oct 26, 2010, at 2:54 PM, Ron Adam wrote: > >> I've worked on pydoc to make it much nicer to use in a browser. > > While you're at it. Can you please modernize the html > and create a style sheet? Right now, all of formatting > is deeply intertwined with content generation. > > Fixing that would be a *huge* improvement. Half way there! The server will read one if it exists. ;-) I'd really like to get this part in before 3.2 beta, and then I'll add a basic style sheet and update the html code to use it for 3.3. The present patch fixes and updates all the functional parts and allows you to do every thing that you can do on the command line, but a LOT easier. I think You, Nick. or one of the other Core developers could probably have this finished up in an afternoon if you really wanted. All the parts work. It's more about checking and adjusting the packaging at this point. Cheers, Ron From alexander.belopolsky at gmail.com Wed Oct 27 04:06:37 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 26 Oct 2010 22:06:37 -0400 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <20101026190538.7E9791FE543@kimball.webabinitio.net> <4CC73E17.1000501@voidspace.org.uk> Message-ID: On Tue, Oct 26, 2010 at 5:37 PM, Fred Drake wrote: > On Tue, Oct 26, 2010 at 4:46 PM, Michael Foord > wrote: >> I find the big-ball-of-mud style development, where everything lives inside >> huge monolithic modules, very painful. I also think that it is an extremely >> bad example for new developers. > > Gadzooks, Michael! ?Something else we agree on. ?2000-line modules are > a source of maintenance pain. > While I appreciate your and Michael's eloquence, I don't really see why five 400-line modules are necessarily easier to maintain than one 2000-line module. Splitting code into modules is certainly a good thing when the resulting modules can be used independently. This helps users write leaner programs, reduces mental footprint of individual modules, etc, etc. The split unittest module does not bring any such benefits. It still presents a single "big-ball-of-mud" namespace, only rather than implemented in a single file, it is now swept in from eight different files. If instead, unittest module was split into several top level modules so that test cases can be defined without pulling in test runner machinery and third party test runners could avoid importing irrelevant text runners, that would qualify as an improvement in my book, but probably not big enough to justify breaking compatibility. While maintainers' convenience is a valid valid concern and some level of idiosyncrasy is healthy to allow active maintainers to code in their preferred style, I think users' convenience should come first when it conflicts with that of maintainers. Remember, code is written once and read many. This is particularly true about stdlib. A minor inconvenience of finding the right place to stick a new function in a large file does not in my view overweights a major inconvenience of not having all pieces in one place neatly organized in a linear order. From kristjan at ccpgames.com Wed Oct 27 04:13:12 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 27 Oct 2010 10:13:12 +0800 Subject: [Python-Dev] new buffer in python2.7 Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> Although 2.7 has the new buffer interface and memoryview objects, these are widely not accepted in the built in modules. Examples are the structmodule, some of the socketmodule apis, structmodule, etc. IMHO this is unfortunate. For example when doign network io, you would want code like this: Buffer = bytearray(10) Socket.recv_into(Buffer) Header = struct.unpack("i", memoryview(Buffer)[:4])[0] In other words, you want to minimize coping the data around. Currently in 2.7 you have to cast the memoryview to str() which copies the data. In 3.0 you don't. Is there any chance of getting changes like these in ? (swapping s# for s* in PyArg_ParseTuple in a few strategic places) My current list of places would be: _strucmodule.c arraymodule.c zlibmodule.c marshal.c audioop.c and imageop.c are probably less important. K -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristjan at ccpgames.com Wed Oct 27 05:07:09 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 27 Oct 2010 11:07:09 +0800 Subject: [Python-Dev] new buffer in python2.7 In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB4B@exchcn.ccp.ad.local> Not forgetggin the StringI object in cStringIO. IMHO, not accepting buffers by these objects can be consided a bug, that needs fixing. K From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Kristj?n Valur J?nsson Sent: Wednesday, October 27, 2010 10:13 To: Python-Dev (python-dev at python.org) Subject: [Python-Dev] new buffer in python2.7 Although 2.7 has the new buffer interface and memoryview objects, these are widely not accepted in the built in modules. Examples are the structmodule, some of the socketmodule apis, structmodule, etc. IMHO this is unfortunate. For example when doign network io, you would want code like this: Buffer = bytearray(10) Socket.recv_into(Buffer) Header = struct.unpack("i", memoryview(Buffer)[:4])[0] In other words, you want to minimize coping the data around. Currently in 2.7 you have to cast the memoryview to str() which copies the data. In 3.0 you don't. Is there any chance of getting changes like these in ? (swapping s# for s* in PyArg_ParseTuple in a few strategic places) My current list of places would be: _strucmodule.c arraymodule.c zlibmodule.c marshal.c audioop.c and imageop.c are probably less important. K -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Wed Oct 27 05:23:32 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 26 Oct 2010 23:23:32 -0400 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <4CC73E17.1000501@voidspace.org.uk> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <20101026190538.7E9791FE543@kimball.webabinitio.net> <4CC73E17.1000501@voidspace.org.uk> Message-ID: <20101026232332.04ae5058@mission> On Oct 26, 2010, at 04:46 PM, Michael Foord wrote: >I find the big-ball-of-mud style development, where everything lives inside >huge monolithic modules, very painful. I also think that it is an extremely >bad example for new developers. There is something to be said for consistency >within the standard library, but I don't think it is a price worth paying in >this particular case. +1 (or more). :) or-maybe-you'd-prefer-more-^L-y y'rs, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From rdmurray at bitdance.com Wed Oct 27 05:35:33 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 26 Oct 2010 23:35:33 -0400 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <4CC73E17.1000501@voidspace.org.uk> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <20101026190538.7E9791FE543@kimball.webabinitio.net> <4CC73E17.1000501@voidspace.org.uk> Message-ID: <20101027033533.21CAF21AD6C@kimball.webabinitio.net> On Tue, 26 Oct 2010 16:46:15 -0400, Michael Foord wrote: > On 26/10/2010 15:05, R. David Murray wrote: > > On Tue, 26 Oct 2010 10:39:19 -0700, Raymond Hettinger wrote: > >> If someone wants to reorganize code for clarity, I would prefer keeping > >> it within one file, bringing related functions together and using > >> comment lines to mark the major sections. ISTM, this is cleaner than > >> introducing a new directory and having multiple files that cross-import > >> one another. > > +1 (or more) > Really, you prefer to work in single multi-thousand line modules than > cleanly split smaller packages? (Is that how you develop your own > projects, out of interest?) Yes. Note that this is a more recent tendency, a few years ago I used to split things pretty finely right off the bat, but I found I wasted a lot of time on py-file interdependencies doing it that way. Now I generally only split when the pieces end up generalized enough that I want to reuse them. Which, granted, is often enough. But what I'm doing in that case is moving code out of a file at the top level of my library....into a new file at the top level of my library. Now, it's true that I haven't written any huge applications lately, but looking at a couple of typical projects the main files run between 1500 and 2000 lines (including comments), and I don't find those to be particularly big and certainly not awkward. I do understand the attraction of putting "related stuff" into separate files. I just think the balance may have tilted too far in the direction of nested and could stand to be nudged back toward flat :) Unittest is only 2600 lines total including comments. Typical file sizes are between 200 and 300 lines. Those feel tiny to me :) (And email is worse...a number of files only have a single class in them, and some of those classes consist of maybe 20 lines...) -- R. David Murray www.bitdance.com From barry at python.org Wed Oct 27 05:37:10 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 26 Oct 2010 23:37:10 -0400 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <20101026215443.0c78fc29@pitrou.net> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> <20101026215443.0c78fc29@pitrou.net> Message-ID: <20101026233710.6b313cf7@mission> On Oct 26, 2010, at 09:54 PM, Antoine Pitrou wrote: >I think it comes down to the preference of whoever works the most >actively on it. Michael is the most active contributor to unittest by >far, and I suppose he prefers it to be split into several submodules. And that seems perfectly reasonable to me, especially if the alternative were that Michael was so frustrated with a massive single .py file that it discouraged him from working on it. If done well, a split can help improve the readability and discoverability of the code. I shudder at the madness that a single email.py file would cause. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From rrr at ronadam.com Wed Oct 27 06:33:26 2010 From: rrr at ronadam.com (Ron Adam) Date: Tue, 26 Oct 2010 23:33:26 -0500 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> Message-ID: <4CC7AB96.3010404@ronadam.com> On 10/26/2010 05:35 PM, Raymond Hettinger wrote: > > On Oct 26, 2010, at 2:54 PM, Ron Adam wrote: >> I wonder what you may think. Keep it in pydoc or move it to the >> HTTP package? Document it or not? I still would like to know what your thoughts are concerning where to put, and/or how to package, the simple threaded text server? Cheers, Ron From rrr at ronadam.com Wed Oct 27 06:33:26 2010 From: rrr at ronadam.com (Ron Adam) Date: Tue, 26 Oct 2010 23:33:26 -0500 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> Message-ID: <4CC7AB96.3010404@ronadam.com> On 10/26/2010 05:35 PM, Raymond Hettinger wrote: > > On Oct 26, 2010, at 2:54 PM, Ron Adam wrote: >> I wonder what you may think. Keep it in pydoc or move it to the >> HTTP package? Document it or not? I still would like to know what your thoughts are concerning where to put, and/or how to package, the simple threaded text server? Cheers, Ron From techtonik at gmail.com Wed Oct 27 09:25:48 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 27 Oct 2010 10:25:48 +0300 Subject: [Python-Dev] r85838 - python/branches/py3k/.gitignore In-Reply-To: <20101026085124.4c68459c@mission> References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> Message-ID: On Tue, Oct 26, 2010 at 3:51 PM, Barry Warsaw wrote: > On Oct 26, 2010, at 09:19 PM, Nick Coghlan wrote: > >>On Tue, Oct 26, 2010 at 5:31 PM, Georg Brandl wrote: >>> This looks more like "Add gitignore". ?Do we really want to check in >>> ignore files for every possible DVCS? >> >>No, but supporting the current big four open source ones (svn, hg, >>bzr, git) seems reasonable enough. > > +1. ?A couple of extra dot files never hurt anyone. :) Why hg and bzr can't be tuned to understand .svnignore files? -- anatoly t. From eckhardt at satorlaser.com Wed Oct 27 09:33:18 2010 From: eckhardt at satorlaser.com (Ulrich Eckhardt) Date: Wed, 27 Oct 2010 09:33:18 +0200 Subject: [Python-Dev] new buffer in python2.7 In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> Message-ID: <201010270933.18420.eckhardt@satorlaser.com> On Wednesday 27 October 2010, Kristj?n Valur J?nsson wrote: > Although 2.7 has the new buffer interface and memoryview objects, these are > widely not accepted in the built in modules. Examples are the structmodule, > some of the socketmodule apis, structmodule, etc. > > IMHO this is unfortunate. For example when doign network io, you would > want code like this: Buffer = bytearray(10) > Socket.recv_into(Buffer) > Header = struct.unpack("i", memoryview(Buffer)[:4])[0] Actually I would like code like s = socket() ... header = struct.unpack("i", s) In other words, struct should interact with files/streams directly, instead of requiring me to first read a chunk who's size I manually have to determine etc. Otherwise, I'm +1 on your suggestion, avoiding copying is a good thing. Uli (who's going to shut up, because he doesn't have a patch either) -- Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at ************************************************************************************** Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden. E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese Folgen nicht verantwortlich. ************************************************************************************** From scott+python-dev at scottdial.com Wed Oct 27 09:54:09 2010 From: scott+python-dev at scottdial.com (Scott Dial) Date: Wed, 27 Oct 2010 03:54:09 -0400 Subject: [Python-Dev] r85838 - python/branches/py3k/.gitignore In-Reply-To: References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> Message-ID: <4CC7DAA1.6010800@scottdial.com> On 10/27/2010 3:25 AM, anatoly techtonik wrote: > On Tue, Oct 26, 2010 at 3:51 PM, Barry Warsaw wrote: >> On Oct 26, 2010, at 09:19 PM, Nick Coghlan wrote: >> >>> On Tue, Oct 26, 2010 at 5:31 PM, Georg Brandl wrote: >>>> This looks more like "Add gitignore". Do we really want to check in >>>> ignore files for every possible DVCS? >>> >>> No, but supporting the current big four open source ones (svn, hg, >>> bzr, git) seems reasonable enough. >> >> +1. A couple of extra dot files never hurt anyone. :) > > Why hg and bzr can't be tuned to understand .svnignore files? Is there such a thing? I think you mean the svn:ignore metadata, which I doubt you can get to through either mirror. Even if you could, svn:ignore is pretty weak (only applies to the directory it is set on and can only do glob patterns). .hgignore is significantly more robust (and only has to be written to a single spot) since matches entire paths and supports regex. Eventually, we'll need to build an appropriate .hgignore file for the python tree anyways. As with others, I don't see the harm in committers who use those tools adding and maintaining these files. Seems akin to having Misc/python-mode.el and Misc/Vim/python.vim. It's all in the spirit of supporting the tools that people are actually using. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From hrvoje.niksic at avl.com Wed Oct 27 09:54:59 2010 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Wed, 27 Oct 2010 09:54:59 +0200 Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <866196.92064.qm@web34402.mail.mud.yahoo.com> References: <866196.92064.qm@web34402.mail.mud.yahoo.com> Message-ID: <4CC7DAD3.5030507@avl.com> On 10/26/2010 07:11 PM, Peter Ingebretson wrote: > The main argument is that preserving immutable objects increases the > complexity of remapping and does not actually solve many problems. > The primary reason for objects to be immutable is so that their > comparison operators and hash value can remain consistent. There are other reasons as well (thread-safety), but I guess those don't really apply to python. I guess one could defend the position that the tuple hasn't really "changed" if its elements merely get upgraded in this way, but it still feels wrong. > Changing, > for example, the contents of a tuple that a dictionary key references > has the same effect as changing the identity of the tuple -- both > modify the hash value of the key and thus invalidate the dictionary. > The full reload processs needs to rehash collections invalidated by > hash values changing, so we might as well modify the contents of > tuples. Do you also rehash when tuples of upgraded objects are used as dict keys? From raymond.hettinger at gmail.com Wed Oct 27 10:01:20 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Wed, 27 Oct 2010 01:01:20 -0700 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <20101026233710.6b313cf7@mission> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> <20101026215443.0c78fc29@pitrou.net> <20101026233710.6b313cf7@mission> Message-ID: <6ED1032E-D515-4065-91D0-F822303C97B1@gmail.com> On Oct 26, 2010, at 8:37 PM, Barry Warsaw wrote: > If done well, a split can help improve the readability and discoverability of > the code. No doubt that is true. The problem is that splitting can also impair discoverability. When unittest was in one file, you knew the filename was unittest.py from just the module name. Now, there are ten files and you have no way to guess any of their names or how the author grouped the functions and methods inside them. Let's say that I'm mystified about the difference between assertSameElements() and assertItemsEqual() and assertSetEqual(). So, I go to http://svn.python.org/view/python/branches/py3k/Lib/unittest/ . Now, how do you know which file contains the source? Formerly, I could call-up the one source file and do a find. Or, I could do OpenModule: unittest with IDLE and instantly see the source and let the class browser analyze the structure of the file. Now, that's not possible either. The source for those methods is now less discoverable. In the unittest/case.py file, the safe_repr() function function is called over 40 times and it is not used in any other file. So how do be benefit from it being defined in the utils.py file? ISTM, this saved nothing. The case.py file is over 1000 lines long and utils is 80. How did we benefit from that split? For me, it makes it harder to read the code because I have to go looking elsewhere for commonly called functions. The unittest module grew from one file in Py2.6 to ten files and one directory with 2500 lines in Py2.7. Was that a win? I've spent time trying to read through the changes and cannot follow it without having most of those ten files open in my editor. For me, it's a PITA to read the code. It doesn't help that the file split blew away the svn blame history, so I have a harder time being about to tell what changed. All that being said, I started this thread with: "Packaging is not always wrong. Maybe it was the right thing to do for unittest, maybe not". The goal of the post was just to raise awareness that splitting modules is not a straight win. There are costs to discoverability and searchability. It makes pyclbr less useful. It makes svn viewer less useful. And it loses history. The split of unittest is a done deal. Who knows, it may have been a net win. Am just hoping that we all understand that it was not cost free. My hope is that the splitting modules unnecessarily does not become a trend. Packages should be the last tool we reach for, not the first. In many cases, flat is better than nested. Raymond P.S. I liked your qualifier, "if done well". There's a lot in those three words. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Wed Oct 27 10:40:19 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 27 Oct 2010 10:40:19 +0200 Subject: [Python-Dev] r85838 - python/branches/py3k/.gitignore In-Reply-To: References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> Message-ID: Am 27.10.2010 09:25, schrieb anatoly techtonik: > On Tue, Oct 26, 2010 at 3:51 PM, Barry Warsaw wrote: >> On Oct 26, 2010, at 09:19 PM, Nick Coghlan wrote: >> >>>On Tue, Oct 26, 2010 at 5:31 PM, Georg Brandl wrote: >>>> This looks more like "Add gitignore". Do we really want to check in >>>> ignore files for every possible DVCS? >>> >>>No, but supporting the current big four open source ones (svn, hg, >>>bzr, git) seems reasonable enough. >> >> +1. A couple of extra dot files never hurt anyone. :) > > Why hg and bzr can't be tuned to understand .svnignore files? As Scott said, this isn't possible because there are no .svnignore files. However, it would be nice to see those VCS handling ignores as a dotfile in the repo root developing support for a single .ignore file... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Wed Oct 27 12:36:22 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Oct 2010 12:36:22 +0200 Subject: [Python-Dev] new buffer in python2.7 References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> Message-ID: <20101027123622.0ab194f9@pitrou.net> On Wed, 27 Oct 2010 10:13:12 +0800 Kristj?n Valur J?nsson wrote: > Although 2.7 has the new buffer interface and memoryview > objects, these are widely not accepted in the built in modules. That's true, and slightly unfortunate. It could be a reason for switching to 3.1/3.2 :-) > IMHO this is unfortunate. For example when doign network io, you would want code like this: > Buffer = bytearray(10) > Socket.recv_into(Buffer) > Header = struct.unpack("i", memoryview(Buffer)[:4])[0] This can be an useless micro-optimization. People are often misled by the implicit analogy with C. In Python, a "lazy slice" still allocates memory for a whole new PyObject (for example a memoryview). So lazy slices are only a win if they are actually big (because a raw memcpy() is fast). Actually, lazy slices can be *slower* if they instantatiate an object whose allocation is less optimized than the built-in bytes object's. Here are micro-benchmarks under 3.2: $ ./python -m timeit -s "x = b'x'*10000" "x[:100]" 10000000 loops, best of 3: 0.134 usec per loop $ ./python -m timeit -s "x = memoryview(b'x'*10000)" "x[:100]" 10000000 loops, best of 3: 0.151 usec per loop $ ./python -m timeit -s "x = b'x'*10000" "x[:1000]" 1000000 loops, best of 3: 0.228 usec per loop $ ./python -m timeit -s "x = memoryview(b'x'*10000)" "x[:1000]" 10000000 loops, best of 3: 0.151 usec per loop So, as you see, creating a 100-byte slice from a 10 KB bytestring is faster when using normal (eager) slices. It becomes slower when creating a 1KB slice, but is still very fast (under one microsecond). > Not forgetggin the StringI object in cStringIO. > IMHO, not accepting buffers by these objects can be consided a bug, > that needs fixing. It is often tempting to say that a "necessary" feature is a bug, but it's a slippery slope. I would say it's only a bug when it's been documented to work. I don't think StringIO objects have ever supported the buffer protocol. In 3.2, though, you can use the BytesIO.getbuffer() method: http://docs.python.org/dev/library/io.html#io.BytesIO.getbuffer (another reason to switch perhaps :-)) In any case, I think it should be the release manager's decision here. Regards Antoine. From solipsis at pitrou.net Wed Oct 27 12:42:23 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Oct 2010 12:42:23 +0200 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <20101026190538.7E9791FE543@kimball.webabinitio.net> <4CC73E17.1000501@voidspace.org.uk> Message-ID: <20101027124223.119ce5b1@pitrou.net> On Tue, 26 Oct 2010 22:06:37 -0400 Alexander Belopolsky wrote: > > While I appreciate your and Michael's eloquence, I don't really see > why five 400-line modules are necessarily easier to maintain than one > 2000-line module. Splitting code into modules is certainly a good > thing when the resulting modules can be used independently. This > helps users write leaner programs, reduces mental footprint of > individual modules, etc, etc. The split unittest module does not > bring any such benefits. It still presents a single "big-ball-of-mud" > namespace, only rather than implemented in a single file, it is now > swept in from eight different files. Are you saying that it has become a pile of medium-sized balls of mud? I would like to say thanks for the mud, Michael! It's high quality mud for sure. Regards Antoine. From kristjan at ccpgames.com Wed Oct 27 13:06:49 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 27 Oct 2010 19:06:49 +0800 Subject: [Python-Dev] new buffer in python2.7 In-Reply-To: <20101027123622.0ab194f9@pitrou.net> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> <20101027123622.0ab194f9@pitrou.net> Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC37@exchcn.ccp.ad.local> -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Antoine Pitrou Sent: Wednesday, October 27, 2010 18:36 >Here are micro-benchmarks under 3.2: > $ ./python -m timeit -s "x = b'x'*10000" "x[:100]" > 10000000 loops, best of 3: 0.134 usec per loop > $ ./python -m timeit -s "x = memoryview(b'x'*10000)" "x[:100]" > 10000000 loops, best of 3: 0.151 usec per loop That's weird. The greedy slice needs two memory allocations. One for the ByteArray object itself, one for its cargo. In total, more that 100 bytes. In contrast, creating the MemoryView object requires only one allocation of a few dozen bytes. The performance difference must come from some other weird overhead, such as initializing the new MemoryView object. This would be pretty cool to profile using a proper profiler. I'll see what my MS tools can come up with. Meanwhile, a patch is in the tracker: http://bugs.python.org/issue10212 Also this: http://bugs.python.org/issue10211 There is a precedent of treating the failure to accept the Py_buffer interface as bugs in 2.7. After all, this is a supported internal buffer. See for example: http://bugs.python.org/issue8104 K From solipsis at pitrou.net Wed Oct 27 13:15:42 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Oct 2010 13:15:42 +0200 Subject: [Python-Dev] new buffer in python2.7 In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC37@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> <20101027123622.0ab194f9@pitrou.net> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC37@exchcn.ccp.ad.local> Message-ID: <1288178142.3533.9.camel@localhost.localdomain> > >Here are micro-benchmarks under 3.2: > > > $ ./python -m timeit -s "x = b'x'*10000" "x[:100]" > > 10000000 loops, best of 3: 0.134 usec per loop > > $ ./python -m timeit -s "x = memoryview(b'x'*10000)" "x[:100]" > > 10000000 loops, best of 3: 0.151 usec per loop > > That's weird. The greedy slice needs two memory allocations. One for > the ByteArray object itself, one for its cargo. In total, more that > 100 bytes. In contrast, creating the MemoryView object requires only > one allocation of a few dozen bytes. It's not a bytearray object, it's a bytes object. It requires only a single allocation since the data is allocated inline. The memoryview object must also call getbuffer() again on the original object. Regards Antoine. From barry at python.org Wed Oct 27 13:35:46 2010 From: barry at python.org (Barry Warsaw) Date: Wed, 27 Oct 2010 07:35:46 -0400 Subject: [Python-Dev] Misc/python-mode.el (was Re: r85838 - python/branches/py3k/.gitignore) In-Reply-To: <4CC7DAA1.6010800@scottdial.com> References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> <4CC7DAA1.6010800@scottdial.com> Message-ID: <20101027073546.7a28339b@mission> On Oct 27, 2010, at 03:54 AM, Scott Dial wrote: >As with others, I don't see the harm in committers who use those tools >adding and maintaining these files. Seems akin to having >Misc/python-mode.el and Misc/Vim/python.vim. It's all in the spirit of >supporting the tools that people are actually using. Oh crap, Misc/python-mode.el is still there and way out of date. Please, if anybody's using it, switch to the latest one at http://launchpad.net/python-mode I'd like to delete the one in Misc for the py3k branch and replace it with a README.python-mode that points to the new location. What do y'all think about doing the same for the Python 2.7 and 3.1 branches? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From kristjan at ccpgames.com Wed Oct 27 14:00:10 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 27 Oct 2010 20:00:10 +0800 Subject: [Python-Dev] new buffer in python2.7 In-Reply-To: <1288178142.3533.9.camel@localhost.localdomain> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> <20101027123622.0ab194f9@pitrou.net> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC37@exchcn.ccp.ad.local> <1288178142.3533.9.camel@localhost.localdomain> Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC40@exchcn.ccp.ad.local> Ah, well in 2.7 you don't have the luxury of a bytes object. A str() would be similar in 2.7 Anyway, isn't the "bytes" object immutable? In that case, it is not a useful target for a sock.recv_into() call. Calling getbuffer on a bytearray or a bytes object should be really cheap, so I still don't accept this as a matter of fact situation. Btw, going to 3.2 isn't a real option for us any time soon. A lot of companies probably find themselves in a similar situation. This is why I spend so much effort applying some love to 2.7. Most of the stuff I locally do to 2.7 I then contribute to python as 3.2 patches. Someday they'll get backported, no doubt :) K -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Antoine Pitrou Sent: Wednesday, October 27, 2010 19:16 To: python-dev at python.org Subject: Re: [Python-Dev] new buffer in python2.7 > >Here are micro-benchmarks under 3.2: > > > $ ./python -m timeit -s "x = b'x'*10000" "x[:100]" > > 10000000 loops, best of 3: 0.134 usec per loop $ ./python -m timeit > > -s "x = memoryview(b'x'*10000)" "x[:100]" > > 10000000 loops, best of 3: 0.151 usec per loop > > That's weird. The greedy slice needs two memory allocations. One for > the ByteArray object itself, one for its cargo. In total, more that > 100 bytes. In contrast, creating the MemoryView object requires only > one allocation of a few dozen bytes. It's not a bytearray object, it's a bytes object. It requires only a single allocation since the data is allocated inline. The memoryview object must also call getbuffer() again on the original object. Regards Antoine. _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com From exarkun at twistedmatrix.com Wed Oct 27 14:00:58 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 27 Oct 2010 12:00:58 -0000 Subject: [Python-Dev] Issue 10194 - Adding a gc.remap() function In-Reply-To: <665617.76443.qm@web34408.mail.mud.yahoo.com> References: <20101026215358.2040.1902248436.divmod.xquotient.42@localhost.localdomain> <665617.76443.qm@web34408.mail.mud.yahoo.com> Message-ID: <20101027120058.2040.1988856126.divmod.xquotient.45@localhost.localdomain> On 26 Oct, 11:31 pm, pingebre at yahoo.com wrote: >--- On Tue, 10/26/10, exarkun at twistedmatrix.com > wrote: >>This can be implemented with ctypes right now (I half did >>it several years ago). >> >>Jean-Paul > >Is there a trick to doing it this way, or are you suggesting >building a ctypes wrapper for each C type in the Python >library, and then effectively reimplementing tp_traverse >in Python? That's the idea, yes. Jean-Paul > > > >_______________________________________________ >Python-Dev mailing list >Python-Dev at python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: http://mail.python.org/mailman/options/python- >dev/exarkun%40twistedmatrix.com From solipsis at pitrou.net Wed Oct 27 14:15:19 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 27 Oct 2010 14:15:19 +0200 Subject: [Python-Dev] new buffer in python2.7 In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC40@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> <20101027123622.0ab194f9@pitrou.net> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC37@exchcn.ccp.ad.local> <1288178142.3533.9.camel@localhost.localdomain> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC40@exchcn.ccp.ad.local> Message-ID: <20101027141519.5beaae26@pitrou.net> On Wed, 27 Oct 2010 20:00:10 +0800 Kristj?n Valur J?nsson wrote: > Calling getbuffer on a bytearray or a bytes object should > be really cheap, so I still don't accept this as a matter of fact > situation. It *is* cheap. It's just that copying a short slice is dirt cheap as well. Of course, it you manipulate 1 KB slices or longer, it's different. Regards Antoine. From benjamin at python.org Wed Oct 27 14:23:54 2010 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 27 Oct 2010 07:23:54 -0500 Subject: [Python-Dev] Misc/python-mode.el (was Re: r85838 - python/branches/py3k/.gitignore) In-Reply-To: <20101027073546.7a28339b@mission> References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> <4CC7DAA1.6010800@scottdial.com> <20101027073546.7a28339b@mission> Message-ID: 2010/10/27 Barry Warsaw : > On Oct 27, 2010, at 03:54 AM, Scott Dial wrote: > >>As with others, I don't see the harm in committers who use those tools >>adding and maintaining these files. Seems akin to having >>Misc/python-mode.el and Misc/Vim/python.vim. It's all in the spirit of >>supporting the tools that people are actually using. > > Oh crap, Misc/python-mode.el is still there and way out of date. ?Please, if > anybody's using it, switch to the latest one at > > http://launchpad.net/python-mode > > I'd like to delete the one in Misc for the py3k branch and replace it with a > README.python-mode that points to the new location. ?What do y'all think about > doing the same for the Python 2.7 and 3.1 branches? +1 and this should probably happen to the other editor related things in Misc. -- Regards, Benjamin From merwok at netwok.org Wed Oct 27 14:54:34 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Wed, 27 Oct 2010 14:54:34 +0200 Subject: [Python-Dev] Misc/python-mode.el (was Re: r85838 - python/branches/py3k/.gitignore) In-Reply-To: References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> <4CC7DAA1.6010800@scottdial.com> <20101027073546.7a28339b@mission> Message-ID: <4CC8210A.8020700@netwok.org> I remember a discussion about Vim files some time ago. I use Vim with the files from Misc/Vim, which are up-to-date and more useful than the files shipped with Vim. I don?t use plugins or external files, so I?m -1 on removing Misc/Vim. Regards From barry at python.org Wed Oct 27 14:55:51 2010 From: barry at python.org (Barry Warsaw) Date: Wed, 27 Oct 2010 08:55:51 -0400 Subject: [Python-Dev] Misc/python-mode.el (was Re: r85838 - python/branches/py3k/.gitignore) In-Reply-To: <4CC8210A.8020700@netwok.org> References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> <4CC7DAA1.6010800@scottdial.com> <20101027073546.7a28339b@mission> <4CC8210A.8020700@netwok.org> Message-ID: <20101027085551.19ab4495@mission> On Oct 27, 2010, at 02:54 PM, ?ric Araujo wrote: >I remember a discussion about Vim files some time ago. I use Vim with >the files from Misc/Vim, which are up-to-date and more useful than the >files shipped with Vim. I don?t use plugins or external files, so I?m >-1 on removing Misc/Vim. FWIW, Python support in Emacs is muddled. GNU Emacs comes with python.el which is an alternative implementation that shares some features with, but also has differences to, python-mode.el. Many of us hard-core Python Emacsers prefer the latter, but sadly the FSF won't bundle it and a merge has proven impossible so far. I think I'll replace Misc/python-mode.el with Misc/README.Emacs that attempts to discuss the issue in a neutral way. I'll fix this in Python 2.7, 3.1, and 3.2. Thanks, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From facundobatista at gmail.com Wed Oct 27 15:30:16 2010 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 27 Oct 2010 10:30:16 -0300 Subject: [Python-Dev] MemoryError... how much memory? Message-ID: There are a lot of places where Python or modules do something like: self->buf = (char *)malloc(size); if (!self->buf) { PyErr_SetString(PyExc_MemoryError, "out of memory"); At job, we're having some MemoryErrors, and one thing that we would love to know, if how much memory it was asking when that happened. So, I thought about doing something like: char message[50]; ... self->buf = (char *)malloc(size); if (!self->buf) { snprintf(message, 50, "out of memory (asked: %lld)", size); PyErr_SetString(PyExc_MemoryError, &message); Is any inherent problem in doing this? May it be a good idea to make it generic, like providing a PyErr_MemoryError that could accept a message and a number, and stores that number in the exception objects internals? Thanks! -- .? ? Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From ncoghlan at gmail.com Wed Oct 27 15:51:05 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 27 Oct 2010 23:51:05 +1000 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <4CC7AB96.3010404@ronadam.com> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> <4CC7AB96.3010404@ronadam.com> Message-ID: On Wed, Oct 27, 2010 at 2:33 PM, Ron Adam wrote: > I still would like to know what your thoughts are concerning where to put, > and/or how to package, the simple threaded text server? Given the time frame until beta 1, I'd suggest leaving it in pydoc for now. We can look at possibly documenting it and moving it to http.servers for 3.3. (The patch is attached to issue 2001 for anyone else that wants to take a look at it) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From rrr at ronadam.com Wed Oct 27 16:22:00 2010 From: rrr at ronadam.com (Ron Adam) Date: Wed, 27 Oct 2010 09:22:00 -0500 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> <4CC7AB96.3010404@ronadam.com> Message-ID: <4CC83588.9000504@ronadam.com> On 10/27/2010 08:51 AM, Nick Coghlan wrote: > On Wed, Oct 27, 2010 at 2:33 PM, Ron Adam wrote: >> I still would like to know what your thoughts are concerning where to put, >> and/or how to package, the simple threaded text server? > > Given the time frame until beta 1, I'd suggest leaving it in pydoc for > now. We can look at possibly documenting it and moving it to > http.servers for 3.3. > > (The patch is attached to issue 2001 for anyone else that wants to > take a look at it) > > Cheers, > Nick. Fantastic! Thanks Nick From rrr at ronadam.com Wed Oct 27 16:22:00 2010 From: rrr at ronadam.com (Ron Adam) Date: Wed, 27 Oct 2010 09:22:00 -0500 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> <4CC7AB96.3010404@ronadam.com> Message-ID: <4CC83588.9000504@ronadam.com> On 10/27/2010 08:51 AM, Nick Coghlan wrote: > On Wed, Oct 27, 2010 at 2:33 PM, Ron Adam wrote: >> I still would like to know what your thoughts are concerning where to put, >> and/or how to package, the simple threaded text server? > > Given the time frame until beta 1, I'd suggest leaving it in pydoc for > now. We can look at possibly documenting it and moving it to > http.servers for 3.3. > > (The patch is attached to issue 2001 for anyone else that wants to > take a look at it) > > Cheers, > Nick. Fantastic! Thanks Nick From kristjan at ccpgames.com Wed Oct 27 16:28:16 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 27 Oct 2010 22:28:16 +0800 Subject: [Python-Dev] new buffer in python2.7 In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC40@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> <20101027123622.0ab194f9@pitrou.net> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC37@exchcn.ccp.ad.local> <1288178142.3533.9.camel@localhost.localdomain> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC40@exchcn.ccp.ad.local> Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC45@exchcn.ccp.ad.local> So, I did some profiling. It turns out that the difference is not due to the creation of the MemoryView object as such, but the much more complicated slice handling for generic objects. Here is the interesting part of the inclusive sample measurements for the MemoryView: Function Name Inclusive Samples Exclusive Samples Inclusive Samples % Exclusive Samples % apply_slice 8.997 468 62,23 3,24 _PyObject_GetItem 6.257 400 43,28 2,77 memory_subscript 5.857 1.051 40,51 7,27 _PyMemoryView_FromBuffer 2.642 455 18,27 3,15 memory_dealloc 1.572 297 10,87 2,05 _PyObject_Malloc 1.374 1.374 9,50 9,50 __PyObject_GC_New 1.256 236 8,69 1,63 _PySlice_New 1.211 333 8,38 2,30 slice_dealloc 1.061 769 7,34 5,32 __PyObject_GC_Malloc 1.022 293 7,07 2,03 bytearray_getbuffer 987 354 6,83 2,45 dup_buffer 932 932 6,45 6,45 Notice how a Slice object is generated. Then a PyObject_GetItem() is done. The salient code path is from apply_slice(). A slice object must be constructed and destroyed. In contrast, when done on a string directly (or a bytes object in py3k) you go directly to PySequence_GetSlice. Function Name Inclusive Samples Exclusive Samples Inclusive Samples % Exclusive Samples % apply_slice 3.888 502 48,44 6,25 _PySequence_GetSlice 3.039 350 37,86 4,36 string_slice 2.689 281 33,50 3,50 _PyString_FromStringAndSize 2.409 575 30,01 7,16 [MSVCR90.dll] 1.413 1.407 17,61 17,53 string_dealloc 467 150 5,82 1,87 _PyObject_Malloc 379 379 4,72 4,72 _PyObject_Free 378 378 4,71 4,71 __PyEval_SliceIndex 347 347 4,32 4,32 (The measurements for the MemoryView above already contain some optimizations I've done on na?ve functions). This area is ripe for improvements. K -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Kristj?n Valur J?nsson Sent: Wednesday, October 27, 2010 20:00 To: Antoine Pitrou; python-dev at python.org Subject: Re: [Python-Dev] new buffer in python2.7 Ah, well in 2.7 you don't have the luxury of a bytes object. A str() would be similar in 2.7 Anyway, isn't the "bytes" object immutable? In that case, it is not a useful target for a sock.recv_into() call. Calling getbuffer on a bytearray or a bytes object should be really cheap, so I still don't accept this as a matter of fact situation. Btw, going to 3.2 isn't a real option for us any time soon. A lot of companies probably find themselves in a similar situation. This is why I spend so much effort applying some love to 2.7. Most of the stuff I locally do to 2.7 I then contribute to python as 3.2 patches. Someday they'll get backported, no doubt :) K -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Antoine Pitrou Sent: Wednesday, October 27, 2010 19:16 To: python-dev at python.org Subject: Re: [Python-Dev] new buffer in python2.7 > >Here are micro-benchmarks under 3.2: > > > $ ./python -m timeit -s "x = b'x'*10000" "x[:100]" > > 10000000 loops, best of 3: 0.134 usec per loop $ ./python -m timeit > > -s "x = memoryview(b'x'*10000)" "x[:100]" > > 10000000 loops, best of 3: 0.151 usec per loop > > That's weird. The greedy slice needs two memory allocations. One for > the ByteArray object itself, one for its cargo. In total, more that > 100 bytes. In contrast, creating the MemoryView object requires only > one allocation of a few dozen bytes. It's not a bytearray object, it's a bytes object. It requires only a single allocation since the data is allocated inline. The memoryview object must also call getbuffer() again on the original object. Regards Antoine. _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com From kristjan at ccpgames.com Wed Oct 27 16:32:02 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 27 Oct 2010 22:32:02 +0800 Subject: [Python-Dev] new buffer in python2.7 In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC40@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> <20101027123622.0ab194f9@pitrou.net> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC37@exchcn.ccp.ad.local> <1288178142.3533.9.camel@localhost.localdomain> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC40@exchcn.ccp.ad.local> Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC46@exchcn.ccp.ad.local> Sorry, here the tables properly formatted: Function Name Inclusive Samples Exclusive Samples Inclusive Samples % Exclusive Samples % apply_slice 8.997 468 62,23 3,24 _PyObject_GetItem 6.257 400 43,28 2,77 memory_subscript 5.857 1.051 40,51 7,27 _PyMemoryView_FromBuffer 2.642 455 18,27 3,15 memory_dealloc 1.572 297 10,87 2,05 _PyObject_Malloc 1.374 1.374 9,50 9,50 __PyObject_GC_New 1.256 236 8,69 1,63 _PySlice_New 1.211 333 8,38 2,30 slice_dealloc 1.061 769 7,34 5,32 __PyObject_GC_Malloc 1.022 293 7,07 2,03 bytearray_getbuffer 987 354 6,83 2,45 dup_buffer 932 932 6,45 6,45 Function Name Inclusive Samples Exclusive Samples Inclusive Samples % Exclusive Samples % apply_slice 3.888 502 48,44 6,25 _PySequence_GetSlice 3.039 350 37,86 4,36 string_slice 2.689 281 33,50 3,50 _PyString_FromStringAndSize 2.409 575 30,01 7,16 [MSVCR90.dll] 1.413 1.407 17,61 17,53 string_dealloc 467 150 5,82 1,87 _PyObject_Malloc 379 379 4,72 4,72 _PyObject_Free 378 378 4,71 4,71 __PyEval_SliceIndex 347 347 4,32 4,32 -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Kristj?n Valur J?nsson Sent: Wednesday, October 27, 2010 20:00 To: Antoine Pitrou; python-dev at python.org Subject: Re: [Python-Dev] new buffer in python2.7 Ah, well in 2.7 you don't have the luxury of a bytes object. A str() would be similar in 2.7 Anyway, isn't the "bytes" object immutable? In that case, it is not a useful target for a sock.recv_into() call. Calling getbuffer on a bytearray or a bytes object should be really cheap, so I still don't accept this as a matter of fact situation. Btw, going to 3.2 isn't a real option for us any time soon. A lot of companies probably find themselves in a similar situation. This is why I spend so much effort applying some love to 2.7. Most of the stuff I locally do to 2.7 I then contribute to python as 3.2 patches. Someday they'll get backported, no doubt :) K -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Antoine Pitrou Sent: Wednesday, October 27, 2010 19:16 To: python-dev at python.org Subject: Re: [Python-Dev] new buffer in python2.7 > >Here are micro-benchmarks under 3.2: > > > $ ./python -m timeit -s "x = b'x'*10000" "x[:100]" > > 10000000 loops, best of 3: 0.134 usec per loop $ ./python -m timeit > > -s "x = memoryview(b'x'*10000)" "x[:100]" > > 10000000 loops, best of 3: 0.151 usec per loop > > That's weird. The greedy slice needs two memory allocations. One for > the ByteArray object itself, one for its cargo. In total, more that > 100 bytes. In contrast, creating the MemoryView object requires only > one allocation of a few dozen bytes. It's not a bytearray object, it's a bytes object. It requires only a single allocation since the data is allocated inline. The memoryview object must also call getbuffer() again on the original object. Regards Antoine. _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristjan at ccpgames.com Wed Oct 27 16:33:32 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 27 Oct 2010 22:33:32 +0800 Subject: [Python-Dev] new buffer in python2.7 In-Reply-To: <20101027141519.5beaae26@pitrou.net> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> <20101027123622.0ab194f9@pitrou.net> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC37@exchcn.ccp.ad.local> <1288178142.3533.9.camel@localhost.localdomain> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC40@exchcn.ccp.ad.local> <20101027141519.5beaae26@pitrou.net> Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC47@exchcn.ccp.ad.local> Not cheap enough. There is no reason why creating another memoryview shouldn't be as cheap as creating a string slice. I'm investigating why. K -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Antoine Pitrou Sent: Wednesday, October 27, 2010 20:15 To: python-dev at python.org Subject: Re: [Python-Dev] new buffer in python2.7 On Wed, 27 Oct 2010 20:00:10 +0800 Kristj?n Valur J?nsson > wrote: > Calling getbuffer on a bytearray or a bytes object should be really > cheap, so I still don't accept this as a matter of fact situation. It *is* cheap. It's just that copying a short slice is dirt cheap as well. Of course, it you manipulate 1 KB slices or longer, it's different. Regards Antoine. _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Wed Oct 27 16:34:52 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 27 Oct 2010 10:34:52 -0400 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <20101026233710.6b313cf7@mission> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> <20101026215443.0c78fc29@pitrou.net> <20101026233710.6b313cf7@mission> Message-ID: <20101027143452.231052274EE@kimball.webabinitio.net> On Tue, 26 Oct 2010 23:37:10 -0400, Barry Warsaw wrote: > On Oct 26, 2010, at 09:54 PM, Antoine Pitrou wrote: > >I think it comes down to the preference of whoever works the most > >actively on it. Michael is the most active contributor to unittest by > >far, and I suppose he prefers it to be split into several submodules. > > And that seems perfectly reasonable to me, especially if the alternative were > that Michael was so frustrated with a massive single .py file that it > discouraged him from working on it. > > If done well, a split can help improve the readability and discoverability of > the code. I shudder at the madness that a single email.py file would cause. To put your mind at ease, Barry, I'd not want to do that either :) But by (IMO good) design Generator, FeedParser, and Message are all supposed to be independent (use only each other's public APIs). And Header can be (and is, I think) used without the other pieces of email, as is true for other of the helper modules (base64mime, quoprimime, etc). On the other hand, I have no clue why 'iterators.py' exists :) The one that bugs me most, though, is MIME. Combining all the mime stuff into one file seems like it would be a big win (not that it's possible, now). What is the benefit of email.mime.text.MIMEText over email.mime.MIMEText, when each of the files in the mime package consists of a single subclass? So, to clarify, like Raymond I'm not saying that packages are always bad. I'm just saying that packages are also not always good; and, further, that the number of lines of code in a file should, IMO, have nothing to do with the decision as to whether or not to create a package. -- R. David Murray www.bitdance.com From benjamin at python.org Wed Oct 27 17:05:23 2010 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 27 Oct 2010 10:05:23 -0500 Subject: [Python-Dev] MemoryError... how much memory? In-Reply-To: References: Message-ID: 2010/10/27 Facundo Batista : > There are a lot of places where Python or modules do something like: > > ? ?self->buf = (char *)malloc(size); > ? ?if (!self->buf) { > ? ? ? ? ? ? ?PyErr_SetString(PyExc_MemoryError, "out of memory"); > > At job, we're having some MemoryErrors, and one thing that we would > love to know, if how much memory it was asking when that happened. Isn't this usually when you do something like [None]*2**300? In that case, wouldn't you know how much memory you're requesting? Also, why is that useful? -- Regards, Benjamin From guido at python.org Wed Oct 27 17:29:58 2010 From: guido at python.org (Guido van Rossum) Date: Wed, 27 Oct 2010 08:29:58 -0700 Subject: [Python-Dev] Misc/python-mode.el (was Re: r85838 - python/branches/py3k/.gitignore) In-Reply-To: <20101027085551.19ab4495@mission> References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> <4CC7DAA1.6010800@scottdial.com> <20101027073546.7a28339b@mission> <4CC8210A.8020700@netwok.org> <20101027085551.19ab4495@mission> Message-ID: On Wed, Oct 27, 2010 at 5:55 AM, Barry Warsaw wrote: > On Oct 27, 2010, at 02:54 PM, ?ric Araujo wrote: > >>I remember a discussion about Vim files some time ago. ?I use Vim with >>the files from Misc/Vim, which are up-to-date and more useful than the >>files shipped with Vim. ?I don?t use plugins or external files, so I?m >>-1 on removing Misc/Vim. > > FWIW, Python support in Emacs is muddled. ?GNU Emacs comes with python.el > which is an alternative implementation that shares some features with, but > also has differences to, python-mode.el. ?Many of us hard-core Python Emacsers > prefer the latter, but sadly the FSF won't bundle it and a merge has proven > impossible so far. > > I think I'll replace Misc/python-mode.el with Misc/README.Emacs that attempts > to discuss the issue in a neutral way. ?I'll fix this in Python 2.7, 3.1, and > 3.2. Why "in a neutral way"? I don't see any problem with us recommending python-mode.el, since it comes from our community and it works better. I have recently had to go out of my way to make sure it is used in my Google setups. I can't live without pdbtrack. -- --Guido van Rossum (python.org/~guido) From barry at python.org Wed Oct 27 17:39:45 2010 From: barry at python.org (Barry Warsaw) Date: Wed, 27 Oct 2010 11:39:45 -0400 Subject: [Python-Dev] Misc/python-mode.el (was Re: r85838 - python/branches/py3k/.gitignore) In-Reply-To: References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> <4CC7DAA1.6010800@scottdial.com> <20101027073546.7a28339b@mission> <4CC8210A.8020700@netwok.org> <20101027085551.19ab4495@mission> Message-ID: <20101027113945.6a54a88f@mission> On Oct 27, 2010, at 08:29 AM, Guido van Rossum wrote: >Why "in a neutral way"? I don't see any problem with us recommending >python-mode.el, since it comes from our community and it works better. >I have recently had to go out of my way to make sure it is used in my >Google setups. I can't live without pdbtrack. pdbtrack does rock! (Hi Ken :). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From alexander.belopolsky at gmail.com Wed Oct 27 18:05:58 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 27 Oct 2010 12:05:58 -0400 Subject: [Python-Dev] [Python-ideas] Move Demo scripts under Lib In-Reply-To: References: Message-ID: I would like to report a conclusion reached on the tracker to a wider audience before committing the changes. The new home for Demo/turtle is Lib/turtledemo. (Lib/turtle/demo alternative received no support and Lib/demo/turtle was not even in the running.) If anyone is interested in reviewing the patch, please see http://bugs.python.org/issue10199. Note that I tried to limit changes to what was necessary for running the demo script as python -m demoturtle. Running the scripts as unit tests and from python prompt will be subject of a separate issue. On Tue, Oct 26, 2010 at 11:49 AM, Alexander Belopolsky wrote: > On Tue, Oct 26, 2010 at 11:18 AM, Guido van Rossum wrote: >> On Tue, Oct 26, 2010 at 8:13 AM, Alexander Belopolsky >> wrote: >>> The one demo that I want to find a better place for is Demo/turtle. >> >> Sure, go for it. It is a special case because the turtle module is >> also in the stdlib and these are intended for a particular novice >> audience. > > Please see http://bugs.python.org/issue10199 for further discussion. > From brett at python.org Wed Oct 27 18:19:13 2010 From: brett at python.org (Brett Cannon) Date: Wed, 27 Oct 2010 09:19:13 -0700 Subject: [Python-Dev] Misc/python-mode.el (was Re: r85838 - python/branches/py3k/.gitignore) In-Reply-To: <4CC8210A.8020700@netwok.org> References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> <4CC7DAA1.6010800@scottdial.com> <20101027073546.7a28339b@mission> <4CC8210A.8020700@netwok.org> Message-ID: On Wed, Oct 27, 2010 at 05:54, ?ric Araujo wrote: > I remember a discussion about Vim files some time ago. ?I use Vim with > the files from Misc/Vim, which are up-to-date and more useful than the > files shipped with Vim. ?I don?t use plugins or external files, so I?m > -1 on removing Misc/Vim. In the name of completeness for people not aware of the issue, http://bugs.python.org/issue9893 discusses actually removing these files in preference to files maintained by others. If Misc/Vim were to be dropped we could place a text file much like Barry is planning to do for Emacs and point to the files Antoine is recommending. From merwok at netwok.org Wed Oct 27 18:26:26 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Wed, 27 Oct 2010 18:26:26 +0200 Subject: [Python-Dev] Fixing zipfile.BadZipfile to zipfile.BadZipFile In-Reply-To: References: <4CBB63FA.8080108@netwok.org> Message-ID: <4CC852B2.1000900@netwok.org> [Reply received off-list quoted here] Le 18/10/2010 09:25, Bo?tjan Mejak a ?crit : > Shoot. Well, too bad. I thought Python is all about readability, but I think > you developers don't take it very seriously. Readability has to be balanced with other important things: maintainability, discoverability, compatibility, rapidity and other things ending in -ity. I?m personally glad that readability gets a huge place in the language definition itself (no braces, standard indentation), I follow 98 % of PEP 8 where I can, and have learned to accept that sometimes, a piece of code in the standard lib won?t get more readable. Still beats some languages that I won?t name because dissing Java is too easy. > If I was a developer, I would > certainly go to the trouble of all the rewrittes of > package/module/class/method/function names that do not comply to PEP 8 and > have them done by the time the first sub-version of Python 3 would be > released. You are free to break compat in your code but it python-dev has a duty to its users. Renames in threading have been deemed a good thing, renames in unittest not. You can read the python-3000 archives to get an idea of the years of work and thousands of messages that went into py3k. At some point, a release had to be done. Now that there is a stable release in the 3.x line, compatibility rules apply. Practicality beats purity; now is better than never. Regards From bostjan.mejak at gmail.com Wed Oct 27 20:15:02 2010 From: bostjan.mejak at gmail.com (=?UTF-8?Q?Bo=C5=A1tjan_Mejak?=) Date: Wed, 27 Oct 2010 20:15:02 +0200 Subject: [Python-Dev] Fwd: Fixing zipfile.BadZipfile to zipfile.BadZipFile In-Reply-To: <4CC85E56.3040208@netwok.org> References: <4CBB63FA.8080108@netwok.org> <4CC852B2.1000900@netwok.org> <4CC85E56.3040208@netwok.org> Message-ID: Forwarded conversation Subject: Fixing zipfile.BadZipfile to zipfile.BadZipFile ------------------------ From: *Bo?tjan Mejak* Date: Fri, Oct 15, 2010 at 11:02 PM To: python-dev at python.org I am very glad you're reorganizing the Standard Library. Thumbs up! I hope everything will comply to PEP 8 after you're done. Since you're reorganizing, I have my own contribution. I have attached a patch. The issue7351 was not accepted at the time, so I hope you'll accept this fix now. My point is that every class name in the module zipfile is like this: - LargeZipFile - ZipFile - PyZipFile . . . So apply my patch to make the class name BadZipfile consistent to other class names in the zipfile module and let it be named BadZipFile. Thank you. Best regards, Bo?tjan Mejak ---------- From: *?ric Araujo* Date: Sun, Oct 17, 2010 at 11:00 PM To: python-dev at python.org Cc: Bo?tjan Mejak Hello (A bit of context: The original message comes from bug #2775, ?Implement PEP 3108?, a meta-bug tracking stdlib reorganization for py3k.) You may have missed the timeline: Most of the PEP 3108 changes have been done before the first 3.x release went out. Now that we have 3.1 out as a stable and supported, we cannot reorganize and break compatibility anymore. (A note about PEP 8 compliance: Module names have been mostly fixed, but not all function/method names, for example in logging and unittest. If I recall correctly, readability did not seem to make all the rewrites worth it.) > a patch. The issue7351 was not I?ve just re-read the answers there and they are still valid. Ezio and me: ?Your patch need to include an alias (BadZipfile = BadZipFile) to preserve compatibility with old pickles, as explains msg95477.? Antoine: ?I don't think changing it for the sake of aesthetics is a good deal given that many existing programs will have to be converted to the new spelling.? Regards ---------- From: *Bo?tjan Mejak* Date: Mon, Oct 18, 2010 at 9:25 AM To: ?ric Araujo Shoot. Well, too bad. I thought Python is all about readability, but I think you developers don't take it very seriously. If I was a developer, I would certainly go to the trouble of all the rewrittes of package/module/class/method/function names that do not comply to PEP 8 and have them done by the time the first sub-version of Python 3 would be released. ---------- From: *?ric Araujo* Date: Mon, Oct 18, 2010 at 2:14 PM To: Bo?tjan Mejak Hi Could you repost your message to the mailing list, so that other people can see it and eventually react? Thanks. (P.S. please make sure not to top-post on the mailing list) Regards ---------- From: *?ric Araujo* Date: Mon, Oct 18, 2010 at 2:45 PM To: python-dev at python.org Hello [Sorry if this comes twice, connection errors here] > a patch. The issue7351 was not _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/bostjan.mejak%40gmail.com ---------- From: *Nick Coghlan* Date: Mon, Oct 18, 2010 at 3:13 PM To: ?ric Araujo Cc: python-dev at python.org Correct. We went through this for one module that I recall (threading) and that was annoying enough that we mostly left things alone after that unless they were truly obnoxious. For threading we were able to clean a lot of things up in the process (such as adding properties where appropriate), but even so, we still made sure all the old names continued to work. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia ---------- From: *Bo?tjan Mejak* Date: Mon, Oct 18, 2010 at 4:36 PM To: Nick Coghlan Then please make an alias for this custom BadZipfile exception class. Add BadZipfile = BadZipFile in the zipfile module. ---------- From: *Bo?tjan Mejak* Date: Tue, Oct 19, 2010 at 10:34 PM To: Nick Coghlan Sorry, let me correct myself: Add BadZipFile = BadZipfile in the zipfile module. Please add the above assignment statement in the right place to the zipfile module and create an alias. Thank you. ---------- From: *Bo?tjan Mejak* Date: Wed, Oct 20, 2010 at 11:13 PM To: Nick Coghlan Ah, leave it as is. If in the future you will be able to rename BadZipfile to BadZipFile, please do. ---------- From: *?ric Araujo* Date: Wed, Oct 27, 2010 at 6:26 PM To: python-dev Cc: Bo?tjan Mejak [Reply received off-list quoted here] Le 18/10/2010 09:25, Bo?tjan Mejak a ?crit : Readability has to be balanced with other important things: maintainability, discoverability, compatibility, rapidity and other things ending in -ity. I?m personally glad that readability gets a huge place in the language definition itself (no braces, standard indentation), I follow 98 % of PEP 8 where I can, and have learned to accept that sometimes, a piece of code in the standard lib won?t get more readable. Still beats some languages that I won?t name because dissing Java is too easy. You are free to break compat in your code but it python-dev has a duty to its users. Renames in threading have been deemed a good thing, renames in unittest not. You can read the python-3000 archives to get an idea of the years of work and thousands of messages that went into py3k. At some point, a release had to be done. Now that there is a stable release in the 3.x line, compatibility rules apply. Practicality beats purity; now is better than never. Regards ---------- From: *Bo?tjan Mejak* Date: Wed, Oct 27, 2010 at 7:13 PM To: ?ric Araujo Since Python 3.2 accepts feature requests, take this fix as a feature request. Please forget about preserving the compatibility with old pickles. ---------- From: *?ric Araujo* Date: Wed, Oct 27, 2010 at 7:16 PM To: Bo?tjan Mejak Again, please send a copy to your messages to the mailing list (choose ?reply to all? or ?reply to list?), and don?t leave the whole message after your reply. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: zipfile-patch.diff Type: application/octet-stream Size: 3786 bytes Desc: not available URL: From merwok at netwok.org Wed Oct 27 20:59:14 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Wed, 27 Oct 2010 20:59:14 +0200 Subject: [Python-Dev] Fixing zipfile.BadZipfile to zipfile.BadZipFile In-Reply-To: References: <4CBB63FA.8080108@netwok.org> <4CC852B2.1000900@netwok.org> <4CC85E56.3040208@netwok.org> Message-ID: <4CC87682.2070405@netwok.org> > From: *Bo?tjan Mejak* > Since Python 3.2 accepts feature requests, take this fix as a feature > request. Please forget about preserving the compatibility with old pickles. You can reopen #7351 as a feature request for 3.2. A serious proposal has to take compatibility into account, note. It is simply not possible to break everyone?s pickles for a very minor inconsistency in one name. There is a wide Python world out there. I?d like to suggest again that the time you want to give to Python (thanks!) would be better employed on something else than BadZipfile. For example, if you don?t think you can contribute code now, you could suggest improvements for unclear sections in the documentation. Recent reports sent by beginners can be found on the archives of the docs at python.org mailing list. Regards From facundobatista at gmail.com Wed Oct 27 21:09:37 2010 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 27 Oct 2010 16:09:37 -0300 Subject: [Python-Dev] MemoryError... how much memory? In-Reply-To: References: Message-ID: On Wed, Oct 27, 2010 at 12:05 PM, Benjamin Peterson wrote: > Isn't this usually when you do something like [None]*2**300? In that > case, wouldn't you know how much memory you're requesting? It could happen on any malloc. It depends on how much you have free. Don't think on getting a MemoryError on a python you just opened in the console. Think about a server with a month of uptime, where you have all the memory fragmented, etc. > Also, why is that useful? It helps to determine why we're having some Memory Errors on our long-lived server, how is the behaviour when that happens, etc. Regards, -- .? ? Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From exarkun at twistedmatrix.com Wed Oct 27 23:08:53 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 27 Oct 2010 21:08:53 -0000 Subject: [Python-Dev] MemoryError... how much memory? In-Reply-To: References: Message-ID: <20101027210853.2040.1837327595.divmod.xquotient.50@localhost.localdomain> On 07:09 pm, facundobatista at gmail.com wrote: >On Wed, Oct 27, 2010 at 12:05 PM, Benjamin Peterson > wrote: >>Isn't this usually when you do something like [None]*2**300? In that >>case, wouldn't you know how much memory you're requesting? > >It could happen on any malloc. It depends on how much you have free. > >Don't think on getting a MemoryError on a python you just opened in >the console. Think about a server with a month of uptime, where you >have all the memory fragmented, etc. >>Also, why is that useful? > >It helps to determine why we're having some Memory Errors on our >long-lived server, how is the behaviour when that happens, etc. But... If you allocated all of your memory to some garbage, and then a 5 byte string can't be allocated, you don't really care about the 5 byte string, you care about the garbage that's wasting your memory. Tools like heapy will give you a lot of information. Maybe it wouldn't hurt anyone to have more information in a MemoryError. But I don't think it's going to help a lot either. It's not the information that you're really interested in. Jean-Paul From ben+python at benfinney.id.au Thu Oct 28 00:00:32 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 28 Oct 2010 09:00:32 +1100 Subject: [Python-Dev] MemoryError... how much memory? References: Message-ID: <87bp6fmh33.fsf@benfinney.id.au> Facundo Batista writes: > On Wed, Oct 27, 2010 at 12:05 PM, Benjamin Peterson wrote: > > > Isn't this usually when you do something like [None]*2**300? In that > > case, wouldn't you know how much memory you're requesting? > > It could happen on any malloc. It depends on how much you have free. It also depends on how much is being requested. The caller knows that amount, surely? -- \ ?If you do not trust the source do not use this program.? | `\ ?Microsoft Vista security dialogue | _o__) | Ben Finney From ncoghlan at gmail.com Thu Oct 28 01:27:15 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 28 Oct 2010 09:27:15 +1000 Subject: [Python-Dev] MemoryError... how much memory? In-Reply-To: <87bp6fmh33.fsf@benfinney.id.au> References: <87bp6fmh33.fsf@benfinney.id.au> Message-ID: On Thu, Oct 28, 2010 at 8:00 AM, Ben Finney wrote: > Facundo Batista writes: > >> On Wed, Oct 27, 2010 at 12:05 PM, Benjamin Peterson wrote: >> >> > Isn't this usually when you do something like [None]*2**300? In that >> > case, wouldn't you know how much memory you're requesting? >> >> It could happen on any malloc. It depends on how much you have free. > > It also depends on how much is being requested. The caller knows that > amount, surely? For a server process, the MemoryError in the log won't always have the context information showing what the values were in the calling frames. The idea behind Facundo's request is similar to the reason why we print the type names in a lot of TypeErrors. If you see MemoryError (5 bytes), the things you go looking for are very different from those you look for when you see MemoryError(1 gajillion bytes). (i.e. for the former, you look for a memory or other resource leak, for the latter, you look for the reason your code is trying to get 1 gajillion bytes from the OS). If a long-lived server isn't crashing but is still getting MemoryError occasionally, problems with specific oversized requests are much more likely than a general resource leak (as those usually bring the whole process down eventually). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From kristjan at ccpgames.com Thu Oct 28 04:22:51 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Thu, 28 Oct 2010 10:22:51 +0800 Subject: [Python-Dev] Continuing 2.x Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Hello all. So, python 2.7 is in bugfix only mode. 'trunk' is off limit. So, where does one make improvements to the distinguished, and still very much alive, 2.x series of Python? The answer would seem to be "one doesn't". But must it be that way? When Morris stopped producing the Oxford III model back in '57 in favor of new developments, it didn't spell the end for it. The plant was sold to India and the Hindustan Ambassador continues to be developed and produced to this day. It even has fuel injection. The Morris Motor Company isn't around anymore. So, here is my suggestion: Let's move the current 'trunk' into /branches/afterlife-27. Open it for submissions from people such as myself that use 2.7 on a regular basis and are willing to give it some extra love. Host it there without the usual stringent python quality assurance, buildbot support, release management and all that rigmarole. Open-source it, if you will. Svn.python.org already plays host to some other, less official, projects such as stackless, so why not this? What do you think? Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.curtin at gmail.com Thu Oct 28 04:44:51 2010 From: brian.curtin at gmail.com (Brian Curtin) Date: Wed, 27 Oct 2010 21:44:51 -0500 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Message-ID: 2010/10/27 Kristj?n Valur J?nsson > > So, here is my suggestion: > > Let?s move the current ?trunk? into /branches/afterlife-27. Open it for > submissions from people such as myself that use 2.7 on a regular basis and > are willing to give it some extra love. Host it there without the usual > stringent python quality assurance, buildbot support, release management and > all that rigmarole. > Without that rigmarole I wouldn't consider using such a product. I also don't know who would, and I think you'll have a hard time marketing a branch that doesn't receive the attention that the others do. I think it would be bad to associate an untested, free-for-all version of Python with python.org. -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel at stutzbachenterprises.com Thu Oct 28 05:44:45 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 27 Oct 2010 20:44:45 -0700 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Message-ID: 2010/10/27 Kristj?n Valur J?nsson > Svn.python.org already plays host to some other, less official, projects > such as stackless, so why not this? > What are the benefits of hosting such a project on svn.python.org instead of somewhere else? (such as GitHub or BitBucket) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From foom at fuhm.net Thu Oct 28 05:12:09 2010 From: foom at fuhm.net (James Y Knight) Date: Wed, 27 Oct 2010 23:12:09 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Message-ID: On Oct 27, 2010, at 10:22 PM, Kristj?n Valur J?nsson wrote: > Hello all. > > So, python 2.7 is in bugfix only mode. ?trunk? is off limit. So, where does one make improvements to the distinguished, and still very much alive, 2.x series of Python? > The answer would seem to be ?one doesn?t?. But must it be that way? > > When Morris stopped producing the Oxford III model back in ?57 in favor of new developments, it didn?t spell the end for it. The plant was sold to India and the Hindustan Ambassador continues to be developed and produced to this day. It even has fuel injection. > The Morris Motor Company isn?t around anymore. > > So, here is my suggestion: > Let?s move the current ?trunk? into /branches/afterlife-27. Open it for submissions from people such as myself that use 2.7 on a regular basis and are willing to give it some extra love. Host it there without the usual stringent python quality assurance, buildbot support, release management and all that rigmarole. Open-source it, if you will. > Svn.python.org already plays host to some other, less official, projects such as stackless, so why not this? The python community has already decided many times over that Python2 is dead and Python3 is the future. So if you want to continue maintaining Python2, that means you need to fork it. I think you'd be best off doing so on your own infrastructure: convincing the python developers to support such a thing is quite unlikely, and furthermore, completely unnecessary. Unlike the Oxford III, you don't need to be "sold" python2: it's open source, you can fork it without any official approval. So, just do it. I wish you best of luck, though: most unofficial forks die a lonely death. But, if enough people feel like you do, it could become successful. But I really doubt anyone else is going to want to use it any python2 afterlife without stringent quality assurance, multi-platform support releases, and other rigamarole. You'd have to set up all that stuff for yourself if you possibly hope to attract users. I can't think of any possible use for an unreliably maintained version of python2... James From kristjan at ccpgames.com Thu Oct 28 06:00:56 2010 From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=) Date: Thu, 28 Oct 2010 12:00:56 +0800 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> Firstly, the ease of integrating changes. It would be possible to port those bugfixes that release-27 gets, and also backport selected things from py3k using the tools already in place such as svnmerge. Second, it would be an official nod from the python community that, yes, we are not actively developing 2.x anymore, we want to focus on 3.x but we acknowledge that there are members of our community that cannot, for various reasons, move to 3.x, but still want to be able to improve their platform and share those improvements with others. James Y Knight said: The python community has already decided many times over that Python2 is dead and Python3 is the future But the patient is very much alive and kicking, no matter what the good doctor declares. Python 2.x is in widespread use, with gazillion lines of .py code. In, there is another gazillion lines of .c and .cpp code both in extensions and embedding applications in use. I?m quite happy with the community at large moving its development focus to 3.x but it is a bit harsh to deprive those left behind of the keys to the old house. Cheers, K From: Daniel Stutzbach [mailto:daniel at stutzbachenterprises.com] Sent: Thursday, October 28, 2010 11:45 To: Kristj?n Valur J?nsson Cc: Python-Dev (python-dev at python.org) Subject: Re: [Python-Dev] Continuing 2.x already plays host to some other, less official, projects such as stackless, so why not this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Thu Oct 28 06:05:37 2010 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 27 Oct 2010 23:05:37 -0500 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> Message-ID: 2010/10/27 Kristj?n Valur J?nsson : > Firstly, the ease of integrating changes.? It would be possible to port > those bugfixes that release-27 gets, and also backport selected things from > py3k using the tools already in place such as svnmerge. svn lets you merge across repos, I believe. > > Second, it would be an official nod How would it "official" if you were the only one maintaining it? -- Regards, Benjamin From daniel at stutzbachenterprises.com Thu Oct 28 06:13:52 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 27 Oct 2010 21:13:52 -0700 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> Message-ID: 2010/10/27 Kristj?n Valur J?nsson > Firstly, the ease of integrating changes. It would be possible to port > those bugfixes that release-27 gets, and also backport selected things from > py3k using the tools already in place such as svnmerge. > py3k will soon be moving to Mercurial, so svnmerge would not be helpful for much longer. On the plus side, since Mercurial is a Distributed Version Control System, if you setup an unofficial continuation of Python 2 on the host of your choice, it will be easy for you to pull patches from py3k. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Thu Oct 28 07:30:40 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 28 Oct 2010 14:30:40 +0900 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> Message-ID: <8762wmdgu7.fsf@uwakimon.sk.tsukuba.ac.jp> Kristj?n Valur J?nsson writes: > Second, it would be an official nod from the python community that, > yes, we are not actively developing 2.x anymore, we want to focus > on 3.x but we acknowledge that there are members of our community > that cannot, for various reasons, move to 3.x, but still want to be > able to improve their platform and share those improvements with > others. I don't see an insuperable problem in principle with hosting it on python.org (for the practical reasons you give, it would be helpful), but I think the repository belongs in /projects/v2-afterlife, not in /projects/python. -------------- next part -------------- > I?m quite happy with the community at large moving its development > focus to 3.x but it is a bit harsh to deprive those left behind of > the keys to the old house. If you mean the current set of 2.x branches in the repository, no, that's not an "old" house, that's the same house we still live in. It contains the official history of each 2.x branch. The continuing development you propose is unofficial, and doesn't belong there. From kristjan at ccpgames.com Thu Oct 28 08:27:55 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Thu, 28 Oct 2010 14:27:55 +0800 Subject: [Python-Dev] http://bugs.python.org/issue9981 Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FCAF@exchcn.ccp.ad.local> I didn't get any response to this. I know that it is a 2.7 patch, but I?m willing to port it to 3.x, if there is interest. Supporting parallel compilation is a big win. K -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Thu Oct 28 08:48:43 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 28 Oct 2010 08:48:43 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> Message-ID: Am 28.10.2010 06:13, schrieb Daniel Stutzbach: > 2010/10/27 Kristj?n Valur J?nsson > > > Firstly, the ease of integrating changes. It would be possible to port > those bugfixes that release-27 gets, and also backport selected things from > py3k using the tools already in place such as svnmerge. > > > py3k will soon be moving to Mercurial, so svnmerge would not be helpful for much > longer. On the plus side, since Mercurial is a Distributed Version Control > System, if you setup an unofficial continuation of Python 2 on the host of your > choice, it will be easy for you to pull patches from py3k. I believe we'll eventually have the ability to create user repos as well, so that Kristjan can simply put his branch into one of these and still have it on hg.python.org. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From dirkjan at ochtman.nl Thu Oct 28 09:32:15 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 28 Oct 2010 09:32:15 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> Message-ID: On Thu, Oct 28, 2010 at 08:48, Georg Brandl wrote: > I believe we'll eventually have the ability to create user repos as well, so > that Kristjan can simply put his branch into one of these and still have it > on hg.python.org. +1. Cheers, Dirkjan From hodgestar+pythondev at gmail.com Thu Oct 28 10:26:28 2010 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Thu, 28 Oct 2010 10:26:28 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> Message-ID: 2010/10/28 Kristj?n Valur J?nsson : > But the patient is very much alive and kicking, no matter what the good > doctor declares. No no! 'E's pining! Schiavo Simon From sable at users.sourceforge.net Thu Oct 28 11:01:12 2010 From: sable at users.sourceforge.net (=?UTF-8?B?U8OpYmFzdGllbiBTYWJsw6k=?=) Date: Thu, 28 Oct 2010 11:01:12 +0200 Subject: [Python-Dev] Buildbot for AIX In-Reply-To: <4CB87587.9020100@users.sourceforge.net> References: <1284586127.79.0.767793857364.issue1633863@psf.upfronthosting.co.za> <4C93377C.4040300@users.sourceforge.net> <4C936225.1000500@v.loewis.de> <4C9770FB.1010103@users.sourceforge.net> <4C97B422.4040606@v.loewis.de> <4CB87587.9020100@users.sourceforge.net> Message-ID: <4CC93BD8.1020702@users.sourceforge.net> Hi Martin, I got no reply concerning those modifications to the buildbot script so that I could run 2 slaves on AIX. I am able to spend some time at the moment on Python for AIX because we are migrating the product in my company from AIX 5.3 to AIX 6.1. However once this is done, by the end of November, it will be more difficult for me to justify spending too much time on this builbot installation and on Python AIX bugs. So it would be nice if I could get this slave running as soon as possible, so that I could spend a few more days solving some of the remaining issues on this platform. regards -- S?bastien Sabl? Le 15/10/2010 17:38, S?bastien Sabl? a ?crit : > Hi Martin, > > I finally got the authorization to run some buildbot slaves on our AIX > servers. > > I made some tests with a buildbot script as close as possible to what > you already use. Here is a patch that show the kind of modifications I > had to do in order to get the buildbot slave to correctly run on AIX. > > It is also necessary to apply the patch provided in issue 9862 so that > the tests won't hang forever. > > It also would help if the corrections provided in the following issues > could be commited: 4499, 678250, 730467. > > Could you please take a look at those modifications in master.cfg, > provide me some password for the bot slaves and apply the corrections in > those issues? > > Once this is done, I can run 2 buildbot slaves for AIX 6.1 and AIX 5.3 > with build plans for gcc and xlc. > > I can't guarantee that those bots will run forever, and I may have to > ask to schedule them only at night if the activity it generates on the > server appears to be too high. > But I think it could greatly help to improve the state of Python on AIX > (which is far from perfect at the moment). > > regards > > -- > S?bastien Sabl? > > > > Le 20/09/2010 21:21, "Martin v. L?wis" a ?crit : >>> Also could you provide me the master.cfg file (with obfuscated >>> passwords) that is used by the Python buildbot master or tell me if it >>> is in subversion somewhere? >> >> Attached! >> >> Regards, >> Martin > From victor.stinner at haypocalc.com Thu Oct 28 11:38:20 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 28 Oct 2010 11:38:20 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Message-ID: <201010281138.21065.victor.stinner@haypocalc.com> Le jeudi 28 octobre 2010 05:12:09, James Y Knight a ?crit : > The python community has already decided many times over that Python2 is > dead and Python3 is the future. ... I think you'd be best off doing > so on your own infrastructure: convincing the python developers to support > such a thing is quite unlikely, and furthermore, completely unnecessary. *I* don't really care to Python 2.x anymore: I consider Python 2.7 as mature and very stable. New features can still be developed as Python or C extensions (browse the Python package index to get some examples). I don't want to touch the Python2 core (the interpreter or the standard library) because it is too expensive (in time). I prefer to focus on Python3 because Python3 core has a better design: strict separation between bytes and characters, no more short integer type, no more old style class, etc. It's easier to work on Python3 core. Backport a patch from Python3 to Python2 takes between 10 minutes and 3 hours (or maybe more on complex patches) because the function names, C macros, even file names, (...), has changed. And I don't know automatic tools to convert a Python3 patch to a Python2 patch (eg. a "3to2" tool, for patches). I don't want to spend 3 hours or more on a "dead project". But when I find a bug in Python3, I immediatly check if it does also exist in Python2. And if both version are affected, I try to fix all versions (if it doesn't break the API). -- Victor Stinner http://www.haypocalc.com/ From solipsis at pitrou.net Thu Oct 28 12:55:06 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 28 Oct 2010 12:55:06 +0200 Subject: [Python-Dev] Continuing 2.x References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> Message-ID: <20101028125506.11d9ccd6@pitrou.net> On Wed, 27 Oct 2010 23:05:37 -0500 Benjamin Peterson wrote: > 2010/10/27 Kristj?n Valur J?nsson : > > Firstly, the ease of integrating changes.? It would be possible to port > > those bugfixes that release-27 gets, and also backport selected things from > > py3k using the tools already in place such as svnmerge. > > svn lets you merge across repos, I believe. And, most of all, we're soon switching to hg (real soon now :-)). Regards Antoine. From facundobatista at gmail.com Thu Oct 28 13:14:54 2010 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 28 Oct 2010 08:14:54 -0300 Subject: [Python-Dev] MemoryError... how much memory? In-Reply-To: References: <87bp6fmh33.fsf@benfinney.id.au> Message-ID: On Wed, Oct 27, 2010 at 8:27 PM, Nick Coghlan wrote: > If you see MemoryError (5 bytes), the things you go looking for are > very different from those you look for when you see MemoryError(1 > gajillion bytes). (i.e. for the former, you look for a memory or other > resource leak, for the latter, you look for the reason your code is > trying to get 1 gajillion bytes from the OS). If a long-lived server > isn't crashing but is still getting MemoryError occasionally, problems > with specific oversized requests are much more likely than a general > resource leak (as those usually bring the whole process down > eventually). Very well explained, you're all right. Furthermore, our server is fairly complex: we're using quite some libraries to do different jobs, and one of the approaches (not the only one) that we're taking to deal with this beast is to analyze its memory-related behaviour from an external POV (thinking it as a black box). So, beyond it's arguable utility, do you think that having that information could harm us in some way? Regards, -- .? ? Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From ncoghlan at gmail.com Thu Oct 28 14:58:36 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 28 Oct 2010 22:58:36 +1000 Subject: [Python-Dev] MemoryError... how much memory? In-Reply-To: References: <87bp6fmh33.fsf@benfinney.id.au> Message-ID: On Thu, Oct 28, 2010 at 9:14 PM, Facundo Batista wrote: > So, beyond it's arguable utility, do you think that having that > information could harm us in some way? I think the idea is sound in principle, but may run into some practical implementation problems due to special cases when raising MemoryError. But creating a patch and putting on the tracker sounds like something worth trying. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From martin at v.loewis.de Thu Oct 28 15:03:26 2010 From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 28 Oct 2010 15:03:26 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Message-ID: <4CC9749E.1030200@v.loewis.de> > Let?s move the current ?trunk? into /branches/afterlife-27. Open it for > submissions from people such as myself that use 2.7 on a regular basis > and are willing to give it some extra love. Host it there without the > usual stringent python quality assurance, buildbot support, release > management and all that rigmarole. Open-source it, if you will. If you don't plan to make a release eventually, why would anybody care? If you do plan to make a release eventually, please say so. That would then be the point where I can point out that I will not be available to make Windows binaries for such a release. Also, if you do plan to make a release, please also indicate how you would label it. Regards, Martin From martin at v.loewis.de Thu Oct 28 15:14:22 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 28 Oct 2010 15:14:22 +0200 Subject: [Python-Dev] MemoryError... how much memory? In-Reply-To: References: <87bp6fmh33.fsf@benfinney.id.au> Message-ID: <4CC9772E.8030804@v.loewis.de> > Furthermore, our server is fairly complex: we're using quite some > libraries to do different jobs, and one of the approaches (not the > only one) that we're taking to deal with this beast is to analyze its > memory-related behaviour from an external POV (thinking it as a black > box). > > So, beyond it's arguable utility, do you think that having that > information could harm us in some way? I think implementing it might do harm. When a memory error is raised, you are typically out of memory, so allocating more memory might fail (it just did). Therefore, allocating more objects or doing string formatting will likely fail (unless the requested size is much larger than the memory required for these operations). So the chance increases that you trigger a fatal error. Regards, Martin From ncoghlan at gmail.com Thu Oct 28 15:33:51 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 28 Oct 2010 23:33:51 +1000 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Message-ID: 2010/10/28 Kristj?n Valur J?nsson : > Hello all. > So, python 2.7 is in bugfix only mode.? ?trunk? is off limit.? So, where > does one make improvements to the distinguished, and still very much alive, > 2.x series of Python? > > The answer would seem to be ?one doesn?t?. ?But must it be that way? When python-dev chose to switch our own focus for new features to 3.x only, we were quite aware that a new group forming to continue with 2.8 was a definite possibility. If you do decide to go ahead with the idea, I have a few suggestions: 1. Since the distinguishing feature is that this branch is a 2.x version that will accept new features, in contrast to the python-dev maintained bugfix-only 2.7 maintenance branch, please call the branch something-or-other-2.8, rather than any form of 2.7. 2. Check with Benjamin how he plans to handle 2.7 maintenance releases. If he plans to release from SVN, use that as your upstream master. If 2.7.1 will instead be released from hg.python.org, wait until the switch happens then use the relevant hg branch as the upstream. 3. Choose your target audience early (if the target is only developers with existing 2.x installations that can build their own version of Python, then that simplifies release management significantly, since you don't need to build binaries any more). 4. Decide whether or not you need a buildbot farm (this relates to point 3: you may choose to limit your audience to people that are happy to run the test suite themselves on their own target platforms of interest). 5. Give some thought to how you will handle controversial design decisions (since you won't have the fallback of appealing to the BDFL, and feedback from python-dev is likely to be limited). 6. Asking for a python-org SIG mailing list may be a reasonable idea as well (e.g. py2x-sig) 7. As 3.x usage grows, such a group may have a vested interest in helping with 3to2 development as well as simplifying backporting of 3.x extension modules to 2.x. A 2.8 branch that sells itself as being suitable only for people willing to run their own builds and QA could definitely have a place in the world (CCP at least would obviously find it useful, but I wouldn't be surprised to find other companies that might consider adopting such a branch if the benefits of the new features over the official 2.7 releases were sufficiently compelling). I don't believe anyone here is implacably opposed to the idea of 2.8 development continuing - it's just that the "collective we" of python-dev isn't interested in making it happen, so a new crop of developers will need to step up if it is going to become more than a CCP-specific 2.x fork. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Thu Oct 28 15:39:12 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 28 Oct 2010 23:39:12 +1000 Subject: [Python-Dev] MemoryError... how much memory? In-Reply-To: <4CC9772E.8030804@v.loewis.de> References: <87bp6fmh33.fsf@benfinney.id.au> <4CC9772E.8030804@v.loewis.de> Message-ID: On Thu, Oct 28, 2010 at 11:14 PM, "Martin v. L?wis" wrote: >> Furthermore, our server is fairly complex: we're using quite some >> libraries to do different jobs, and one of the approaches (not the >> only one) that we're taking to deal with this beast is to analyze its >> memory-related behaviour from an external POV (thinking it as a black >> box). >> >> So, beyond it's arguable utility, do you think that having that >> information could harm us in some way? > > I think implementing it might do harm. When a memory error is raised, > you are typically out of memory, so allocating more memory might fail > (it just did). Therefore, allocating more objects or doing string > formatting will likely fail (unless the requested size is much larger > than the memory required for these operations). > > So the chance increases that you trigger a fatal error. What Martin describes here is a more explicit description of what I meant by "practical implementation problems" and "special cases when raising MemoryError". However, I think thresholding the additional error formatting to only kick in the requested amount of memory exceeds a certain size would be an adequate safeguard without reducing the utility in Facundo's use case (the pre-allocated instance can have a generic error message saying an allocation of less than the threshold value failed). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From g.brandl at gmx.net Thu Oct 28 15:54:50 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 28 Oct 2010 15:54:50 +0200 Subject: [Python-Dev] MemoryError... how much memory? In-Reply-To: <4CC9772E.8030804@v.loewis.de> References: <87bp6fmh33.fsf@benfinney.id.au> <4CC9772E.8030804@v.loewis.de> Message-ID: Am 28.10.2010 15:14, schrieb "Martin v. L?wis": >> Furthermore, our server is fairly complex: we're using quite some >> libraries to do different jobs, and one of the approaches (not the >> only one) that we're taking to deal with this beast is to analyze its >> memory-related behaviour from an external POV (thinking it as a black >> box). >> >> So, beyond it's arguable utility, do you think that having that >> information could harm us in some way? > > I think implementing it might do harm. When a memory error is raised, > you are typically out of memory, so allocating more memory might fail > (it just did). Therefore, allocating more objects or doing string > formatting will likely fail (unless the requested size is much larger > than the memory required for these operations). > > So the chance increases that you trigger a fatal error. Especially since we have a MemoryError instance preallocated to avoid exactly this problem. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Thu Oct 28 16:19:34 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 28 Oct 2010 16:19:34 +0200 Subject: [Python-Dev] MemoryError... how much memory? References: <87bp6fmh33.fsf@benfinney.id.au> <4CC9772E.8030804@v.loewis.de> Message-ID: <20101028161934.3de73f11@pitrou.net> On Thu, 28 Oct 2010 15:54:50 +0200 Georg Brandl wrote: > Am 28.10.2010 15:14, schrieb "Martin v. L?wis": > >> Furthermore, our server is fairly complex: we're using quite some > >> libraries to do different jobs, and one of the approaches (not the > >> only one) that we're taking to deal with this beast is to analyze its > >> memory-related behaviour from an external POV (thinking it as a black > >> box). > >> > >> So, beyond it's arguable utility, do you think that having that > >> information could harm us in some way? > > > > I think implementing it might do harm. When a memory error is raised, > > you are typically out of memory, so allocating more memory might fail > > (it just did). Therefore, allocating more objects or doing string > > formatting will likely fail (unless the requested size is much larger > > than the memory required for these operations). > > > > So the chance increases that you trigger a fatal error. > > Especially since we have a MemoryError instance preallocated to avoid > exactly this problem. And which creates other problems of its own, such as keeping many objects alive: http://bugs.python.org/issue5437 ;) Antoine. From martin at v.loewis.de Thu Oct 28 16:26:58 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 28 Oct 2010 16:26:58 +0200 Subject: [Python-Dev] Buildbot for AIX In-Reply-To: <4CC93BD8.1020702@users.sourceforge.net> References: <1284586127.79.0.767793857364.issue1633863@psf.upfronthosting.co.za> <4C93377C.4040300@users.sourceforge.net> <4C936225.1000500@v.loewis.de> <4C9770FB.1010103@users.sourceforge.net> <4C97B422.4040606@v.loewis.de> <4CB87587.9020100@users.sourceforge.net> <4CC93BD8.1020702@users.sourceforge.net> Message-ID: <4CC98832.9000500@v.loewis.de> > I got no reply concerning those modifications to the buildbot script so > that I could run 2 slaves on AIX. I'm really really reluctant here. The proposed/requested changes are fairly intrusive, even though AIX is just a minority platform. So I need to find some time to rewrite them, which I haven't been able to, yet. Regards, Martin From solipsis at pitrou.net Thu Oct 28 16:45:15 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 28 Oct 2010 16:45:15 +0200 Subject: [Python-Dev] Buildbot for AIX References: <1284586127.79.0.767793857364.issue1633863@psf.upfronthosting.co.za> <4C93377C.4040300@users.sourceforge.net> <4C936225.1000500@v.loewis.de> <4C9770FB.1010103@users.sourceforge.net> <4C97B422.4040606@v.loewis.de> <4CB87587.9020100@users.sourceforge.net> Message-ID: <20101028164515.33ac8412@pitrou.net> On Fri, 15 Oct 2010 17:38:47 +0200 S?bastien Sabl? wrote: > > Could you please take a look at those modifications in master.cfg, > provide me some password for the bot slaves and apply the corrections in > those issues? About the master.cfg modifications: there should be no need for separate environment variables. Instead, you should be able to specify them as command-line arguments to ./configure, e.g.: ["--with-pydebug", "--without-computed-gotos", "CC=xlc", 'OPT="-O2 -qmaxmem=18000"'] Can you check this works for you? Also, there's no need to complicate the buildbot naming procedure. You should be able to run several buildslaves on a single machine, provided we give you separate credentials: one per compiler type. Regards Antoine. From barry at python.org Thu Oct 28 18:04:14 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Oct 2010 12:04:14 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <4CC9749E.1030200@v.loewis.de> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> Message-ID: <20101028120414.1f4f3016@mission> Who is the target audience for a Python 2.8? What exactly would a Python 2.8 accomplish? If Python 2.8 doesn't include new features, well, then what's the point? Python 2.7 will be bug fix maintained for a long time, longer in fact than previous Python 2 versions. So a no-feature Python 2.8 can't be about improving Python 2 stability over time (i.e. just fix the bug in Python 2.7). If Python 2.8 is about adding new features, then it has to be about backporting those features from Python 3. Adding new feature only to a Python 2.8 *isn't* Python, it's a fork of Python. Of course, it's open source and you're always allowed to do that, but you would need to be clear that this isn't "Python". IOW, a distro like Ubuntu would likely never package such a thing as "/usr/bin/python". What specific features that are showing up in say Python 3.2 are so compelling that they need to show up in Python 2.8, *and* would compel folks who are pinned to Python 2 to spend the resources to support it? Porting and certifying a code base against a new Python version is always work, sometimes a significant amount of work. What would be so compelling about a Python 2.8 that users, package authors, and distros would be willing to undertake this work? I'd *much* rather this enthusiasm be spent on making Python 3 rock, and in porting third party code to Python 3. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From lutz at rmi.net Thu Oct 28 18:07:50 2010 From: lutz at rmi.net (lutz at rmi.net) Date: Thu, 28 Oct 2010 16:07:50 -0000 Subject: [Python-Dev] Continuing 2.x Message-ID: Kristj?n Valur J?nsson writes: > James Y Knight said: > The python community has already decided many times over that Python2 is dead > and Python3 is the future > > But the patient is very much alive and kicking, no matter what the good doctor > declares. Python 2.x is in widespread use, with gazillion lines of .py code. > In, there is another gazillion lines of .c and .cpp code both in extensions and > embedding applications in use. I?m quite happy with the community at large > moving its development focus to 3.x but it is a bit harsh to deprive those left > behind of the keys to the old house. Exactly. Has anyone here analyzed download stats on py.org lately? Please feel free to prove me wrong, but by my reckoning, and at least for Windows MSI installer files, people are still downloading Python 2.X roughly 3 to 4 times more often than Python 3.X today, some 2 years after 3.X's release. This is from http://www.python.org/webstats for September and October, based on file size and bytes fetched for all significant versions. As one metric, roughly 439K people fetched 2.X MSI files in September, and 124K went for 3.X. Granted, there are plenty of variables such as preinstalled Pythons on Macs and Linux, though many would tend to skew 2.X dominance even higher. Moreover, downloads may be more reflective of new users, than existing users who are likely in the 2.X camp. But clearly, the 2.X patient is hardly dead; it still reflects the vast majority of the Python world today. I hope 3.X use expands; in fact, I've bet the future of at least one book on it. And even 1/4 of new users seems a large enough subset to care about too. But one can't help but wonder if most of the development community is focused on some imaginary future user base, at the expense of the much larger current user base. Then again, there's still plenty of Fortran77 code out there, so... --Mark Lutz (http://learning-python.com, http://rmi.net/~lutz) From exarkun at twistedmatrix.com Thu Oct 28 18:17:45 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 28 Oct 2010 16:17:45 -0000 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101028120414.1f4f3016@mission> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> Message-ID: <20101028161745.2040.1157174389.divmod.xquotient.52@localhost.localdomain> On 04:04 pm, barry at python.org wrote: > >I'd *much* rather this enthusiasm be spent on making Python 3 rock, and >in >porting third party code to Python 3. Enthusiasm isn't fungible. Jean-Paul From barry at python.org Thu Oct 28 18:31:32 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Oct 2010 12:31:32 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101028161745.2040.1157174389.divmod.xquotient.52@localhost.localdomain> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <20101028161745.2040.1157174389.divmod.xquotient.52@localhost.localdomain> Message-ID: <20101028123132.3b5e8c4f@mission> On Oct 28, 2010, at 04:17 PM, exarkun at twistedmatrix.com wrote: >On 04:04 pm, barry at python.org wrote: >> >>I'd *much* rather this enthusiasm be spent on making Python 3 rock, and >in >>porting third party code to Python 3. > >Enthusiasm isn't fungible. Maybe so, but I think it's actually more fun to be working on something other people will actually use. ;) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From g.brandl at gmx.net Thu Oct 28 18:32:55 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 28 Oct 2010 18:32:55 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: Message-ID: Am 28.10.2010 18:07, schrieb lutz at rmi.net: > Kristj?n Valur J?nsson writes: >> James Y Knight said: >> The python community has already decided many times over that Python2 is dead >> and Python3 is the future >> >> But the patient is very much alive and kicking, no matter what the good doctor >> declares. Python 2.x is in widespread use, with gazillion lines of .py code. >> In, there is another gazillion lines of .c and .cpp code both in extensions and >> embedding applications in use. I?m quite happy with the community at large >> moving its development focus to 3.x but it is a bit harsh to deprive those left >> behind of the keys to the old house. > > Exactly. > > Has anyone here analyzed download stats on py.org lately? > Please feel free to prove me wrong, but by my reckoning, > and at least for Windows MSI installer files, people are > still downloading Python 2.X roughly 3 to 4 times more often > than Python 3.X today, some 2 years after 3.X's release. This doesn't worry me too much. Just look at how long it usually takes for 2.(x+1) to actually get used over 2.x, or even 2.(x-1) -- and it's fairly obvious that this time will be a bit longer for 2.x -> 3.x. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From janssen at parc.com Thu Oct 28 18:34:03 2010 From: janssen at parc.com (Bill Janssen) Date: Thu, 28 Oct 2010 09:34:03 PDT Subject: [Python-Dev] Continuing 2.x In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Message-ID: <64043.1288283643@parc.com> Kristj?n Valur J?nsson wrote: > Let's move the current 'trunk' into /branches/afterlife-27. Open it > for submissions from people such as myself that use 2.7 on a regular > basis and are willing to give it some extra love. Though I'm not personally convinced it's a good idea, I can see where some could find it useful. > Host it there > without the usual stringent python quality assurance, buildbot > support, release management and all that rigmarole. Open-source it, > if you will. No real need to go that far, I think. The tests after all are part of the source tree, buildbots are still running them, etc. And if the buildbot master stops doing that, there are community buildbots for testing things like this. Release management is going to happen one way or another. Bill From alexander.belopolsky at gmail.com Thu Oct 28 18:39:15 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 28 Oct 2010 12:39:15 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: Message-ID: On Thu, Oct 28, 2010 at 12:07 PM, wrote: .. > Has anyone here analyzed download stats on py.org lately? > Please feel free to prove me wrong, but by my reckoning, > and at least for Windows MSI installer files, people are > still downloading Python 2.X roughly 3 to 4 times more often > than Python 3.X today, some 2 years after 3.X's release. > I don't think this is a fair comparison. At least not until 3.2 final is out for some time. Note that 2.7 is at the moment the latest stable release and 3.x releases so far have suffered from developers' attention divided between 2.x and 3.x series. I believe the trend will change with 3.2 release. From barry at python.org Thu Oct 28 18:42:49 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Oct 2010 12:42:49 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: Message-ID: <20101028124249.3cb5ff75@mission> On Oct 28, 2010, at 04:07 PM, lutz at rmi.net wrote: >I hope 3.X use expands; in fact, I've bet the future of at >least one book on it. And even 1/4 of new users seems a >large enough subset to care about too. But one can't help >but wonder if most of the development community is focused >on some imaginary future user base, at the expense of the >much larger current user base. Then again, there's still >plenty of Fortran77 code out there, so... Python 2 will live on for a long time. Other than promising to bug-fix maintain Python 2.7 for much longer than usual, which we've already done, what specifically should we do? A no-new-feature Python 2.8 doesn't make sense, and I'm not convinced that a new-feature Python 2.8 really helps folks who are stuck on Python 2 for whatever reason. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From rdmurray at bitdance.com Thu Oct 28 19:20:05 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 28 Oct 2010 13:20:05 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: Message-ID: <20101028172005.097512345C9@kimball.webabinitio.net> On Thu, 28 Oct 2010 16:07:50 -0000, lutz at rmi.net wrote: > I hope 3.X use expands; in fact, I've bet the future of at > least one book on it. And even 1/4 of new users seems a > large enough subset to care about too. But one can't help > but wonder if most of the development community is focused > on some imaginary future user base, at the expense of the > much larger current user base. Then again, there's still > plenty of Fortran77 code out there, so... Given the existing rate of Python3 adoption (which by the signs we see in the tracker is increasing), you can hardly call the user base imaginary. Further, Python development (and development in general!) is *always* focused on a "future user base" in the sense you are using it, not the "current user base". That's pretty much part of the definition of development :) But the reality is that almost all those Python2 users are future Python3 users, so they *are* the future user base. And like Georg said, as far as we can see Python3 uptake is pretty much right on the schedule that was predicted when it was first released. -- R. David Murray www.bitdance.com From ianb at colorstudy.com Thu Oct 28 19:25:28 2010 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 28 Oct 2010 10:25:28 -0700 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101028120414.1f4f3016@mission> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> Message-ID: On Thu, Oct 28, 2010 at 9:04 AM, Barry Warsaw wrote: > Who is the target audience for a Python 2.8? What exactly would a Python > 2.8 > accomplish? > > If Python 2.8 doesn't include new features, well, then what's the point? > Python 2.7 will be bug fix maintained for a long time, longer in fact than > previous Python 2 versions. So a no-feature Python 2.8 can't be about > improving Python 2 stability over time (i.e. just fix the bug in Python > 2.7). > > If Python 2.8 is about adding new features, then it has to be about > backporting those features from Python 3. Adding new feature only to a > Python > 2.8 *isn't* Python, it's a fork of Python. Thinking about language features and core type this seems reasonable, but with the standard library this seems less reasonable -- there's lots of conservative changes to the standard library which aren't bug fixes, and the more the standard library is out of sync between Python 2 and 3 the harder maintaining software that works across those versions becomes. Though one opportunity is to distribute modules from the standard library under new names (e.g., unittest2), and at least in Python 2 you don't have to do anything fancy or worry about the standard library has catching up to the standard library forked module. Library installers seem particularly apropos to this discussion, as everyone seems excited to get them into the standard library and distributed with Python, but with the current plan that will never happen with Python 2. -- Ian Bicking | http://blog.ianbicking.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Thu Oct 28 19:33:56 2010 From: debatem1 at gmail.com (geremy condra) Date: Thu, 28 Oct 2010 10:33:56 -0700 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <2E034B571A5CE44E949B9FCC3B6D24EE5761FC80@exchcn.ccp.ad.local> Message-ID: On Wed, Oct 27, 2010 at 11:48 PM, Georg Brandl wrote: > Am 28.10.2010 06:13, schrieb Daniel Stutzbach: >> 2010/10/27 Kristj?n Valur J?nsson > > >> >> ? ? Firstly, the ease of integrating changes. ?It would be possible to port >> ? ? those bugfixes that release-27 gets, and also backport selected things from >> ? ? py3k using the tools already in place such as svnmerge. >> >> >> py3k will soon be moving to Mercurial, so svnmerge would not be helpful for much >> longer. ?On the plus side, since Mercurial is a Distributed Version Control >> System, if you setup an unofficial continuation of Python 2 on the host of your >> choice, it will be easy for you to pull patches from py3k. > > I believe we'll eventually have the ability to create user repos as well, so > that Kristjan can simply put his branch into one of these and still have it > on hg.python.org. > > Georg Huge +1 from me- I think this would be an excellent development. Geremy Condra PS- this should not be taken as an endorsement of the original proposal, I simply don't use 2.x anymore and don't have an opinion on it. From stephen at xemacs.org Thu Oct 28 19:43:05 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 29 Oct 2010 02:43:05 +0900 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101028123132.3b5e8c4f@mission> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <20101028161745.2040.1157174389.divmod.xquotient.52@localhost.localdomain> <20101028123132.3b5e8c4f@mission> Message-ID: <87vd4mxlg6.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > Maybe so, but I think it's actually more fun to be working on > something other people will actually use. ;) I think that the point is that the people will be doing this are supporting software to pay for Johnny's piano lessons, not for personal pleasure. I imagine many, perhaps most, of them would rather be coding and/or Python 3, too! From tseaver at palladion.com Thu Oct 28 19:47:00 2010 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 28 Oct 2010 13:47:00 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101028120414.1f4f3016@mission> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/28/2010 12:04 PM, Barry Warsaw wrote: > Who is the target audience for a Python 2.8? What exactly would a Python 2.8 > accomplish? > > If Python 2.8 doesn't include new features, well, then what's the point? > Python 2.7 will be bug fix maintained for a long time, longer in fact than > previous Python 2 versions. So a no-feature Python 2.8 can't be about > improving Python 2 stability over time (i.e. just fix the bug in Python 2.7). > > If Python 2.8 is about adding new features, then it has to be about > backporting those features from Python 3. I think that assumption may not be warranted. If the current core folks are focused only on developing Python 3, but others are working on a notional 2.8, there is no necessary correlation any longer between the two. In particular, the judgement of the current core about various tradeoffs in the Python 2 codebase won't be as relevant as it has been, in particular because the overarching drive (add features / warnings etc. which ease / encourage migration to Python 3) won't be in the forefront of the new group's perspective. > Adding new feature only to a Python 2.8 *isn't* Python, it's a fork of Python. - From the perspective of this notional group of 2.8 developers, that particular horse is out of the barn already: it is called "Python 3". > Of course, it's open source and > you're always allowed to do that, but you would need to be clear that this > isn't "Python". Pythonic is in the eye of the beholder. > IOW, a distro like Ubuntu would likely never package such a > thing as "/usr/bin/python". For the set of folks who might care about the retro-forked 2.8, I doubt that will matter. For instance, although I'm not (necessarily) in that camp, I choose not to use the system python for any app I deploy: the system packagers make tradeoffs which are inappropriate for my applications, and I don't want to risk having a sysadmin-driven update break them. I always build Python from source and install under '/opt'. Distros who desire to package not-yet-or-maybe-ever-ported-to-Python-3 apps will have to make their own choices. Perhaps the retro-fork is available via a PPA or "extras" repository, and installs explictly as '/usr/bin/python2'. > What specific features that are showing up in say Python 3.2 are so compelling > that they need to show up in Python 2.8, *and* would compel folks who are > pinned to Python 2 to spend the resources to support it? Porting and > certifying a code base against a new Python version is always work, sometimes > a significant amount of work. What would be so compelling about a Python 2.8 > that users, package authors, and distros would be willing to undertake this > work? I can imagine features (and particularly library changes) which aren't even on the radar for Python 3, which provide real value to to folks maintaining the notional 2.8, and hence which get developed there first; perhaps they get forward-ported later to a future Python 3 release. > I'd *much* rather this enthusiasm be spent on making Python 3 rock, and in > porting third party code to Python 3. Sure you would -- you're already invested there. I would like to be there, but can't take off the several months required to port the whole stack under my own code. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkzJtxMACgkQ+gerLs4ltQ7iTACglNlfMd+zEx0isiTAvTECGT6h VccAmgMHBMGkLQaqONU9CC5wY9uso63V =Qpxy -----END PGP SIGNATURE----- From tseaver at palladion.com Thu Oct 28 19:47:04 2010 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 28 Oct 2010 13:47:04 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/28/2010 09:33 AM, Nick Coghlan wrote: > 2010/10/28 Kristj?n Valur J?nsson : >> Hello all. >> So, python 2.7 is in bugfix only mode. ?trunk? is off limit. So, where >> does one make improvements to the distinguished, and still very much alive, >> 2.x series of Python? >> >> The answer would seem to be ?one doesn?t?. But must it be that way? > > When python-dev chose to switch our own focus for new features to 3.x > only, we were quite aware that a new group forming to continue with > 2.8 was a definite possibility. If you do decide to go ahead with the > idea, I have a few suggestions: > > 1. Since the distinguishing feature is that this branch is a 2.x > version that will accept new features, in contrast to the python-dev > maintained bugfix-only 2.7 maintenance branch, please call the branch > something-or-other-2.8, rather than any form of 2.7. > 2. Check with Benjamin how he plans to handle 2.7 maintenance > releases. If he plans to release from SVN, use that as your upstream > master. If 2.7.1 will instead be released from hg.python.org, wait > until the switch happens then use the relevant hg branch as the > upstream. > 3. Choose your target audience early (if the target is only developers > with existing 2.x installations that can build their own version of > Python, then that simplifies release management significantly, since > you don't need to build binaries any more). > 4. Decide whether or not you need a buildbot farm (this relates to > point 3: you may choose to limit your audience to people that are > happy to run the test suite themselves on their own target platforms > of interest). > 5. Give some thought to how you will handle controversial design > decisions (since you won't have the fallback of appealing to the BDFL, > and feedback from python-dev is likely to be limited). > 6. Asking for a python-org SIG mailing list may be a reasonable idea > as well (e.g. py2x-sig) > 7. As 3.x usage grows, such a group may have a vested interest in > helping with 3to2 development as well as simplifying backporting of > 3.x extension modules to 2.x. > > A 2.8 branch that sells itself as being suitable only for people > willing to run their own builds and QA could definitely have a place > in the world (CCP at least would obviously find it useful, but I > wouldn't be surprised to find other companies that might consider > adopting such a branch if the benefits of the new features over the > official 2.7 releases were sufficiently compelling). > > I don't believe anyone here is implacably opposed to the idea of 2.8 > development continuing - it's just that the "collective we" of > python-dev isn't interested in making it happen, so a new crop of > developers will need to step up if it is going to become more than a > CCP-specific 2.x fork. Thanks for the helpful guidance to such prospective volunteers! Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkzJtxgACgkQ+gerLs4ltQ6OKgCcCH1Wt0Bg1COSqMdBm7whSL8H JOMAnRqA9jy8eazZnTMV+Q/gvKNX7zLb =yu2I -----END PGP SIGNATURE----- From solipsis at pitrou.net Thu Oct 28 19:50:08 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 28 Oct 2010 19:50:08 +0200 Subject: [Python-Dev] Continuing 2.x References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> Message-ID: <20101028195008.671c0aeb@pitrou.net> On Thu, 28 Oct 2010 10:25:28 -0700 Ian Bicking wrote: > > Thinking about language features and core type this seems reasonable, but > with the standard library this seems less reasonable -- there's lots of > conservative changes to the standard library which aren't bug fixes, and the > more the standard library is out of sync between Python 2 and 3 the harder > maintaining software that works across those versions becomes. For the same reason that new features only get in 3.2 and not in 3.1 or 2.7, for example. I know people would like both stability *and* new features in the same codebase, but that doesn't work. There's a reason most decently managed software projects have separate bugfix branches and feature branches. That's the same old discussion and it isn't specific to Python. (and, believe me, not having to backport new 3.x features to the 2.x branch makes our work much easier than it was; people generally seem to underestimate the amount of care needed for such things, especially in areas where 2.x is significantly more complex - old-style classes, two parallel buffer APIs, misleading implicit conversions...) Regards Antoine. From fuzzyman at voidspace.org.uk Thu Oct 28 19:55:49 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 28 Oct 2010 13:55:49 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101028172005.097512345C9@kimball.webabinitio.net> References: <20101028172005.097512345C9@kimball.webabinitio.net> Message-ID: <4CC9B925.1020501@voidspace.org.uk> On 28/10/2010 13:20, R. David Murray wrote: > On Thu, 28 Oct 2010 16:07:50 -0000, lutz at rmi.net wrote: >> I hope 3.X use expands; in fact, I've bet the future of at >> least one book on it. And even 1/4 of new users seems a >> large enough subset to care about too. But one can't help >> but wonder if most of the development community is focused >> on some imaginary future user base, at the expense of the >> much larger current user base. Then again, there's still >> plenty of Fortran77 code out there, so... > Given the existing rate of Python3 adoption (which by the signs we see in > the tracker is increasing), The Wing IDE guys get a lot of feedback from the "report issue" dialog that is built in to the IDE. This sends information to them which includes which version of Python the user is working with. They are seeing an ever increasing proportion number of users working with Python 3 (I don't have numbers though). All the best, Michael Foord > you can hardly call the user base imaginary. > Further, Python development (and development in general!) is *always* > focused on a "future user base" in the sense you are using it, not the > "current user base". That's pretty much part of the definition of > development :) > > But the reality is that almost all those Python2 users are future Python3 > users, so they *are* the future user base. And like Georg said, as far > as we can see Python3 uptake is pretty much right on the schedule that > was predicted when it was first released. > > -- > R. David Murray www.bitdance.com > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ From stephen at xemacs.org Thu Oct 28 20:43:51 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 29 Oct 2010 03:43:51 +0900 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: Message-ID: <87tyk6ximw.fsf@uwakimon.sk.tsukuba.ac.jp> lutz at rmi.net writes: > But one can't help but wonder if most of the development community > is focused on some imaginary future user base, at the expense of > the much larger current user base. Of course not. Most of the development community is *focused* on a very real, very current, and very *small* user base. Themselves and their employers and/or customers. What attention they do pay to other user bases is relatively unconcentrated (but far from negligible!) And that's as it should be. And so what if it's at the "expense" of the current user base. By and large, the "user base" has paid no expenses of the developers to date. Nobody has a problem with that, but there is no promise that free ride will continue forever. Eventually the Piper presents his bill.... From martin at v.loewis.de Thu Oct 28 20:55:48 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 28 Oct 2010 20:55:48 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101028195008.671c0aeb@pitrou.net> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <20101028195008.671c0aeb@pitrou.net> Message-ID: <4CC9C734.1050005@v.loewis.de> > (and, believe me, not having to backport new 3.x features to the 2.x > branch makes our work much easier than it was; people generally seem > to underestimate the amount of care needed for such things, especially > in areas where 2.x is significantly more complex - old-style classes, > two parallel buffer APIs, misleading implicit conversions...) I completely agree with that point. I find it unlikely that those who do regular maintenance of Python will join a continuing-2.x effort. Regards, Martin From martin at v.loewis.de Thu Oct 28 21:05:18 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 28 Oct 2010 21:05:18 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: Message-ID: <4CC9C96E.4080708@v.loewis.de> Am 28.10.2010 18:07, schrieb lutz at rmi.net: > Kristj?n Valur J?nsson writes: >> James Y Knight said: >> The python community has already decided many times over that Python2 is dead >> and Python3 is the future >> >> But the patient is very much alive and kicking, no matter what the good doctor >> declares. Python 2.x is in widespread use, with gazillion lines of .py code. >> In, there is another gazillion lines of .c and .cpp code both in extensions and >> embedding applications in use. I?m quite happy with the community at large >> moving its development focus to 3.x but it is a bit harsh to deprive those left >> behind of the keys to the old house. > > Exactly. > > Has anyone here analyzed download stats on py.org lately? I don't think anybody here questions that usage of 2.x is orders of magnitude larger than that of 3.x, and that it will stay that way for quite some time. If, by "Exactly", you also supported "it is a bit harsh", then I disagree. It's not harsh at all. Existing 2.x users are *not* deprived at all. 2.7 releases are still being made, and existing 2.x code will continue to run just fine for many years to come. Users who chose to ignore 3.x can well continue to work in their projects, without having to worry that bugs won't be fixed anymore. > Please feel free to prove me wrong, but by my reckoning, > and at least for Windows MSI installer files, people are > still downloading Python 2.X roughly 3 to 4 times more often > than Python 3.X today, some 2 years after 3.X's release. Again, no doubt about that - I readily believe you without checking, and you could have said that the factor was 10 and I still would have believe it. It just doesn't worry me. > But one can't help > but wonder if most of the development community is focused > on some imaginary future user base, at the expense of the > much larger current user base. Yes, we do focus on future users, but we are also working on future releases. But not at the expense of the much larger current user base. They are being given much time to convert their code to 3.x. So far, there has been no pressure at all to migrate. Now, we are telling them that there won't be new features in 2.x anymore - but many haven't switched to 2.7, either. Debian still ships with 2.5, and the next Debian release will be shipping with 2.6. So any theoretical 2.8 release would be just as irrelevant to existing users for many years to come (e.g. the *next* Debian release would switch to 2.7). Regards, Martin From alexander.belopolsky at gmail.com Thu Oct 28 22:48:16 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 28 Oct 2010 16:48:16 -0400 Subject: [Python-Dev] A sad state of doctests in the python manual Message-ID: I have just discovered that sphinx supports running doctests embedded in ReST documentation. It looks like it is as simple as "cd Doc; make doctest". The result, however is not encouraging: $ make doctest ... Doctest summary =============== 1162 tests 262 failures in tests 0 failures in setup code ... From alexander.belopolsky at gmail.com Thu Oct 28 22:52:51 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 28 Oct 2010 16:52:51 -0400 Subject: [Python-Dev] A sad state of doctests in the python manual In-Reply-To: References: Message-ID: On the second look, the problem may not be that bad - "make doctest" picks up system python instead of the one from the source tree. I'll try to figure out how to rerun the doctests properly. On Thu, Oct 28, 2010 at 4:48 PM, Alexander Belopolsky wrote: > I have just discovered that sphinx supports running doctests embedded > in ReST documentation. ? It looks like it is as simple as "cd Doc; > make doctest". ?The result, however is not encouraging: > > $ make doctest > ... > Doctest summary > =============== > ?1162 tests > ?262 failures in tests > ? ?0 failures in setup code > ... > From techtonik at gmail.com Thu Oct 28 22:52:58 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 28 Oct 2010 23:52:58 +0300 Subject: [Python-Dev] Fixing zipfile.BadZipfile to zipfile.BadZipFile In-Reply-To: <4CC87682.2070405@netwok.org> References: <4CBB63FA.8080108@netwok.org> <4CC852B2.1000900@netwok.org> <4CC85E56.3040208@netwok.org> <4CC87682.2070405@netwok.org> Message-ID: Can anybody summarize the outcome? Is it that renaming BadZipfile to BadZipFile with backward compatible alias and deprecation note breaks something? -- anatoly t. From alexander.belopolsky at gmail.com Thu Oct 28 22:57:43 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 28 Oct 2010 16:57:43 -0400 Subject: [Python-Dev] A sad state of doctests in the python manual In-Reply-To: References: Message-ID: On Thu, Oct 28, 2010 at 4:52 PM, Alexander Belopolsky wrote: > On the second look, the problem may not be that bad ... Nope, the problem is even worse. It looks like Sphinx in py3k requires 2.x python: $ ../python.exe tools/sphinx-build.py -b doctest -d build/doctrees -D latex_paper_size= . build/doctest Traceback (most recent call last): File "tools/sphinx-build.py", line 27, in from sphinx import main File "tools/sphinx/__init__.py", line 44 except ImportError, err: ^ SyntaxError: invalid syntax From barry at python.org Thu Oct 28 23:18:49 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Oct 2010 17:18:49 -0400 Subject: [Python-Dev] A sad state of doctests in the python manual In-Reply-To: References: Message-ID: <20101028171849.4c3d66fb@mission> On Oct 28, 2010, at 04:57 PM, Alexander Belopolsky wrote: >On Thu, Oct 28, 2010 at 4:52 PM, Alexander Belopolsky > wrote: >> On the second look, the problem may not be that bad ... > >Nope, the problem is even worse. It looks like Sphinx in py3k >requires 2.x python: > >$ ../python.exe tools/sphinx-build.py -b doctest -d build/doctrees -D >latex_paper_size= . build/doctest >Traceback (most recent call last): > File "tools/sphinx-build.py", line 27, in > from sphinx import main > File "tools/sphinx/__init__.py", line 44 > except ImportError, err: > ^ >SyntaxError: invalid syntax It would be really cool if you fixed this! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From alexander.belopolsky at gmail.com Thu Oct 28 23:30:34 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 28 Oct 2010 17:30:34 -0400 Subject: [Python-Dev] A sad state of doctests in the python manual In-Reply-To: <20101028171849.4c3d66fb@mission> References: <20101028171849.4c3d66fb@mission> Message-ID: On Thu, Oct 28, 2010 at 5:18 PM, Barry Warsaw wrote: .. > > It would be really cool if you fixed this! > Working on it. Stay tuned. :-) From alexander.belopolsky at gmail.com Fri Oct 29 00:39:38 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 28 Oct 2010 18:39:38 -0400 Subject: [Python-Dev] A sad state of doctests in the python manual In-Reply-To: References: <20101028171849.4c3d66fb@mission> Message-ID: On Thu, Oct 28, 2010 at 5:30 PM, Alexander Belopolsky wrote: > On Thu, Oct 28, 2010 at 5:18 PM, Barry Warsaw wrote: > .. >> >> It would be really cool if you fixed this! >> > Working on it. ?Stay tuned. :-) > See http://bugs.python.org/issue10224 From merwok at netwok.org Fri Oct 29 00:49:47 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 29 Oct 2010 00:49:47 +0200 Subject: [Python-Dev] Fixing zipfile.BadZipfile to zipfile.BadZipFile In-Reply-To: References: <4CBB63FA.8080108@netwok.org> <4CC852B2.1000900@netwok.org> <4CC85E56.3040208@netwok.org> <4CC87682.2070405@netwok.org> Message-ID: <4CC9FE0B.6090808@netwok.org> Le 28/10/2010 22:52, anatoly techtonik a ?crit : > Can anybody summarize the outcome? > Is it that renaming BadZipfile to BadZipFile with backward compatible > alias and deprecation note breaks something? See #7351 or r85874. Regards From ncoghlan at gmail.com Fri Oct 29 03:46:14 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Oct 2010 11:46:14 +1000 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> Message-ID: On Fri, Oct 29, 2010 at 3:47 AM, Tres Seaver wrote: > I think that assumption may not be warranted. ?If the current core folks > are focused only on developing Python 3, but others are working on a > notional 2.8, there is no necessary correlation any longer between the > two. ?In particular, the judgement of the current core about various > tradeoffs in the Python 2 codebase won't be as relevant as it has been, > in particular because the overarching drive (add features / warnings > etc. which ease / encourage migration to Python 3) won't be in the > forefront of the new group's perspective. That's a fair point actually, but it would be a decision for the possible-but-not-yet-existing group to take as they formed. Given the likely divergence in design goals, it would probably be best to just bite the bullet and declare it a fork of Python 2.7 (py2x 2.8? RetroPython 2.8?). It would hardly be the first such fork - other flavours of 2.x with design goals that differ from those of python-dev certainly have a long history (Stackless, wpython, etc). There are also IP issues to consider in setting up such a group though. The PSF takes care of it for python.org, but those contributor agreements wouldn't necessarily cover a new fork. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From kristjan at ccpgames.com Fri Oct 29 04:10:31 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Fri, 29 Oct 2010 10:10:31 +0800 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101028120414.1f4f3016@mission> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Hi all. This has been a lively discussion. My desire to keep 2.x alive in some sense is my own and I don't know if anyone shares it but as a member of this community I think I'm allowed to voice it. So, just to clarify my particular position, let me explain where all this comes from. I am the maintainer of a private fork of Stackless Python 2.7, used by EVE Online. Since I started doing this, we have moved from version 2.1 to 2.3, 2.5 and finally a few months ago to 2.7. Python is embedded into the C application so we use the C API extensively. For a long while now I have been contributing a few improvements and patches to both the interpreter core and the standard library. These are changes that stem from solving particular problems that we face in our rather extensive use of python, and range from crash bugs to performance optimizations as well as the occasional feature. Usually the process is something like this: 1) We identify a problem that needs fixing. 2) We then spend some development effort on finding the right fix for it. 3) We then reflect: Shouldn't this be contributed back to standard Python? That means a) others will benefit b) It reduces the diff of our fork from the central branch. 4) I spend some time reworking the change, submit a patch to the 2.x version that eventually gets accepted or rejected by the community 5) And in the last few years, should a change be accepted, I am then often asked port the change to 3.x, which I normally do. (and sometimes even correctly using svnmerge...) This has been a happy and a personally fulfilling process, because who doesn't like to contribute to Python? Now, of late some of this has changed. Changes, (except those that pass as "bugfixes") are no longer accepted into the 2.x branches. Should the change apply to 3.x, then I have to locally port it to 3.x, and submit it to that branch. Some changes don't really apply to 3.x at all and have no place to go. So people using a platform similar to mine won't benefit. The result is that there is a much higher threshold for any of our improvements to make it back to the community and much less personal pleasure derived from it. What finally drove me to write the original post, was that working with the new bytearray and memoryview object in 2.7 made me realize that they don't interoperate with other classes in a natural way and so nullify their advantages. My straightforward patches to 2.7 to remedy this situation (issues 10211 and 10212) were met with the usual "it's not a bugfix" reply. So, I'm frustrated. I work with 2.7 on a day to day basis, and will continue to do so for quite some time. It's a great product with some shortcomings, and I'd like to contribute to it but such contributions aren't welcome anymore. I'm not sure what I'm actually proposing. But I certainly wasn't thinking of a new "fork" of python. And not a new version 2.8 that gets all new 3.x features backported. I'm more thinking of a place where usability improvements, C API improvements, performance improvements, Library improvments, can go. Cheers, Kristj?n -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Barry Warsaw Sent: Friday, October 29, 2010 0:04 To: python-dev at python.org Subject: Re: [Python-Dev] Continuing 2.x Who is the target audience for a Python 2.8? What exactly would a Python 2.8 accomplish? If Python 2.8 doesn't include new features, well, then what's the point? Python 2.7 will be bug fix maintained for a long time, longer in fact than previous Python 2 versions. So a no-feature Python 2.8 can't be about improving Python 2 stability over time (i.e. just fix the bug in Python 2.7). From benjamin at python.org Fri Oct 29 04:31:42 2010 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 28 Oct 2010 21:31:42 -0500 Subject: [Python-Dev] [Python-checkins] r85902 - in python/branches/py3k/Lib: os.py test/test_os.py In-Reply-To: <20101029003858.7D584EEA52@mail.python.org> References: <20101029003858.7D584EEA52@mail.python.org> Message-ID: 2010/10/28 victor.stinner : > Author: victor.stinner > Date: Fri Oct 29 02:38:58 2010 > New Revision: 85902 > > Log: > Issue #10210: os.get_exec_path() ignores BytesWarning warnings > > > Modified: > ? python/branches/py3k/Lib/os.py > ? python/branches/py3k/Lib/test/test_os.py > > Modified: python/branches/py3k/Lib/os.py > ============================================================================== > --- python/branches/py3k/Lib/os.py ? ? ?(original) > +++ python/branches/py3k/Lib/os.py ? ? ?Fri Oct 29 02:38:58 2010 > @@ -382,18 +382,32 @@ > ? ? *env* must be an environment variable dict or None. ?If *env* is None, > ? ? os.environ will be used. > ? ? """ > + ? ?# Use a local import instead of a global import to avoid bootstrap issue: > + ? ?# the os module is used to build Python extensions. > + ? ?import warnings This sort of function import should be avoided. -- Regards, Benjamin From brett at python.org Fri Oct 29 04:51:58 2010 From: brett at python.org (Brett Cannon) Date: Thu, 28 Oct 2010 19:51:58 -0700 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: 2010/10/28 Kristj?n Valur J?nsson : > Hi all. > This has been a lively discussion. > My desire to keep 2.x alive in some sense is my own and I don't know if anyone shares it but as a member of this community I think I'm allowed to voice it. ?So, just to clarify my particular position, let me explain where all this comes from. > > I am the maintainer of a private fork of Stackless Python 2.7, used by EVE Online. ?Since I started doing this, we have moved from version 2.1 to 2.3, 2.5 and finally a few months ago to 2.7. > Python is embedded into the C application so we use the C API extensively. > For a long while now I have been contributing a few improvements and patches to both the interpreter core and the standard library. ?These are changes that stem from solving particular problems that we face in our rather extensive use of python, and range from crash bugs to performance optimizations as well as the occasional feature. > Usually the process is something like this: > 1) We identify a problem that needs fixing. > 2) We then spend some development effort on finding the right fix for it. > 3) We then reflect: ?Shouldn't this be contributed back to standard Python? That means > ?a) others will benefit > ?b) It reduces the diff of our fork from the central branch. > 4) I spend some time reworking the change, submit a patch to the 2.x version that eventually ?gets accepted or rejected by the community > 5) And in the last few years, should a change be accepted, I am then often asked port the change to 3.x, which I normally do. ?(and sometimes even correctly using svnmerge...) > This has been a happy and a personally fulfilling process, because who doesn't like to contribute to Python? > And all of that work has been appreciated obviously (in case that wasn't obvious =). > Now, of late some of this has changed. ? Changes, (except those that pass as "bugfixes") are no longer accepted into the 2.x branches. ?Should the change apply to 3.x, then I have to locally port it to 3.x, and submit it to that branch. ?Some changes don't really apply to 3.x at all and have no place to go. ?So people using a platform similar to mine won't benefit. > The result is that there is a much higher threshold for any of our improvements to make it back to the community and much less personal pleasure derived from it. > > What finally drove me to write the original post, was that working with the new bytearray and memoryview object in 2.7 made me realize that they don't interoperate with other classes in a natural way and so nullify their advantages. ?My straightforward patches to 2.7 to remedy this situation (issues 10211 and 10212) were met with the usual "it's not a bugfix" reply. > Which is the correct reply. > So, I'm frustrated. ?I work with 2.7 on a day to day basis, and will continue to do so for quite some time. ?It's a great product with some shortcomings, and I'd like to contribute to it but such contributions aren't welcome anymore. Well, they are welcome, just in Python 3.2 instead of Python 2.7 (when they apply to Python 3.2 in the first place). We have had a bugfix version in development as well as the new features version for ages. It just so happens that a bunch of people have not decided when they plan to switch to the new features version yet. > > I'm not sure what I'm actually proposing. ?But I certainly wasn't thinking of a new "fork" of python. ?And not a new version 2.8 that gets all new 3.x features backported. > I'm more thinking of a place where usability improvements, C API improvements, performance improvements, Library improvments, can go. It's called a fork. I realize you are trying to avoid that "dirty" word, Kristj?n, and I appreciate it, but you are describing a fork. Python 2.7 is the last sanctioned version of the Python 2.x series, period. Any non-bugfix changes will not go in there as policy dictates. And with there being no way Python 2.8 will happen (I know we as a group have said "slim chance" since Python 3.0 came out, uptake of Python 3 is such I am willing to personally say "never" for a python-dev sanctioned Python 2.8), that means it will take a fork, whether it be internal to CCP or public somewhere, it will still be a fork. And as everyone has said so far (and with which I agree), that's fine. As long as it is not called Python 2.8 -- EVE-Python 2.8 or some Monty Python reference -- then that's fine. And as pointed out by folks, once Hg kicks in and we have user repos you can even host it on hg.python.org yourself to give it some semblance of authority if you want. I understand the desire to have the final version of the Python 2.x to be as perfect as it can be, but that will never happen. Every release of Python is imperfect. There will always be mistakes we made that require incompatible changes to rectify. It's life and that's just a part of software development. I think people need to stop viewing the difference between Python 2.7 and Python 3.2 as this crazy shift and view it from python-dev's perspective; it should be viewed one follows from the other at this point. You can view it as Python 3.2 is the next version after Python 2.7 just like 2.7 followed to 2.6, which makes the policies we follow for releases make total sense and negates this discussion. It just so happens people don't plan to switch to the newest release immediately as the backward-incompatible changes are more involved than what people are used to from past releases. So to summarize, we are not changing our minds on Python 2.8; there won't be a Python 2.8 sanctioned by python-dev ever. Python 3.2 is the next version after Python 2.7 and the typical policy of bugfix/feature release rules apply as normal. People who want to iron out some inconsistencies in Python 2.7 by forking it and renaming it to prevent people from thinking python-dev made the release are welcome to and there won't be any ill will or hard feelings if that does occur. -Brett > > Cheers, > > Kristj?n > > -----Original Message----- > From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Barry Warsaw > Sent: Friday, October 29, 2010 0:04 > To: python-dev at python.org > Subject: Re: [Python-Dev] Continuing 2.x > > Who is the target audience for a Python 2.8? ?What exactly would a Python 2.8 accomplish? > > If Python 2.8 doesn't include new features, well, then what's the point? > Python 2.7 will be bug fix maintained for a long time, longer in fact than previous Python 2 versions. ?So a no-feature Python 2.8 can't be about improving Python 2 stability over time (i.e. just fix the bug in Python 2.7). > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From stephen at xemacs.org Fri Oct 29 06:59:27 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 29 Oct 2010 13:59:27 +0900 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: <87eib9r3v4.fsf@uwakimon.sk.tsukuba.ac.jp> Brett Cannon writes: > I think people need to stop viewing the difference between Python 2.7 > and Python 3.2 as this crazy shift and view it from python-dev's > perspective; That phrasing *is* harsh. People also need to work with code bases that are incompatible with Python 3.2 for various reasons, and will be very expensive to port to 3.2. Some, perhaps many, of those people *do* consider Python 3 to be the way to go, and I imaginge they are already going that way when they can. Nevertheless, their bread-and- butter projects are feeling pain; their world is going out of whack. It is a "crazy shift" (or "crazy-making" shift) for them. And for now the book writers have to feel the same way; a lot of Python-2-based applications are going to be perfectly happy to stay that way with Python 2.7 for years to come. The book writers need (as a commercial matter) to serve the new engineers who will be hired to maintain *and extend* those products. I suspect there will be a substantial market for Python 2 content in Python books until 2015 (although Mark Lutz should be able to sit back and just collect royalties on that part of his book starting in 2012). > You can view it as Python 3.2 is the next version after Python > 2.7 just like 2.7 followed to 2.6, Kristj?n acknowledges that. He's looking for some relief from his *current* headache. Mark's position is different. His words suggest that he thinks that Python.org owes the users something, although if pressed I imagine he'd present some argument that more users will lead to development of a better language. I think the developers universally consider that to be objectively false: Python 3 is a much better language, and is on track to be a much better environment for development -- of itself and of applications -- in 2013 than Python 2 could conceivably be. From solipsis at pitrou.net Fri Oct 29 08:34:14 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Oct 2010 08:34:14 +0200 Subject: [Python-Dev] issue 10212 References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: <20101029083414.6e94d519@pitrou.net> On Fri, 29 Oct 2010 10:10:31 +0800 Kristj?n Valur J?nsson wrote: > > What finally drove me to write the original post, was that working with > the new bytearray and memoryview object in 2.7 made me realize that > they don't interoperate with other classes in a natural way and so > nullify their advantages. My straightforward patches to 2.7 to remedy > this situation (issues 10211 and 10212) were met with the usual "it's > not a bugfix" reply. For the record, and that's not because you are using it here as an argument, I misevaluated issue 10212. I thought you wanted to make cStringIO objects *provide* the buffer protocol. Actually, your patch simply makes it *accept* new buffer-compliant objects in places where it already accepts old buffer-compliant objects (such as the write() method). There is then much less contention against letting the patch in (provided it gets reviewed :-)). That doesn't change the rest of your argument, though; and I agree with other people's voiced opinions that what you are proposing ("usability improvements", etc.) is what we usually consider a new feature, in QA terms. Regards Antoine. From solipsis at pitrou.net Fri Oct 29 08:52:16 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Oct 2010 08:52:16 +0200 Subject: [Python-Dev] x86 Leopard buildbot Message-ID: <20101029085216.07aeb2bf@pitrou.net> Hello, As a leap of faith, I have added Stephen Hansen's x86 Leopard buildbot to the list of stable bots. Stephen has been very proactive in diagnosing and fixing issues (thank you!). Antoine. From glyph at twistedmatrix.com Fri Oct 29 08:55:55 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Fri, 29 Oct 2010 02:55:55 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: On Oct 28, 2010, at 10:51 PM, Brett Cannon wrote: > I think people need to stop viewing the difference between Python 2.7 > and Python 3.2 as this crazy shift and view it from python-dev's > perspective; it should be viewed one follows from the other at this > point. You can view it as Python 3.2 is the next version after Python > 2.7 just like 2.7 followed to 2.6, which makes the policies we follow > for releases make total sense and negates this discussion. It just so > happens people don't plan to switch to the newest release immediately > as the backward-incompatible changes are more involved than what > people are used to from past releases. Brett, with all due respect, this is not a reasonable position. You are making it sound like the popular view of 3.2 is a "crazy shift" is based on a personal dislike of python-dev or something. The fact is that the amount of effort required to port to 3.2 is extreme compared to previous upgrades, and most people still aren't willing to deal with it. It is a crazy shift. Let's take PyPI numbers as a proxy. There are ~8000 packages with a "Programming Language::Python" classifier. There are ~250 with "Programming Langauge::Python::3". Roughly speaking, we can say that is 3% of Python code which has been ported so far. Python 3.0 was released at the end of 2008, so people have had roughly 2 years to port, which comes up with 1.5% per year. Let's say that 20% of the code on PyPI is just junk; it's unfair to expect 100% of all code ever to get ported. But, still: with this back-of-the-envelope estimate of the rate of porting, it will take over 50 years before a decisive majority of Python code is on Python 3. By contrast, there are 536 packages with ::2.6, and 177 with ::2.7. (Trying to compare apples to apples here, since I assume the '2' tag is much more lightly used than '3' to identify supported versions; I figure someone likely to tag one micro-version would also tag the other.) 2.7 was released on July 3rd, so let's be generous and say approximately 6 months. That's 30% of packages, ported in 6 months, or 60% per year. This means that Python 3 is two orders of magnitude crazier of a shift than 2.7. I know that the methods involved at arriving at these numbers are not particularly good. But, I think that if their accuracy differs from that of the download stats, it's better: it takes a much more significant commitment to actually write some code and upload it than to accidentally download 3.x because it's the later version. Right now, Kristj?n is burning off his (non-fungible) enthusiasm in this discussion rather than addressing more 2.x maintenance issues. If 3.x adoption takes off and makes a nice hockey stick graph, then few people will care about this in retrospect. In the intervening hypothetical half-century while we wait to see how it pans out, isn't it better to just have an official Python branch for the "maybe 2.8" release? Nobody from the current core team needs to work on it, necessarily; either other, new maintainers will show up or they won't. For that matter, Kristj?n is still talking about porting much of his work to 3.x anyway. In the best case (3.x takes over the world in 6 months) a 2.x branch won't be needed and nobody will show up to do the work of a release; some small amount of this work (the stuff not ported to 3.x) will be lost. In the medium case (3.x adoption is good, but there are still millions of 2.x users in 5 years) it will accumulate some helpers that will make migrating to 3.x even smoother than with 2.7. In the worst case (straw man: 3.x adoption actually declines, and distros start maintaining their own branches of 2.7) I'm sure everyone will be glad that some of this maintenance effort took place and there's some central place to continue it. I'm perfectly willing to admit that I'm still too pessimistic about this and I could be wrong. But given the relatively minimal amount of effort required to let 2.x bugs continue to get fixed under the aegis of Python.org rather than going through the painful negotiation process of figuring out where else to host it (and thereby potentially losing a bunch of maintenance that would not otherwise happen), it seems foolhardy to insist that those of us who think 2.x is going to necessitate another release must necessarily be wrong. -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Fri Oct 29 09:11:12 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Oct 2010 09:11:12 +0200 Subject: [Python-Dev] Continuing 2.x References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: <20101029091112.3aa04bb1@pitrou.net> On Fri, 29 Oct 2010 02:55:55 -0400 Glyph Lefkowitz wrote: > > Let's say that 20% of the code on PyPI is just junk; > it's unfair to expect 100% of all code ever to get ported. But, still: > with this back-of-the-envelope estimate of the rate of porting, it will > take over 50 years before a decisive majority of Python code is on > Python 3. Well, no. A decisive majority would be much smaller than that. There are probably between 2% and 5% of the CheeseShop entries which are widely used dependencies. When these 2 to 5% all get ported, you have a decisive majority. Yes, perhaps more than 50% of 2.x code will never get ported. But, perhaps more than 50% of 1.5.2 code never got upgraded either. That doesn't make it any decisive; just dead (or pining for security fixes in some old rusty "RedHat Enterprise Linux" server, if you prefer). > Right now, Kristj?n is burning off his (non-fungible) enthusiasm > in this discussion rather than addressing more 2.x maintenance issues. By the same argument, we are also burning off our (non-fungible) enthusiasm trying to answer people like you and Kristj?n. Perhaps we should stop answering you and instead concentrate on improving 3.x? ;) Regards Antoine. From martin at v.loewis.de Fri Oct 29 09:22:08 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 29 Oct 2010 09:22:08 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: <4CCA7620.9070104@v.loewis.de> > Right now, Kristj?n is burning off his (non-fungible) enthusiasm in this > discussion rather than addressing more 2.x maintenance issues. If 3.x > adoption takes off and makes a nice hockey stick graph, then few people > will care about this in retrospect. In the intervening hypothetical > half-century while we wait to see how it pans out, isn't it better to > just have an official Python branch for the "maybe 2.8" release? Nobody > from the current core team needs to work on it, necessarily; either > other, new maintainers will show up or they won't. For that matter, > Kristj?n is still talking about porting much of his work to 3.x anyway. That might be - however, I think it is (now) clear that talking about this on python-dev isn't going to make it happen, at least not now. Nobody has really stepped forward to manage a Python 2.8 project; more specifically, Kristjan explicitly said that he will *not* do a Python 2.8 release. As Brett (IMHO correctly) analyzed: Kristjan wants a fork, a place where he can publish his Python changes. Not sure if anybody else has that desire, but if so, these people should get together and set something up. As for the specific implementation of the fork: people proposed that Kristjan should wait for the hg switchover, and can then easily host the fork anywhere (e.g. on bitbucket, or as a private clone on python.org). Assuming there are other contributors (outside of python-committers) to this fork as well, hosting it on bitbucket is probably easier since it will allow to give push permissions to non-core committers. > I'm perfectly willing to admit that I'm still too pessimistic about this > and I could be wrong. But given the relatively minimal amount of effort > required to let 2.x bugs continue to get fixed under the aegis of > Python.org I think here you are strongly mistaken. It is not a relatively minimal effort. Having to support another branch would be very very painful. > it seems foolhardy to insist that those of us who think 2.x is > going to necessitate another release must /necessarily/ be wrong. Predictions are always difficult to make. It may be that 2.x will necessitate another release (by some criteria), but I truly hope that you are wrong in predicting that such a release will actually happen. Regards, Martin From vinay_sajip at yahoo.co.uk Fri Oct 29 09:46:33 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Fri, 29 Oct 2010 07:46:33 +0000 (UTC) Subject: [Python-Dev] Continuing 2.x References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Message-ID: Kristj?n Valur J?nsson ccpgames.com> writes: > Let?s move the current ?trunk? into /branches/afterlife-27.? Open it for > submissions from people such as myself that use 2.7 on a regular basis and are > willing to give it some extra love. Just curious - what specific new features or backwards-incompatible fixes do you need to add, i.e. things which cannot be catered for by release27-maint? Or is this just about the *principle* of having a 2.8? Regards, Vinay Sajip From martin at v.loewis.de Fri Oct 29 12:15:04 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 29 Oct 2010 12:15:04 +0200 Subject: [Python-Dev] new buffer in python2.7 In-Reply-To: <201010270933.18420.eckhardt@satorlaser.com> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FB17@exchcn.ccp.ad.local> <201010270933.18420.eckhardt@satorlaser.com> Message-ID: <4CCA9EA8.5060300@v.loewis.de> > Actually I would like code like > s = socket() > ... > header = struct.unpack("i", s) > > In other words, struct should interact with files/streams directly, instead of > requiring me to first read a chunk who's size I manually have to determine > etc. That is easy to achieve using the existing API: def read_and_unpack(stream, format): data = stream.read(struct.calcsize(format)) return struct.unpack(format, data) > Otherwise, I'm +1 on your suggestion, avoiding copying is a good thing. I believe my function also doesn't involve any unnecessary copies. Regards, Martin From martin at v.loewis.de Fri Oct 29 12:26:06 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 29 Oct 2010 12:26:06 +0200 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <20101026190538.7E9791FE543@kimball.webabinitio.net> <4CC73E17.1000501@voidspace.org.uk> Message-ID: <4CCAA13E.8090104@v.loewis.de> > While maintainers' convenience is a valid valid concern and some level > of idiosyncrasy is healthy to allow active maintainers to code in > their preferred style, I think users' convenience should come first > when it conflicts with that of maintainers. Remember, code is written > once and read many. This is particularly true about stdlib. A minor > inconvenience of finding the right place to stick a new function in a > large file does not in my view overweights a major inconvenience of > not having all pieces in one place neatly organized in a linear order. I agree. While investigating an incompatibility in unittest2, I found that the breakage into multiple files makes it much harder to find out how things fit together, and where specifically a certain functionality is implemented. So join those who would have preferred this module to stay as a single 2000-line file. Regards, Martin From vinay_sajip at yahoo.co.uk Fri Oct 29 13:03:40 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Fri, 29 Oct 2010 11:03:40 +0000 (UTC) Subject: [Python-Dev] Continuing 2.x References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> Message-ID: Vinay Sajip yahoo.co.uk> writes: > need to add, i.e. things which cannot be catered for by release27-maint? Or is > this just about the *principle* of having a 2.8? Never mind - I've just picked up the extra posts on this thread, which for some reason didn't show up in my reader before. Sorry for the noise. From mal at egenix.com Fri Oct 29 15:42:42 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Oct 2010 15:42:42 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: <4CCACF52.8090202@egenix.com> Brett Cannon wrote: > 2010/10/28 Kristj?n Valur J?nsson : >> I'm not sure what I'm actually proposing. But I certainly wasn't thinking of a new "fork" of python. And not a new version 2.8 that gets all new 3.x features backported. >> I'm more thinking of a place where usability improvements, C API improvements, performance improvements, Library improvments, can go. > > It's called a fork. I realize you are trying to avoid that "dirty" > word, Kristj?n, and I appreciate it, but you are describing a fork. > Python 2.7 is the last sanctioned version of the Python 2.x series, > period. Any non-bugfix changes will not go in there as policy > dictates. And with there being no way Python 2.8 will happen (I know > we as a group have said "slim chance" since Python 3.0 came out, > uptake of Python 3 is such I am willing to personally say "never" for > a python-dev sanctioned Python 2.8), that means it will take a fork, > whether it be internal to CCP or public somewhere, it will still be a > fork. > > And as everyone has said so far (and with which I agree), that's fine. > As long as it is not called Python 2.8 -- EVE-Python 2.8 or some Monty > Python reference -- then that's fine. I don't see why we should not welcome a team of new developers who want to continue working on the 2.x series. It's obvious that a large proportion of the existing python-dev'ers will not participate in such a project, but why should we try to stop someone else to work on it ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 29 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From bostjan.mejak at gmail.com Fri Oct 29 15:45:43 2010 From: bostjan.mejak at gmail.com (=?UTF-8?Q?Bo=C5=A1tjan_Mejak?=) Date: Fri, 29 Oct 2010 15:45:43 +0200 Subject: [Python-Dev] Fixing zipfile.BadZipfile to zipfile.BadZipFile In-Reply-To: <4CC9FE0B.6090808@netwok.org> References: <4CBB63FA.8080108@netwok.org> <4CC852B2.1000900@netwok.org> <4CC85E56.3040208@netwok.org> <4CC87682.2070405@netwok.org> <4CC9FE0B.6090808@netwok.org> Message-ID: The tests prove that r85874 does not break the build. On Fri, Oct 29, 2010 at 12:49 AM, ?ric Araujo wrote: > Le 28/10/2010 22:52, anatoly techtonik a ?crit : > > Can anybody summarize the outcome? > > Is it that renaming BadZipfile to BadZipFile with backward compatible > > alias and deprecation note breaks something? > > See #7351 or r85874. > > Regards > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/bostjan.mejak%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Fri Oct 29 16:05:45 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 29 Oct 2010 09:05:45 -0500 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <4CCACF52.8090202@egenix.com> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> Message-ID: 2010/10/29 M.-A. Lemburg : > Brett Cannon wrote: >> 2010/10/28 Kristj?n Valur J?nsson : >>> I'm not sure what I'm actually proposing. ?But I certainly wasn't thinking of a new "fork" of python. ?And not a new version 2.8 that gets all new 3.x features backported. >>> I'm more thinking of a place where usability improvements, C API improvements, performance improvements, Library improvments, can go. >> >> It's called a fork. I realize you are trying to avoid that "dirty" >> word, Kristj?n, and I appreciate it, but you are describing a fork. >> Python 2.7 is the last sanctioned version of the Python 2.x series, >> period. Any non-bugfix changes will not go in there as policy >> dictates. And with there being no way Python 2.8 will happen (I know >> we as a group have said "slim chance" since Python 3.0 came out, >> uptake of Python 3 is such I am willing to personally say "never" for >> a python-dev sanctioned Python 2.8), that means it will take a fork, >> whether it be internal to CCP or public somewhere, it will still be a >> fork. >> >> And as everyone has said so far (and with which I agree), that's fine. >> As long as it is not called Python 2.8 -- EVE-Python 2.8 or some Monty >> Python reference -- then that's fine. > > I don't see why we should not welcome a team of new developers who want > to continue working on the 2.x series. He's not saying we shouldn't welcome them; we just don't want to it attached to python-dev. -- Regards, Benjamin From martin at v.loewis.de Fri Oct 29 16:12:38 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 29 Oct 2010 16:12:38 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <4CCACF52.8090202@egenix.com> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> Message-ID: <4CCAD656.7070408@v.loewis.de> > It's obvious that a large proportion of the existing python-dev'ers will > not participate in such a project, but why should we try to stop someone > else to work on it ? I propose to stop this discussion of theoretical projects, and only restart it when someone actually proposes to lead such a project. I might not personally stop anybody from doing such a project, but I surely will not support him. This thread was started by a specific proposal from Kristjan, and Kristjan got a specific suggestion on how to proceed (namely, wait for the Mercurial switchover, then publish his changes in a branch). So despite the more general subject (which I think is still mostly hypothetical), the real issue Kristjan raised has been resolved, AFAICT (although Kristjan has not yet voiced an opinion of whether he finds that resolution acceptable). Regards, Martin From mal at egenix.com Fri Oct 29 16:13:56 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Oct 2010 16:13:56 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> Message-ID: <4CCAD6A4.9040503@egenix.com> Benjamin Peterson wrote: > 2010/10/29 M.-A. Lemburg : >> Brett Cannon wrote: >>> 2010/10/28 Kristj?n Valur J?nsson : >>>> I'm not sure what I'm actually proposing. But I certainly wasn't thinking of a new "fork" of python. And not a new version 2.8 that gets all new 3.x features backported. >>>> I'm more thinking of a place where usability improvements, C API improvements, performance improvements, Library improvments, can go. >>> >>> It's called a fork. I realize you are trying to avoid that "dirty" >>> word, Kristj?n, and I appreciate it, but you are describing a fork. >>> Python 2.7 is the last sanctioned version of the Python 2.x series, >>> period. Any non-bugfix changes will not go in there as policy >>> dictates. And with there being no way Python 2.8 will happen (I know >>> we as a group have said "slim chance" since Python 3.0 came out, >>> uptake of Python 3 is such I am willing to personally say "never" for >>> a python-dev sanctioned Python 2.8), that means it will take a fork, >>> whether it be internal to CCP or public somewhere, it will still be a >>> fork. >>> >>> And as everyone has said so far (and with which I agree), that's fine. >>> As long as it is not called Python 2.8 -- EVE-Python 2.8 or some Monty >>> Python reference -- then that's fine. >> >> I don't see why we should not welcome a team of new developers who want >> to continue working on the 2.x series. > > He's not saying we shouldn't welcome them; we just don't want to it > attached to python-dev. That new team could be part of python-dev, couldn't it ? Not necessarily the mailing list, but the team of Python developers. Much like the (new) py3k developers joined in when that project was kicked off. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 29 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From barry at python.org Fri Oct 29 16:14:25 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Oct 2010 10:14:25 -0400 Subject: [Python-Dev] Misc/python-mode.el (was Re: r85838 - python/branches/py3k/.gitignore) In-Reply-To: References: <20101025173718.7CF19F753@mail.python.org> <20101026085124.4c68459c@mission> <4CC7DAA1.6010800@scottdial.com> <20101027073546.7a28339b@mission> <4CC8210A.8020700@netwok.org> Message-ID: <20101029101425.082b8c8a@mission> On Oct 27, 2010, at 09:19 AM, Brett Cannon wrote: >In the name of completeness for people not aware of the issue, >http://bugs.python.org/issue9893 discusses actually removing these >files in preference to files maintained by others. If Misc/Vim were to >be dropped we could place a text file much like Barry is planning to >do for Emacs and point to the files Antoine is recommending. The Emacs changes are committed now in py3k, release31-maint, and release27-maint. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From mal at egenix.com Fri Oct 29 16:15:23 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Oct 2010 16:15:23 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <4CCAD656.7070408@v.loewis.de> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <4CCAD656.7070408@v.loewis.de> Message-ID: <4CCAD6FB.70705@egenix.com> "Martin v. L?wis" wrote: >> It's obvious that a large proportion of the existing python-dev'ers will >> not participate in such a project, but why should we try to stop someone >> else to work on it ? > > I propose to stop this discussion of theoretical projects, and only > restart it when someone actually proposes to lead such a project. Fair enough. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 29 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From benjamin at python.org Fri Oct 29 16:21:41 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 29 Oct 2010 09:21:41 -0500 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <4CCAD6A4.9040503@egenix.com> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <4CCAD6A4.9040503@egenix.com> Message-ID: 2010/10/29 M.-A. Lemburg : > Benjamin Peterson wrote: >> He's not saying we shouldn't welcome them; we just don't want to it >> attached to python-dev. > > That new team could be part of python-dev, couldn't it ? Not necessarily > the mailing list, but the team of Python developers. Much like the > (new) py3k developers joined in when that project was kicked off. Perhaps, but it would debatable how much infrastructure (ie buildbots, tracker) would be available to them. -- Regards, Benjamin From rdmurray at bitdance.com Fri Oct 29 16:23:41 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 29 Oct 2010 10:23:41 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: <20101029142341.A8BE51F437B@kimball.webabinitio.net> On Fri, 29 Oct 2010 02:55:55 -0400, Glyph Lefkowitz wrote: > I'm perfectly willing to admit that I'm still too pessimistic about this > and I could be wrong. But given the relatively minimal amount of effort > required to let 2.x bugs continue to get fixed under the aegis of > Python.org rather than going through the painful negotiation process of > figuring out where else to host it (and thereby potentially losing a > bunch of maintenance that would not otherwise happen), it seems > foolhardy to insist that those of us who think 2.x is going to > necessitate another release must necessarily be wrong. Your argument was interesting, but you conclude by talking only about bugs. We are continuing to bugfix 2.x in the form of 2.7 bugfix releases. If at the end of the five years the 2.x user community is large enough that additional *bugfix* releases of 2.7 are worth the effort, we will, I'm sure, continue to produce them. What *new features* are needed in 2.x? I think the effort required to set up and maintain a fork is a good measure of whether or not such features are *valuable enough* to be worth doing. If they are, someone will do it. If not....not. -- R. David Murray www.bitdance.com From barry at python.org Fri Oct 29 16:35:00 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Oct 2010 10:35:00 -0400 Subject: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/ In-Reply-To: <20101027143452.231052274EE@kimball.webabinitio.net> References: <1288110692.31.0.0560369831336.issue10199@psf.upfronthosting.co.za> <9011AC5F-4121-4BBE-9427-02D2A1C24259@gmail.com> <33B13F0C-F6B9-4CFE-A52D-0071474F3D71@gmail.com> <20101026215443.0c78fc29@pitrou.net> <20101026233710.6b313cf7@mission> <20101027143452.231052274EE@kimball.webabinitio.net> Message-ID: <20101029103500.6aaab676@mission> On Oct 27, 2010, at 10:34 AM, R. David Murray wrote: >To put your mind at ease, Barry, I'd not want to do that either :) Phew! But I wasn't worried, 'cause I know you're not insane. (Though the fact that you've effectively inherited the email package does bring that into question. :) >But by (IMO good) design Generator, FeedParser, and Message are all >supposed to be independent (use only each other's public APIs). And >Header can be (and is, I think) used without the other pieces of email, >as is true for other of the helper modules (base64mime, quoprimime, etc). Agreed. >On the other hand, I have no clue why 'iterators.py' exists :) They're there for some corner use cases, but they do tend to be helpful. body_line_iterator() is probably the one I've used the most. The iterators are intended to be used independently. >The one that bugs me most, though, is MIME. Combining all the mime >stuff into one file seems like it would be a big win (not that it's >possible, now). What is the benefit of email.mime.text.MIMEText over >email.mime.MIMEText, when each of the files in the mime package consists >of a single subclass? I think you're right that the extra level of module path is probably unnecessary. I'm not sure that means all the .py files in email/mime should be combined though. OTOH, `wc -l Lib/email/mime/*.py` is only 314 lines so I'm happy to defer to you on that. >So, to clarify, like Raymond I'm not saying that packages are always bad. >I'm just saying that packages are also not always good; and, further, >that the number of lines of code in a file should, IMO, have nothing to >do with the decision as to whether or not to create a package. +1 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Fri Oct 29 17:07:02 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Oct 2010 11:07:02 -0400 Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles In-Reply-To: References: Message-ID: <20101029110702.414c298d@mission> On Oct 25, 2010, at 02:28 PM, Vinay Sajip wrote: >I've just checked in a change to logging into the py3k branch (r85835), >including doc changes and tests, for providing slightly more flexibility in >alternative format styles for logging. > >Basically, Formatter.__init__ gets an extra optional keyword arg style='%' (default), '{' or '$'>. This is then used to merge the format string with >the LogRecord: either fmt % record.__dict__, or fmt.format(**record.dict), or >string.Template(fmt).substitute(**record.dict). Backward compatibility is >maintained (unless I've missed something). This sounds like a reasonable solution that provides the flexibility we want, while maintaining backward compatibility. Thanks! I haven't played with it yet, but do you think it makes sense to add a 'style' keyword argument to basicConfig()? That would make it pretty easy to get the formatting style you want without having to explicitly instantiate a Formatter, at least for simple logging clients. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From olemis at gmail.com Fri Oct 29 17:40:45 2010 From: olemis at gmail.com (Olemis Lang) Date: Fri, 29 Oct 2010 10:40:45 -0500 Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles In-Reply-To: References: Message-ID: On Tue, Oct 26, 2010 at 6:15 AM, Nick Coghlan wrote: > On Tue, Oct 26, 2010 at 9:08 PM, Nick Coghlan wrote: >> On Tue, Oct 26, 2010 at 12:28 AM, Vinay Sajip wrote: >>> Comments welcome. Assuming there are no strong objections asking for reversion >>> of this change, I'll publicise to the wider community in a few days. >> >> It strikes me as a solid, pragmatic solution to a thorny problem. >> >> Looking at your checkin though, I wonder if it might be worth >> implementing some little formatting style classes to get rid of the >> if/elif chains from the Formatter code. Something like: > > Some syntax highlighting may make that wall-o'-code a little easier to > read: http://pastebin.com/SG0Qr3r5 > FWIW +1 -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: From olemis at gmail.com Fri Oct 29 17:44:03 2010 From: olemis at gmail.com (Olemis Lang) Date: Fri, 29 Oct 2010 10:44:03 -0500 Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles In-Reply-To: <20101029110702.414c298d@mission> References: <20101029110702.414c298d@mission> Message-ID: On Fri, Oct 29, 2010 at 10:07 AM, Barry Warsaw wrote: > On Oct 25, 2010, at 02:28 PM, Vinay Sajip wrote: > >>I've just checked in a change to logging into the py3k branch (r85835), >>including doc changes and tests, for providing slightly more flexibility in >>alternative format styles for logging. >> >>Basically, Formatter.__init__ gets an extra optional keyword arg style=>'%' (default), '{' or '$'>. This is then used to merge the format string with >>the LogRecord: either fmt % record.__dict__, or fmt.format(**record.dict), or >>string.Template(fmt).substitute(**record.dict). Backward compatibility is >>maintained (unless I've missed something). > > This sounds like a reasonable solution that provides the flexibility we want, > while maintaining backward compatibility. ?Thanks! > > I haven't played with it yet, but do you think it makes sense to add a 'style' > keyword argument to basicConfig()? ?That would make it pretty easy to get the > formatting style you want without having to explicitly instantiate a > Formatter, at least for simple logging clients. > Since this may be considered as a little sophisticated, I'd rather prefer these new classes to be added to configuration sections using fileConfig (and default behavior if missing), and still leave `basicConfig` unchanged (i.e. *basic*) . PS: Good work ! -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: From status at bugs.python.org Fri Oct 29 18:07:20 2010 From: status at bugs.python.org (Python tracker) Date: Fri, 29 Oct 2010 18:07:20 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20101029160720.1840E7826A@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2010-10-22 - 2010-10-29) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues stats: open 2496 (+41) closed 19519 (+56) total 22015 (+61) Open issues with patches: 1038 Issues opened (41) ================== #9935: Faster pickling of instances http://bugs.python.org/issue9935 reopened by pitrou #10173: Don't pickle TestCase instances in test_multiprocessing http://bugs.python.org/issue10173 opened by pitrou #10174: multiprocessing expects sys.stdout to have a fileno/close meth http://bugs.python.org/issue10174 opened by ikanobori #10175: vs version for win32 compilation of extension modules is undoc http://bugs.python.org/issue10175 opened by tfogal #10177: PyUnicode_AsWideCharString and PyMem_Free http://bugs.python.org/issue10177 opened by ocean-city #10179: os.stat fails on mapped network drive http://bugs.python.org/issue10179 opened by pitrou #10180: File objects should not pickleable http://bugs.python.org/issue10180 opened by pitrou #10181: get_shape0 in memoryobject.c not checked for error return http://bugs.python.org/issue10181 opened by rupole #10182: match_start truncates large values http://bugs.python.org/issue10182 opened by loewis #10183: test_concurrent_futures failure on Windows http://bugs.python.org/issue10183 opened by pitrou #10184: tarfile touches directories twice http://bugs.python.org/issue10184 opened by loewis #10188: tempfile.TemporaryDirectory may throw errors at shutdown http://bugs.python.org/issue10188 opened by ncoghlan #10189: SyntaxError: no binding for nonlocal doesn't contain a useful http://bugs.python.org/issue10189 opened by alex #10190: Can argparse._AttributeHolder._get_kwargs become a public API? http://bugs.python.org/issue10190 opened by dsuch #10191: scripts files are not RECORDed. http://bugs.python.org/issue10191 opened by Atsushi.Odagiri #10195: Memory allocation fault-injection? http://bugs.python.org/issue10195 opened by dmalcolm #10196: distutils.get_python_lib() returns /usr/local/python3/dist-pac http://bugs.python.org/issue10196 opened by Bruce.Sherwood #10197: subprocess.getoutput fails on win32 http://bugs.python.org/issue10197 opened by jldm #10199: Move Demo/turtle under Lib/ http://bugs.python.org/issue10199 opened by belopolsky #10202: ftplib doesn't check close status after sending file http://bugs.python.org/issue10202 opened by nagle #10203: sqlite3.Row doesn't support sequence protocol http://bugs.python.org/issue10203 opened by pfalcon #10205: Can't have two tags with the same QName http://bugs.python.org/issue10205 opened by bersace #10206: python program starting with unmatched quote spews spaces to s http://bugs.python.org/issue10206 opened by r.david.murray #10211: BufferObject doesn't support new buffer interface http://bugs.python.org/issue10211 opened by krisvale #10212: struct.unpack and cStringIO.StringIO don't support new buffer http://bugs.python.org/issue10212 opened by krisvale #10213: tests shouldn't fail with unset timezone http://bugs.python.org/issue10213 opened by djc #10217: python-2.7.amd64.msi install fails http://bugs.python.org/issue10217 opened by Manfred.Bartz #10219: BufferedReader.read1 does not check for closed file http://bugs.python.org/issue10219 opened by amaury.forgeotdarc #10220: Make generator state easier to introspect http://bugs.python.org/issue10220 opened by ncoghlan #10221: {}.pop('a') raises non-standard KeyError exception http://bugs.python.org/issue10221 opened by djc #10224: Build 3.x documentation using python3.x http://bugs.python.org/issue10224 opened by belopolsky #10225: Fix doctest runable examples in python manual http://bugs.python.org/issue10225 opened by belopolsky #10226: urlparse example is wrong http://bugs.python.org/issue10226 opened by belopolsky #10227: Improve performance of MemoryView slicing http://bugs.python.org/issue10227 opened by krisvale #10228: Refleak run of test_dbm fails when several dbm modules are ava http://bugs.python.org/issue10228 opened by pitrou #10229: Refleak run of test_gettext fails http://bugs.python.org/issue10229 opened by pitrou #10230: test_tarfile failure (test_extractall) on AMD64 debian paralle http://bugs.python.org/issue10230 opened by haypo #10231: SimpleHTTPRequestHandler directory bugs http://bugs.python.org/issue10231 opened by hfuru #10232: Tkinter issues with Scrollbar and custom widget list http://bugs.python.org/issue10232 opened by Robert.Lerche #10233: fix test_tarfile ResourceWarnings http://bugs.python.org/issue10233 opened by pitrou #1610654: cgi.py multipart/form-data http://bugs.python.org/issue1610654 reopened by r.david.murray Most recent 15 issues with no replies (15) ========================================== #10233: fix test_tarfile ResourceWarnings http://bugs.python.org/issue10233 #10232: Tkinter issues with Scrollbar and custom widget list http://bugs.python.org/issue10232 #10231: SimpleHTTPRequestHandler directory bugs http://bugs.python.org/issue10231 #10228: Refleak run of test_dbm fails when several dbm modules are ava http://bugs.python.org/issue10228 #10219: BufferedReader.read1 does not check for closed file http://bugs.python.org/issue10219 #10205: Can't have two tags with the same QName http://bugs.python.org/issue10205 #10183: test_concurrent_futures failure on Windows http://bugs.python.org/issue10183 #10182: match_start truncates large values http://bugs.python.org/issue10182 #10181: get_shape0 in memoryobject.c not checked for error return http://bugs.python.org/issue10181 #10177: PyUnicode_AsWideCharString and PyMem_Free http://bugs.python.org/issue10177 #10173: Don't pickle TestCase instances in test_multiprocessing http://bugs.python.org/issue10173 #10170: Relationship between turtle speed setting and actual speed is http://bugs.python.org/issue10170 #10169: socket.sendto raises incorrect exception when passed incorrect http://bugs.python.org/issue10169 #10138: calendar module does not support years outside [1, 9999] range http://bugs.python.org/issue10138 #10136: kill_python doesn't work with short path http://bugs.python.org/issue10136 Most recent 15 issues waiting for review (15) ============================================= #10233: fix test_tarfile ResourceWarnings http://bugs.python.org/issue10233 #10229: Refleak run of test_gettext fails http://bugs.python.org/issue10229 #10227: Improve performance of MemoryView slicing http://bugs.python.org/issue10227 #10225: Fix doctest runable examples in python manual http://bugs.python.org/issue10225 #10221: {}.pop('a') raises non-standard KeyError exception http://bugs.python.org/issue10221 #10212: struct.unpack and cStringIO.StringIO don't support new buffer http://bugs.python.org/issue10212 #10211: BufferObject doesn't support new buffer interface http://bugs.python.org/issue10211 #10206: python program starting with unmatched quote spews spaces to s http://bugs.python.org/issue10206 #10205: Can't have two tags with the same QName http://bugs.python.org/issue10205 #10199: Move Demo/turtle under Lib/ http://bugs.python.org/issue10199 #10189: SyntaxError: no binding for nonlocal doesn't contain a useful http://bugs.python.org/issue10189 #10184: tarfile touches directories twice http://bugs.python.org/issue10184 #10180: File objects should not pickleable http://bugs.python.org/issue10180 #10179: os.stat fails on mapped network drive http://bugs.python.org/issue10179 #10173: Don't pickle TestCase instances in test_multiprocessing http://bugs.python.org/issue10173 Top 10 most discussed issues (10) ================================= #4111: Add Systemtap/DTrace probes http://bugs.python.org/issue4111 15 msgs #10199: Move Demo/turtle under Lib/ http://bugs.python.org/issue10199 12 msgs #6715: xz compressor support http://bugs.python.org/issue6715 11 msgs #10227: Improve performance of MemoryView slicing http://bugs.python.org/issue10227 9 msgs #10179: os.stat fails on mapped network drive http://bugs.python.org/issue10179 8 msgs #10224: Build 3.x documentation using python3.x http://bugs.python.org/issue10224 8 msgs #6011: python doesn't build if prefix contains non-ascii characters http://bugs.python.org/issue6011 7 msgs #7061: Improve 24.5. turtle doc http://bugs.python.org/issue7061 7 msgs #10142: Support for SEEK_HOLE/SEEK_DATA http://bugs.python.org/issue10142 7 msgs #10180: File objects should not pickleable http://bugs.python.org/issue10180 7 msgs Issues closed (56) ================== #2703: SimpleXMLRPCDispatcher.__init__ is not python2.4-backward-comp http://bugs.python.org/issue2703 closed by georg.brandl #3018: tkinter demos fixed http://bugs.python.org/issue3018 closed by georg.brandl #3362: locale.getpreferredencoding() gives bus error on Mac OS X 10.4 http://bugs.python.org/issue3362 closed by ned.deily #4828: patch suggestion for webbrowser http://bugs.python.org/issue4828 closed by georg.brandl #5027: xml namespace not understood by xml.sax.saxutils.XMLGenerator http://bugs.python.org/issue5027 closed by pitrou #5178: Add context manager for temporary directory http://bugs.python.org/issue5178 closed by ncoghlan #5251: contextlib.nested inconsistent with, well, nested with stateme http://bugs.python.org/issue5251 closed by ncoghlan #5437: Singleton MemoryError can hold traceback data and locals indef http://bugs.python.org/issue5437 closed by pitrou #5639: Support TLS SNI extension in ssl module http://bugs.python.org/issue5639 closed by pitrou #5688: inability to ignore multiline warnings -- request to add re.DO http://bugs.python.org/issue5688 closed by georg.brandl #5975: csv unix file format ('\n' line terminator) http://bugs.python.org/issue5975 closed by georg.brandl #6008: Idle should be installed as `idle3.1` and not `idle3` http://bugs.python.org/issue6008 closed by ned.deily #6518: Enable 'with' statement in ossaudiodev module http://bugs.python.org/issue6518 closed by georg.brandl #6645: multiprocessing build fails on AIX - /dev/urandom (or equivale http://bugs.python.org/issue6645 closed by jnoller #6668: locale.py: can't parse sr_RS at latin locale http://bugs.python.org/issue6668 closed by r.david.murray #7167: Smarter FTP passive mode http://bugs.python.org/issue7167 closed by georg.brandl #7696: Improve Memoryview/Buffer documentation http://bugs.python.org/issue7696 closed by benjamin.peterson #7761: telnetlib Telnet.interact fails on Windows but not Linux http://bugs.python.org/issue7761 closed by r.david.murray #8332: regrtest single TestClass/test_method http://bugs.python.org/issue8332 closed by sandro.tosi #8423: tiger buildbot: test_pep277 failures http://bugs.python.org/issue8423 closed by haypo #8755: idle crash UnicodeDecodeError utf-8 codec http://bugs.python.org/issue8755 closed by ned.deily #8761: PyUnicode_CompareWithASCIIString name is not mangled (UCS2, UC http://bugs.python.org/issue8761 closed by haypo #8777: Add threading.Barrier http://bugs.python.org/issue8777 closed by krisvale #8852: _socket fails to build on OpenSolaris x64 http://bugs.python.org/issue8852 closed by pitrou #9295: test_close_open_print_buffered(test_file) sometimes crashes http://bugs.python.org/issue9295 closed by pitrou #9762: PEP 3149 related build failures http://bugs.python.org/issue9762 closed by barry #10049: Add a "no-op" (null) context manager to contextlib http://bugs.python.org/issue10049 closed by ncoghlan #10077: Python 3.1: site error is not logged http://bugs.python.org/issue10077 closed by haypo #10093: Warn when files are not explicitly closed http://bugs.python.org/issue10093 closed by pitrou #10143: Update "os.pathconf" values http://bugs.python.org/issue10143 closed by jcea #10153: Memory leak in pythonrun.c http://bugs.python.org/issue10153 closed by skrah #10161: unittest: use ascii() instead of repr() to display values on e http://bugs.python.org/issue10161 closed by haypo #10176: telnetlib.Telnet.read_very_eager() performance http://bugs.python.org/issue10176 closed by r.david.murray #10178: PEP 378 uses replace where translate may work better http://bugs.python.org/issue10178 closed by rhettinger #10185: Py_hash_t declaration needed in _testcapimodule http://bugs.python.org/issue10185 closed by pitrou #10186: Invalid SyntaxError pointer offset http://bugs.python.org/issue10186 closed by benjamin.peterson #10187: exec encode unicode to utf-8 str automatically in GBK environm http://bugs.python.org/issue10187 closed by loewis #10192: 'from urllib.parse import *' does not import urlencode functio http://bugs.python.org/issue10192 closed by orsenthil #10193: Simplify instrospection used by turtle module http://bugs.python.org/issue10193 closed by belopolsky #10194: Add gc.remap() function to the gc module. http://bugs.python.org/issue10194 closed by pingebretson #10198: wave module writes corrupt wav file for zero-length writeframe http://bugs.python.org/issue10198 closed by georg.brandl #10200: Documentation: "link use"? http://bugs.python.org/issue10200 closed by georg.brandl #10201: Fix building of socket module under Solaris http://bugs.python.org/issue10201 closed by pitrou #10204: exec string fails with trailing whitespace http://bugs.python.org/issue10204 closed by benjamin.peterson #10207: test_file2k crashes under Windows http://bugs.python.org/issue10207 closed by pitrou #10208: add mazovia.py to encodings/ http://bugs.python.org/issue10208 closed by lemburg #10209: Mac OS X: Decompose filenames on encode, and precompose filena http://bugs.python.org/issue10209 closed by haypo #10210: os.get_exec_path() should not produce any warning http://bugs.python.org/issue10210 closed by haypo #10214: Misc/python-mode.el is out of date. http://bugs.python.org/issue10214 closed by barry #10215: Mac Python 3.1 installer does less than earlier installers http://bugs.python.org/issue10215 closed by r.david.murray #10216: json.loads() on str erroneously returns str http://bugs.python.org/issue10216 closed by pitrou #10218: Add a return value to threading.Condition.wait() http://bugs.python.org/issue10218 closed by georg.brandl #10222: 3.2 on AIX - Unexpected text ',' encountered. http://bugs.python.org/issue10222 closed by georg.brandl #10223: socket.gethostname() fail http://bugs.python.org/issue10223 closed by EcmaXp #1349106: email.Generator does not separate headers with "\r\n" http://bugs.python.org/issue1349106 closed by r.david.murray #1772788: chr(128) in u'only ascii' -> TypeError with misleading msg http://bugs.python.org/issue1772788 closed by georg.brandl From tseaver at palladion.com Fri Oct 29 18:22:09 2010 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 29 Oct 2010 12:22:09 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <4CCAD6A4.9040503@egenix.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/29/2010 10:21 AM, Benjamin Peterson wrote: > 2010/10/29 M.-A. Lemburg : >> Benjamin Peterson wrote: >>> He's not saying we shouldn't welcome them; we just don't want to it >>> attached to python-dev. >> >> That new team could be part of python-dev, couldn't it ? Not necessarily >> the mailing list, but the team of Python developers. Much like the >> (new) py3k developers joined in when that project was kicked off. > > Perhaps, but it would debatable how much infrastructure (ie buildbots, > tracker) would be available to them. "Infrastructure" sounds to me like code for "money". How much of the PSF's money, for instance, comes from organizations whose primary interest is still Python2? How many of them are only or principally only interested in Python3? Then again, how much of the PSF's budget goes toward infrastructure? (I honestly don't know the answers to any of these questions). For "donated" infrastructure, surely the individuals providing CPU / bandwidth / diskspace make that call, and not python-dev. If a new retro-fork development community emerges, its members will include folks who have a vested interest in continuing Python 2.x development, and hence can donate (or recruit) such in-kind contributions. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkzK9LEACgkQ+gerLs4ltQ5obQCdEaSgkZ+vG8RlzCHQTzhLEyCb jkYAn00pBS0aPZ0AS05hKqbP3/TpA4pb =AHme -----END PGP SIGNATURE----- From exarkun at twistedmatrix.com Fri Oct 29 18:41:19 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 29 Oct 2010 16:41:19 -0000 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: <20101029164119.2040.1998748350.divmod.xquotient.291@localhost.localdomain> On 02:51 am, brett at python.org wrote: >2010/10/28 Kristj?n Valur J?nsson : >>Hi all. >>This has been a lively discussion. >>My desire to keep 2.x alive in some sense is my own and I don't know >>if anyone shares it but as a member of this community I think I'm >>allowed to voice it. ?So, just to clarify my particular position, let >>me explain where all this comes from. >[snip] > >And as everyone has said so far (and with which I agree), that's fine. >As long as it is not called Python 2.8 -- EVE-Python 2.8 or some Monty >Python reference -- then that's fine. And as pointed out by folks, >once Hg kicks in and we have user repos you can even host it on >hg.python.org yourself to give it some semblance of authority if you >want. In case anyone was discouraged by the idea that a 2.x continuation would not be allowed to bear the name "Python" as Brett suggests here, I want to make a clarification. Brett is speaking for himself here (and he never claimed otherwise!). However, decisions about where to allow the use of the "Python" trademark are made by the Python Software Foundation. The PSF has not decided to reject any request by a 2.x continuation project to use the name "Python". Of course, this does not mean they would necessarily allow such a use. I just wanted to point out that they have not categorically rejected it, as one might be tempted to infer from Brett's message. Jean-Paul From martin at v.loewis.de Fri Oct 29 18:57:54 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 29 Oct 2010 18:57:54 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <4CCAD6A4.9040503@egenix.com> Message-ID: <4CCAFD12.7090300@v.loewis.de> > "Infrastructure" sounds to me like code for "money". No, it's rather "volunteer time". Of course, people keep proposing that this should be replaced by hired time that gets paid from donations, but all such proposals so far got stuck at implementation details (i.e. it's actual work that nobody has done). > How much of the > PSF's money, for instance, comes from organizations whose primary > interest is still Python2? How many of them are only or principally > only interested in Python3? Then again, how much of the PSF's budget > goes toward infrastructure? The first two questions are difficult to answer: the PSF doesn't maintain records of what Python versions are of primary interest to sponsor members. A significant portion of the donations comes from the conference surplus (being saved for the also-likely risk of a massive conference loss); in this case, it's even difficult to identify the donors (as you can't really attribute the surplus to being from, say, attendee fees, as opposed to conference sponsors). As for the budget that goes into infrastructure: you'll find the details in the treasurer reports, but it is comparatively minor and goes primarily into hardware purchases. Connectivity and colocation is donated by companies who may not have an actual interest in Python at all (e.g. XS4ALL, which do this out of a general support for free software and in positive recollection of their former employee Thomas Wouters). > For "donated" infrastructure, surely the individuals providing CPU / > bandwidth / diskspace make that call, and not python-dev. Yes, and I have already stated my opinion. Other pydotorg'ers will surely voice their opinion when they get asked to help. Regards, Martin From solipsis at pitrou.net Fri Oct 29 18:58:11 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Oct 2010 18:58:11 +0200 Subject: [Python-Dev] Continuing 2.x References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <20101029164119.2040.1998748350.divmod.xquotient.291@localhost.localdomain> Message-ID: <20101029185811.0487ac32@pitrou.net> On Fri, 29 Oct 2010 16:41:19 -0000 exarkun at twistedmatrix.com wrote: > > Brett is speaking for himself here (and he never claimed otherwise!). > However, decisions about where to allow the use of the "Python" > trademark are made by the Python Software Foundation. The point is not to allow the use of a trademark ("EVE-Python" is already an use of the trademark, as are "IronPython", "Cython", "VPython", etc.), it is to respect the original project and to keep things clear. Even if there were no trademark, I think it would be wrong for a separate project to adopt the same name without agreement from the original group of contributors. I have never seen a fork which didn't change the name of the project. Regards Antoine. From ctb at msu.edu Fri Oct 29 19:07:53 2010 From: ctb at msu.edu (C. Titus Brown) Date: Fri, 29 Oct 2010 10:07:53 -0700 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <4CCAFD12.7090300@v.loewis.de> References: <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <4CCAD6A4.9040503@egenix.com> <4CCAFD12.7090300@v.loewis.de> Message-ID: <20101029170753.GA22274@idyll.org> On Fri, Oct 29, 2010 at 06:57:54PM +0200, "Martin v. L?wis" wrote: > > "Infrastructure" sounds to me like code for "money". > > No, it's rather "volunteer time". Of course, people keep proposing > that this should be replaced by hired time that gets paid from > donations, but all such proposals so far got stuck at implementation > details (i.e. it's actual work that nobody has done). > > > How much of the > > PSF's money, for instance, comes from organizations whose primary > > interest is still Python2? How many of them are only or principally > > only interested in Python3? Then again, how much of the PSF's budget > > goes toward infrastructure? > > The first two questions are difficult to answer: the PSF doesn't > maintain records of what Python versions are of primary interest > to sponsor members. A significant portion of the donations comes > from the conference surplus (being saved for the also-likely risk > of a massive conference loss); in this case, it's even difficult to > identify the donors (as you can't really attribute the surplus to > being from, say, attendee fees, as opposed to conference sponsors). > > As for the budget that goes into infrastructure: you'll find the details > in the treasurer reports, but it is comparatively minor and goes > primarily into hardware purchases. Connectivity and colocation is > donated by companies who may not have an actual interest in Python > at all (e.g. XS4ALL, which do this out of a general support for > free software and in positive recollection of their former employee > Thomas Wouters). I'd just like to add my 2c that AFAICT the volunteer effort that goes into Python, and in particular into python-dev and the infrastructure foo, absolutely *dwarfs* all other aspects of "official" Python and PSF (including $$ in all forms). So, good job, -dev guys! But they're already pretty overwhelmed. Independent of talk, unless there's a proposal to continue 2.x that actually involves someone *new* stepping up to put in hugely substantial and ridiculously large amounts of seriously expert time, I don't see the point of talking about it. cheers, --titus p.s. I would be happy to enter into discussions on how to clone Martin and others, though. I just need some epithelial cells, I think. And about $20 bn dollars, and relocation to Israel (which I think has the best combination of tech and human use guidelines for cloning). Martin's permission is not *strictly* necessary but should probably be obtained, too. p.p.s. The PSF isn't foolish enough to let me speak for them, in case anyone is wondering. -- C. Titus Brown, ctb at msu.edu From dmalcolm at redhat.com Fri Oct 29 19:46:03 2010 From: dmalcolm at redhat.com (David Malcolm) Date: Fri, 29 Oct 2010 13:46:03 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101029091112.3aa04bb1@pitrou.net> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <20101029091112.3aa04bb1@pitrou.net> Message-ID: <1288374363.28582.42.camel@radiator.bos.redhat.com> On Fri, 2010-10-29 at 09:11 +0200, Antoine Pitrou wrote: > On Fri, 29 Oct 2010 02:55:55 -0400 > Glyph Lefkowitz wrote: > > > > Let's say that 20% of the code on PyPI is just junk; > > it's unfair to expect 100% of all code ever to get ported. But, > still: > > with this back-of-the-envelope estimate of the rate of porting, it > will > > take over 50 years before a decisive majority of Python code is on > > Python 3. > > Well, no. A decisive majority would be much smaller than that. There > are probably between 2% and 5% of the CheeseShop entries which are > widely used dependencies. When these 2 to 5% all get ported, you have > a > decisive majority. > > Yes, perhaps more than 50% of 2.x code will never get ported. But, > perhaps more than 50% of 1.5.2 code never got upgraded either. That > doesn't make it any decisive; just dead (or pining for security fixes > in some old rusty "RedHat Enterprise Linux" server, if you prefer). Ouch! Having spent much of the last week doublechecking fixes for CVEs in the builds of python 2.2, 2.3 and 2.4 in the various older RHEL releases, that cuts deep :) Red Hat's security team monitors vulnerabilities in Python, and we do continue to support these releases in the context of our products, even though they're no longer supported by the wider Python development community. As with the the security work done by python-dev on the more up-to-date Python releases, it's tedious and painstaking work (we do have customers paying us to do it, though) If you have concerns about specific security flaws that may affect the older releases of python that are no longer supported by python.org but are within a product supported by Red Hat, please email secalert at redhat.com See: https://www.redhat.com/security/team/contact/ for more information. Hope this is helpful Dave From eric at trueblade.com Fri Oct 29 20:03:34 2010 From: eric at trueblade.com (Eric Smith) Date: Fri, 29 Oct 2010 14:03:34 -0400 Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles In-Reply-To: References: Message-ID: <4CCB0C76.8070305@trueblade.com> On 10/26/10 7:08 AM, Nick Coghlan wrote: > On Tue, Oct 26, 2010 at 12:28 AM, Vinay Sajip wrote: >> Comments welcome. Assuming there are no strong objections asking for reversion >> of this change, I'll publicise to the wider community in a few days. > > It strikes me as a solid, pragmatic solution to a thorny problem. I keep meaning to review this but haven't had time. One thing I want to look at specifically is the ability to put the time formatting into the str.format version of the format string. Now that the time format specifier can be included in the format string, it's no longer necessary to have the asctime inspection hack that's currently used in order to avoid formatting the time. It would be good if we could remove the formatTime logic for str.format, although I'm not sure how practical it is. I suspect it's too baked-in to the design, but I'm hopeful. I'm +1 on the overall concept of allow the other string formatters. -- Eric. From debatem1 at gmail.com Fri Oct 29 20:12:28 2010 From: debatem1 at gmail.com (geremy condra) Date: Fri, 29 Oct 2010 11:12:28 -0700 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: On Thu, Oct 28, 2010 at 11:55 PM, Glyph Lefkowitz wrote: > On Oct 28, 2010, at 10:51 PM, Brett Cannon wrote: > > I think people need to stop viewing the difference between Python 2.7 > and Python 3.2 as this crazy shift and view it from python-dev's > perspective; it should be viewed one follows from the other at this > point. You can view it as Python 3.2 is the next version after Python > 2.7 just like 2.7 followed to 2.6, which makes the policies we follow > for releases make total sense and negates this discussion. It just so > happens people don't plan to switch to the newest release immediately > as the backward-incompatible changes are more involved than what > people are used to from past releases. > > Brett, with all due respect, this is not a reasonable position. ?You are > making it sound like the popular view of 3.2 is a "crazy shift" is based on > a personal dislike of python-dev or something. ?The fact is that the amount > of effort required to port to 3.2 is extreme compared to previous upgrades, > and most people still aren't willing to deal with it. ?It is a crazy shift. > Let's take PyPI numbers as a proxy. ?There are ~8000 packages with a > "Programming Language::Python" classifier. ?There are ~250 with "Programming > Langauge::Python::3". ?Roughly speaking, we can say that is 3% of Python > code which has been ported so far. ?Python 3.0 was released at the end of > 2008, so people have had roughly 2 years to port, which comes up with 1.5% > per year. > Let's say that 20% of the code on PyPI is just junk; it's unfair to expect > 100% of all code ever to get ported. ?But, still: with this > back-of-the-envelope estimate of the rate of porting, it will take over 50 > years before a decisive majority of Python code is on Python 3. > By contrast, there are 536 packages with ::2.6, and 177 with ::2.7. ?(Trying > to compare apples to apples here, since I assume the '2' tag is much more > lightly used than '3' to identify supported versions; I figure someone > likely to tag one micro-version would also tag the other.) Just my two cents: First off, unless you have a lot of information I don't, there's no reason at all to believe that Python3's adoption will be linear- if anything it seems very likely to be a power law curve, just like the adoption trends we see for other software projects. Secondly, speaking of power law curves, not all packages on PyPI are equally important as measured by downloads. If even the top 2% of most downloaded projects on PyPI got ported to Python3 it would represent a majority of downloads, and that pattern seems consistent with what you see in other software repositories. The net result of this is that even if you're right and the growth is linear AND the 1.5% statistic is accurate- again, I doubt it- it is conceivable that within two years an overwhelming majority of downloads of Python software could be of projects with full Python3 support. From a user perspective that's fairly close to indistinguishable from a community-wide transition to Python3. Thirdly, only counting PyPI is probably not a very good way to measure popularity in the wider community, as PyPI's overall usage is fairly small. Just to name a few examples, sqlalchemy had been pulled just 2,000 times from PyPI, but 86,000 times from Ubuntu's repo, pyparsing had just 295 downloads on PyPI but 62,000 from Ubuntu, etc. With a few exceptions- especially modules that pride themselves on being pure Python- this is pretty indicative of the general relationship between PyPI and these other repositories. I don't know how the adoption rate figures would look if we took those into account, but if you want to portray the trend accurately that's something you're likely to have to do. So, to cut down a long-winded and overly pedantic response: I'm pretty sure that you're not going to get accurate estimates out of that methodology, assuming that accurate results are what you're after. Geremy Condra From debatem1 at gmail.com Fri Oct 29 20:24:46 2010 From: debatem1 at gmail.com (geremy condra) Date: Fri, 29 Oct 2010 11:24:46 -0700 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: On Fri, Oct 29, 2010 at 11:12 AM, geremy condra wrote: > On Thu, Oct 28, 2010 at 11:55 PM, Glyph Lefkowitz > wrote: >> On Oct 28, 2010, at 10:51 PM, Brett Cannon wrote: [snip] > First off, unless you have a lot of information I don't, there's no > reason at all to believe that Python3's adoption will be linear- if > anything it seems very likely to be a power law curve, just like the > adoption trends we see for other software projects. [snip] Just a correction, I'm predicting that Python3's adoption will be exponential and that it's popularity relative to other software projects will move up according to a power law curve, not that it's overall adoption will be power law. Thanks for pointing this out, Titus. Geremy Condra From benjamin at python.org Fri Oct 29 20:36:04 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 29 Oct 2010 13:36:04 -0500 Subject: [Python-Dev] [Python-checkins] r85934 - in python/branches/py3k: Misc/NEWS Modules/socketmodule.c In-Reply-To: <20101029182008.DFE35EEBE1@mail.python.org> References: <20101029182008.DFE35EEBE1@mail.python.org> Message-ID: 2010/10/29 martin.v.loewis : > Author: martin.v.loewis > Date: Fri Oct 29 20:20:08 2010 > New Revision: 85934 > > Log: > Issue #9377: Use Unicode API for gethostname on Windows. > > Modified: > python/branches/py3k/Misc/NEWS > python/branches/py3k/Modules/socketmodule.c > > Modified: python/branches/py3k/Misc/NEWS > ============================================================================== > --- python/branches/py3k/Misc/NEWS (original) > +++ python/branches/py3k/Misc/NEWS Fri Oct 29 20:20:08 2010 > @@ -160,6 +160,8 @@ > Extensions > ---------- > > +- Issue #9377: Use Unicode API for gethostname on Windows. > + > - Issue #10143: Update "os.pathconf" values. > > - Issue #6518: Support context manager protcol for ossaudiodev types. > > Modified: python/branches/py3k/Modules/socketmodule.c > ============================================================================== > --- python/branches/py3k/Modules/socketmodule.c (original) > +++ python/branches/py3k/Modules/socketmodule.c Fri Oct 29 20:20:08 2010 > @@ -3093,6 +3093,27 @@ > static PyObject * > socket_gethostname(PyObject *self, PyObject *unused) > { > +#ifdef MS_WINDOWS > + /* Don't use winsock's gethostname, as this returns the ANSI > + version of the hostname, whereas we need a Unicode string. > + Otherwise, gethostname apparently also returns the DNS name. */ > + wchar_t buf[MAX_COMPUTERNAME_LENGTH]; > + DWORD size = sizeof(buf); > + if (!GetComputerNameExW(ComputerNamePhysicalDnsHostname, buf, &size)) { > + if (GetLastError() == ERROR_MORE_DATA) { > + /* MSDN says this may occur "because DNS allows longer names */ > + PyObject *result = PyUnicode_FromUnicode(NULL, size); > + if (!result) > + return NULL; > + if (GetComputerName(ComputerNamePhysicalDnsHostname, > + PyUnicode_AS_UNICODE(result), > + size+1)) > + return result; A reference leak (of result) occurs here if GetComputerName fails again. -- Regards, Benjamin From a.badger at gmail.com Fri Oct 29 20:41:04 2010 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 29 Oct 2010 11:41:04 -0700 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> Message-ID: <20101029184104.GW14816@unaka.lan> On Fri, Oct 29, 2010 at 11:12:28AM -0700, geremy condra wrote: > On Thu, Oct 28, 2010 at 11:55 PM, Glyph Lefkowitz > > Let's take PyPI numbers as a proxy. ?There are ~8000 packages with a > > "Programming Language::Python" classifier. ?There are ~250 with "Programming > > Langauge::Python::3". ?Roughly speaking, we can say that is 3% of Python > > code which has been ported so far. ?Python 3.0 was released at the end of > > 2008, so people have had roughly 2 years to port, which comes up with 1.5% > > per year. > Just my two cents: > Just one further informational note about using pypi in this way for statistics... In the porting work we've done within Fedora, I've noticed that a lot of packages are python3 ready or even officially support python3 but the language classifier on pypi does not reflect this. Here's just a few since I looked them up when working on the python porting wiki pages: http://pypi.python.org/pypi/Beaker/ http://pypi.python.org/pypi/pycairo http://pypi.python.org/pypi/docutils -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From casey at pandora.com Fri Oct 29 20:43:28 2010 From: casey at pandora.com (Casey Duncan) Date: Fri, 29 Oct 2010 12:43:28 -0600 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <87eib9r3v4.fsf@uwakimon.sk.tsukuba.ac.jp> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <87eib9r3v4.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <7ED03BC9-40A5-4974-8D02-7DFD6CC6711D@pandora.com> On Oct 28, 2010, at 10:59 PM, Stephen J. Turnbull wrote: > Mark's position is different. His words suggest that he thinks that > Python.org owes the users something, although if pressed I imagine > he'd present some argument that more users will lead to development of > a better language. I think the developers universally consider that > to be objectively false: Python 3 is a much better language, and is on > track to be a much better environment for development -- of itself and > of applications -- in 2013 than Python 2 could conceivably be. There is tension here. python-dev wants Python to succeed, and now Python == Python 3.x. That means end-of-lifing Python 2.x, for many reasons, not the least of which is that more Python 2.x releases are a disincentive for folks to move their projects to Python 3.x. However there are many many more users of Python 2.x than Python 3.x. Many may never upgrade for the life of these projects, because if it ain't broke, why fix it? It doesn't matter how much better Python 3 is than Python 2. It isn't better enough. I like Python 3, I am using it for my latest projects, but I am also keeping Python 2 compatibility. This incurs some overhead, and basically means I am still really only using Python 2 features. So in some respects, my Python 3.x support is only tacit, it works as well as for Python 2, but it's not taking advantage of Python 3 really. I haven't run into a situation yet where I really want to or have to use Python 3 exclusive features, but then again I'm not really learning to use Python 3 either, short of the new C api. In this regard the existence of Python 3 is a disadvantage, not an advantage for my new code, regardless of how much better a language or dev environment it may be. Of course I made the choice to support both 2 and 3, but it was largely informed by the fact that other dependancies for my projects currently only support Python 2 and I don't have the spare time to port them right now. So at least right now, for me, Python 3 is not helping make new projects easier, it is the contrary unfortunately. Yeah, I am getting older and the years are going by faster, but gosh 2013 still feels like a ways off. -Casey From barry at python.org Fri Oct 29 21:21:25 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Oct 2010 15:21:25 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <7ED03BC9-40A5-4974-8D02-7DFD6CC6711D@pandora.com> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <87eib9r3v4.fsf@uwakimon.sk.tsukuba.ac.jp> <7ED03BC9-40A5-4974-8D02-7DFD6CC6711D@pandora.com> Message-ID: <20101029152125.04609bb1@mission> On Oct 29, 2010, at 12:43 PM, Casey Duncan wrote: >I like Python 3, I am using it for my latest projects, but I am also keeping >Python 2 compatibility. This incurs some overhead, and basically means I am >still really only using Python 2 features. So in some respects, my Python 3.x >support is only tacit, it works as well as for Python 2, but it's not taking >advantage of Python 3 really. I haven't run into a situation yet where I >really want to or have to use Python 3 exclusive features, but then again I'm >not really learning to use Python 3 either, short of the new C api. One thing that *might* be interesting to explore for Python 3.3 would be something like `python3 --1` or some such switch that would help Python 2 code run more easily in Python 3. This might be a hook to 2to3 or other internal changes that help some of the trickier bits of writing cross-compatible code. I don't know what those things enabled by --1 would be. Some syntactic things might be fairly easy though largely inconsequential (e.g. print() -> print). It might be that large changes like bytes/string dwarfs syntactic sugar. I had a brief conversation with Michael Foord yesterday and he's writing code that works in 2.4 through 3.2, so for *some* code bases, it's tricky and ugly, but possible. IMHO, those are the kinds of directions we should be thinking about, to help existing code more easily make the jump to Python 3. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ianb at colorstudy.com Fri Oct 29 21:31:25 2010 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 29 Oct 2010 12:31:25 -0700 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101029152125.04609bb1@mission> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <87eib9r3v4.fsf@uwakimon.sk.tsukuba.ac.jp> <7ED03BC9-40A5-4974-8D02-7DFD6CC6711D@pandora.com> <20101029152125.04609bb1@mission> Message-ID: On Fri, Oct 29, 2010 at 12:21 PM, Barry Warsaw wrote: > On Oct 29, 2010, at 12:43 PM, Casey Duncan wrote: > > >I like Python 3, I am using it for my latest projects, but I am also > keeping > >Python 2 compatibility. This incurs some overhead, and basically means I > am > >still really only using Python 2 features. So in some respects, my Python > 3.x > >support is only tacit, it works as well as for Python 2, but it's not > taking > >advantage of Python 3 really. I haven't run into a situation yet where I > >really want to or have to use Python 3 exclusive features, but then again > I'm > >not really learning to use Python 3 either, short of the new C api. > > One thing that *might* be interesting to explore for Python 3.3 would be > something like `python3 --1` or some such switch that would help Python 2 > code > run more easily in Python 3. This might be a hook to 2to3 or other > internal > changes that help some of the trickier bits of writing cross-compatible > code. > More useful IMHO would be things like "from __past__ import print_statement", still requiring some annotation of code to make it run, but less invasive than translating code itself. There's still major things you can't handle like that, but if something is syntactically acceptable in both Python 2 and 3 then it's a lot easier to apply simple conditionals around semantics. This would remove the need, for example, for people to use sys.exc_info() to avoid using "except Exception as e". -- Ian Bicking | http://blog.ianbicking.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Fri Oct 29 21:38:13 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Oct 2010 15:38:13 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <4CCACF52.8090202@egenix.com> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> Message-ID: On 10/29/2010 9:42 AM, M.-A. Lemburg wrote: > I don't see why we should not welcome a team of new developers who want > to continue working on the 2.x series. Given the number of issues on the tracker, I think it would be great if there were some new 2.7-focused developers that would work on fixing 2.7-specific bugs and helping with fixes (including by backporting) to 2.7/3.x bugs. > It's obvious that a large proportion of the existing python-dev'ers will > not participate in such a project, but why should we try to stop someone > else to work on it ? Where is such a team? It is a moot point until they show up. As to a possible successor to 2.7: this seems hardly worth discussing until 1) 2.7 has been out for at year and maybe more; 2) there actually are such new developers working on 2.7 maintenance; and 3) there actually is a proposal to respond to. If new features were limited to backports of features in 3.x, especially in the library, then I *personally* could see something being released as 'python 2.8'. I will be surprised it these preconditions come about. I suspect that most 2.7 users and most *nix distributions would be happy to have a stable increasingly de-bugged 2.7 be Python 2 for several years. -- Terry Jan Reedy From barry at python.org Fri Oct 29 21:54:19 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Oct 2010 15:54:19 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> Message-ID: <07F03E8C-347C-4E07-A6BC-7D0029F056EC@python.org> Another quick thought. What would people think about regular timed releases if python 2.7? This is probably more a question for Benjamin but doing sonmight provide better predictability and "customer service" to our users. I might like to see monthly releases but even quarterly would probably be useful. Doing timed releases might also incentivize folks to fix more outstanding 2.7 bugs. Sent from my digital lollipop. On Oct 29, 2010, at 3:38 PM, Terry Reedy wrote: > On 10/29/2010 9:42 AM, M.-A. Lemburg wrote: > >> I don't see why we should not welcome a team of new developers who want >> to continue working on the 2.x series. > > Given the number of issues on the tracker, I think it would be great if there were some new 2.7-focused developers that would work on fixing 2.7-specific bugs and helping with fixes (including by backporting) to 2.7/3.x bugs. > >> It's obvious that a large proportion of the existing python-dev'ers will >> not participate in such a project, but why should we try to stop someone >> else to work on it ? > > Where is such a team? It is a moot point until they show up. > > As to a possible successor to 2.7: this seems hardly worth discussing until 1) 2.7 has been out for at year and maybe more; 2) there actually are such new developers working on 2.7 maintenance; and 3) there actually is a proposal to respond to. If new features were limited to backports of features in 3.x, especially in the library, then I *personally* could see something being released as 'python 2.8'. > > I will be surprised it these preconditions come about. I suspect that most 2.7 users and most *nix distributions would be happy to have a stable increasingly de-bugged 2.7 be Python 2 for several years. > > -- > Terry Jan Reedy > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/barry%40python.org From tjreedy at udel.edu Fri Oct 29 21:55:33 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 29 Oct 2010 15:55:33 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101029184104.GW14816@unaka.lan> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <20101029184104.GW14816@unaka.lan> Message-ID: On 10/29/2010 2:41 PM, Toshio Kuratomi wrote: > On Fri, Oct 29, 2010 at 11:12:28AM -0700, geremy condra wrote: >> On Thu, Oct 28, 2010 at 11:55 PM, Glyph Lefkowitz >>> Let's take PyPI numbers as a proxy. There are ~8000 packages with a >>> "Programming Language::Python" classifier. There are ~250 with "Programming >>> Langauge::Python::3". Roughly speaking, we can say that is 3% of Python >>> code which has been ported so far. Python 3.0 was released at the end of >>> 2008, so people have had roughly 2 years to port, which comes up with 1.5% >>> per year. >> Just my two cents: >> > Just one further informational note about using pypi in this way for > statistics... In the porting work we've done within Fedora, I've noticed > that a lot of packages are python3 ready or even officially support python3 > but the language classifier on pypi does not reflect this. Here's just > a few since I looked them up when working on the python porting wiki pages: > > http://pypi.python.org/pypi/Beaker/ > http://pypi.python.org/pypi/pycairo > http://pypi.python.org/pypi/docutils If you could (successfully) encourage the authors of such packages to update their PyPI classifiers, I and other Python 3 users would greatly appreciate it. That is aside from having better data for this and similar discussions. -- Terry Jan Reedy From barry at python.org Fri Oct 29 21:56:03 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Oct 2010 15:56:03 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <87eib9r3v4.fsf@uwakimon.sk.tsukuba.ac.jp> <7ED03BC9-40A5-4974-8D02-7DFD6CC6711D@pandora.com> <20101029152125.04609bb1@mission> Message-ID: <49173690-CBB7-4760-923D-A88948A376F0@python.org> That's a much better idea! Sent from my digital lollipop. On Oct 29, 2010, at 3:31 PM, Ian Bicking wrote: > On Fri, Oct 29, 2010 at 12:21 PM, Barry Warsaw wrote: > On Oct 29, 2010, at 12:43 PM, Casey Duncan wrote: > > >I like Python 3, I am using it for my latest projects, but I am also keeping > >Python 2 compatibility. This incurs some overhead, and basically means I am > >still really only using Python 2 features. So in some respects, my Python 3.x > >support is only tacit, it works as well as for Python 2, but it's not taking > >advantage of Python 3 really. I haven't run into a situation yet where I > >really want to or have to use Python 3 exclusive features, but then again I'm > >not really learning to use Python 3 either, short of the new C api. > > One thing that *might* be interesting to explore for Python 3.3 would be > something like `python3 --1` or some such switch that would help Python 2 code > run more easily in Python 3. This might be a hook to 2to3 or other internal > changes that help some of the trickier bits of writing cross-compatible code. > > More useful IMHO would be things like "from __past__ import print_statement", still requiring some annotation of code to make it run, but less invasive than translating code itself. There's still major things you can't handle like that, but if something is syntactically acceptable in both Python 2 and 3 then it's a lot easier to apply simple conditionals around semantics. This would remove the need, for example, for people to use sys.exc_info() to avoid using "except Exception as e". > > -- > Ian Bicking | http://blog.ianbicking.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.rodola at gmail.com Fri Oct 29 21:56:37 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Fri, 29 Oct 2010 21:56:37 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101029152125.04609bb1@mission> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <87eib9r3v4.fsf@uwakimon.sk.tsukuba.ac.jp> <7ED03BC9-40A5-4974-8D02-7DFD6CC6711D@pandora.com> <20101029152125.04609bb1@mission> Message-ID: 2010/10/29 Barry Warsaw : > I had a brief conversation with Michael Foord yesterday and he's writing code > that works in 2.4 through 3.2, so for *some* code bases, it's tricky and ugly, > but possible. If the application does not involve a lot of I/O, 2.4 -> 3.2 support by using a unique code base is possible and not that ugly. At least, that's what happened with psutil: http://code.google.com/p/psutil/issues/detail?id=75&can=1&q=python%203&colspec=ID%20Summary%20Type%20Opsys%20Status%20Milestone%20Opened%20Owner%20Progress#c9 My personal choice is to encourage the same approach where possible. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil From solipsis at pitrou.net Fri Oct 29 22:06:32 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 29 Oct 2010 22:06:32 +0200 Subject: [Python-Dev] Continuing 2.x References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <07F03E8C-347C-4E07-A6BC-7D0029F056EC@python.org> Message-ID: <20101029220632.170464f5@pitrou.net> On Fri, 29 Oct 2010 15:54:19 -0400 Barry Warsaw wrote: > Another quick thought. What would people think about regular timed releases if python 2.7? > This is probably more a question for Benjamin but doing sonmight > provide better predictability and "customer service" to our users. I > might like to see monthly releases but even quarterly would probably > be useful. Doing timed releases might also incentivize folks to fix > more outstanding 2.7 bugs. Why would it only apply to 2.7? Regards Antoine. From barry at python.org Fri Oct 29 22:32:36 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Oct 2010 16:32:36 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101029220632.170464f5@pitrou.net> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <07F03E8C-347C-4E07-A6BC-7D0029F056EC@python.org> <20101029220632.170464f5@pitrou.net> Message-ID: <645A477A-EB2A-4C6C-97A4-E169C492F6AA@python.org> It certainly doesn't have to. Sent from my digital lollipop. On Oct 29, 2010, at 4:06 PM, Antoine Pitrou wrote: > On Fri, 29 Oct 2010 15:54:19 -0400 > Barry Warsaw wrote: >> Another quick thought. What would people think about regular timed releases if python 2.7? >> This is probably more a question for Benjamin but doing sonmight >> provide better predictability and "customer service" to our users. I >> might like to see monthly releases but even quarterly would probably >> be useful. Doing timed releases might also incentivize folks to fix >> more outstanding 2.7 bugs. > > Why would it only apply to 2.7? > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/barry%40python.org From benjamin at python.org Fri Oct 29 23:23:24 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 29 Oct 2010 16:23:24 -0500 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <07F03E8C-347C-4E07-A6BC-7D0029F056EC@python.org> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <07F03E8C-347C-4E07-A6BC-7D0029F056EC@python.org> Message-ID: 2010/10/29 Barry Warsaw : > Another quick thought. What would people think about regular timed releases if python 2.7? ?This is probably more a question for Benjamin but doing sonmight provide better predictability and "customer service" to our users. I might like to see monthly releases but even quarterly would probably be useful. Doing timed releases might also incentivize folks to fix more outstanding 2.7 bugs. At the moment, I'm planning to do regular maintenance releases for 3.1 and 2.7 roughly every 6 months. -- Regards, Benjamin From dirkjan at ochtman.nl Fri Oct 29 23:39:45 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 29 Oct 2010 23:39:45 +0200 Subject: [Python-Dev] Regular scheduled releases (was: Continuing 2.x) Message-ID: On Fri, Oct 29, 2010 at 21:54, Barry Warsaw wrote: > Another quick thought. What would people think about regular timed releases if python 2.7? ?This is probably more a question for Benjamin but doing sonmight provide better predictability and "customer service" to our users. I might like to see monthly releases but even quarterly would probably be useful. Doing timed releases might also incentivize folks to fix more outstanding 2.7 bugs. I would really like to see a more regular and frequent release schedule. Most of my experience with this is with Mercurial, where we have a time-based schedule with a feature release every four months and a bugfix release at least every month (usually at the first of each month) and more often if we have bad regressions. It's nice because (a) release are practiced more and therefore become easier to do, (b) regressions can be fixed in a shorter timeframe. A predictable schedule is also just nice for all parties involved. In Gentoo, we actually started taking backports from the maintenance branches to fix issues (regressions) in our packages, but didn't work out so well. Obviously a random snapshot from SVN (even from a stable branch) isn't exercised as well as an actual release, so we ended up having some issues due to that. Also releasing packages with a version number that doesn't fully correspond to the tarball is less than ideal (we mitigated this somewhat by adding a date tag to the packages, but still). Here are the bugfix releases from the 2.6 branch: 2.6.1: 64 days 2.6.2: 131 days 2.6.3: 174 days 2.6.4: 23 days (critical regressions) 2.6.5: 145 days 2.6.6: 158 days That's an average of 4 (if you include .4) or 4.5 months (PEP 6 specifies 6 months, but some of the parts seem outdated). I think releasing each month might be a bit ambitious, but it would be great to drive down the release interval towards 2-3 months instead of 4-5. Cheers, Dirkjan From mal at egenix.com Fri Oct 29 23:42:41 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Oct 2010 23:42:41 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101029152125.04609bb1@mission> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <87eib9r3v4.fsf@uwakimon.sk.tsukuba.ac.jp> <7ED03BC9-40A5-4974-8D02-7DFD6CC6711D@pandora.com> <20101029152125.04609bb1@mission> Message-ID: <4CCB3FD1.1020101@egenix.com> Barry Warsaw wrote: > On Oct 29, 2010, at 12:43 PM, Casey Duncan wrote: > >> I like Python 3, I am using it for my latest projects, but I am also keeping >> Python 2 compatibility. This incurs some overhead, and basically means I am >> still really only using Python 2 features. So in some respects, my Python 3.x >> support is only tacit, it works as well as for Python 2, but it's not taking >> advantage of Python 3 really. I haven't run into a situation yet where I >> really want to or have to use Python 3 exclusive features, but then again I'm >> not really learning to use Python 3 either, short of the new C api. > > One thing that *might* be interesting to explore for Python 3.3 would be > something like `python3 --1` or some such switch that would help Python 2 code > run more easily in Python 3. This might be a hook to 2to3 or other internal > changes that help some of the trickier bits of writing cross-compatible code. > > I don't know what those things enabled by --1 would be. Some syntactic things > might be fairly easy though largely inconsequential (e.g. print() -> print). > It might be that large changes like bytes/string dwarfs syntactic sugar. I > had a brief conversation with Michael Foord yesterday and he's writing code > that works in 2.4 through 3.2, so for *some* code bases, it's tricky and ugly, > but possible. > > IMHO, those are the kinds of directions we should be thinking about, to help > existing code more easily make the jump to Python 3. +1 -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 29 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From martin at v.loewis.de Sat Oct 30 01:12:28 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 30 Oct 2010 01:12:28 +0200 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <07F03E8C-347C-4E07-A6BC-7D0029F056EC@python.org> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <07F03E8C-347C-4E07-A6BC-7D0029F056EC@python.org> Message-ID: <4CCB54DC.3060708@v.loewis.de> Am 29.10.2010 21:54, schrieb Barry Warsaw: > Another quick thought. What would people think about regular timed > releases if python 2.7? This is probably more a question for > Benjamin but doing sonmight provide better predictability and > "customer service" to our users. I might like to see monthly releases > but even quarterly would probably be useful. Doing timed releases > might also incentivize folks to fix more outstanding 2.7 bugs. Ah, timed releases :-) I know this is bike-shedding, but PY_MINOR_VERSION has never used two digits, so far. More seriously - I think that monthly releases would be a *dis-service* to users, and I base that on personal experience with both Bazaar, and TortoiseSVN. For less-than-daily users, the user experience will be that they should upgrade their installation *every* time they want to use it. People providing support will always ask "are you using the latest version", to which the answer will be "of course not, I am using an installation that is five weeks old". Regards, Martin From brett at python.org Sat Oct 30 02:35:36 2010 From: brett at python.org (Brett Cannon) Date: Fri, 29 Oct 2010 17:35:36 -0700 Subject: [Python-Dev] closing files and sockets in a timely manner in the stdlib Message-ID: For those of you who have not noticed, Antoine committed a patch that raises a ResourceWarning under a pydebug build if a file or socket is closed through garbage collection instead of being explicitly closed. I have started to go through the test suite to fix as many of these cases as possible, but it's time for me to stop. For those of you interested in helping out, at the end of this email is a list of tests which have some ResourceWarning raised. Some of them are probably listed here because of some other module causing the problem. Everything from test_mailbox and *up* I already tried to silence but it the solution was non-obvious to me (or was very network-specific which I don't have much experience with). from test_os *down* I have not touched (although Antoine has gotten to some). Just a little tip: if testMethod() comes up as the warning in unittest it isn't unittest. =) Because the warning gets triggered at odd times thanks to gc it doesn't always output a helpful line. Best thing is to search for obvious things like "open(" or "socket(" to find things that need a context manager if the line where the warning triggered seems odd (both in the test and in the module being tested). [ 92/349] test_distutils [105/349] test_file [108/349] test_fileio [116/349] test_ftplib [143/349] test_httplib [144/349] test_httpservers [154/349] test_io [169/349] test_logging [173/349] test_mailbox [197/349] test_os [214/349] test_pkgimport [219/349] test_popen [247/349] test_sax [255/349] test_shutil [260/349] test_smtplib [263/349] test_socket [267/349] test_ssl [278/349] test_subprocess [279/349] test_sunau [289/349] test_tarfile [291/349] test_telnetlib [297/349] test_threading [303/349] test_tokenize [314/349] test_unicodedata [320/349] test_urllib2_localnet [328/349] test_uu [343/349] test_xmlrpc [345/349] test_zipfile From steve at pearwood.info Sat Oct 30 03:04:49 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 30 Oct 2010 12:04:49 +1100 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <20101029185811.0487ac32@pitrou.net> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <20101029164119.2040.1998748350.divmod.xquotient.291@localhost.localdomain> <20101029185811.0487ac32@pitrou.net> Message-ID: <4CCB6F31.80900@pearwood.info> Antoine Pitrou wrote: > Even if there were no trademark, I think it would be wrong for a > separate project to adopt the same name without agreement from the > original group of contributors. I have never seen a fork which didn't > change the name of the project. +1 -- Steven From raymond.hettinger at gmail.com Sat Oct 30 05:14:27 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Fri, 29 Oct 2010 20:14:27 -0700 Subject: [Python-Dev] Cleaning-up the new unittest API Message-ID: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> The API for the unittest module has grown fat (the documentation is approaching 2,000 lines and 10,000 words like a small book). I think we can improve learnability by focusing on the most important parts of the API. I would like to simplify and clean-up the API for the unittest module by de-documenting assertSetEqual(), assertDictEqual(), assertListEqual, and assertTupleEqual(). All of those methods are already called automatically by assertEqual(), so those methods never need to be called directly. Or, if you need to be more explicit about the type checking for sequences, the assertSequenceEqual() method will suffice. Either way, there's no need to expose the four type specific methods. Besides de-documenting those four redundant methods, I propose that assertItemsEqual() be deprecated just like its brother assertSameElements(). I haven't found anyone who accurately guesses what those methods entail based on their method names ("items" usually implies key/value pairs elsewhere in the language; nor is it clear whether order is important, whether the elements need to be hashable or orderable or just define equality tests, nor is is clear whether duplicates cause the test to fail). Given the purpose of the unittest module, it's important that the reader have a crystal clear understanding of what a test is doing. Their attention needs to be focused on the subject of the test, not on questioning the semantics of the test method. IMO, users are far better-off sticking with assertEqual() so they can be specific about the nature of the test: # hashable elements; ignore dups assertEqual(set(a), set(b)) # orderable elements; dups matter, order doesn't assertEqual(sorted(a), sorted(b)) # eq tested elements, dups matter, order matters assertEqual(list(a), list(b)) # hashable keys, eq tested values # ignore dups, ignore order assertEqual(dict(a), dict(b)) These take just a few more characters than assertSameElements() and assertItemsEqual(), but they are far more clear about their meaning. You won't have to second guess what semantics are hidden behind the abstraction. There are a couple other problems with the new API but it is probably too late to do anything about it. * elsewhere in Python we spell comparison names with abbreviations like eq, ne, lt, le, gt, ge. In unittest, those are spelled in an awkward, not easily remembered manner: assertLessEqual(a, b), etc. Fortunately, it's clear what the mean; however, it's not easy to guess their spelling. * the names for assertRegexpMatches() and assertNotRegexpMatches are deeply misleading since they are implemented in terms of re.search(), not re.match(). Raymond P.S. I also looked ar assertDictContainsSubset(a,b). It is a bit over-specialized, but at least it is crystal clear what is does and it beats the awkward alternative using dict views: assertLessEqual(a.items(), b.items()) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Oct 30 05:22:10 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 30 Oct 2010 13:22:10 +1000 Subject: [Python-Dev] Regular scheduled releases (was: Continuing 2.x) In-Reply-To: References: Message-ID: On Sat, Oct 30, 2010 at 7:39 AM, Dirkjan Ochtman wrote: > That's an average of 4 (if you include .4) or 4.5 months (PEP 6 > specifies 6 months, but some of the parts seem outdated). I think > releasing each month might be a bit ambitious, but it would be great > to drive down the release interval towards 2-3 months instead of 4-5. Ultimately, the frequency of releases comes down to the burden on the release manager and the folks that build the binary installers. Any given RM is usually only responsible for one or two branches, but the same two people (Martin and Ronald) typically build the Windows and Mac OS X binaries for all of them. So if you add 2.6 and 3.1 together, as well as the releases for 2.7 and 3.2 development, I think you'll find releases happening a lot more often than an average of 1 every 4 months. I suspect the most significant thing that needs to be done in making more regular bug fix releases possible is solid, reliable automated creation of Windows and Mac OS X binaries. We also need to consider the impact on downstream - switching to a new compiler or interpreter version generally has a much higher chance of breaking things than switching to a new version of almost any other software development tool. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From fuzzyman at voidspace.org.uk Sat Oct 30 05:29:10 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 29 Oct 2010 23:29:10 -0400 Subject: [Python-Dev] Cleaning-up the new unittest API In-Reply-To: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> Message-ID: <4CCB9106.4020302@voidspace.org.uk> On 29/10/2010 23:14, Raymond Hettinger wrote: > The API for the unittest module has grown fat (the documentation > is approaching 2,000 lines and 10,000 words like a small book). > I think we can improve learnability by focusing on the most > important parts of the API. > > I would like to simplify and clean-up the API for the unittest module > by de-documenting assertSetEqual(), assertDictEqual(), > assertListEqual, and assertTupleEqual(). > > All of those methods are already called automatically by > assertEqual(), so those methods never need to be called directly. > Or, if you need to be more explicit about the type checking for > sequences, the assertSequenceEqual() method will suffice. > Either way, there's no need to expose the four type specific methods. > I'm fine with dedocumenting these. These methods should not need to be called directly. > Besides de-documenting those four redundant methods, > I propose that assertItemsEqual() be deprecated just like > its brother assertSameElements(). I haven't found anyone > who accurately guesses what those methods entail based > on their method names ("items" usually implies key/value > pairs elsewhere in the language; nor is it clear whether order is > important, whether the elements need to be hashable or > orderable or just define equality tests, nor is is clear whether > duplicates cause the test to fail). > > Given the purpose of the unittest module, it's important that > the reader have a crystal clear understanding of what a test > is doing. Their attention needs to be focused on the subject > of the test, not on questioning the semantics of the test method. > assertItemsEqual compares two iterables and tests that they have the same elements irrespective of order. A relatively straightforward definition. Hopefully the docstring and documentation make this clear. If the members are all of the same type then indeed comparing two sorted lists is only slightly more typing. If the members are of different types checking that the members are the same is a much harder problem in Python 3, and this method can be very useful. -1 for deprecating. All the best, Michael Foord > IMO, users are far better-off sticking with assertEqual() so they > can be specific about the nature of the test: > > # hashable elements; ignore dups > assertEqual(set(a), set(b)) > > # orderable elements; dups matter, order doesn't > assertEqual(sorted(a), sorted(b)) > > # eq tested elements, dups matter, order matters > assertEqual(list(a), list(b)) > > # hashable keys, eq tested values > # ignore dups, ignore order > assertEqual(dict(a), dict(b)) > > These take just a few more characters than assertSameElements() > and assertItemsEqual(), but they are far more clear about their meaning. > You won't have to second guess what semantics are hidden > behind the abstraction. > > There are a couple other problems with the new API but it is probably > too late to do anything about it. > > * elsewhere in Python we spell comparison names with abbreviations > like eq, ne, lt, le, gt, ge. In unittest, those are spelled in > an awkward, > not easily remembered manner: assertLessEqual(a, b), etc. > Fortunately, it's clear what the mean; however, it's not easy to guess > their spelling. > > * the names for assertRegexpMatches() and assertNotRegexpMatches > are deeply misleading since they are implemented in terms of > re.search(), not re.match(). > > > Raymond > > > P.S. I also looked ar assertDictContainsSubset(a,b). It is a bit > over-specialized, but at least it is crystal clear what is does > and it beats the awkward alternative using dict views: > > assertLessEqual(a.items(), b.items()) > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Sat Oct 30 05:56:04 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 29 Oct 2010 23:56:04 -0400 Subject: [Python-Dev] Cleaning-up the new unittest API In-Reply-To: <4CCB9106.4020302@voidspace.org.uk> References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> <4CCB9106.4020302@voidspace.org.uk> Message-ID: <4CCB9754.5000508@voidspace.org.uk> On 29/10/2010 23:29, Michael Foord wrote: > [snip...] >> Besides de-documenting those four redundant methods, >> I propose that assertItemsEqual() be deprecated just like >> its brother assertSameElements(). I haven't found anyone >> who accurately guesses what those methods entail based >> on their method names ("items" usually implies key/value >> pairs elsewhere in the language; nor is it clear whether order is >> important, whether the elements need to be hashable or >> orderable or just define equality tests, nor is is clear whether >> duplicates cause the test to fail). >> >> Given the purpose of the unittest module, it's important that >> the reader have a crystal clear understanding of what a test >> is doing. Their attention needs to be focused on the subject >> of the test, not on questioning the semantics of the test method. >> > > assertItemsEqual compares two iterables and tests that they have the > same elements irrespective of order. A relatively straightforward > definition. Hopefully the docstring and documentation make this clear. > > If the members are all of the same type then indeed comparing two > sorted lists is only slightly more typing. If the members are of > different types checking that the members are the same is a much > harder problem in Python 3, and this method can be very useful. > Just to clarify. The following fails in Python 3: sorted([3, 1, 2, None]) If you want to compare that two iterables containing heterogeneous types have the same members then it is tricky to implement correctly and assertItemsEqual does it for you. I agree that the name is not ideal and would be happy to change the name (deprecating the old name as it was released in 2.7). API churn is as bad as API bloat, but at least changing the name is something only done once. All the best, Michael > -1 for deprecating. > > All the best, > > Michael Foord > >> IMO, users are far better-off sticking with assertEqual() so they >> can be specific about the nature of the test: >> >> # hashable elements; ignore dups >> assertEqual(set(a), set(b)) >> >> # orderable elements; dups matter, order doesn't >> assertEqual(sorted(a), sorted(b)) >> >> # eq tested elements, dups matter, order matters >> assertEqual(list(a), list(b)) >> >> # hashable keys, eq tested values >> # ignore dups, ignore order >> assertEqual(dict(a), dict(b)) >> >> These take just a few more characters than assertSameElements() >> and assertItemsEqual(), but they are far more clear about their meaning. >> You won't have to second guess what semantics are hidden >> behind the abstraction. >> >> There are a couple other problems with the new API but it is probably >> too late to do anything about it. >> >> * elsewhere in Python we spell comparison names with abbreviations >> like eq, ne, lt, le, gt, ge. In unittest, those are spelled in >> an awkward, >> not easily remembered manner: assertLessEqual(a, b), etc. >> Fortunately, it's clear what the mean; however, it's not easy to >> guess >> their spelling. >> >> * the names for assertRegexpMatches() and assertNotRegexpMatches >> are deeply misleading since they are implemented in terms of >> re.search(), not re.match(). >> >> >> Raymond >> >> >> P.S. I also looked ar assertDictContainsSubset(a,b). It is a bit >> over-specialized, but at least it is crystal clear what is does >> and it beats the awkward alternative using dict views: >> >> assertLessEqual(a.items(), b.items()) >> >> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe:http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > > -- > http://www.voidspace.org.uk/ > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Sat Oct 30 06:11:03 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 30 Oct 2010 00:11:03 -0400 Subject: [Python-Dev] Cleaning-up the new unittest API In-Reply-To: <4CCB9754.5000508@voidspace.org.uk> References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> <4CCB9106.4020302@voidspace.org.uk> <4CCB9754.5000508@voidspace.org.uk> Message-ID: <4CCB9AD7.8020504@voidspace.org.uk> On 29/10/2010 23:56, Michael Foord wrote: > On 29/10/2010 23:29, Michael Foord wrote: >> [snip...] >>> Besides de-documenting those four redundant methods, >>> I propose that assertItemsEqual() be deprecated just like >>> its brother assertSameElements(). I haven't found anyone >>> who accurately guesses what those methods entail based >>> on their method names ("items" usually implies key/value >>> pairs elsewhere in the language; nor is it clear whether order is >>> important, whether the elements need to be hashable or >>> orderable or just define equality tests, nor is is clear whether >>> duplicates cause the test to fail). >>> >>> Given the purpose of the unittest module, it's important that >>> the reader have a crystal clear understanding of what a test >>> is doing. Their attention needs to be focused on the subject >>> of the test, not on questioning the semantics of the test method. >>> >> >> assertItemsEqual compares two iterables and tests that they have the >> same elements irrespective of order. A relatively straightforward >> definition. Hopefully the docstring and documentation make this clear. >> >> If the members are all of the same type then indeed comparing two >> sorted lists is only slightly more typing. If the members are of >> different types checking that the members are the same is a much >> harder problem in Python 3, and this method can be very useful. >> > Just to clarify. The following fails in Python 3: > > sorted([3, 1, 2, None]) > > If you want to compare that two iterables containing heterogeneous > types have the same members then it is tricky to implement correctly > and assertItemsEqual does it for you. > > I agree that the name is not ideal and would be happy to change the > name (deprecating the old name as it was released in 2.7). API churn > is as bad as API bloat, but at least changing the name is something > only done once. Sorry for the noise. Suggested alternative name: assertElementsEqual The docs need updating to make it clear that the method isn't just a synonym for assertEqual(sorted(iter1), sorted(iter2)) and that it works with unorderable types. As for "assertLessEqual" and friends, I don't find those names intuitive. In fact whilst typing this email I initially called the method "assertLessThan". For "assertRegexpMatches" I don't find it hard to understand (in natural English it makes sense even if the standard terminology for regular expressions is different). I would have preferred "assertRegex" though. All the best, Michael > > All the best, > > Michael >> -1 for deprecating. >> >> All the best, >> >> Michael Foord >> >>> IMO, users are far better-off sticking with assertEqual() so they >>> can be specific about the nature of the test: >>> >>> # hashable elements; ignore dups >>> assertEqual(set(a), set(b)) >>> >>> # orderable elements; dups matter, order doesn't >>> assertEqual(sorted(a), sorted(b)) >>> >>> # eq tested elements, dups matter, order matters >>> assertEqual(list(a), list(b)) >>> >>> # hashable keys, eq tested values >>> # ignore dups, ignore order >>> assertEqual(dict(a), dict(b)) >>> >>> These take just a few more characters than assertSameElements() >>> and assertItemsEqual(), but they are far more clear about their >>> meaning. >>> You won't have to second guess what semantics are hidden >>> behind the abstraction. >>> >>> There are a couple other problems with the new API but it is probably >>> too late to do anything about it. >>> >>> * elsewhere in Python we spell comparison names with abbreviations >>> like eq, ne, lt, le, gt, ge. In unittest, those are spelled in >>> an awkward, >>> not easily remembered manner: assertLessEqual(a, b), etc. >>> Fortunately, it's clear what the mean; however, it's not easy to >>> guess >>> their spelling. >>> >>> * the names for assertRegexpMatches() and assertNotRegexpMatches >>> are deeply misleading since they are implemented in terms of >>> re.search(), not re.match(). >>> >>> >>> Raymond >>> >>> >>> P.S. I also looked ar assertDictContainsSubset(a,b). It is a bit >>> over-specialized, but at least it is crystal clear what is does >>> and it beats the awkward alternative using dict views: >>> >>> assertLessEqual(a.items(), b.items()) >>> >>> >>> >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe:http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk >> >> >> -- >> http://www.voidspace.org.uk/ >> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe:http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > > -- > http://www.voidspace.org.uk/ > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.hettinger at gmail.com Sat Oct 30 07:56:38 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Fri, 29 Oct 2010 22:56:38 -0700 Subject: [Python-Dev] Cleaning-up the new unittest API In-Reply-To: <4CCB9AD7.8020504@voidspace.org.uk> References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> <4CCB9106.4020302@voidspace.org.uk> <4CCB9754.5000508@voidspace.org.uk> <4CCB9AD7.8020504@voidspace.org.uk> Message-ID: On Oct 29, 2010, at 9:11 PM, Michael Foord wrote: >> Just to clarify. The following fails in Python 3: >> >> sorted([3, 1, 2, None]) >> >> If you want to compare that two iterables containing heterogeneous types have the same members then it is tricky to implement correctly and assertItemsEqual does it for you. >> >> I agree that the name is not ideal and would be happy to change the name (deprecating the old name as it was released in 2.7). API churn is as bad as API bloat, but at least changing the name is something only done once. > > Sorry for the noise. Suggested alternative name: > > assertElementsEqual > > The docs need updating to make it clear that the method isn't just a synonym for assertEqual(sorted(iter1), sorted(iter2)) and that it works with unorderable types. I looked at this again and think we should just remove assertItemsEqual() from Py3.2 and dedocument it in Py2.7. It is listed as being new in 3.2 so nothing is lost. A new name like assertElementsEqual is an improvement because it doesn't suggest something like assertEqual(d.items(), d.items()), but it falls short in describing its key features: * the method doesn't care about order * it does care about duplicates * it don't need hashability * it can handle sequences of non-comparable types Also, I think the O(n**2) behavior is unexpected. There is a O(n log n) fast-path but it has a bug and needs to be removed. See issue 10242. The sole benefit over the more explicit variants like assertEqual(set(a), set(b)) and assertEqual(sorted(a), sorted(b)) is that it handles a somewhat rare corner case where neither of those work (unordered comparison of non-compable types when you do care about duplicates). That particular case doesn't come-up much and isn't suggested by either the current name or its proposed replacement. FWIW, I checked-out some other unittest suites in other languages and did not find an equivalent. That strongly suggests this is YAGNI and it shouldn't be added in Py3.2. There needs to be more evidence of need before putting this in. And if it goes in, it needs a really good name that tells what operations are hidden behind the abstraction. When reading test assertion, it is vital that the reader understand exactly what is being tested. It's an API fail if a reader guesses that assertElementsEqual(a,b) means list(a)==list(b); the test will pass unintentionally. See: http://www.phpunit.de/manual/3.4/en/api.html http://kentbeck.github.com/junit/javadoc/latest/ Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Oct 30 08:41:29 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 30 Oct 2010 17:41:29 +1100 Subject: [Python-Dev] Cleaning-up the new unittest API In-Reply-To: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> Message-ID: <4CCBBE19.1040107@pearwood.info> Raymond Hettinger wrote: > The API for the unittest module has grown fat (the documentation > is approaching 2,000 lines and 10,000 words like a small book). > I think we can improve learnability by focusing on the most > important parts of the API. > > I would like to simplify and clean-up the API for the unittest module > by de-documenting assertSetEqual(), assertDictEqual(), > assertListEqual, and assertTupleEqual(). While you're at it, what do you think of assertAlmostEqual? Is there a use-case for it as it exists now? As I see it, it suffers from a number of problems: (1) Comparing floats for equality to some number of decimal places is rarely (never?) the right way to do it. Comparison to some number of significant figures would be better. Specifying an absolute or relative error would be better still. (2) This leads people to write the wrong test, because assertAlmostEqual exists and is easy to use, while the right test is tricky and requires better understanding of floating point issues. Example: I recently wrote tests using assertAlmostEqual where I specified places=12, because my code should have been accurate to 12 significant figures. Worked fine for small floating point, but when I started testing values of the order of 10000, my tests started failing, because although it was correct to > 12 significant figures, five of those figures were not decimal places. (3) In the rare case that mere rounding is the right way to compare for approximate equality, assertAlmostEqual doesn't save you much: a handful of characters, that's about all. self.assertAlmostEqual(x, y, places=n) vs. self.assertEqual(round(x, n), round(y, n)) (Specifying the number of decimal places is keyword-only in 3.1. To me, that seems to be a gratuitous change from 2.x, where it could be specified as a positional argument.) -- Steven From stephen at xemacs.org Sat Oct 30 13:21:38 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 30 Oct 2010 20:21:38 +0900 Subject: [Python-Dev] Continuing 2.x In-Reply-To: <7ED03BC9-40A5-4974-8D02-7DFD6CC6711D@pandora.com> References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <87eib9r3v4.fsf@uwakimon.sk.tsukuba.ac.jp> <7ED03BC9-40A5-4974-8D02-7DFD6CC6711D@pandora.com> Message-ID: <874oc4c4e5.fsf@uwakimon.sk.tsukuba.ac.jp> Casey Duncan writes: > However there are many many more users of Python 2.x than Python > 3.x. Many may never upgrade for the life of these projects, > because if it ain't broke, why fix it? It doesn't matter how much > better Python 3 is than Python 2. It isn't better enough. And the "don't fix what ain't broke" people are well supported, by explicit policy and by the personal dedication of developers like Victor Stinner. > In this regard the existence of Python 3 is a disadvantage, not an > advantage for my new code, regardless of how much better a language > or dev environment it may be. You mean, at this very instant. Pay now, profit later is the definition of "investment." In the past, while I'm sure you worked real hard, you didn't have to pay the price of the *community's* investment because you were free to start new projects in the way best fitted to current features. Now, with a large codebase to maintain and extend, you have to share the community's costs by either living with Python 2, which won't improve from now on, or shifting to Python 3, which entails big porting costs for many projects.[1] Or you can participate in a new project to fork from python.org and continue development of Python 2. You have a wealth of choice there. The problem[2] is that you're not getting a free ride on the fast track this time. Well, that party was nice while it lasted, but now it's over. The next party is going to be at the Python 3 Lounge, and you're invited. Sure, you gotta get up and go to work tomorrow, but Friday's comin'! Footnotes: [1] But I can't say it's obvious that they're huge on average. There's clearly a lot of variance in porting costs. [2] This is a community problem; it's not something you did to yourself. But it's also been judged unavoidable if Python is to continue to grow. From dirkjan at ochtman.nl Sat Oct 30 13:44:16 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Sat, 30 Oct 2010 13:44:16 +0200 Subject: [Python-Dev] Regular scheduled releases (was: Continuing 2.x) In-Reply-To: References: Message-ID: On Sat, Oct 30, 2010 at 05:22, Nick Coghlan wrote: > Ultimately, the frequency of releases comes down to the burden on the > release manager and the folks that build the binary installers. Any > given RM is usually only responsible for one or two branches, but the > same two people (Martin and Ronald) typically build the Windows and > Mac OS X binaries for all of them. So if you add 2.6 and 3.1 together, > as well as the releases for 2.7 and 3.2 development, I think you'll > find releases happening a lot more often than an average of 1 every 4 > months. Right, the effort of those people is obviously the limiting factor here. Automating builds sounds like a good step forward. What are the sticky bits here? Martin, Ronald, how much of the process is not automated, and why is automating hard? Cheers, Dirkjan From martin at v.loewis.de Sat Oct 30 14:09:25 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 30 Oct 2010 14:09:25 +0200 Subject: [Python-Dev] Regular scheduled releases In-Reply-To: References: Message-ID: <4CCC0AF5.4070701@v.loewis.de> > Right, the effort of those people is obviously the limiting factor > here. Automating builds sounds like a good step forward. What are the > sticky bits here? Martin, Ronald, how much of the process is not > automated, and why is automating hard? I don't feel like producing a complete list of build steps; the entire process takes about four hours. The steps that are difficult to automate are: - code signing - testing the resulting installer - uploading However, in a significant number of releases, some of the build steps failed, so it requires some investigation to find the source of the problem. Regards, Martin From solipsis at pitrou.net Sat Oct 30 14:12:05 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Oct 2010 14:12:05 +0200 Subject: [Python-Dev] r85960 - python/branches/py3k/Lib/test/test_mailbox.py References: <20101030001301.16B7BEEA46@mail.python.org> Message-ID: <20101030141205.3a8cc725@pitrou.net> On Sat, 30 Oct 2010 02:13:01 +0200 (CEST) brett.cannon wrote: > Author: brett.cannon > Date: Sat Oct 30 02:13:00 2010 > New Revision: 85960 > > Log: > Silence some ResourceWarning in test_mailbox by using file context managers. Unfortunately, these file-like objects don't support the context management protocol: ====================================================================== ERROR: test_get_file (test.test_mailbox.TestMaildir) ---------------------------------------------------------------------- Traceback (most recent call last): File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_mailbox.py", line 168, in test_get_file with self._box.get_file(key0) as file: AttributeError: __exit__ ====================================================================== ERROR: test_get_file (test.test_mailbox.TestMbox) ---------------------------------------------------------------------- Traceback (most recent call last): File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_mailbox.py", line 168, in test_get_file with self._box.get_file(key0) as file: AttributeError: __exit__ ====================================================================== ERROR: test_get_file (test.test_mailbox.TestMMDF) ---------------------------------------------------------------------- Traceback (most recent call last): File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_mailbox.py", line 168, in test_get_file with self._box.get_file(key0) as file: AttributeError: __exit__ ====================================================================== ERROR: test_get_file (test.test_mailbox.TestMH) ---------------------------------------------------------------------- Traceback (most recent call last): File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_mailbox.py", line 168, in test_get_file with self._box.get_file(key0) as file: AttributeError: __exit__ From dirkjan at ochtman.nl Sat Oct 30 14:29:15 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Sat, 30 Oct 2010 14:29:15 +0200 Subject: [Python-Dev] Regular scheduled releases In-Reply-To: <4CCC0AF5.4070701@v.loewis.de> References: <4CCC0AF5.4070701@v.loewis.de> Message-ID: On Sat, Oct 30, 2010 at 14:09, "Martin v. L?wis" wrote: > I don't feel like producing a complete list of build steps; the entire > process takes about four hours. So is most of this scripted, or is there just a process in your head? > The steps that are difficult to automate are: > - code signing > - testing the resulting installer > - uploading Why are those steps difficult to automate? I'd think that uploading is just some ftp commands. Signing might be a thing because you don't want the keys distributed. Testing the resulting installer could probably be automated in part, and people could still download the nightlies to do more testing. > However, in a significant number of releases, some of the build steps > failed, so it requires some investigation to find the source of the > problem. Well, sure, but if we do continuous builds we find failures like that sooner. Cheers, Dirkjan From solipsis at pitrou.net Sat Oct 30 14:51:28 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Oct 2010 14:51:28 +0200 Subject: [Python-Dev] Cleaning-up the new unittest API References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> Message-ID: <20101030145128.6ab69935@pitrou.net> On Fri, 29 Oct 2010 20:14:27 -0700 Raymond Hettinger wrote: > > I would like to simplify and clean-up the API for the unittest module > by de-documenting assertSetEqual(), assertDictEqual(), > assertListEqual, and assertTupleEqual(). +1 from me. Regards Antoine. From martin at v.loewis.de Sat Oct 30 15:29:59 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 30 Oct 2010 15:29:59 +0200 Subject: [Python-Dev] Regular scheduled releases In-Reply-To: References: <4CCC0AF5.4070701@v.loewis.de> Message-ID: <4CCC1DD7.30305@v.loewis.de> Am 30.10.2010 14:29, schrieb Dirkjan Ochtman: > On Sat, Oct 30, 2010 at 14:09, "Martin v. L?wis" wrote: >> I don't feel like producing a complete list of build steps; the entire >> process takes about four hours. > > So is most of this scripted, or is there just a process in your head? Define "most". There hundreds of commands that run automatically (like all compiler invocations). But still, there are about 20 steps that are in my head, some written down in plain English. But yes, "most" of the steps (by sheer number) are automated. > Why are those steps difficult to automate? I'd think that uploading is > just some ftp commands. No, it's ssh, and you need access to the right key. More importantly, it's editing content.ht, to update the links, compute and copy the md5sums and size, and svn add the GPG signatures. It *could* be automated, I suppose. It just isn't, and it would take quite some time to automate it. > Testing the resulting installer could > probably be automated in part, and people could still download the > nightlies to do more testing. People will never ever test nightly builds. Been there, done that. Instead, the nightly build process will break, and nobody will fix it for months (or even complain, for that matter). I also doubt that the installer could be automatically tested with reasonable effort. Regards, Martin From g.brandl at gmx.net Sat Oct 30 16:33:01 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 30 Oct 2010 14:33:01 +0000 Subject: [Python-Dev] r85960 - python/branches/py3k/Lib/test/test_mailbox.py In-Reply-To: <20101030141205.3a8cc725@pitrou.net> References: <20101030001301.16B7BEEA46@mail.python.org> <20101030141205.3a8cc725@pitrou.net> Message-ID: Am 30.10.2010 12:12, schrieb Antoine Pitrou: > On Sat, 30 Oct 2010 02:13:01 +0200 (CEST) > brett.cannon wrote: >> Author: brett.cannon >> Date: Sat Oct 30 02:13:00 2010 >> New Revision: 85960 >> >> Log: >> Silence some ResourceWarning in test_mailbox by using file context managers. > > Unfortunately, these file-like objects don't support the context > management protocol Should be fixed in r85979. Do we want Mailboxes themselves to be context managers? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ocean-city at m2.ccsnet.ne.jp Sat Oct 30 16:48:20 2010 From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto) Date: Sat, 30 Oct 2010 23:48:20 +0900 Subject: [Python-Dev] [Python-checkins] r85934 - in python/branches/py3k: Misc/NEWS Modules/socketmodule.c In-Reply-To: <20101029182008.DFE35EEBE1@mail.python.org> References: <20101029182008.DFE35EEBE1@mail.python.org> Message-ID: <4CCC3034.20008@m2.ccsnet.ne.jp> On 2010/10/30 3:20, martin.v.loewis wrote: > Modified: python/branches/py3k/Modules/socketmodule.c > ============================================================================== > --- python/branches/py3k/Modules/socketmodule.c (original) > +++ python/branches/py3k/Modules/socketmodule.c Fri Oct 29 20:20:08 2010 > @@ -3093,6 +3093,27 @@ > static PyObject * > socket_gethostname(PyObject *self, PyObject *unused) > { > +#ifdef MS_WINDOWS > + /* Don't use winsock's gethostname, as this returns the ANSI > + version of the hostname, whereas we need a Unicode string. > + Otherwise, gethostname apparently also returns the DNS name. */ > + wchar_t buf[MAX_COMPUTERNAME_LENGTH]; > + DWORD size = sizeof(buf); > + if (!GetComputerNameExW(ComputerNamePhysicalDnsHostname, buf,&size)) { > + if (GetLastError() == ERROR_MORE_DATA) { > + /* MSDN says this may occur "because DNS allows longer names */ > + PyObject *result = PyUnicode_FromUnicode(NULL, size); > + if (!result) > + return NULL; > + if (GetComputerName(ComputerNamePhysicalDnsHostname, > + PyUnicode_AS_UNICODE(result), > + size+1)) > + return result; > + } > + return PyErr_SetExcFromWindowsErr(PyExc_WindowsError, GetLastError()); > + } > + return PyUnicode_FromUnicode(buf, size); > +#else > char buf[1024]; > int res; > Py_BEGIN_ALLOW_THREADS > @@ -3102,6 +3123,7 @@ > return set_error(); > buf[sizeof buf - 1] = '\0'; > return PyUnicode_FromString(buf); > +#endif > } > > PyDoc_STRVAR(gethostname_doc, > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > I think size should be in TCHARs, not in bytes. (MSDN says so) And GetComputerName's signature differs from MSDN. (Maybe should use GetComputerNameExW again?) From fuzzyman at voidspace.org.uk Sat Oct 30 17:42:58 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 30 Oct 2010 11:42:58 -0400 Subject: [Python-Dev] Cleaning-up the new unittest API In-Reply-To: <4CCBBE19.1040107@pearwood.info> References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> <4CCBBE19.1040107@pearwood.info> Message-ID: <4CCC3D02.5080307@voidspace.org.uk> On 30/10/2010 02:41, Steven D'Aprano wrote: > Raymond Hettinger wrote: >> The API for the unittest module has grown fat (the documentation >> is approaching 2,000 lines and 10,000 words like a small book). I >> think we can improve learnability by focusing on the most important >> parts of the API. >> >> I would like to simplify and clean-up the API for the unittest module >> by de-documenting assertSetEqual(), assertDictEqual(), >> assertListEqual, and assertTupleEqual(). > > While you're at it, what do you think of assertAlmostEqual? Is there a > use-case for it as it exists now? > > As I see it, it suffers from a number of problems: > > (1) Comparing floats for equality to some number of decimal places is > rarely (never?) the right way to do it. Comparison to some number of > significant figures would be better. Specifying an absolute or > relative error would be better still. > I agree, comparing floats to a specific number of decimal places has almost always been useless to me. What would be nice would be a 'delta' keyword argument allowing you to specify a difference instead. That could also allow you to compare non-numeric objects like times as well. Now where are the keys to that time machine... http://docs.python.org/dev/library/unittest.html#unittest.TestCase.assertAlmostEqual All the best, Michael Foord > (2) This leads people to write the wrong test, because > assertAlmostEqual exists and is easy to use, while the right test is > tricky and requires better understanding of floating point issues. > > Example: I recently wrote tests using assertAlmostEqual where I > specified places=12, because my code should have been accurate to 12 > significant figures. Worked fine for small floating point, but when I > started testing values of the order of 10000, my tests started > failing, because although it was correct to > 12 significant figures, > five of those figures were not decimal places. > > (3) In the rare case that mere rounding is the right way to compare > for approximate equality, assertAlmostEqual doesn't save you much: a > handful of characters, that's about all. > > self.assertAlmostEqual(x, y, places=n) > > vs. > > self.assertEqual(round(x, n), round(y, n)) > > (Specifying the number of decimal places is keyword-only in 3.1. To > me, that seems to be a gratuitous change from 2.x, where it could be > specified as a positional argument.) > > > -- http://www.voidspace.org.uk/ From ocean-city at m2.ccsnet.ne.jp Sat Oct 30 17:00:25 2010 From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto) Date: Sun, 31 Oct 2010 00:00:25 +0900 Subject: [Python-Dev] PyMem_MALLOC vs PyMem_Malloc Message-ID: <4CCC3309.1020104@m2.ccsnet.ne.jp> Hello. I found several codes using PyMem_Free to free allocated memory with PyMem_MALLOC (ie: PyUnicode_AsWideCharString) Is it safe? From rdmurray at bitdance.com Sat Oct 30 18:02:27 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Sat, 30 Oct 2010 12:02:27 -0400 Subject: [Python-Dev] Cleaning-up the new unittest API In-Reply-To: <20101030145128.6ab69935@pitrou.net> References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> <20101030145128.6ab69935@pitrou.net> Message-ID: <20101030160228.C27A621ADFC@kimball.webabinitio.net> On Sat, 30 Oct 2010 14:51:28 +0200, Antoine Pitrou wrote: > On Fri, 29 Oct 2010 20:14:27 -0700 > Raymond Hettinger wrote: > > > > I would like to simplify and clean-up the API for the unittest module > > by de-documenting assertSetEqual(), assertDictEqual(), > > assertListEqual, and assertTupleEqual(). > > +1 from me. I don't disagree with this simplification, but given that you all want to pare down the unittest API, I'd be interested in your opinions on issue 10164. Because the assertBytesEqual method takes an optional argument, it seems like it would need to be documented, even though it would in a lot of cases just be used through assertEqual. -- R. David Murray www.bitdance.com From solipsis at pitrou.net Sat Oct 30 18:36:45 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Oct 2010 18:36:45 +0200 Subject: [Python-Dev] Cleaning-up the new unittest API References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> <20101030145128.6ab69935@pitrou.net> <20101030160228.C27A621ADFC@kimball.webabinitio.net> Message-ID: <20101030183645.3e96a83e@pitrou.net> On Sat, 30 Oct 2010 12:02:27 -0400 "R. David Murray" wrote: > > I don't disagree with this simplification, but given that you all want > to pare down the unittest API, I'd be interested in your opinions on > issue 10164. Because the assertBytesEqual method takes an optional > argument, it seems like it would need to be documented, even though > it would in a lot of cases just be used through assertEqual. The optional argument doesn't look very useful. I imagine there are plenty of special cases where you could need custom splitting of bytestrings on a given byte, a regexp pattern, or along some fixed chunk length, but they are special cases. Regards Antoine. From mal at egenix.com Sat Oct 30 19:28:29 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Sat, 30 Oct 2010 19:28:29 +0200 Subject: [Python-Dev] PyMem_MALLOC vs PyMem_Malloc In-Reply-To: <4CCC3309.1020104@m2.ccsnet.ne.jp> References: <4CCC3309.1020104@m2.ccsnet.ne.jp> Message-ID: <4CCC55BD.5000606@egenix.com> Hirokazu Yamamoto wrote: > Hello. I found several codes using PyMem_Free to free > allocated memory with PyMem_MALLOC (ie: PyUnicode_AsWideCharString) > > Is it safe? Within the interpreter: yes. In extensions: depends on the platform, but probably not. The macros provide faster access to the C lib malloc calls. The functions need to be used in extensions in case the interpreter will free the resource or the extension wants to free an interpreter allocated resource. They provide access to the malloc calls used by the interpreter, which may operate on a different heap than the extensions. Within an extension the macros use the extension heap. A subtle, but important difference. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 30 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Sat Oct 30 19:32:43 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Sat, 30 Oct 2010 19:32:43 +0200 Subject: [Python-Dev] PyMem_MALLOC vs PyMem_Malloc In-Reply-To: <4CCC55BD.5000606@egenix.com> References: <4CCC3309.1020104@m2.ccsnet.ne.jp> <4CCC55BD.5000606@egenix.com> Message-ID: <4CCC56BB.9010202@egenix.com> M.-A. Lemburg wrote: > Hirokazu Yamamoto wrote: >> Hello. I found several codes using PyMem_Free to free >> allocated memory with PyMem_MALLOC (ie: PyUnicode_AsWideCharString) >> >> Is it safe? > > Within the interpreter: yes. > > In extensions: depends on the platform, but probably not. > > The macros provide faster access to the C lib malloc calls. > > The functions need to be used in extensions in case the interpreter will > free the resource or the extension wants to free an interpreter > allocated resource. They provide access to the malloc calls > used by the interpreter, which may operate on a different heap > than the extensions. > > Within an extension the macros use the extension heap. > > A subtle, but important difference. BTW: If you were referring to extensions using PyMem_Free() to deallocate memory allocated in the interpreter using PyMem_MALLOC(), then that's exactly how things should be done. I was referring to use of the two mentioned APIs within an extension. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 30 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From rdmurray at bitdance.com Sat Oct 30 20:24:10 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Sat, 30 Oct 2010 14:24:10 -0400 Subject: [Python-Dev] Cleaning-up the new unittest API In-Reply-To: <20101030183645.3e96a83e@pitrou.net> References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> <20101030145128.6ab69935@pitrou.net> <20101030160228.C27A621ADFC@kimball.webabinitio.net> <20101030183645.3e96a83e@pitrou.net> Message-ID: <20101030182411.0EC02209AAF@kimball.webabinitio.net> On Sat, 30 Oct 2010 18:36:45 +0200, Antoine Pitrou wrote: > On Sat, 30 Oct 2010 12:02:27 -0400 > "R. David Murray" wrote: > > > > I don't disagree with this simplification, but given that you all want > > to pare down the unittest API, I'd be interested in your opinions on > > issue 10164. Because the assertBytesEqual method takes an optional > > argument, it seems like it would need to be documented, even though > > it would in a lot of cases just be used through assertEqual. > > The optional argument doesn't look very useful. I imagine there are > plenty of special cases where you could need custom splitting of > bytestrings on a given byte, a regexp pattern, or along some fixed > chunk length, but they are special cases. Well, I have a specific special case I need it for: comparing byte strings that are wire-format email messages. Considering how much of a pain it was to get right, I'd hate to see people have to reimplement the guts of it for each special case. Maybe a 'make_chunks' argument that takes a function that returns a list? -- R. David Murray www.bitdance.com From solipsis at pitrou.net Sat Oct 30 20:35:22 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Oct 2010 20:35:22 +0200 Subject: [Python-Dev] Cleaning-up the new unittest API In-Reply-To: <20101030182411.0EC02209AAF@kimball.webabinitio.net> References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> <20101030145128.6ab69935@pitrou.net> <20101030160228.C27A621ADFC@kimball.webabinitio.net> <20101030183645.3e96a83e@pitrou.net> <20101030182411.0EC02209AAF@kimball.webabinitio.net> Message-ID: <20101030203522.3c518c8f@pitrou.net> On Sat, 30 Oct 2010 14:24:10 -0400 "R. David Murray" wrote: > On Sat, 30 Oct 2010 18:36:45 +0200, Antoine Pitrou wrote: > > On Sat, 30 Oct 2010 12:02:27 -0400 > > "R. David Murray" wrote: > > > > > > I don't disagree with this simplification, but given that you all want > > > to pare down the unittest API, I'd be interested in your opinions on > > > issue 10164. Because the assertBytesEqual method takes an optional > > > argument, it seems like it would need to be documented, even though > > > it would in a lot of cases just be used through assertEqual. > > > > The optional argument doesn't look very useful. I imagine there are > > plenty of special cases where you could need custom splitting of > > bytestrings on a given byte, a regexp pattern, or along some fixed > > chunk length, but they are special cases. > > Well, I have a specific special case I need it for: comparing byte > strings that are wire-format email messages. Considering how much of > a pain it was to get right, I'd hate to see people have to reimplement > the guts of it for each special case. Maybe a 'make_chunks' argument > that takes a function that returns a list? Well, I was hoping that we don't need to make assertBytesEqual a public API ;) Regards Antoine. From jackdied at gmail.com Sat Oct 30 20:39:35 2010 From: jackdied at gmail.com (Jack Diederich) Date: Sat, 30 Oct 2010 14:39:35 -0400 Subject: [Python-Dev] closing files and sockets in a timely manner in the stdlib In-Reply-To: References: Message-ID: On Fri, Oct 29, 2010 at 8:35 PM, Brett Cannon wrote: > For those of you who have not noticed, Antoine committed a patch that > raises a ResourceWarning under a pydebug build if a file or socket is > closed through garbage collection instead of being explicitly closed. Just yesterday I discovered /proc//fd/ which is a list of open file descriptors for your PID on *nix and includes all open files, pipes, and sockets. Very handy, I filed some tickets about company internal libs that were opening file handles as a side effect of import (logging mostly). I tried to provoke standard python imports (non-test) to leave some open handles and came up empty. -Jack From glyph at twistedmatrix.com Sat Oct 30 21:06:45 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Sat, 30 Oct 2010 15:06:45 -0400 Subject: [Python-Dev] closing files and sockets in a timely manner in the stdlib In-Reply-To: References: Message-ID: <2AC21E21-FB91-4A6E-917B-3EAEB784FDB1@twistedmatrix.com> On Oct 30, 2010, at 2:39 PM, Jack Diederich wrote: > On Fri, Oct 29, 2010 at 8:35 PM, Brett Cannon wrote: >> For those of you who have not noticed, Antoine committed a patch that >> raises a ResourceWarning under a pydebug build if a file or socket is >> closed through garbage collection instead of being explicitly closed. > > Just yesterday I discovered /proc//fd/ which is a list > of open file descriptors for your PID on *nix and includes all open > files, pipes, and sockets. Very handy, I filed some tickets about > company internal libs that were opening file handles as a side effect > of import (logging mostly). I tried to provoke standard python > imports (non-test) to leave some open handles and came up empty. That path (and anything below /proc, really) is a list of open file descriptors specifically on Linux, not "*nix". Also on linux, you can avoid "" by just doing "/proc/self". A more portable (albeit not standard) path for "what file descriptors do I have open" is /dev/fd/. This is supported via a symlink to /proc/self on all the Linuxes I've tested on. There's no portable standard equivalent for not-yourself processes that I'm aware of, though. See more discussion here: . -------------- next part -------------- An HTML attachment was scrubbed... URL: From jackdied at gmail.com Sat Oct 30 22:04:39 2010 From: jackdied at gmail.com (Jack Diederich) Date: Sat, 30 Oct 2010 16:04:39 -0400 Subject: [Python-Dev] closing files and sockets in a timely manner in the stdlib In-Reply-To: <2AC21E21-FB91-4A6E-917B-3EAEB784FDB1@twistedmatrix.com> References: <2AC21E21-FB91-4A6E-917B-3EAEB784FDB1@twistedmatrix.com> Message-ID: On Sat, Oct 30, 2010 at 3:06 PM, Glyph Lefkowitz wrote: > > On Oct 30, 2010, at 2:39 PM, Jack Diederich wrote: > > On Fri, Oct 29, 2010 at 8:35 PM, Brett Cannon wrote: > > For those of you who have not noticed, Antoine committed a patch that > > raises a ResourceWarning under a pydebug build if a file or socket is > > closed through garbage collection instead of being explicitly closed. > > Just yesterday I discovered /proc//fd/ which is a list > of open file descriptors for your PID on *nix and includes all open > files, pipes, and sockets. ?Very handy, I filed some tickets about > company internal libs that were opening file handles as a side effect > of import (logging mostly). ?I tried to provoke standard python > imports (non-test) to leave some open handles and came up empty. > > That path (and anything below /proc, really) is a list of open file > descriptors specifically on Linux, not "*nix". ?Also on linux, you can avoid > "" by just doing "/proc/self". I was happy to find out that the /proc system came from Plan9 because I always thought Plan9 was dead water. But in this particular case Plan9 outdid System7 in the the realm of "everything is a file" by making everything a file. From martin at v.loewis.de Sat Oct 30 22:08:57 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 30 Oct 2010 22:08:57 +0200 Subject: [Python-Dev] [Python-checkins] r85934 - in python/branches/py3k: Misc/NEWS Modules/socketmodule.c In-Reply-To: <4CCC3034.20008@m2.ccsnet.ne.jp> References: <20101029182008.DFE35EEBE1@mail.python.org> <4CCC3034.20008@m2.ccsnet.ne.jp> Message-ID: <4CCC7B59.2030408@v.loewis.de> > I think size should be in TCHARs, not in bytes. (MSDN says so) > And GetComputerName's signature differs from MSDN. (Maybe should > use GetComputerNameExW again?) You are right. So how about this patch? Index: Modules/socketmodule.c =================================================================== --- Modules/socketmodule.c (Revision 85983) +++ Modules/socketmodule.c (Arbeitskopie) @@ -3098,16 +3098,16 @@ version of the hostname, whereas we need a Unicode string. Otherwise, gethostname apparently also returns the DNS name. */ wchar_t buf[MAX_COMPUTERNAME_LENGTH]; - DWORD size = sizeof(buf); + DWORD size = sizeof(buf)/sizeof(wchar_t); if (!GetComputerNameExW(ComputerNamePhysicalDnsHostname, buf, &size)) { if (GetLastError() == ERROR_MORE_DATA) { /* MSDN says this may occur "because DNS allows longer names */ PyObject *result = PyUnicode_FromUnicode(NULL, size); if (!result) return NULL; - if (GetComputerName(ComputerNamePhysicalDnsHostname, - PyUnicode_AS_UNICODE(result), - size+1)) + if (GetComputerNameExW(ComputerNamePhysicalDnsHostname, + PyUnicode_AS_UNICODE(result), + size+1)) return result; Py_DECREF(result); } Regards, Martin From martin at v.loewis.de Sat Oct 30 23:20:56 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 30 Oct 2010 23:20:56 +0200 Subject: [Python-Dev] closing files and sockets in a timely manner in the stdlib In-Reply-To: References: <2AC21E21-FB91-4A6E-917B-3EAEB784FDB1@twistedmatrix.com> Message-ID: <4CCC8C38.9030308@v.loewis.de> > I was happy to find out that the /proc system came from Plan9 because > I always thought Plan9 was dead water. But in this particular case > Plan9 outdid System7 in the the realm of "everything is a file" by > making everything a file. However, on Plan 9, /proc//fd is not a directory, but a regular text file. There would be one line in this file per open file descriptor, see http://plan9.bell-labs.com/magic/man2html/3/proc Also notice that the /proc filesystem did *not* come from Plan 9 originally, see http://en.wikipedia.org/wiki/Procfs It originally came from Unix V8, in 1984. In that version, each entry in /proc was a file, essentially giving access to the process' address space. Supposedly, it was still possible to find out the list of open files using that interface, see http://man.cat-v.org/unix_8th/4/proc Regards, Martin From bobbyi at gmail.com Sun Oct 31 00:12:13 2010 From: bobbyi at gmail.com (Bobby Impollonia) Date: Sat, 30 Oct 2010 15:12:13 -0700 Subject: [Python-Dev] closing files and sockets in a timely manner in the stdlib In-Reply-To: <2AC21E21-FB91-4A6E-917B-3EAEB784FDB1@twistedmatrix.com> References: <2AC21E21-FB91-4A6E-917B-3EAEB784FDB1@twistedmatrix.com> Message-ID: On Sat, Oct 30, 2010 at 12:06 PM, Glyph Lefkowitz wrote: > That path (and anything below /proc, really) is a list of open file > descriptors specifically on Linux, not "*nix". ?Also on linux, you can avoid > "" by just doing "/proc/self". > A more portable (albeit not standard) path for "what file descriptors do I > have open" is /dev/fd/. ?This is supported via a symlink to /proc/self on > all the Linuxes I've tested on. ?There's no portable standard equivalent for > not-yourself processes that I'm aware of, though. > See more discussion here: . lsof(8) is available for Linux, FreeBSD, Mac any many other *nixes: http://en.wikipedia.org/wiki/Lsof From nas at arctrix.com Sun Oct 31 00:32:54 2010 From: nas at arctrix.com (Neil Schemenauer) Date: Sat, 30 Oct 2010 22:32:54 +0000 (UTC) Subject: [Python-Dev] Continuing 2.x References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <4CCAD656.7070408@v.loewis.de> Message-ID: I have a specific, easy to implement proposal. I would like one more version tag added to the Roundup tracker. My proposed name is "Python 2.7+" but I don't care what it is called. It would be used to tag bug reports and patches that apply only to the 2.x line and are considered not appropriate for the 2.7.x release. In order to keep them out of the way, I suppose they could be marked as "postponed" and closed. It's possible that no one will step up to handle these issues and integrate them into a Python 2.7+ release (official or fork). However, given the cost to add such a tag, I think it would be a shame to lose the manpower used to produce such bug reports and patches. Neil From tjreedy at udel.edu Sun Oct 31 02:41:18 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 30 Oct 2010 20:41:18 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <4CCAD656.7070408@v.loewis.de> Message-ID: On 10/30/2010 6:32 PM, Neil Schemenauer wrote: > I have a specific, easy to implement proposal. I would like one > more version tag added to the Roundup tracker. My proposed name is > "Python 2.7+" but I don't care what it is called. As a tracker gardener, I disagree. I would expect such to cause more trouble than it is worth. > It would be used to tag bug reports and patches that apply only to > the 2.x line and are considered not appropriate for the 2.7.x > release. Bug reports *are* appropriate. > In order to keep them out of the way, I suppose they could > be marked as "postponed" and closed. The few issues that would get such a 2.7+ tag can just as well be marked 2.7/closed/postponed. > It's possible that no one will step up to handle these issues Which would make new the version tag YAGNI -- Terry Jan Reedy From db3l.net at gmail.com Sun Oct 31 02:58:07 2010 From: db3l.net at gmail.com (David Bolen) Date: Sat, 30 Oct 2010 20:58:07 -0400 Subject: [Python-Dev] Regular scheduled releases References: <4CCC0AF5.4070701@v.loewis.de> <4CCC1DD7.30305@v.loewis.de> Message-ID: "Martin v. L?wis" writes: > People will never ever test nightly builds. Been there, done that. > Instead, the nightly build process will break, and nobody will fix > it for months (or even complain, for that matter). Certainly seems to be past experience. I know Martin knows this, but for other readers who may not, my Windows XP buildbot built nightly windows installers (the basic MSI package, close but not necessarily a fully signed new release as Martin makes) starting in September, 2007. It ran successfully for about 6 months, at which point it started to fail fairly consistently. Nobody noticed, and Martin and I finally just shut it down in December, deciding it wasn't worth the effort to try to fix. My OSX buildbot has been building nightly DMG images (though again, I suspect Ronald has a few extra steps beyond what its doing for a full release) since April. I'd actually be interested in knowing if anyone is using them - I suspect perhaps not. In both cases, getting the process going actually took quite a bit of effort (even stuff like having to fix the buildbot upload code in the Windows case), not just on my part, but with the help of Martin and Ronald. But without actual use of the result, it's hard to think it was worth it. I'm pretty sure my default reaction to a break-down in the current OSX build process at this point would be to first suggest disabling it unless there were real users. -- David From ncoghlan at gmail.com Sun Oct 31 09:11:59 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 31 Oct 2010 18:11:59 +1000 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <4CCAD656.7070408@v.loewis.de> Message-ID: On Sun, Oct 31, 2010 at 10:41 AM, Terry Reedy wrote: > The few issues that would get such a 2.7+ tag can just as well be marked > 2.7/closed/postponed. Using closed+postponed as the resolution for 2.x specific feature requests sounds fine. Feature requests that are also applicable to 3.x can just be bumped to refer to the in-development 3.x version (with beta 1 less than 2 weeks away, that will typically be 3.3 now). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sun Oct 31 15:01:47 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 1 Nov 2010 00:01:47 +1000 Subject: [Python-Dev] [Python-checkins] r85902 - in python/branches/py3k/Lib: os.py test/test_os.py In-Reply-To: <20101029003858.7D584EEA52@mail.python.org> References: <20101029003858.7D584EEA52@mail.python.org> Message-ID: On Fri, Oct 29, 2010 at 10:38 AM, victor.stinner wrote: > ? ? try: > - ? ? ? ?path_list = env.get('PATH') > + ? ? ? ?# ignore BytesWarning warning > + ? ? ? ?with warnings.catch_warnings(record=True): > + ? ? ? ? ? ?path_list = env.get('PATH') This looks odd to me. You're requesting that the warnings be saved, but not actually retrieving the list object where they're recorded from the __enter__ method. The correct way to suppress a specific warning type is: with warnings.catch_warnings(): warnings.simplefilter("ignore", BytesWarning) path_list = env.get('PATH') I'll also echo Benjamin's concern with the embedded import. Of such things, deadlocks are created. If there's a dependency problem between os and the warnings build process in a fresh build, then it is better to simply fix that rather than risking the deadlock. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sun Oct 31 15:20:05 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 1 Nov 2010 00:20:05 +1000 Subject: [Python-Dev] [Python-checkins] r86000 - python/branches/py3k/Lib/test/test_fileio.py In-Reply-To: <20101030235645.65C08EE996@mail.python.org> References: <20101030235645.65C08EE996@mail.python.org> Message-ID: On Sun, Oct 31, 2010 at 9:56 AM, brian.curtin wrote: > Author: brian.curtin > Date: Sun Oct 31 01:56:45 2010 > New Revision: 86000 > > Log: > Fix ResourceWarning about unclosed file > > > Modified: > ? python/branches/py3k/Lib/test/test_fileio.py > > Modified: python/branches/py3k/Lib/test/test_fileio.py > ============================================================================== > --- python/branches/py3k/Lib/test/test_fileio.py ? ? ? ?(original) > +++ python/branches/py3k/Lib/test/test_fileio.py ? ? ? ?Sun Oct 31 01:56:45 2010 > @@ -260,7 +260,6 @@ > ? ? ? ? ? ? ? ? ? ? # OS'es that don't support /dev/tty. > ? ? ? ? ? ? ? ? ? ? pass > ? ? ? ? ? ? ? ? else: > - ? ? ? ? ? ? ? ? ? ?f = _FileIO("/dev/tty", "a") > ? ? ? ? ? ? ? ? ? ? self.assertEquals(f.readable(), False) > ? ? ? ? ? ? ? ? ? ? self.assertEquals(f.writable(), True) > ? ? ? ? ? ? ? ? ? ? if sys.platform != "darwin" and \ Doesn't that delete the file object that the next two lines are trying to test? Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sun Oct 31 15:28:51 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 1 Nov 2010 00:28:51 +1000 Subject: [Python-Dev] [Python-checkins] r85902 - in python/branches/py3k/Lib: os.py test/test_os.py In-Reply-To: References: <20101029003858.7D584EEA52@mail.python.org> Message-ID: On Mon, Nov 1, 2010 at 12:01 AM, Nick Coghlan wrote: > On Fri, Oct 29, 2010 at 10:38 AM, victor.stinner > wrote: >> ? ? try: >> - ? ? ? ?path_list = env.get('PATH') >> + ? ? ? ?# ignore BytesWarning warning >> + ? ? ? ?with warnings.catch_warnings(record=True): >> + ? ? ? ? ? ?path_list = env.get('PATH') > > This looks odd to me. You're requesting that the warnings be saved, > but not actually retrieving the list object where they're recorded > from the __enter__ method. > > The correct way to suppress a specific warning type is: > > ? ? ? ?with warnings.catch_warnings(): > ? ? ? ? ? ?warnings.simplefilter("ignore", BytesWarning) > ? ? ? ? ? ?path_list = env.get('PATH') Alternatively, specifically in the test suite, you can use: with test.support.check_warnings(('', BytesWarning), quiet=True): path_list = env.get('PATH') (Without the "quiet=True" the test would fail in release mode, when the warning *isn't* issued) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From vinay_sajip at yahoo.co.uk Sun Oct 31 15:35:56 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sun, 31 Oct 2010 14:35:56 +0000 (UTC) Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles References: <4CCB0C76.8070305@trueblade.com> Message-ID: Eric Smith trueblade.com> writes: > I keep meaning to review this but haven't had time. One thing I want to > look at specifically is the ability to put the time formatting into the > str.format version of the format string. Now that the time format > specifier can be included in the format string, it's no longer necessary > to have the asctime inspection hack that's currently used in order to > avoid formatting the time. It would be good if we could remove the > formatTime logic for str.format, although I'm not sure how practical it > is. I suspect it's too baked-in to the design, but I'm hopeful. Well, asctime checks would be in there for the other format styles anyway, and you don't gain anything by making str.format a special case where the check is avoided. The %-style will be around for a while, particularly as AFAICT {}-formatting is still nominally slower than %-formatting [it *is* a lot more flexible, so I can understand that]and people still cling to the "logging is slow" meme, despite there being no hard numbers presented in evidence. You don't have to specify asctime in the format string, but remember that since Python logging predates datetime [both came in at Python 2.3 but the logging package was independently available before, and compatible with Python 1.5.2 for a while after introduction], times in logging are floats as per time.time(). IIUC you need a datetime.datetime object to do the formatting automatically using str.format, so if one could e.g. use a Filter or LoggerAdapter to get a datetime value into the LogRecord, then you could use {}-formatting without {asctime} but with some other LogRecord attribute. I think the way it is now for 3.2 is the path of least resistance, though. BTW as it is now, the asctime "hack" is less of a hack than it used to be; it was an inline check before, but now can be easily reimplemented e.g. via subclassing and overriding the usesTime() method. Regards, Vinay Sajip From brian.curtin at gmail.com Sun Oct 31 15:55:32 2010 From: brian.curtin at gmail.com (Brian Curtin) Date: Sun, 31 Oct 2010 09:55:32 -0500 Subject: [Python-Dev] [Python-checkins] r86000 - python/branches/py3k/Lib/test/test_fileio.py In-Reply-To: References: <20101030235645.65C08EE996@mail.python.org> Message-ID: On Sun, Oct 31, 2010 at 09:20, Nick Coghlan wrote: > On Sun, Oct 31, 2010 at 9:56 AM, brian.curtin > wrote: > > Author: brian.curtin > > Date: Sun Oct 31 01:56:45 2010 > > New Revision: 86000 > > > > Log: > > Fix ResourceWarning about unclosed file > > > > > > Modified: > > python/branches/py3k/Lib/test/test_fileio.py > > > > Modified: python/branches/py3k/Lib/test/test_fileio.py > > > ============================================================================== > > --- python/branches/py3k/Lib/test/test_fileio.py (original) > > +++ python/branches/py3k/Lib/test/test_fileio.py Sun Oct 31 > 01:56:45 2010 > > @@ -260,7 +260,6 @@ > > # OS'es that don't support /dev/tty. > > pass > > else: > > - f = _FileIO("/dev/tty", "a") > > self.assertEquals(f.readable(), False) > > self.assertEquals(f.writable(), True) > > if sys.platform != "darwin" and \ > > Doesn't that delete the file object that the next two lines are trying to > test? > > Cheers, > Nick. It gets created in the try block above, so the file was being created twice but deleted once. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinay_sajip at yahoo.co.uk Sun Oct 31 15:55:34 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sun, 31 Oct 2010 14:55:34 +0000 (UTC) Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles References: <20101029110702.414c298d@mission> Message-ID: Olemis Lang gmail.com> writes: > > On Fri, Oct 29, 2010 at 10:07 AM, Barry Warsaw python.org> wrote: > > I haven't played with it yet, but do you think it makes sense to add a > > 'style' keyword argument to basicConfig()? ?That would make it pretty easy > > to get the formatting style you want without having to explicitly > > instantiate a Formatter, at least for simple logging clients. > > > > Since this may be considered as a little sophisticated, I'd rather > prefer these new classes to be added to configuration sections using > fileConfig (and default behavior if missing), and still leave > `basicConfig` unchanged (i.e. *basic*) . > Actually it's no biggie to have an optional style argument for basicConfig. People who don't use it don't have to specify it; the style argument would only apply if format was specified. For some people, use of {} over % is more about personal taste than about the actual usage of str.format's flexibility; we may as well accommodate that preference, as it encourages in a small way the use of {}-formatting. Regards, Vinay Sajip From ncoghlan at gmail.com Sun Oct 31 16:24:52 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 1 Nov 2010 01:24:52 +1000 Subject: [Python-Dev] [Python-checkins] r86000 - python/branches/py3k/Lib/test/test_fileio.py In-Reply-To: References: <20101030235645.65C08EE996@mail.python.org> Message-ID: On Mon, Nov 1, 2010 at 12:55 AM, Brian Curtin wrote: > > It gets created in the try block above, so the file was being created twice > but deleted once. Ah, right. My brain read that "else:" as an except clause for some reason (possibly a sign that I should have gone to bed a while ago...). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From rdmurray at bitdance.com Sun Oct 31 16:54:26 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 31 Oct 2010 11:54:26 -0400 Subject: [Python-Dev] Change to logging Formatters: support for alternative format styles In-Reply-To: References: <20101029110702.414c298d@mission> Message-ID: <20101031155426.D37032194FE@kimball.webabinitio.net> On Sun, 31 Oct 2010 14:55:34 -0000, Vinay Sajip wrote: > Olemis Lang gmail.com> writes: > > On Fri, Oct 29, 2010 at 10:07 AM, Barry Warsaw python.org> wrote: > > > I haven't played with it yet, but do you think it makes sense to add a > > > 'style' keyword argument to basicConfig()? ??That would make it pretty easy > > > to get the formatting style you want without having to explicitly > > > instantiate a Formatter, at least for simple logging clients. > > > > > > > Since this may be considered as a little sophisticated, I'd rather > > prefer these new classes to be added to configuration sections using > > fileConfig (and default behavior if missing), and still leave > > `basicConfig` unchanged (i.e. *basic*) . > > > > Actually it's no biggie to have an optional style argument for basicConfig. > People who don't use it don't have to specify it; the style argument would only > apply if format was specified. > > For some people, use of {} over % is more about personal taste than about the > actual usage of str.format's flexibility; we may as well accommodate that > preference, as it encourages in a small way the use of {}-formatting. +1 -- R. David Murray www.bitdance.com From barry at python.org Sun Oct 31 17:11:50 2010 From: barry at python.org (Barry Warsaw) Date: Sun, 31 Oct 2010 12:11:50 -0400 Subject: [Python-Dev] Cleaning-up the new unittest API In-Reply-To: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> Message-ID: <20101031121150.3dee7990@mission> On Oct 29, 2010, at 08:14 PM, Raymond Hettinger wrote: >I would like to simplify and clean-up the API for the unittest module >by de-documenting assertSetEqual(), assertDictEqual(), >assertListEqual, and assertTupleEqual(). As a general principle, I think all public API methods should be documented. That still leaves plenty of room for simplification. Some ways to address both concerns could be: - moving the documentation to an "advanced" or "complete reference" section - make the methods non-public by prepending an underscore - leaving them public but adding deprecation warnings to the code >All of those methods are already called automatically by >assertEqual(), so those methods never need to be called directly. >Or, if you need to be more explicit about the type checking for >sequences, the assertSequenceEqual() method will suffice. >Either way, there's no need to expose the four type specific methods. It sounds like those methods should not be public then. >Given the purpose of the unittest module, it's important that >the reader have a crystal clear understanding of what a test >is doing. Their attention needs to be focused on the subject >of the test, not on questioning the semantics of the test method. That's different documentation than a complete reference manual. A reference manual should contain all public methods, functions, classes and attributes. It's there so that when you see some third party code that uses it, you have an authoritative description of the semantics of that method. We owe it to our users to have complete reference material. OTOH, we also owe them clear guidelines on best practices for the use of our API. Often, this is either obvious or can live in the same documentation. By sectioning the documentation, the module docs can be organized to give both a user guide with opinionated recommendations, and a complete reference manual. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Sun Oct 31 17:25:39 2010 From: barry at python.org (Barry Warsaw) Date: Sun, 31 Oct 2010 12:25:39 -0400 Subject: [Python-Dev] Continuing 2.x In-Reply-To: References: <2E034B571A5CE44E949B9FCC3B6D24EE5761FC57@exchcn.ccp.ad.local> <4CC9749E.1030200@v.loewis.de> <20101028120414.1f4f3016@mission> <2E034B571A5CE44E949B9FCC3B6D24EE5761FD38@exchcn.ccp.ad.local> <4CCACF52.8090202@egenix.com> <07F03E8C-347C-4E07-A6BC-7D0029F056EC@python.org> Message-ID: <20101031122539.39faf730@mission> On Oct 29, 2010, at 04:23 PM, Benjamin Peterson wrote: >At the moment, I'm planning to do regular maintenance releases for 3.1 and >2.7 roughly every 6 months. Cool. The actual interval doesn't matter as much as the regularity. I say that speaking as a semi-former RM who sadly didn't adhere to much regularity. ;/ Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From greg at krypto.org Sun Oct 31 17:54:11 2010 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 31 Oct 2010 09:54:11 -0700 Subject: [Python-Dev] Cleaning-up the new unittest API In-Reply-To: <20101031121150.3dee7990@mission> References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> <20101031121150.3dee7990@mission> Message-ID: On Sun, Oct 31, 2010 at 9:11 AM, Barry Warsaw wrote: > On Oct 29, 2010, at 08:14 PM, Raymond Hettinger wrote: > > >I would like to simplify and clean-up the API for the unittest module > >by de-documenting assertSetEqual(), assertDictEqual(), > >assertListEqual, and assertTupleEqual(). > > As a general principle, I think all public API methods should be > documented. > > That still leaves plenty of room for simplification. Some ways to address > both concerns could be: > > - moving the documentation to an "advanced" or "complete reference" section > Agreed, I perfer simply deemphasizing these methods by reorganizing the documentation and mentioning in their documentation to, "just use assertEqual." De-documenting them is the first step towards causing unnecessary pain by taking either of the next two steps: - make the methods non-public by prepending an underscore > - leaving them public but adding deprecation warnings to the code > Please do not make any existing released methods from the unittest module non-public or add any deprecation warnings. That will simply cause unnecessary code churn and pain for people porting their code from one version to the next without benefiting anyone. > >All of those methods are already called automatically by > >assertEqual(), so those methods never need to be called directly. > For new code I agree and we should document them as such. They _do_ have value on their own when called directly. They offer an explicit type check as part of their assertion and are much easier to read for that than manually passing the parameter to assertSequenceEqual(a, b, seq_type=xxx). How often they are used for that reason is hard for me to measure as the giant code base we have at work that uses these was written over the years prior to assertEqual automagically calling these methods based on input types so its uses tend to be a mix of cases where the type check doesn't matter and a small fraction of cases where it is important. The smarts about automagically calling an appropriate method from assertEqual were added during the sprints while contributing them at PyCon 2009. > >Or, if you need to be more explicit about the type checking for > >sequences, the assertSequenceEqual() method will suffice. > >Either way, there's no need to expose the four type specific methods. > > It sounds like those methods should not be public then. > I strongly prefer de-documenting them to making anything already released as public be nonpublic. That leads to pain for people moving to new versions of Python. There is no maintenance burden to keeping these trivial methods for the convenience of code that has already used them. > >Given the purpose of the unittest module, it's important that > >the reader have a crystal clear understanding of what a test > >is doing. Their attention needs to be focused on the subject > >of the test, not on questioning the semantics of the test method. > > That's different documentation than a complete reference manual. A > reference > manual should contain all public methods, functions, classes and > attributes. > It's there so that when you see some third party code that uses it, you > have > an authoritative description of the semantics of that method. We owe it to > our users to have complete reference material. > > OTOH, we also owe them clear guidelines on best practices for the use of > our > API. Often, this is either obvious or can live in the same documentation. > By > sectioning the documentation, the module docs can be organized to give both > a > user guide with opinionated recommendations, and a complete reference > manual. > Agreed. -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Sun Oct 31 18:44:54 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 31 Oct 2010 18:44:54 +0100 Subject: [Python-Dev] r86046 - python/branches/py3k/Lib/test/test_smtplib.py References: <20101031171543.1EB17E885@mail.python.org> Message-ID: <20101031184454.4f09e18e@pitrou.net> On Sun, 31 Oct 2010 18:15:43 +0100 (CET) benjamin.peterson wrote: > > # SimSMTPChannel doesn't fully support LOGIN or CRAM-MD5 auth because they > # require a synchronous read to obtain the credentials...so instead smtpd > @@ -503,6 +504,7 @@ > except smtplib.SMTPAuthenticationError as err: > if sim_auth_login_password not in str(err): > raise "expected encoded password not found in error message" > + smtp.close() Perhaps the string-raising above should be converted to 3.x-compliant code? > def testAUTH_CRAM_MD5(self): > self.serv.add_feature("AUTH CRAM-MD5") > @@ -512,6 +514,7 @@ > except smtplib.SMTPAuthenticationError as err: > if sim_auth_credentials['cram-md5'] not in str(err): > raise "expected encoded credentials not found in error message" > + smtp.close() Same here. From bobbyi at gmail.com Sun Oct 31 19:10:13 2010 From: bobbyi at gmail.com (Bobby Impollonia) Date: Sun, 31 Oct 2010 11:10:13 -0700 Subject: [Python-Dev] test_grp regression test fails with NIS entries present Message-ID: The regression tests for py3k (or, I think, any branch) fail on one of my machines because test_grp chokes if /etc/group contains a "+" line, which is a directive to pull information from NIS. The test enumerates all entries in /etc/group using grp.getgrall() and verifies that it can look up each entry by name using grp.getgrnam(). The current behavior of grp.getgrall() is to return entries whose names start with plus or minus signs (NIS-related lines) as if they were regular entries. The result is that the test tries to look up the name "+" and fails because no entry is found. It turns out that a bug on this issue has existed since 2003: http://bugs.python.org/issue775964 The bug originally indicated that the problem is specific to Red Hat, but that is not the case because I ran into it on Debian Squeeze. According to a comment on the bug, this syntax in the group file has been deprecated for a long time, which is why the issue rarely comes up. I believe the right thing to do at this point is to keep the behavior of grp.getgrall(), but document that it will return NIS-related lines along with regular entries and to modify the test to not try to look up those entries with grp.getgrnam(). I've attached a patch to the bug that makes these changes and results in the test passing. Is it possible that someone can review/ checkin the patch? Thanks. From eric at trueblade.com Sun Oct 31 21:39:44 2010 From: eric at trueblade.com (Eric Smith) Date: Sun, 31 Oct 2010 16:39:44 -0400 Subject: [Python-Dev] str.format_from_mapping Message-ID: <4CCDD410.6020104@trueblade.com> What are your thoughts on adding a str.format_from_mapping (or similar name, maybe the suggested "format_map") to 3.2? See http://bugs.python.org/issue6081 . This method would be similar to "%(foo)s %(bar)s" % d, where d is a dict (or rather any mapping object), but of course would use str.format syntax: "{foo} {bar}".format_from_mapping(d). I like the idea. It's particularly handy when converting from %-formatting. Eric. From solipsis at pitrou.net Sun Oct 31 21:55:51 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 31 Oct 2010 21:55:51 +0100 Subject: [Python-Dev] str.format_from_mapping References: <4CCDD410.6020104@trueblade.com> Message-ID: <20101031215551.3d4c2aec@pitrou.net> On Sun, 31 Oct 2010 16:39:44 -0400 Eric Smith wrote: > What are your thoughts on adding a str.format_from_mapping (or similar > name, maybe the suggested "format_map") to 3.2? See > http://bugs.python.org/issue6081 . This method would be similar to > "%(foo)s %(bar)s" % d, where d is a dict (or rather any mapping object), > but of course would use str.format syntax: "{foo} > {bar}".format_from_mapping(d). I must be missing something, but what's the difference with XXX.format(**d)? From benjamin at python.org Sun Oct 31 22:02:08 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 31 Oct 2010 16:02:08 -0500 Subject: [Python-Dev] str.format_from_mapping In-Reply-To: <20101031215551.3d4c2aec@pitrou.net> References: <4CCDD410.6020104@trueblade.com> <20101031215551.3d4c2aec@pitrou.net> Message-ID: 2010/10/31 Antoine Pitrou : > On Sun, 31 Oct 2010 16:39:44 -0400 > Eric Smith wrote: > >> What are your thoughts on adding a str.format_from_mapping (or similar >> name, maybe the suggested "format_map") to 3.2? See >> http://bugs.python.org/issue6081 . This method would be similar to >> "%(foo)s %(bar)s" % d, where d is a dict (or rather any mapping object), >> but of course would use str.format syntax: "{foo} >> {bar}".format_from_mapping(d). > > I must be missing something, but what's the difference with > XXX.format(**d)? It allows arbitrary mappings. -- Regards, Benjamin From solipsis at pitrou.net Sun Oct 31 22:08:17 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 31 Oct 2010 22:08:17 +0100 Subject: [Python-Dev] str.format_from_mapping In-Reply-To: References: <4CCDD410.6020104@trueblade.com> <20101031215551.3d4c2aec@pitrou.net> Message-ID: <20101031220817.1436b003@pitrou.net> On Sun, 31 Oct 2010 16:02:08 -0500 Benjamin Peterson wrote: > 2010/10/31 Antoine Pitrou : > > On Sun, 31 Oct 2010 16:39:44 -0400 > > Eric Smith wrote: > > > >> What are your thoughts on adding a str.format_from_mapping (or similar > >> name, maybe the suggested "format_map") to 3.2? See > >> http://bugs.python.org/issue6081 . This method would be similar to > >> "%(foo)s %(bar)s" % d, where d is a dict (or rather any mapping object), > >> but of course would use str.format syntax: "{foo} > >> {bar}".format_from_mapping(d). > > > > I must be missing something, but what's the difference with > > XXX.format(**d)? > > It allows arbitrary mappings. Ah, yes, sorry. I should have read the issue before posting. It's a bit embarassing to add a method for that, though, while formatting regular dicts doesn't need it. Regards Antoine. From tjreedy at udel.edu Sun Oct 31 22:24:37 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 31 Oct 2010 17:24:37 -0400 Subject: [Python-Dev] r86046 - python/branches/py3k/Lib/test/test_smtplib.py In-Reply-To: <20101031184454.4f09e18e@pitrou.net> References: <20101031171543.1EB17E885@mail.python.org> <20101031184454.4f09e18e@pitrou.net> Message-ID: On 10/31/2010 1:44 PM, Antoine Pitrou wrote: > On Sun, 31 Oct 2010 18:15:43 +0100 (CET) > benjamin.peterson wrote: >> >> # SimSMTPChannel doesn't fully support LOGIN or CRAM-MD5 auth because they >> # require a synchronous read to obtain the credentials...so instead smtpd >> @@ -503,6 +504,7 @@ >> except smtplib.SMTPAuthenticationError as err: >> if sim_auth_login_password not in str(err): >> raise "expected encoded password not found in error message" >> + smtp.close() > > Perhaps the string-raising above should be converted to 3.x-compliant > code? Since raise 'string' itself raises a TypeError in 3.x, it must be that the raise statement has never been executed in 3.x testing or that the TypeError has not been noticed to be an erroneous error. >> def testAUTH_CRAM_MD5(self): >> self.serv.add_feature("AUTH CRAM-MD5") >> @@ -512,6 +514,7 @@ >> except smtplib.SMTPAuthenticationError as err: >> if sim_auth_credentials['cram-md5'] not in str(err): >> raise "expected encoded credentials not found in error message" >> + smtp.close() > > Same here. -- Terry Jan Reedy From alexander.belopolsky at gmail.com Sun Oct 31 22:52:42 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 31 Oct 2010 17:52:42 -0400 Subject: [Python-Dev] [Python-checkins] r86066 - python/branches/py3k/Doc/library/functions.rst In-Reply-To: <20101031212324.D92E1EE99E@mail.python.org> References: <20101031212324.D92E1EE99E@mail.python.org> Message-ID: On Sun, Oct 31, 2010 at 5:23 PM, raymond.hettinger wrote: .. > + ? For some use cases, there a good alternatives to :func:`sum`. This sentence is missing a verb. From v+python at g.nevcal.com Sun Oct 31 23:28:10 2010 From: v+python at g.nevcal.com (Glenn Linderman) Date: Sun, 31 Oct 2010 15:28:10 -0700 Subject: [Python-Dev] str.format_from_mapping In-Reply-To: References: <4CCDD410.6020104@trueblade.com> <20101031215551.3d4c2aec@pitrou.net> Message-ID: <4CCDED7A.6080706@g.nevcal.com> On 10/31/2010 2:02 PM, Benjamin Peterson wrote: > 2010/10/31 Antoine Pitrou: >> > On Sun, 31 Oct 2010 16:39:44 -0400 >> > Eric Smith wrote: >> > >>> >> What are your thoughts on adding a str.format_from_mapping (or similar >>> >> name, maybe the suggested "format_map") to 3.2? See >>> >> http://bugs.python.org/issue6081 . This method would be similar to >>> >> "%(foo)s %(bar)s" % d, where d is a dict (or rather any mapping object), >>> >> but of course would use str.format syntax: "{foo} >>> >> {bar}".format_from_mapping(d). >> > >> > I must be missing something, but what's the difference with >> > XXX.format(**d)? > It allows arbitrary mappings. Other than the language moratorium, why are arbitrary mappings not allowed for the (**d) syntax? -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Sun Oct 31 23:34:12 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 31 Oct 2010 17:34:12 -0500 Subject: [Python-Dev] str.format_from_mapping In-Reply-To: <4CCDED7A.6080706@g.nevcal.com> References: <4CCDD410.6020104@trueblade.com> <20101031215551.3d4c2aec@pitrou.net> <4CCDED7A.6080706@g.nevcal.com> Message-ID: 2010/10/31 Glenn Linderman : > On 10/31/2010 2:02 PM, Benjamin Peterson wrote: > > 2010/10/31 Antoine Pitrou : > >> On Sun, 31 Oct 2010 16:39:44 -0400 >> Eric Smith wrote: >> > >>> What are your thoughts on adding a str.format_from_mapping (or similar >>> name, maybe the suggested "format_map") to 3.2? See >>> http://bugs.python.org/issue6081 . This method would be similar to >>> "%(foo)s %(bar)s" % d, where d is a dict (or rather any mapping object), >>> but of course would use str.format syntax: "{foo} >>> {bar}".format_from_mapping(d). > >> >> I must be missing something, but what's the difference with >> XXX.format(**d)? > > It allows arbitrary mappings. > > Other than the language moratorium, why are arbitrary mappings not allowed > for the (**d) syntax? They are; they're just copied into a dictionary. -- Regards, Benjamin From eric at trueblade.com Sun Oct 31 23:32:12 2010 From: eric at trueblade.com (Eric Smith) Date: Sun, 31 Oct 2010 18:32:12 -0400 Subject: [Python-Dev] str.format_from_mapping In-Reply-To: <4CCDED7A.6080706@g.nevcal.com> References: <4CCDD410.6020104@trueblade.com> <20101031215551.3d4c2aec@pitrou.net> <4CCDED7A.6080706@g.nevcal.com> Message-ID: <4CCDEE6C.1010901@trueblade.com> On 10/31/2010 6:28 PM, Glenn Linderman wrote: > On 10/31/2010 2:02 PM, Benjamin Peterson wrote: >> 2010/10/31 Antoine Pitrou: >>> > On Sun, 31 Oct 2010 16:39:44 -0400 >>> > Eric Smith wrote: >>> > >>>> >> What are your thoughts on adding a str.format_from_mapping (or similar >>>> >> name, maybe the suggested "format_map") to 3.2? See >>>> >> http://bugs.python.org/issue6081 . This method would be similar to >>>> >> "%(foo)s %(bar)s" % d, where d is a dict (or rather any mapping object), >>>> >> but of course would use str.format syntax: "{foo} >>>> >> {bar}".format_from_mapping(d). >>> > >>> > I must be missing something, but what's the difference with >>> > XXX.format(**d)? >> It allows arbitrary mappings. > > Other than the language moratorium, why are arbitrary mappings not > allowed for the (**d) syntax? An arbitrary mapping would be converted to a dict. That disallows using a subclass with __missing__ defined, among other things. See the discussion in the issue. From matthew at woodcraft.me.uk Sun Oct 31 23:47:58 2010 From: matthew at woodcraft.me.uk (Matthew Woodcraft) Date: Sun, 31 Oct 2010 22:47:58 +0000 (UTC) Subject: [Python-Dev] Cleaning-up the new unittest API References: <20866584-B4E0-4F97-8086-87455A836053@gmail.com> <4CCB9754.5000508@voidspace.org.uk> <4CCB9AD7.8020504@voidspace.org.uk> Message-ID: Raymond Hettinger wrote: > I looked at this again and think we should just remove > assertItemsEqual() from Py3.2 and dedocument it in Py2.7. It is listed > as being new in 3.2 so nothing is lost. One thing that would be lost is that correct Python 2.7 code using assertItemsEqual would no longer run on 3.2. > The sole benefit over the more explicit variants like > assertEqual(set(a), set(b)) and assertEqual(sorted(a), sorted(b)) is > that it handles a somewhat rare corner case where neither of those > work (unordered comparison of non-compable types when you do care > about duplicates). Another benefit is that it gives better descriptions of differences. See http://bugs.python.org/issue9977 for an example. -M- From v+python at g.nevcal.com Sun Oct 31 23:54:24 2010 From: v+python at g.nevcal.com (Glenn Linderman) Date: Sun, 31 Oct 2010 15:54:24 -0700 Subject: [Python-Dev] str.format_from_mapping In-Reply-To: <4CCDEE6C.1010901@trueblade.com> References: <4CCDD410.6020104@trueblade.com> <20101031215551.3d4c2aec@pitrou.net> <4CCDED7A.6080706@g.nevcal.com> <4CCDEE6C.1010901@trueblade.com> Message-ID: <4CCDF3A0.4020107@g.nevcal.com> On 10/31/2010 3:32 PM, Eric Smith wrote: > On 10/31/2010 6:28 PM, Glenn Linderman wrote: >> On 10/31/2010 2:02 PM, Benjamin Peterson wrote: >>> 2010/10/31 Antoine Pitrou: >>>> > On Sun, 31 Oct 2010 16:39:44 -0400 >>>> > Eric Smith wrote: >>>> > >>>>> >> What are your thoughts on adding a str.format_from_mapping (or >>>>> similar >>>>> >> name, maybe the suggested "format_map") to 3.2? See >>>>> >> http://bugs.python.org/issue6081 . This method would be >>>>> similar to >>>>> >> "%(foo)s %(bar)s" % d, where d is a dict (or rather any >>>>> mapping object), >>>>> >> but of course would use str.format syntax: "{foo} >>>>> >> {bar}".format_from_mapping(d). >>>> > >>>> > I must be missing something, but what's the difference with >>>> > XXX.format(**d)? >>> It allows arbitrary mappings. >> >> Other than the language moratorium, why are arbitrary mappings not >> allowed for the (**d) syntax? > > An arbitrary mapping would be converted to a dict. Yes, but why convert? I suppose it has something to do with the case where there are named parameters, in addition to a **kwargs formal, and the mixing and matching that happens as a result. -------------- next part -------------- An HTML attachment was scrubbed... URL: